[Zope-CMF] Fixing STX for non-ascii

[Charlie Clark]

Hi,

since my patch to support ReST for Documents (& Newsitems - thanks to  
Jens for this), I've gone back to looking at what was my original  
problem: STX choking with non-ascii text. From my tests this  
afternoon this looks surprisingly easy to fix. This is a sample  
method from zope.structuredtext.document.py (which is used for Zope  
2.10 and up but looks little more than a simple reimplementation of  
the StructuredText package)

     def doc_strong(self,
                    s,
                    expr = re.compile('\*\*([\w%s\s]+?)\*\*' %  
(strongem_punc), re.UNICODE).search # works with non-ASCII
                    # expr = re.compile(r'\*\*([%s%s%s\s]+?)\*\*' %  
(letters, digits, strongem_punc)).search # fails with non-ASCII
                    #expr = re.compile(r'\s*\*\*([ \n\r%s0-9.:/;, 
\'\"\?\-\_\/\=\-\>\<\(\)]+)\*\*(?!\*|-)' % letters).search, # old  
expr, inconsistent punc, failed to cross newlines.
         ):

         r=expr(s)
         if r:
             start, end = r.span(1)
             return (stng.StructuredTextStrong(s[start:end]),  
start-2, end+2)

It seems simply adding the re.UNICODE flag and using \w rather than  
string.letters + string.digits is sufficient. However, given my  
relative inexperience with regexes this could simply be naïvety on my  
part.

If this does indeed work does a patch need submitting for both Zope  
2.1x and Zope 3?

Charlie
--
Charlie Clark
Helmholtzstr. 20
Düsseldorf
D- 40215
Tel: +49-211-938-5360
GSM: +49-178-782-6226