[Zope-CMF] Re: Problem with STX rendering

[Tres Seaver]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Charlie Clark wrote:
> Am 11.05.2007 um 01:09 schrieb Alexander Limi:
> 
>>
>> On Thu, 10 May 2007 07:28:49 -0700, Charlie Clark  
>> <charlie-6bEcJwdP2pR+28uErXmBHg at public.gmane.org> wrote:
>>
>>> Anyway before I submit a follow-up to the bug is there any CMF  
>>> setting I can use to avoid this problem?
>> It's a known issue with STX.
> 
> Thanks, does that mean anything's being done about it? I would have  
> thought Plone users would be suffering the most with it.
> 
> I spent some time looking at the underlying code yesterday and can  
> see what's causing the problem. If I take my own medicine the STX  
> code should be fixed.
> 
> STLetters simply makes use of string.letters and string.digits which  
> are then thrown into various re's
> 
> expr = re.compile(r'\*\*([%s%s%s\s]+?)\*\*' % (letters, digits,  
> strongem_punc))
> 
> This will work with non-ascii only if the encoding for string.letters  
> and the text to be compared is the same which, certainly on my  
> machine is not the case, but can be easily fixed.
> 
> letters = string.letters.decode(locale.getlocale()[1].encode 
> (WHATEVERCMFISPUSHING)

It's worse, in that you have to re-encode to the charset being used by
*that* STX document, which there is no good algorithmic way to determine
after the fact.  Only the HTTP Request which produced the document (in
the case of a ZODB-based thing like CMFDefault.Document) has enough
information to establish the encoding being used.

> I thought that the default_zpublisher_encoding would do but that's  
> wrong. Assuming it's possible to get the value from Zope surely this  
> would be a relatively painless fix?
> 
> It might be an idea would be to hook into the ReST encoding  
> directives that are mysteriously included in /etc/zope.conf
> 
> An alternative would be to coerce the use of unicode and simply  
> compare against that:
> re.compile(r'\*\*([\s\w]+?)\*\*', re.U) #extended as necessary for  
> punctuation
> 
> The advantage of this is that it would be entirely independent of  
> locale but this would take more work on STX which seems to choke if  
> fed unicode.

If converted during the upload, you solve the "which encoding is this
document in?" problem.

> Going back briefly to ReST: if Zope seems ready to support this out  
> of the box, wouldn't it useful to support ReST in  
> CMFDefault.Document? Or is the dependency on docutils too precarious?  
> It could be added gracefully to the edit form if import  
> reStructuredText succeeded. Or is this yet another case of me being  
> behind the loop again?

That might be simpler than the change I have wanted to make, which is to
register named utilities for a new utility interface, and then populate
the drop-down for "Format" with all registered utilities.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGRI1d+gerLs4ltQ4RAvDmAJ9O6cIuYqKtmzgN5HpxkctOoO88qgCfc29s
94p20FM36hob+SHq48+YhiI=
=rS7Y
-----END PGP SIGNATURE-----