[Zope] Problem with non-latin character sets in Page Templates

Mon May 17 18:11:57 EDT 2010

On 05/17/2010 10:03 PM, Christopher N. Deckard wrote:
> Hello,
> Due to ICANN's decision to allow for non-Latin characters in domain
> names, I thought I'd give our Zope installation a test to see how it
> would handle it.  We've had problems with people copy and pasting from
> MS Word, and its use of strange characters.  I'm sure at some point the
> faculty here at Purdue will have a need to link to web sites using
> non-Latin characters.
>
> This is my example HTML:
>
> <html>
>   <head>
>     <title>URL Test</title>
>   </head>
>   <body>
>     <a
> href="http://EHB9.H2'1)-'D#*5'D'*.E51">http://EHB9.H2'1)-'D#*5'D'*.E51</a>.
>   </body>
> </html>
>
>
> Not only is it a different character set, but it is a right to left
> character set.  This code works fine while editing a Page Template.
> However, when viewing it, all of those characters in the href are
> converted to question marks.  If the same HTML is pasted into a DTML
> Method or a File object, Zope will convert the characters to ASCII
> characters.  When viewed it will be viewed correctly.
>
> Any reason that Page Templates may fail to render this properly?
>
It is a bug of page template. I patched them

zope.pagetemplate-3.5.0-py2.6.egg/zope/pagetemplate/pagetemplate.py

line 116     return output.getvalue()

to

116     # --- unicode/utf-8 hotfix ---
       for idx in range(0, len(output.buflist)):
           try:
               output.buflist[idx] = unicode(output.buflist[idx], 'utf-8')
           except (UnicodeDecodeError):
               output.buflist[idx] = output.buflist[idx].decode('utf-8',
'replace')
           except:
               pass

       return output.getvalue()
       # --- unicode/utf-8 hotfix ---

_______________________________________________
Zope maillist  -  Zope at zope.org
https://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists -
 https://mail.zope.org/mailman/listinfo/zope-announce
 https://mail.zope.org/mailman/listinfo/zope-dev )

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.zope.org/pipermail/zope/attachments/20100518/cc28bf43/attachment.html 