[zope2-tracker] [Bug 143431] Re: Unicode error in PageTemplate

Thu Sep 24 10:59:15 EDT 2009

We have the same problem with simple 'latin-1' encoding. PageTemplate is
designed in the way that it cannot handle both Unicode and plain strings
with non-ascii characters in the output of the same page. Here is the
simplest way to reproduce that error in Zope 2.9:

<html>
  <body>
    <p tal:content="python: 'Non-ASCII: \xe9'" />
    <p tal:content="python: u'Unicode: \u9053'" />
  </body>
</html>

Here is our monkey-patch. It can handle different encodings in the different parts of Zope site. Separate encoding can be forced in any particular Folder by setting its "zpt_output_encoding" string property. Patch also accepts existing "management_page_charset" properties. Without any encoding set patch reproduces original behavior and raises the same UnicodeDecodeError in the same place.

Put it into something like "Products/ZPTHotfix/__init__.py" to activate:

"""Allow Unicode in the PageTemplates output"""

import new

from Products.PageTemplates.PageTemplate import FasterStringIO, PageTemplate
from Products.PageTemplates.ZopePageTemplate import ZopePageTemplate

class UnicodeAwareStringIO(FasterStringIO):

    def __init__(self, encoding):
        self.encoding = encoding
        FasterStringIO.__init__(self)

    def write(self, s):
        if isinstance(s, unicode):
            s = s.encode(self.encoding, 'xmlcharrefreplace')
        return FasterStringIO.write(self, s)

def patched_StringIO(self):
    # Comment from the original PageTemplate.StringIO():
        # Third-party products wishing to provide a full Unicode-aware
        # StringIO can do so by monkey-patching this method.
    encoding = getattr(self, 'zpt_output_encoding', None) \
               or getattr(self, 'management_page_charset', None)
    if encoding:
        return UnicodeAwareStringIO(encoding)
    else:
        return PageTemplate.StringIO(self)

def patch_zpt():
    # apply monkey-patch
    ZopePageTemplate._originalStringIO = ZopePageTemplate.StringIO
    ZopePageTemplate.StringIO = new.instancemethod(patched_StringIO, None,
                                                   ZopePageTemplate)

def revert_zpt():
    # restore original method
    original_method = getattr(ZopePageTemplate, '_originalStringIO', None)
    if original_method and ZopePageTemplate.StringIO != original_method:
        ZopePageTemplate.StringIO = original_method

patch_zpt()

-- 
Unicode error in PageTemplate
https://bugs.launchpad.net/bugs/143431
You received this bug notification because you are a member of Zope 2
Developers, which is subscribed to Zope 2.