[Zope-dev] UnicodeDecodeError from PageTemplate rendering after upgrading to Zope 2.12.7

[Charlie Clark]

Am 31.08.2010, 12:51 Uhr, schrieb Chris Withers <chris at simplistix.co.uk>:

> Any idea what I should be looking for and in what changelog?

This my horribly patched version of charset.py which we just use via an  
override. I know it's wrong but it was the easiest thing to do at a time.

class HTTPCharsets(object):
     """This is a complete copy of zope.publisher.http.HTTPCharset
     except if there is no charset specified. Zope defaults to
     ISO-8859-1 and we use the portal default_charset"""
     adapts(IHTTPRequest)
     implements(IUserPreferredCharsets)

     def __init__(self, request):
         self.request = request

     def getPreferredCharsets(self):
         '''See interface IUserPreferredCharsets'''
         charsets = []
         sawstar = sawiso88591 = 0
         header_present = 'HTTP_ACCEPT_CHARSET' in self.request
         for charset in self.request.get('HTTP_ACCEPT_CHARSET',  
'').split(','):
             charset = charset.strip().lower()
             if charset:
                 if ';' in charset:
                     charset, quality = charset.split(';')
                     if not quality.startswith('q='):
                         # not a quality parameter
                         quality = 1.0
                     else:
                         try:
                             quality = float(quality[2:])
                         except ValueError:
                             continue
                 else:
                     quality = 1.0
                 if quality == 0.0:
                     continue
                 if charset == '*':
                     sawstar = 1
                 if charset == 'iso-8859-1':
                     sawiso88591 = 1
                 charsets.append((quality, charset))
         # Quoting RFC 2616, $14.2: If no "*" is present in an  
Accept-Charset
         # field, then all character sets not explicitly mentioned get a
         # quality value of 0, except for ISO-8859-1, which gets a quality
         # value of 1 if not explicitly mentioned.
         # And quoting RFC 2616, $14.2: "If no Accept-Charset header is
         # present, the default is that any character set is acceptable."
         if not sawstar and not sawiso88591 and header_present:
             ptool = getUtility(IPropertiesTool)
             default_charset = ptool.getProperty('default_charset', None)
             charsets.append((1.0, default_charset))
             # charsets.append((1.0, 'iso-8859-1'))
         # UTF-8 is **always** preferred over anything else.
         # Reason: UTF-8 is not specific and can encode the entire unicode
         # range , unlike many other encodings. Since Zope can easily use  
very
         # different ranges, like providing a French-Chinese dictionary, it  
is
         # always good to use UTF-8.
         # charsets.sort(sort_charsets)
         charsets = [charset for quality, charset in charsets]
         if sawstar and charsets[0] != 'utf-8':
             if 'utf-8' in charsets:
                 idx = charsets.index('utf-8')
                 del charsets[idx]
             charsets.insert(0, 'utf-8')
         return charsets

The related bug in zope.publisher may or may not have found its way into  
the current release.

https://bugs.launchpad.net/zope2/+bug/143873

In addition, but I don't know how relevant this is in your case, Jens  
improved FSPageTemplate

https://bugs.launchpad.net/zope-cmf/+bug/322263

Charlie
-- 
Charlie Clark
Managing Director
Clark Consulting & Research
German Office
Helmholtzstr. 20
Düsseldorf
D- 40215
Tel: +49-211-600-3657
Mobile: +49-178-782-6226