Zope newbie.... unicode problem
Greetings all, I'm 100% new to Zope. A complete Zope newbie. I've done web Work for more than a dozen years... just never with Zope. When serving a web page, we get an error that involves the fetching and display of a document to the browser. Evidently, it's choking on some unicode bytes when it's expecting plain ascii. I'm assuming it's /usr/lib/zope2.7/lib/python/Products/PageTemplates/PageTemplate.py that's causing the problem as the data isn't somehow massaged correctly before processing arrives at /usr/lib/python2.3/StringIO.py line 203. I don't know if the bad byte is in a zope template, document, or a database record that it's fetching. I do believe this installation of Zope uses Plone... but I've no idea whether Plone is somewhere used in the error stream of all the different calls. Do I need to post more information? Tell me .. and I'll get it. If this comes down to a patch, I've no idea how-to apply a patch once I get my grubby mits upon the patch. So, I'll need a bit of help with that as well. If I've posted this in the wrong place.. please forgive me and point me to the right place that I might ask the proper folk to help me. Thanks a lot .. and thanks in advance Tyler ====================TRACEBACK=============================== 2006-11-29T11:03:37 ERROR(200) SiteError http://www.teamhancock.com/help/bb-install Traceback (most recent call last): File "/usr/lib/zope2.7/lib/python/ZPublisher/Publish.py", line 101, in publish request, bind=1) File "/usr/lib/zope2.7/lib/python/ZPublisher/mapply.py", line 88, in mapply if debug is not None: return debug(object,args,context) File "/usr/lib/zope2.7/lib/python/ZPublisher/Publish.py", line 39, in call_object result=apply(object,args) # Type s<cr> to step into published object. File "/var/lib/zope2.7/instance/zeo/Products/CMFDefault/SkinnedFolder.py", line 80, in __call__ return view() File "/usr/lib/zope2.7/lib/python/Shared/DC/Scripts/Bindings.py", line 306, in __call__ return self._bindAndExec(args, kw, None) File "/usr/lib/zope2.7/lib/python/Shared/DC/Scripts/Bindings.py", line 343, in _bindAndExec return self._exec(bound_data, args, kw) File "/var/lib/zope2.7/instance/zeo/Products/CMFCore/FSPageTemplate.py", line 191, in _exec result = self.pt_render(extra_context=bound_names) File "/var/lib/zope2.7/instance/zeo/Products/CMFCore/FSPageTemplate.py", line 124, in pt_render result = FSPageTemplate.inheritedAttribute('pt_render')( File "/usr/lib/zope2.7/lib/python/Products/PageTemplates/PageTemplate.py", line 97, in pt_render return output.getvalue() File "/usr/lib/python2.3/StringIO.py", line 203, in getvalue self.buf += ''.join(self.buflist) UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 501: ordinal not in range(128) -- Tyler Nally tnally@teamhancock.com
Hi Tyler
When serving a web page, we get an error that involves the fetching and display of a document to the browser. Evidently, it's choking on some unicode bytes when it's expecting plain ascii.
I'm assuming it's /usr/lib/zope2.7/lib/python/Products/PageTemplates/PageTemplate.py that's causing the problem as the data isn't somehow massaged correctly before processing arrives at /usr/lib/python2.3/StringIO.py line 203. So, you are using zope 2.7 and for the location I guess it is a binary.
I don't know if the bad byte is in a zope template, document, or a database record that it's fetching. I do believe this installation of Zope uses Plone... but I've no idea whether Plone is somewhere used in the error stream of all the different calls.
Do I need to post more information? Tell me .. and I'll get it.
Yes, perhaps you could post the template generating that error. If it is really long, try to localize the problem by removing parts of the template and test it continuously till you find the offending code. Recently I got some similar problem. I'm using zope 2.8.8 and I taught I was affected by this issue: http://www.zope.org/Collectors/Zope/2180 However, after adjusting that patch for 2.8.8 (it is for 2.10), I was still getting the problem. Then I figured out that somewhere in my code I was getting the title from an object and comparing it with an unicode string. The title turned to be a normal string with unicode parameters. ie: a = u'This is unicode \xc3\xa4\xc3\xb6\xc3\xbc' #this has unicode characters äöü, but it isn't a Unicode string #Note that the u' is missing b = '\xc3\xa4\xc3\xb6\xc3\xbc' #This will fail print a == b #This is what you need to do a = u'This is unicode äöü' b = 'This is ascii' print a == b.decode('utf-8') I was doing that in a python product, so I could do this: from types import StringType if type(b) == StringType: b = b.decode('utf-8') I don't know if you can do that in zpt. You also have to know which encoding you are using. In my case it is always utf-8 You may also trying to see if your plone version supports your zope version, which is old (Open the README or INSTALL file). Try also to upgrage your zope. Perhaps this will solve your problem. Regards Josef
Tyler Nally wrote at 2006-11-29 15:49 -0500:
... When serving a web page, we get an error that involves the fetching and display of a document to the browser. Evidently, it's choking on some unicode bytes when it's expecting plain ascii.
Unfortunately, this is a *VERY* frequent problem. It occurs when a template mixes unicode and some non unicode string containing non ASCII characters. There are several options: * Do not use unicode at all -- always convert to something like your "site encoding" * Use unicode whenever you handle text * Set Python's "defaultencoding" to your "site encoding" (provided you have a consistent site encoding) Searching the mailing list archives will show you hundreds of similar threads and may give further information. -- Dieter
participants (3)
-
Dieter Maurer -
Josef Meile -
Tyler Nally