[Zope] Re: Zope newbie.... unicode problem

[Josef Meile]

Hi Tyler

> When serving a web page, we get an error that involves the 
> fetching and display of a document to the browser. Evidently, 
> it's choking on some unicode bytes when it's expecting plain 
> ascii.  
> 
> I'm assuming it's
> /usr/lib/zope2.7/lib/python/Products/PageTemplates/PageTemplate.py
> that's causing the problem as the data isn't somehow massaged correctly
> before processing arrives at /usr/lib/python2.3/StringIO.py line 203.
So, you are using zope 2.7 and for the location I guess it is a binary.

> 
> I don't know if the bad byte is in a zope template, document, or
> a database record that it's fetching.  I do believe this installation
> of Zope uses Plone... but I've no idea whether Plone is somewhere
> used in the error stream of all the different calls.
> 
> Do I need to post more information?  Tell me .. and I'll get it.
Yes, perhaps you could post the template generating that error. If it is 
really long, try to localize the problem by removing parts of the 
template and test it continuously till you find the offending code.

Recently I got some similar problem. I'm using zope 2.8.8 and I taught I 
was affected by this issue:
http://www.zope.org/Collectors/Zope/2180

However, after adjusting that patch for 2.8.8 (it is for 2.10), I was 
still getting the problem. Then I figured out that somewhere in my code 
I was getting the title from an object and comparing it with an unicode 
string. The title turned to be a normal string with unicode parameters. ie:

a = u'This is unicode \xc3\xa4\xc3\xb6\xc3\xbc'

#this has unicode characters äöü, but it isn't a Unicode string
#Note that the u' is missing
b = '\xc3\xa4\xc3\xb6\xc3\xbc'

#This will fail
print a == b

#This is what you need to do

a = u'This is unicode äöü'
b = 'This is ascii'
print a == b.decode('utf-8')

I was doing that in a python product, so I could do this:
from types import StringType
if type(b) == StringType:
   b = b.decode('utf-8')

I don't know if you can do that in zpt. You also have to know which 
encoding you are using. In my case it is always utf-8

You may also trying to see if your plone version supports your zope 
version, which is old (Open the README or INSTALL file).

Try also to upgrage your zope. Perhaps this will solve your problem.

Regards
Josef