[Zope] UnicodeDecodeError in Zope 2.10.4 (upgrade from 2.8.4)

Peter Bengtsson peter at fry-it.com
Tue Jul 17 07:06:33 EDT 2007


Both
<em tal:content="python: 'żółć'">template id</em>.
and
<em tal:content="structure python: 'żółć'">template id</em>
is like playing with fire. Don't do it.

What you've got there is unicode characters written down without any 
encoding information.

It will work if you set the internal ZPT encoding to be the same as it 
was entered into the template which I can see is not ASCII which is what 
your system defaults to.

This is internally what happens:
 >>> x="\xc5"
 >>> unicode(x)
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc5 in position 0: 
ordinal not in range(128)
 >>> import sys; sys.getdefaultencoding()
'ascii'




Maciej Wisniowski wrote:
> I've upgraded my installation from Zope2.8.4 to Zope 2.10.4
> (by copying data.fs, Products/ etc.). I have publisher encoding
> and management_page_charset set to utf-8. Also system
> default encoding is utf-8.
> 
> Zope 2.10 is said to have better Unicode support with
> UnicodeEncodeConflictResolver. It is but unfortunatelly in
> some cases this is not enough. Seems that
> code that is 'protected' by Resolver is not the only code
> that may be affected by non unicode strings.
> Simple example is with 'structure' keyword.
> 
> eg:
> This works (resolver resolves conflict):
> <em tal:content="python: 'żółć'">template id</em>.
> 
> This doesn't work:
> <em tal:content="structure python: 'żółć'">template id</em>.
> 
> Also if you have Folder instance and set it's Title
> to the string that contains some i18n characters
> you're not able to even add page template
> inside it.
> 
> Error traceback in both cases is same:
> 
> Error Type: UnicodeDecodeError
> Error Value: 'ascii' codec can't decode byte 0xc5 in position 200:
> ordinal not in range(128)
> 
> Traceback (innermost last):
> Module ZPublisher.Publish, line 119, in publish
> Module ZPublisher.mapply, line 88, in mapply
> Module ZPublisher.Publish, line 42, in call_object
> Module Shared.DC.Scripts.Bindings, line 313, in __call__
> Module Shared.DC.Scripts.Bindings, line 350, in _bindAndExec
> Module Products.PageTemplates.PageTemplateFile, line 129, in _exec
> Module Products.PageTemplates.PageTemplate, line 89, in pt_render
> Module zope.pagetemplate.pagetemplate, line 117, in pt_render
> Module zope.tal.talinterpreter, line 271, in __call__
> Module zope.tal.talinterpreter, line 346, in interpret
> Module zope.tal.talinterpreter, line 534, in do_optTag_tal
> Module zope.tal.talinterpreter, line 516, in no_tag
> Module zope.tal.talinterpreter, line 346, in interpret
> Module zope.tal.talinterpreter, line 754, in do_insertStructure_tal
> UnicodeDecodeError: 'ascii' codec can't decode byte 0xc5 in position
> 200: ordinal not in range(128)
> 
> problem is with:
>   text = unicode(structure)
> at
> Module zope.tal.talinterpreter, line 754, in do_insertStructure_tal
> 
> sitecustomize.py with setdefaultencoding('utf-8') solves this
> but it is not nice solution. I wonder whether this should be
> submitted as a bug, or maybe there is different solution that
> I've missed?
> 
> For the record: one more thing that was wrong
> for me during migration was 'expand' attribute of ZPT
> that in case of few ZPT objects was set to true in
> 2.10.4 while oryginally in 2.8.4 it was false.
> This caused that tales expressions disappeared
> from ZPTs under 2.10.4. I've written a script that
> explicitly set expand=0 to all ZPT instances in
> Zope 2.8.4 and then, after migration, everything is OK.
> 

-- 
Peter Bengtsson,
work www.fry-it.com
home www.peterbe.com
hobby www.issuetrackerproduct.com


More information about the Zope mailing list