[Zope] ZPT problems upgrading 2.9.6 to 2.10.2

[Pablo Ziliani]

Hi all again,

As I mentioned in my previous email I'm in the middle of an upgrade from 
2.9.6 to 2.10.2, and getting (at least) a couple of errors:
Sometimes when a ZPT that gets content from the database (postgres, 
psycopg 2.0.4) is called, it raises the following UnicodeDecodeError:
"""
Module zope.tal.talinterpreter, line 754, in do_insertStructure_tal
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 
164: ordinal not in range(128)
"""
I've also seen it failing in presence of a 0xc2 byte, and I haven't 
tested if this is the usual behavior for a non ascii.

I have the following directives in the zope.conf file:
- rest-input-encoding utf-8
- rest-output-encoding utf-8
- default-zpublisher-encoding utf-8

All my "admin" forms have an accept-charset="utf-8".
So, any hint on why it uses the ascii codec? am I omitting anything? If 
the problem is in the database, is there any way to pass a "replace" or 
"ignore" argument for unicode error handling? (maybe the DB should do 
that?). Again, although I don't really know how much garbage was being 
actually served, the same requests worked under 2.9.6.
BTW, I don't think I can isolate where this offending insertion occurs 
from this traceback (I might be wrong on this, but I think I remember 
that in previous versions of Zope you could get the line number, or at 
least the actual culprit's filename)

Also, my main template (the file that contains the macro that is used by 
most other pages) is showing this message when editing TTW:
"""
Macro expansion failed
exceptions.NameError: name 'widget' is not defined
"""
And that's because I have this code:
<tal:block condition="PAGE/widgets" repeat="widget PAGE/widgets">
    <metal:singleton 
use-macro="python:path('context/widgets_sidebar.html/macros/' + widget)" />
</tal:block>

Of course the expected macro does exist in the file when the variable 
"widget" is evaluated. This not only shows when editing the template 
itself, but also when doing it on every page that uses it. No, the 
checkbox "Expand macros when editing" is not selected an any level.
Fortunately, this doesn't prevent the page from being served properly. 
It does show this warning in the log, though (I can see it when 
something ELSE fails afterwards):
"""
Module zope.pagetemplate.pagetemplate, line 117, in pt_render
   - Warning: Macro expansion failed
   - Warning: exceptions.NameError: name 'widget' is not defined
"""

Thanks in advance and sorry for the length of this post.
Pablo