[Zope3-dev] unicode bug in the TAL interpreter

[Jean-Marc Orliaguet]

Hi!

there is a bug in TAL interpreter (zope2.8 / zope2.9), the following markup

<div tal:attributes="a python:'é'; b python:u'é'">...</div>

(which mixes unicode- and non unicode-encoded attributes) generates an 
exception::

    result = self.pt_render(extra_context=bound_names)
  File "/home/jmo/zope-2.9-cps/Products/CMFCore/FSPageTemplate.py", line 
134, in pt_render
    result = FSPageTemplate.inheritedAttribute('pt_render')(
  File 
"/usr/local/Zope-2.9.0/lib/python/Products/PageTemplates/PageTemplate.py", 
line 104, in pt_render
    tal=not source, strictinsert=0)()
  File "/usr/local/Zope-2.9.0/lib/python/TAL/TALInterpreter.py", line 
238, in __call__
    self.interpret(self.program)
  File "/usr/local/Zope-2.9.0/lib/python/TAL/TALInterpreter.py", line 
281, in interpret
    handlers[opcode](self, args)
  File "/usr/local/Zope-2.9.0/lib/python/TAL/TALInterpreter.py", line 
357, in do_startTag
    self._stream_write(_nulljoin(L))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 3: 
ordinal not in range(128)

it is not necessarily an issue in a pure zope2.8 environment (actually 
the bug was first discovered under zope2.9/Five) but on the migration 
path to zope3 the mixing of unicode and non unicode strings coming from 
different software generations may be a real issue.

/JM