[Zope-dev] Re: RFV: Unicode in Zope 2

[Martijn Faassen]

Florent Guillaume wrote:
> Martijn Faassen wrote:
[snip]
>> What I'm worried about doing this for old code, but some steps will 
>> probably become clear during the brainstorming session. Migration 
>> tools that turn strings in the ZODB into unicode ones magically (with 
>> the ability to spell out exceptions and encoding)? Tricky...
> 
> I'd start by adding some deprecation warnings at the places where 
> PlacelessTranslationService (or Localizer for that matter) do their 
> workarounds so that normally incompatible types are mixed together.

With the most recent Silva under development, we're not using either 
anymore, so someone else would have to do this.

Where would you put the deprecation warning in PTS? It monkey patches 
the ZPT engine, and in particular its StringIO object, to make sure that 
UTF-8 strings and unicode strings, when finally concatenated at the end 
after page template generation, can actually be concatenated as it will 
first encode the unicode strings to UTF-8. The output of the page 
template will then be UTF-8.

In Zope 2 without this in place, the output of the page template will be 
unicode, and if you put in UTF-8 strings (that aren't just ascii), the 
system will give rise to a UnicodeError when this concatenation takes place.

I guess you could put some detection code in PTS's concatenation 
functionality to see whether anything unicode is mixed with non-ascii 
strings, and in that case output one deprecation warning. I have a 
suspicion that you'll see this warning whenever someone views a page 
template in Plone. :)

Regards,

Martijn