[ZPT] Unicode and 8-bit string migration fix

[Stuart Bishop]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


On Friday, October 17, 2003, at 05:53  PM, Tino Wildenhain wrote:

> Hi Toby,
>
> Toby Dickenson wrote:
>> On Friday 17 October 2003 06:29, Stuart Bishop wrote:
> ...
>
>> The rule used in dtml for this type of template (that is, ones that 
>> mix 'string' and u'string' ) is to convert the plain strings into 
>> unicode assuming latin-1, then for the whole template to return a 
>> unicode string.
> ...
>
> The assumption of latin-1 is very dangerous and brings us in a position
> similar to the anybody-uses-ascii assumption as it was before.
>
> What if I type utf-8 into the template? This is very common since
> it is supported by browsers and lets me input for different languages
> without switching encoding.
> Could be content-type with the extension charset used for display
> and evaluation of the encoding?

This is the alternative approach I had considered. It would also
work fine assuming that the Content-Type header has been set already
and won't be changed (a reasonable restriction). However, the 
implementation
would be more complex as we need to somehow find and interrogate the
response headers. It also ties TALInterpreter more closely to 
ZPublisher,
which may not be wise (?). Still, either approach would work for
me and my utf8 encoded strings that Formulator is spitting out at me,
and I can't set the character set to latin1 since I need to support 
Asian
languages...

None of this a particularly major problem to me, as I can always patch
Formulator instead, but others may appreciate the fix going in Zope
for everyone :-)

- -- 
Stuart Bishop <stuart at stuartbishop.net>
http://www.stuartbishop.net/

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.2 (Darwin)

iD8DBQE/j6NJAfqZj7rGN0oRAgURAKCTpq35w+4oE/Gg0pgZrYtPQMDh6gCdGCLg
/x9piV6sRoOEljb681S/q60=
=DDUm
-----END PGP SIGNATURE-----