[Zope-dev] [Zope 2.10] ZPT going Unicode

[Dieter Maurer]

Andreas Jung wrote at 2006-1-6 10:13 +0100:
> ...
>A site encoding as default makes sense however there are situations when 
>the encoding of a string from is different. E.g. when you deal with 
>XMLHTTPRequest the browser expects an utf8 fragment which might conflict 
>with an iso-8859-15 site-encoding. So there should be some mechanism to
>specify the encoding. For PyScript one could introduce a property to 
>specify the encoding and methods of products I can imagine using function 
>properties..and this is possibly something some must be solved on the TAL 
>level.

I do not like this idea:

  If I need to modify individual objects (such as Python Scripts),
  rather than fiddling with properties, I can simply modify the
  script to return Unicode.

I do not think that we need additional special features beside
support for some kind of site encoding.

>> As pointed out in private email ("charset negociation will not work"),
>> the response charset has a significant impact on the charset
>> used in form data sent back to the server.
>> This may pose severe problems when the response charset is
>> not the same as the site encoding (for textual form data).
>
>This issue is connected to the ongoing/planned project to use the Z3 
>publisher in Zope 2.10. There will be a sprint at PyCon next month afaik...

I do not see that the problem is related to any specific publisher.
Instead, it is a conceptual problem:

  When form data comes back, the server *MUST* know the encoding
  used by the client.

  For "POST", the encoding is hopefully specified in the
  "Content-Type" request header, but for "GET" there is almost
  surely no information available: the server must assume that
  the client used the same charset as the page it replied to.
  However, the server does not have this information if it uses
  charset negotiation.

-- 
Dieter