[Zope-dev] Should PageTemplate._text be a unicode or an encoded string in Zope 2.9.3?

[Tino Wildenhain]

Andreas Jung wrote:
> 
> 
> --On 17. Juli 2006 17:11:54 +0100 Chris Withers <chris at simplistix.co.uk>
> wrote:
> 
>> Andreas Jung wrote:
>>>
>>> Zope 2.10 comes with the ZPT implementation of Zope 3 which works nicely
>>> with unicode strings. However the 2.10 won't enforce the use of unicode
>>> strings for backward compatibility. However (at least)  the
>>> ZopePageTemplate
>>> class constructor has a flag 'strict' to enforce the use of unicode.
>>
>> Okay, but what actually gets stored in the ZPT when editing it via ZMI or
>> WebDAV?
> 
> Here's the code:
> 
>    security.declareProtected(change_page_templates, 'PUT')
>    def PUT(self, REQUEST, RESPONSE):
>        """ Handle HTTP PUT requests """
>        self.dav__init(REQUEST, RESPONSE)
>        self.dav__simpleifhandler(REQUEST, RESPONSE, refresh=1)
>        ## XXX this should be unicode or we must pass an encoding
>        self.pt_edit(REQUEST.get('BODY', ''))
>        RESPONSE.setStatus(204)
>        return RESPONSE
> 
> As you can see from the comment..there is some work to do. AFAIK
> WevDAV the encoding is not available from a WebDAV request?!

But it is - in fact my local copies have the hack where I used
management_page_charset here. Now I'm seeing we dont even need
that - default_zpublisher_encoding is much better here.

Kate (as webdav editor client) plays very well with that.

> On the other hand there is code available that tries to obtain
> the encoding from the XML preamble (sniffEncoding)...and the very other
> hand

Yes this double encoder marking is a mess. We need to be able to
provide a fixup (just like the infamous <base>-tag)

> there is still a problem with this method since the encoding can be
> determined by the BOM (if available)...this currently not handled through
> the code...I think I'll have a closer look at the code once again this
> week.

BOM is only for filesystem "unicode" of some 16-bit variants. Nothing
you really want to send over the wire (although you can).
But after all its just another encoding so it would be a matter
of setting the encoding correctly.

Regards
Tino