[Zope-dev] [ZOPE 2.6 B1] Unicode/locale problems with OFS/dtml/properties.dtml

[Toby Dickenson]

On Thursday 26 Sep 2002 4:38 pm, Arnar Lundesgaard wrote:

> We can successfully store Norwegian characters
> ('=E6','=F8', and '=E5') in property fields, but when we reload the pag=
e, we
> get results that display wrong.

Thats bad. I will definetely resolve this asap.

> When studying the generated sources, we see that the data from ZOPE
> inserted into the form is ISO-8859-1.

what do you mean by "inserted into the form"? That the response contains =
a=20
single byte where you properties contain character whose unicode value is=
=20
greater than 127 ? how have you checked this? if so, thats a bug.

> Secondly, the type attribute of all inputfields contain an extra
> ':utf8:' that we assumed is a server directive to interpret the
> contents as UTF-8. This apperently what crashes when storing the
> second time.

This is a directive to tell zope when you submit the form that your brows=
er=20
will have encoded the form response using utf-8. Browsers stupidly dont p=
ut=20
this information anywhere more suitable.

What browser are you using? is it correctly using utf8 for this page?  (f=
or=20
example, Mozilla has a View/Encoding menu that can override server-suppli=
ed=20
encoding information)

> lib/python/OFS/dtml/properties.dtml contains the following that seems
> to us to be debug code:
>    <dtml-call "REQUEST.set('management_page_charset','UTF-8')">
>    <dtml-var "u''">
> and several ':utf8:' directives.
>
> When we removed those, it worked fine. Was this dtml-method merged in
> mistakenly

Those are supposed to be there.  The first inserts the text/html;charset=3D=
utf-8=20
header into the management page. The second ensures that the dtml which=20
computes this page content returns a unicode object.

If the dtml returns a unicode object, then ZPublisher looks at the charse=
t=20
header to determine how to encode it.

>, or are there problems in our setup.

could be.=20

please put ib/python/OFS/dtml/properties.dtml back the way is was origina=
lly,=20
then send me=20

1. which browser you are using
2. a export file containing one object that demonstrates the problem.
3. a copy of the page obtained using wget or similar.  (please dont use y=
our=20
browsers 'save' feature because that sometimes performs transcoding)