[ZOPE 2.6 B1] Unicode/locale problems with OFS/dtml/properties.dtml
We are trying out 2.6 b1. So far it seems very snappy (!!), and the new error-log object is exceptionally usefull! Kudos to Zope Corp and the community. :-) While trying the beta we have found a few problems in the handling of propertysheets. We can successfully store Norwegian characters ('æ','ø', and 'å') in property fields, but when we reload the page, we get results that display wrong. When studying the generated sources, we see that the data from ZOPE inserted into the form is ISO-8859-1. OTOH this field is inserted in the html-header: <meta http-equiv="content-type" content="text/html;charset=UTF-8"> Therefore the browser tries to display utf-8, which of course becomes garbage when you get an ISO-8859-1 character with a larger value than 127. All other pages (than the "properies" tab) in the have <meta http-equiv="content-type" content="text/html;charset=ISO-8859-1"> Secondly, the type attribute of all inputfields contain an extra ':utf8:' that we assumed is a server directive to interpret the contents as UTF-8. This apperently what crashes when storing the second time. lib/python/OFS/dtml/properties.dtml contains the following that seems to us to be debug code: <dtml-call "REQUEST.set('management_page_charset','UTF-8')"> <dtml-var "u''"> and several ':utf8:' directives. When we removed those, it worked fine. Was this dtml-method merged in mistakenly, or are there problems in our setup. Arnar Lundesgaard ------------------------------------------------------- phone: (+47) 982 38 036 mailto:arnar.lundesgaard(at)creuna.no Creuna as Bryggegata 3 NO-0250 Oslo phone office: (+47) 23 23 88 00 fax: (+47) 23 23 88 50 http://www.creuna.no/
On Thursday 26 Sep 2002 4:38 pm, Arnar Lundesgaard wrote:
We can successfully store Norwegian characters ('æ','ø', and 'å') in property fields, but when we reload the page, we get results that display wrong.
Thats bad. I will definetely resolve this asap.
When studying the generated sources, we see that the data from ZOPE inserted into the form is ISO-8859-1.
what do you mean by "inserted into the form"? That the response contains a single byte where you properties contain character whose unicode value is greater than 127 ? how have you checked this? if so, thats a bug.
Secondly, the type attribute of all inputfields contain an extra ':utf8:' that we assumed is a server directive to interpret the contents as UTF-8. This apperently what crashes when storing the second time.
This is a directive to tell zope when you submit the form that your browser will have encoded the form response using utf-8. Browsers stupidly dont put this information anywhere more suitable. What browser are you using? is it correctly using utf8 for this page? (for example, Mozilla has a View/Encoding menu that can override server-supplied encoding information)
lib/python/OFS/dtml/properties.dtml contains the following that seems to us to be debug code: <dtml-call "REQUEST.set('management_page_charset','UTF-8')"> <dtml-var "u''"> and several ':utf8:' directives.
When we removed those, it worked fine. Was this dtml-method merged in mistakenly
Those are supposed to be there. The first inserts the text/html;charset=utf-8 header into the management page. The second ensures that the dtml which computes this page content returns a unicode object. If the dtml returns a unicode object, then ZPublisher looks at the charset header to determine how to encode it.
, or are there problems in our setup.
could be. please put ib/python/OFS/dtml/properties.dtml back the way is was originally, then send me 1. which browser you are using 2. a export file containing one object that demonstrates the problem. 3. a copy of the page obtained using wget or similar. (please dont use your browsers 'save' feature because that sometimes performs transcoding)
Hi! Some time ago I filed this issue as a 2.6a1 bug (http://collector.zope.org/Zope/517), only that it was German characters with me ... The browsers I was using where IE 6.0 (which led to errors storing property sheets and could not display some of the propertysheets correctly) and Mozilla, which just displayed the wrong characters. I am quite sure that the problem is not with the Zope setup, but a real bug in the way the forms are generated and evaluated. My Zope setup uses the German locale settings. But I don't think that should be a problem ... Cheers Joachim ----- Original Message ----- From: "Toby Dickenson" <tdickenson@geminidataloggers.com> To: "Arnar Lundesgaard" <arnar.lundesgaard@creuna.no>; <zope-dev@zope.org> Sent: Thursday, September 26, 2002 6:04 PM Subject: Re: [Zope-dev] [ZOPE 2.6 B1] Unicode/locale problems with OFS/dtml/properties.dtml On Thursday 26 Sep 2002 4:38 pm, Arnar Lundesgaard wrote:
We can successfully store Norwegian characters ('æ','ø', and 'å') in property fields, but when we reload the page, we get results that display wrong.
Thats bad. I will definetely resolve this asap.
When studying the generated sources, we see that the data from ZOPE inserted into the form is ISO-8859-1.
what do you mean by "inserted into the form"? That the response contains a single byte where you properties contain character whose unicode value is greater than 127 ? how have you checked this? if so, thats a bug.
Secondly, the type attribute of all inputfields contain an extra ':utf8:' that we assumed is a server directive to interpret the contents as UTF-8. This apperently what crashes when storing the second time.
This is a directive to tell zope when you submit the form that your browser will have encoded the form response using utf-8. Browsers stupidly dont put this information anywhere more suitable. What browser are you using? is it correctly using utf8 for this page? (for example, Mozilla has a View/Encoding menu that can override server-supplied encoding information)
lib/python/OFS/dtml/properties.dtml contains the following that seems to us to be debug code: <dtml-call "REQUEST.set('management_page_charset','UTF-8')"> <dtml-var "u''"> and several ':utf8:' directives.
When we removed those, it worked fine. Was this dtml-method merged in mistakenly
Those are supposed to be there. The first inserts the text/html;charset=utf-8 header into the management page. The second ensures that the dtml which computes this page content returns a unicode object. If the dtml returns a unicode object, then ZPublisher looks at the charset header to determine how to encode it.
, or are there problems in our setup.
could be. please put ib/python/OFS/dtml/properties.dtml back the way is was originally, then send me 1. which browser you are using 2. a export file containing one object that demonstrates the problem. 3. a copy of the page obtained using wget or similar. (please dont use your browsers 'save' feature because that sometimes performs transcoding) _______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
participants (3)
-
Arnar Lundesgaard -
Joachim Werner -
Toby Dickenson