Re: [Zope] How to convert Zope instance charset?
Sunday, April 24, 2005, 2:36:24 PM, Andreas Jung wrote:
--On Sonntag, 24. April 2005 14:18 Uhr +0200 Daniel Dekany <ddekany@freemail.hu> wrote:
I have a Zope instance that uses utf-8 for everything. Since Python/Zope/etc practically doesn't support utf-8,
Please explain in which sense Zope would not support utf-8. For your information:
It can't sort strings alphabetically *anywhere* (concretely: the accented letters will go to the end of the list -- I guess because 0x80 is mathematically greater than the code of the US-ASCII characters). Kind of basic fundamental thing for a portal, or for text handling in general. I have asked here earlier how to solve it, but there was no answer that could be applied in practice (i.e. the answer was that I should write custom fixes for each individual products, and/or write a patch for Zope, and then maybe for Python... If so, this is equal of saying that UTF-8 is not supported yet).
Plone has UTF8 as default charset.
Aha. Then this is why this Plone site I should fix/maintain used UTF-8 everywhere. I believed it was a bad decision of my predecessor. (But then this problem is even more mysterious for me: *if* it doesn't working (yet), then why did the Plone authors chose that?)
In general Zope does not care much about encoded strings except for some conversions. Dealing with utf8 might be tricky in some cases but saying Zope does not support Utf-8 is wrong.
Believe me, I really hope I'm wrong. So how could I achieve that strings are sorted correctly? If it works for someone, how? (I have locale hu_HU.UTF-8 in zope.conf, I have even printed locale.getlocale(locale.LC_COLLATE) from products and external methods, and it was hu_HU.UTF-8. Note that at least on Python level sorting with hu_HU.ISO-8859-2 works... so I hope it would work with Plone as well.)
-aj
-- Best regards, Daniel Dekany
--On Sonntag, 24. April 2005 16:03 Uhr +0200 Daniel Dekany <ddekany@freemail.hu> wrote:
Sunday, April 24, 2005, 2:36:24 PM, Andreas Jung wrote:
--On Sonntag, 24. April 2005 14:18 Uhr +0200 Daniel Dekany <ddekany@freemail.hu> wrote:
I have a Zope instance that uses utf-8 for everything. Since Python/Zope/etc practically doesn't support utf-8,
Please explain in which sense Zope would not support utf-8. For your information:
It can't sort strings alphabetically *anywhere* (concretely: the accented letters will go to the end of the list -- I guess because 0x80 is mathematically greater than the code of the US-ASCII characters).
This is neither a problem of Zope nor of Python! A Python string has no notion an an encoding. The sort method can not smell the encoding. Instead use Python unicode strings and depend on the sorting order defined by the Unicode standard. This is an application-level problem but not a server-side problem.
Plone has UTF8 as default charset.
Believe me, I really hope I'm wrong. So how could I achieve that strings are sorted correctly? If it works for someone, how? (I have locale hu_HU.UTF-8 in zope.conf, I have even printed locale.getlocale(locale.LC_COLLATE) from products and external methods, and it was hu_HU.UTF-8. Note that at least on Python level sorting with hu_HU.ISO-8859-2 works... so I hope it would work with Plone as well.)
see above..Also the standard sort() methods of Python does not care about your locales (why should it)....strings are streams of bytes...nothing else... sort() accepts a user-defined comparison method of implement user-specific sorting. And there are also methods in Python "locale" module to perform locale-dependent comparison. Once again: you must solve your problem on the application layer...Zope does not help you at this point because it can't. -aj
participants (2)
-
Andreas Jung -
Daniel Dekany