[Zope-dev] Zope 2.7 from cvs and internationalization issues

[Eugene Morozov]

Hello!
I've found today that it is simply impossible to use cyrillic
characters in ZMI in the latest zope from cvs. If I don't have
management_page_charset set on root folder and try to enter cyrillic
title for some folder, I get following error:

Error Type: UnicodeError
Error Value: Latin-1 encoding error: ordinal not in range(256)

If I set management_page_charset=koi8-r for the root folder and then
try to set cyrillic title for some folder, I get the following error:

Error Type: UnicodeError
Error Value: UTF-8 decoding error: invalid data

If I set management_page_charset=utf-8 for the root folder and then
try to set cyrillic title for some folder, I get the following error:

Error Type: UnicodeError
Error Value: Latin-1 encoding error: ordinal not in range(256)

Folders with Russian titles that were created in previous version of
Zope show correct title in the ZMI, but it is impossible to change it
because form for changing properties is inaccessible (it also gives
UnicodeError because it tries to encode unicode string, which by the
way contains cyrillic characters encoded in koi8-r, not unicode, in
koi8-r). I was able to investigate last issue because entry in error
log appears and I was able to insert debugging print statments in Zope
code and see that it somehow manages to store koi8-r characters in
unicode object and then tries to assume that it is really unicode
object and apply conversion from unicode to koi8-r. Please, help me, I
don't want to install older version of Zope now. Sorry if my English
is not good enough to correctly describe the problem, but every
non-english speaking subscriber may install zope 2.7 from cvs and see
that it doesn't handle languages with encodings other than latin-1.