[Zope-dev] Zope 2.6.0 ZMI Problem for CJK(Collector 623) patch.

[Toby Dickenson]

On Friday 29 November 2002 5:44 am, Yusei Tahara wrote:
> Hi.
>
> > The right approach is to make it possible to change the title propert=
y to
> > a unicode string. All my custom products have this already, but it is=
 a
> > deficiency in the standard Zope types such as 'Folder' that their tit=
les
> > type> can not be changed.
>
> This approach is not useful for me.
>
> I often use zope with RDB like MySQL.
> generally japanese encoding text is in RDB.
>
> so if object properties are unicode string,
> I always encode it before publishing.
>
> <title tal:content=3D"python:here.title.encode('euc-jp')">TITLE</title>
>
>
> because, it always mix in RDB data(japanese) and
> properties(python unicode string) in one page.
>
> certainly I will be faced with this ustring problem everytime.
> Japanese charactor is not ascii,
> so if I do not encode ustring before publishing,raise unicode error.

The unicode changes were designed to cope with your scenario, so I would =
like=20
to explore the limits of where this approach has gone wrong.

As I understand it, you dont want to do anything in unicode. You store,=20
manipulate, process, and output strings as 8-bit pre-encoded strings. You=
 =20
are making an assumption that these 8-bit strings use some specific chara=
cter=20
encoding. The scope of this assumption is quite broad. It has to cover:
1. All strings stored by your ZODB
2. All strings stored by your RDB.
3. All processing performed by Zope products. (must follow your assumptio=
ns,=20
or be encoding neutral)

Anything that beaks these assumptions will need special treatment.

Am I right so far?


Im sure you are aware of the limitations of this approach - they are the=20
limitations that unicode was designed to break (not Zope's unicode, but=20
unicode in general). Im not sure your approach is workable long term beca=
use=20
an increasing number of Zope products will naturally use unicode so that =
they=20
can store properties containing text across a range of scripts. Even if t=
his=20
is not workable long term, it was supposed to work in 2.6.

Zope's approach is that your application (excluding the ZMI for now) shou=
ld=20
work as in 2.5, provided you dont use any unicode values. Can you confirm=
=20
that this is working OK?

Zope 2.5 left ZMI character encoding down to browser autodetection, and a=
s a=20
result most ZMI-controlled properties are encoding-neutral. For compatabi=
lity=20
with unicode pages, which are seen as the long term future, the ZMI has t=
o=20
specify *some* character encoding. By default in 2.6 this is latin-1, a=20
change which I think was announced mid-way through the 2.5 development cy=
cle.=20
I understand that some users are very happy overriding this with a=20
management_page_charset property on their root folder. Ive never used thi=
s,=20
and it wasnt designed to work this way, but it looks like it works due to=
=20
happy coincidence. Note that this doesnt work for management pages which=20
explicitly set their own character encoding, or management pages which=20
'accidentally' encounter a unicode string.

(Although this management_page_charset feature was not planned, I guess I=
 will=20
be supporting this feature if it is being used)

One area where this went wrong for you is the properties page, which is=20
explicitly set to utf8 to allow unicode properties to be set. Im not yet =
sure=20
on the best way to fix this for you, but I agree it needs fixing.

As far as I can tell this is the only area which should be causing you=20
problems. Am I missing something?



(The 'title' problem I mentioned in a previous post is also a problem tha=
t=20
needs fixing, but not one that affects you if you dont want to use unicod=
e.)