[Zope] utf-8 over xmlrpc with zope2.9

[Doyon, Jean-Francois]

To be more explicit.

The problem is that the strings received by XML-RPC where the encoding is utf-8 end up internally as unicode u'' strings.

Other strings you're working with internally might not be however.

When you're trying to render a ZPT, if you have a mix of u'' and '' strings, you might end up with such UnicodeDecodeErrors ...

In my case, I make sure I use u'' strnigs everywhere, and set character sets to utf-8 everywhere, and so on.

Basically if needed you can make your whole system utf-8, avoiding the risks of those errors cropping up.

J.F. 

-----Original Message-----
From: zope-bounces+jdoyon=nrcan.gc.ca at zope.org [mailto:zope-bounces+jdoyon=nrcan.gc.ca at zope.org] On Behalf Of Doyon, Jean-Francois
Sent: September 11, 2007 14:01
To: Kevin; zope at zope.org
Subject: RE: [Zope] utf-8 over xmlrpc with zope2.9

Should work, I use XML-RPC pretty heavily with utf-8 without much problems.

This error might be occurring because you're mixing unicode and non-unicode strings on the same page.

Send the full traceback and we'll know more ...

J.F. 

-----Original Message-----
From: zope-bounces at zope.org [mailto:zope-bounces at zope.org] On Behalf Of Kevin
Sent: September 11, 2007 13:56
To: zope at zope.org
Subject: [Zope] utf-8 over xmlrpc with zope2.9

I am trying to send unicode encoded utf-8 or latin1 strings over xmlrpc to a method in zope.  The method in zope tries to decode the string and it seems detects ascii.  I then recieve an error that
"UnicodeDecodeError: 'ascii' codec can't decode byte 0xf3 in position 4: 
ordinal not in range(128)"

I can re-create this error in a python shell by doing something like:
 >>> s = 'helló'
 >>> s2 = unicode(s,'ascii')
so after looking at code, I assume this is what is happening instead of  >>> s2 = unicode(s,'iso-8859-1')

If  I  submit  to this method from  an html form, I have no error and the data is stored perfectly.

I have tried setting the encoding of xmlrpclib.Server to utf-8 and
latin1(iso-8859-1) and making a custom transport for xmlrpclib where I add charset='utf-8' or charset='iso-8859-1'.

Is is not possible to send unicode characters > 128 to zope?


The method is LDAPUserFolder.manage_edit_user()






_______________________________________________
Zope maillist  -  Zope at zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists -
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev ) _______________________________________________
Zope maillist  -  Zope at zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists -
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )