[Zope-dev] XML-RPC - UTF-8 vs. Latin-1 encoding

[Skip Montanaro]

I just stubbed my virtual toe on a problem with XML-RPC that I thought I'd
fixed about a year ago.  It has to do with the use of UTF-8 as the default
encoding for XML-RPC.

I twiddled my Python search path a little yesterday and unwittingly began
picking up the Zope 2.2.1 version of xmlrpclib.py instead of the version I
got from Fredrik Lundh and modified locally.

When non-7bit-ASCII characters started popping up in my XML-RPC responses I
began to get errors because those characters weren't properly encoded as
UTF-8.  The fix I applied for my own code was to observe that everything I
did used Latin-1 and so I simply forced a Latin-1 encoding in all my
requests and responses with

    <?xml version='1.0' encoding='ISO-8859-1' ?>

When I twiddled sys.path yesterday I began picking up the Zope version of
xmlrpclib and began having this Latin-1 vs. UTF-8 problem again.  I
retwiddled the path to get my version of xmlrpclib.py and things are once
again fine.

I have a couple questions: How are encodings handled in Zope (at least vis a
vis XML-RPC)?  I presume that in a Python 1.5.2-based environment such as
Zope currently lives in, this is a potential problem.  If so, where is the
Latin-1 to UTF-8 conversion happening in Zope?  It doesn't appear in Zope's
version of xmlrpclib.py, nor in lib/python/ZPublisher/xmlrpc.py.

Thanks,

-- 
Skip Montanaro (skip@mojam.com)
http://www.mojam.com/
http://www.musi-cal.com/