[Zope] Unicode Error ASCII -> decoding error: ordinal not inrange(128)

[Douwe Osinga]

Hi,

In my experience you get this or similar errors if you try to convert
normal strings in an encoding to unicode or vice versa. As I understand it,
Python represents unicode internally as 16 bit integers. However, most
content
out there is coded as 8 bits chars. In order to convert the 8 bits chars
to 16 bits integers, Python needs to know what encoding to use, ie some
ISO number or utf-8 or so. If you don't supply an encoding, it will only use
the standard ascii characters and give you an error like below on any
char>127.

Now, it seems you add a string msgText1 as unicode to the ZODB. Zope will
try
to convert the string of normal chars to unicode and fails. The best way
around it is to first convert the string to unicode and than add it, like
with

msgText1Unicode = msgText1.decode( <your encoding> )
objRef.manage_addProperty('trString', msgText1Unicode, 'utext')

Now, what you have to figure out, is what encoding you use. Your aliases.py
file in the encodings directory of your Python distribution should give you
an overview what's on offer.

Douwe Osinga
http://douweosinga.com

> -----Original Message-----
> From: zope-bounces at zope.org [mailto:zope-bounces at zope.org]On Behalf Of
> Shashank Ashtikar
> Sent: Thursday, October 02, 2003 10:08 AM
> To: zope at zope.org
> Subject: [Zope] Unicode Error ASCII -> decoding error: ordinal not
> inrange(128)
>
>
> Hello friends,
>
> I am trying to import unicode content in to ZODB using an external method.
> This content is being added as a property.
>
> The error that I am Getting is
>
> Error Type: UnicodeError
> Error Value: ASCII decoding error: ordinal not in range(128)
>
> I did some googling and I found out that this is because of pythons
> default encoding being ascii. But I did not come across any particular
> solution. The text that I am trying to import is Hindi language text
> (i.e. ISO - 10646 encoding.)
>
> The actual traceback of the error is,
> Traceback (innermost last):
>   Module ZPublisher.Publish, line 98, in publish
>   Module ZPublisher.mapply, line 88, in mapply
>   Module ZPublisher.Publish, line 39, in call_object
>   Module Products.ExternalMethod.ExternalMethod, line 224, in __call__
>    - __traceback_info__: ((<TransparentFolder instance at
> 8cf1f68>, <HTTPRequest,
> URL=http://gnu.hbcse.tifr.res.in:9673/IndicTrans/indic2/Programs/i
> mport_po>), {}, (None,))
>   Module
> /usr/lib/zope/lib/python/Products/GNOWSYS/Extensions/po_import.py,
>  line 47, in manage_addPoFile
>   Module OFS.PropertyManager, line 247, in manage_addProperty
>   Module ZPublisher.Converters, line 123, in __call__
> UnicodeError: ASCII decoding error: ordinal not in range(128)
>
> The line 47 in the external method is
>
> objRef.manage_addProperty('trString', msgText1, 'utext')
>
> here "objRef" is the object handle which I got using
> restrictedTraverse API
> "trString" is the property id and "msgText1" is the variable containing
> actual unicode string.
>
> Any help or pointers???
>
> Thanks in Advance.
>
> Shashank
>
> _______________________________________________
> Zope maillist  -  Zope at zope.org
> http://mail.zope.org/mailman/listinfo/zope
> **   No cross posts or HTML encoding!  **
> (Related lists -
>  http://mail.zope.org/mailman/listinfo/zope-announce
>  http://mail.zope.org/mailman/listinfo/zope-dev )