[Zope3-dev] Re: [Zope3-Users] PsycopgDA problem

[Dmitry Vasiliev]

Stuart Bishop wrote:
> [moving to zope3-dev]
> 
> I think I know what might have been going on. The earlier patch neglected to
> change getEncoding() to always return UTF-8. So the code in zope.app.rdb
> that encodes the queries was encoding them to whatever getEncoding()
> happened to return rather than UTF8.
> 
> Would you be happy with this if it works for you?

I agree that the problem possibly was in getEncoding(), unfortunately I didn't 
have a time to explore the problem deeper. I'll try to return 'SET 
client_encoding' back and hide the encoding field for psycopgda in a few days.

> The alternative that meets my requirements would be to make the possible
> encodings a Choice field of the encodings that recent PostgreSQL supports,
> (which needs a mapping from the PostgreSQL naming to the Python naming). The
> default would be UTF-8, and the connection would issue a 'SET
> client_encoding' statement using the selected encoding rather than trusting
> that whatever encoding has been selected in Z3 happens to be the default
> client encoding. There would be a slight performance improvement with this
> method in the case where a non-UTF8 database could receive data in its
> native encoding, but I'd personally rather not have the added complexity.

I've already thought about the Choice field, but I didn't find the easy way to 
implement enumeration of the encodings in Python.

> The getEncoding() approach isn't meaningful with database adapters that
> accept Unicode strings which is becoming the norm. The idea that database
> adapters will never be Unicode aware seems to have been embedded into Z3
> quite deeply though :-(

I guess the 'zope.app.rdb' code is somewhat in beta state so we still need to
figure out better approaches, interfaces, etc.

> I guess we can allow getEncoding() to return None
> for modern database drivers and make Z3 pass through queries as Unicode or
> ASCII strings to driver in this case. I'm going to want to make the switch
> to psycopg2 soon, and I think I'll need to sort this out in Z3 at that
> point.

I think it would be better to add some additional method like
'isUnicodeAware()' because 'getEncoding()' which returns None in the case of
unicode aware adapter seems hackish for me.

> I'm going to want to make the switch
> to psycopg2 soon, and I think I'll need to sort this out in Z3 at that
> point.  I don't know yet if it will be better to maintain a single psycopgda
> or if it will be better to split it into two products.

I think the single product would be better and more flexible approach, in this
case we can select better available underlying adapter at setup-time or/and at
run-time.

By the way, what do you think about the idea of add DB-API2 compliant exception 
hierarchy to 'zope.app.rdb'? Adapters will be responsible to translate its 
exceptions to 'zope.app.rdb' exceptions.

-- 
Dmitry Vasiliev (dima at hlabs.spb.ru)
     http://hlabs.spb.ru