[ZODB-Dev] RelStorage now in Subversion

[Shane Hathaway]

Jim Fulton wrote:
> 
> On Jan 27, 2008, at 3:11 AM, Shane Hathaway wrote:
>> I hope the patch, or a modified version of the patch, will be accepted 
>> for inclusion in ZODB 3.9.  A monkey patch version is possible, but 
>> I'm trying to avoid that.
> 
> 
> I'm tentatively planning on including it in 3.9.  I need to review it 
> carefully. I also plan to make other changes in the invalidation 
> mechanisms, so I'd like to take this into account at the same time.

I'm really glad to hear that.  I tried to make the patch as simple as I 
could.  Here is an overview of the changes the patch makes:

* Connection.__init__() looks for a method of the storage called 
bind_connection.  If it exists, the Connection calls bind_connection() 
and gets back a Connection-specific storage object, which the Connection 
should use in place of the original storage.  This way, each storage 
instance can hold Connection-specific state.

* If the storage has an attribute called propagate_invalidations set to 
a false value, Connection.invalidate() no longer accepts any 
invalidation messages.  The storage.poll_invalidations() method becomes 
the sole source of invalidation information.

* Connection.close() calls storage.connection_closing() if the method 
exists.  This allows the storage to release resources while the 
connection is not in use.

* Connection._flush_invalidations() calls storage.poll_invalidations() 
if the method exists.  poll_invalidations() returns either a sequence of 
OIDs the Connection needs to invalidate, or the value None, which means 
the entire cache needs to be invalidated (which can happen if a 
connection has not been used in a long time).

* The DB class also calls connection_closing() after initialization, to 
free storage resources used during initialization.

I admit that polling for invalidations probably limits scalability, but 
I have not yet found a better way to match ZODB with relational 
databases.  Polling in both PostgreSQL and Oracle appears to cause no 
delays right now, but if the polling becomes a problem, within 
RelStorage I can probably find ways to reduce the impact of polling, 
such as limiting the polling frequency.

Shane