[ZODB-Dev] transactions, data manager order

[Matthias]

Am 09.09.2010, 20:55 Uhr, schrieb Jim Fulton <jim at zope.com>:

>> This means for a sequence such as
>>
>> transaction.begin()
>> transaction.get().join( myDataManager )
>> transaction.savepoint()
>> transaction.rollback()
>> transaction.commit()
>>
>> the data managers are not sorted unless commit is encountered. If I move
>> the sorting code to the join() method instead, I get the proper search
>> order.
>>
>> Is this a bug?
>
> No. sortKey is intentionally used just for commits.
>
> Why do you want savepoints to execute in sortKey order?

I'll try to keep it short.

I've built a spatial index on top of a BTree. The index basically writes  
out its data in chunks of 4kb pages. Each page has an id. So the BTree  
stores pageId -> pageData.

Now consider this situation: A transaction begins. Object A is added to  
the spatial index. The spatial index writes this data to an internal  
memory buffer, nothing is written to the BTree yet. Now you create a  
transaction savepoint. My custom data manager needs to flush the cached  
data to the BTree, then the Connection's data manager should be invoked,  
so it can save the changed BTree.
If the Connection's data manager was executed before my custom data  
manager then the data is written to the BTree after the savepoint was  
created. If you abort the transaction now, there'll be nothing in the  
BTree. Which is wrong since there was an object inserted before the  
savepoint was created.

Currently I use an ugly hack like the following to keep the data managers  
sorted:

t = transaction.get()
org_join = t.join
def join(resource):
     org_join( resource )
     t._resources = sorted( t._resources, transaction._transaction.rm_cmp )
t.join = join
t.join( SpatialDataManager(self) )

-Matthias