[ZODB-Dev] Zope, ZEO, BTrees, and conflicts

[Shane Hathaway]

sean.upton@uniontrib.com wrote:

> I'm really thinking that in theory, it would be nice to put distribute some
> activity - i.e. BTreeFolder._setOb() - simultaneously across several ZEO
> client nodes to speed up my application.  Even if I could do this - the
> objects I add are CatalogAware, and I call index_object() - I worry that I
> wouldn't be able to, since I assume I might run into similar issues with the
> IOBTrees used in ZCatalog indexes.


Actually there's a really cool way to do this that Jim explained once, 
implemented in ZCatalog and BTreeFolder CVS.  You let the BTree folder 
choose the id for each added object.  Each ZEO client chooses a random 
base ID and counts forward for each added object.

So client 1 might add items ob87631, ob87632, and ob87633, while client 
2 adds items ob10362, ob10363, and 10364.  Once the initial negotiations 
are over, each client tends to add items to its own bucket.  There will 
still be conflicts, but if you keep your transactions small then the 
conflicts don't have much impact.

As long as you can allow the folder to choose the IDs, this strategy is 
probably the simplest.

Shane