[ZODB-Dev] Re: ZODB Benchmarks

[David Binger]

On Nov 1, 2007, at 7:05 AM, Matt Hamilton wrote:

> Ie we perhaps look at a catalog data structure
> in which writes are initially done to some kind of queue then moved  
> to the
> BTrees at a later point.

A suggestion: use a pair of BTrees, one with a high branching factor  
(bucket size)
and one with a very low branching factor.  Force all writes into the  
tree with little
buckets.  Make every search look in both trees.  Consolidate  
occasionally.

It seems like this would be a good way to push the conflict rate down  
while
still providing fast item access when the cache is cold.
The little bucket tree seems better than an ordinary queue for this  
purpose
because it can be searched faster, and you can let it grow as much as  
you
like between consolidations.  Also the size of the transaction on  
each insert will
be pretty small.