[Zope-dev] Re: BTreeFolder

[Shane Hathaway]

"Phillip J. Eby" wrote:
> Now, what *I*'d like to know is what the "bold new idea" is, since it
> sounds like it's something even newer than the
> forward-chained-buckets-plus-conflict-resolution stuff that's already been
> released.

It's simple, really.  BTreeFolders play havoc with acquisition because
every time you pass through them, looking for some name, they perform a
lookup in a BTree.  This wouldn't be an issue if it weren't for the fact
that many sites use implicit acquisition heavily--I once measured 4500
lookups in a simple request to the management interface.

But if the items stored in the BTree follow some simple pattern, we
don't have to perform a BTree lookup for things that don't follow the
pattern.  The most useful pattern would be a string prefix followed by
an integer, which would allow us to use an IOBTree (meaning integer keys
and object values) for storage.  Jim claims an IOBTree is faster. :-)

So when scripts go looking all the way up the tree for the name REQUEST
a hundred times (a common thing to do), the BTreeFolder can ignore it.

The other cool idea he came up with is allowing the folder to decide
what the ID of a new object is going to be so the folder can try to
avoid conflicts with other ZEO clients.  Many Zope applications use time
stamps for IDs, but doing that can lead to frequent conflicts.  Random
IDs avoid conflicts.

However, purely random IDs would cause all the buckets to be loaded in
memory all the time, so Jim's third idea was to have each client
increment sequentially from a random ID and move to a new random ID if
conflicts ever occur.

So, all things considered, you get faster access, fewer conflicts, and
less RAM usage.  Can't beat that. :-)

Shane