[ZODB-Dev] PersistentMapping vs OOBTree revisited

[Tim Peters]

[Chris Cioffi]
> I know we've talked about this before, but since we now have v3.3 and
> maybe a little more benchmarking I thought I'd ask again.
>
> How large does a keyspace have to be to tip the preference from a
> PersistentMapping object to a OOBTree?

Well, it's simply impossible to guess without details about _what_ you're
trying to optimize (aggregate clock cycles?  peak memory use?  worst-case
latency? ...), expected access patterns, and details about object structure.

> Right now my rule of thumb as has been that frequently randomly accessed
> objects with more than 2000 keys are better as OOBTrees.  For spaces
> where I mostly access the object by iterating over the entire space I
> leave it as a PersistentMapping until the keyspace grows to 5000+ keys.

If you measured those approaches on your data, using your access patterns,
and judged them according to whatever it is you're trying to optimize, then
I'm not going to argue with your measurements <wink>.

> My interest is mostly for a standalone ZODB, though I guess the answer
> would also apply to Zope as well...

Seems to me that "the question" needs clarification more than "the answer"
here, so far.  If they're truly "frequently randomly accessed", then they're
likely to all be sitting in a Connection's memory cache (provided you've
configured the cache to be big enough) in steady state, and then dict lookup
(PersistentMapping) is generally quicker than BTree lookup.  It's just a
guess that lookup time is what you're asking about, though.

...

> PS:  Great thanks to the ZODB core developers, I really appreciate your
> efforts...storing the objects my code is using as opposed to having to
> make a SQL->object mapping has sped up development time enormously!

Yup, ZODB can be way cool!