[ZODB-Dev] Cache warm up time

[Jim Fulton]

On Sat, Mar 9, 2013 at 9:02 AM, Jim Fulton <jim at zope.com> wrote:
...
> I think a simple method on a storage that gives a hint that a set of
> object ids will be loaded is enough.  A network storage could then
> issue a pipelined request for those oids. The application can then
> proceed as usual.  I think I've proposed such an API before, but am
> too lazy to look it up. Something like:
>
>     load_hint(*oids)
>
> I'd like to see this functionality, but I don't have time to do it soon.
>
> I must say that I think this API is more likely to be abused
> than used effectively.  Prefetching catalog indexes is a sort of
> anti-pattern than only makes sense for small catalogs.  It
> would likely make more sense to have a dedicated catalog
> server that returned oids and possibly object records in
> response to queries (or whimper, use solr ).

I forgot to mention an even simpler way to reduce the number of
round-trips for cataloging data structures is to increase the bucket
and internal node sizes.  You can do this now by patching the
BTree header files. We've done this in the past to reduce the
likelihood of database conflicts when buckets split.  I suspect the
default sizes are too low.

I'd really like BTrees to be subclassible and for sizes to
be read from instance data, falling back to class settings.

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton
Jerky is better than bacon! http://zo.pe/Kqm