[ZODB-Dev] ZEO and replication of BTree based objects
Jeremy Hylton
jeremy at zope.com
Thu Jun 5 19:02:51 EDT 2003
On Thu, 2003-06-05 at 13:53, Shane Hathaway wrote:
> Jeremy Hylton wrote:
> > I think it would be great to have a prefetching mechanism in ZEO,
> > probably coupled with a notification system to let the app know when an
> > object is loaded.
>
> It seems like prefetching is actually easy to do. You just create a
> thread in the ZEO client that draws from some kind of OID queue, calling
> storage.load() for each OID. Each successful load() adds to the local
> ZEO cache and possibly notifies the application. This wouldn't require
> any changes to ZODB.
It's a bit more complicated because ZEO only allows one synchronous call
at a time, which limits the benefits of pipelining. I think it would be
better to have an async load message that gets produces an async
response. Otherwise, as you have proposed.
> What isn't clear is how you fill the OID queue. How can you know the
> OIDs of the objects that are important enough to prefetch?
I agree that this is the harder problem. The best bet is probably
heuristics based on references within an object. If you access object
A, and it has references to B and C, it increases the likelihood that
you'll access B or C.
BTW, I blogged a little about this subject in April:
http://www.python.org/~jeremy/weblog/030418.html
Jeremy
More information about the ZODB-Dev
mailing list