[Zope] Versioned connectors from ZODB

[Etienne Labuschagne]

On 7/11/05, Florent Guillaume <fg at nuxeo.com> wrote:
> ZODB versions are deprecated, unsupported, buggy and hard to use. Don't
> use them.
> 
> Florent

And as I understand, so are temporary connections too.  That leaves me
with getting a "normal" ZODB connection from the pool which I don't
want to do.

I really need a "temporary" connection that I can discard.  This
connection can have a much smaller cache than the normal connections
as it makes very little difference in the speed of data loading. 
Second prize is a connection that will only be used by a specific
process and never used for other processes.  Versions solves this for
me.

I can check out a connection and keep it aside only for data loading. 
But this means that I waste precious memory on a connection that does
not really need to cache the amount of objects that the other
connections should.  In my case, this translates to using 1GB of RAM
on one connection that gets used once a day.

Please believe me that I really need a "special" connection.  For
those who really want to know why, below is an attempt at an
explanation why:

In the application that I have written, I want to be able to get
connections that are not part of the normal connection pool.  Once my
process is finished, I can store these connections for later use, or
discard them.  Currently my application uses the normal connections in
the pool.  The problem is that this process "contaminates" the cache
of the connections with objects that are not used in "normal" client
application use (I use a thick client).  This means that the client
applications are extremely slow the next day and that it takes a long
time before the cache contains the often used objects again.

>From there the reason why I DON'T want to use the connections for my
once a day data loading process.

My ZODB contains about 700`000 objects.  A connection caches about
60`000 objects to give satisfactory client speed.  To start up the
client before the cache is initialized, takes about 5 minutes.  Once
the cache is populated, it takes a client seconds to start up.  Data
loading invalidates all of this, but is worse than a "clean" cache in
that it takes long for the "new" objects in the cache to be flushed
and replaced by the often used objects again.  Data loading does not
need such a big cache since it mostly loads data into the ZODB. 
Unfortunately, the loaded objects also end up in the cache.

Why do I need so many objects in the cache?  Some searches cannot be
done with a mere ZCatalog search and have to run through a subset of
all the objects.  These tend to fit nicely in the cache.