[Zope] Re: Versioned connectors from ZODB

[Tres Seaver]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Etienne Labuschagne wrote:
> On 7/11/05, Florent Guillaume <fg at nuxeo.com> wrote:
> 
>>ZODB versions are deprecated, unsupported, buggy and hard to use. Don't
>>use them.
>>
>>Florent
> 
> 
> And as I understand, so are temporary connections too.  That leaves me
> with getting a "normal" ZODB connection from the pool which I don't
> want to do.
> 
> I really need a "temporary" connection that I can discard.  This
> connection can have a much smaller cache than the normal connections
> as it makes very little difference in the speed of data loading. 
> Second prize is a connection that will only be used by a specific
> process and never used for other processes.  Versions solves this for
> me.
> 
> I can check out a connection and keep it aside only for data loading. 
> But this means that I waste precious memory on a connection that does
> not really need to cache the amount of objects that the other
> connections should.  In my case, this translates to using 1GB of RAM
> on one connection that gets used once a day.
> 
> Please believe me that I really need a "special" connection.  For
> those who really want to know why, below is an attempt at an
> explanation why:
> 
> In the application that I have written, I want to be able to get
> connections that are not part of the normal connection pool.  Once my
> process is finished, I can store these connections for later use, or
> discard them.  Currently my application uses the normal connections in
> the pool.  The problem is that this process "contaminates" the cache
> of the connections with objects that are not used in "normal" client
> application use (I use a thick client).  This means that the client
> applications are extremely slow the next day and that it takes a long
> time before the cache contains the often used objects again.
> 
>>From there the reason why I DON'T want to use the connections for my
> once a day data loading process.
> 
> My ZODB contains about 700`000 objects.  A connection caches about
> 60`000 objects to give satisfactory client speed.  To start up the
> client before the cache is initialized, takes about 5 minutes.  Once
> the cache is populated, it takes a client seconds to start up.  Data
> loading invalidates all of this, but is worse than a "clean" cache in
> that it takes long for the "new" objects in the cache to be flushed
> and replaced by the often used objects again.  Data loading does not
> need such a big cache since it mostly loads data into the ZODB. 
> Unfortunately, the loaded objects also end up in the cache.
> 
> Why do I need so many objects in the cache?  Some searches cannot be
> done with a mere ZCatalog search and have to run through a subset of
> all the objects.  These tend to fit nicely in the cache.

Your query would be better served on the zodb-dev list, where Tim Peters
hangs out;  he can probably explain how to get what you want without
guessing.  If I had to guess, I would suggest constructing your
connection programmatically, where you can specify the object cache size
for instance, and then closing / discarding the connection when you are
done.


Tres.
- --
===================================================================
Tres Seaver          +1 202-558-7113          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFC0p6H+gerLs4ltQ4RAkxMAKCYMC7HKTddmCgog+yip3GZd/AChgCgr0k8
APQ337C2zCxBQBOYIuWFKNU=
=MVtV
-----END PGP SIGNATURE-----