-----Original Message----- From: Andrew M. Kuchling [mailto:akuchlin@mems-exchange.org] Sent: Monday, September 27, 1999 7:29 PM To: zope-dev@zope.org Subject: [Zope-dev] ZODB and # of objects
FileStorage's property of requiring memory proportional to the number of objects in the ZODB may be causing me problems. I'm running into trouble with a ceiling of 150,000 items, but the memory usage becomes too high, 60 or 70 or 80Mb well before reaching that point. I don't think the problem is a memory leak in my code, but perhaps I'm doing something wrong; has anyone stored that many items in a ZODB before? What was the memory usage like?
Are you using substransactions? If you are using 2.0.x, then you might not notice any improvement with sub-transactions because of a deactivation bug that was fixed just after the release (sigh). If you use the CVS version and tune your cache agressively, you should be able to keep the memory below a ceiling for any number of objects. ZCatalog is pretty agressive about this, not only does it sub-transact but it also kicks the cache with self._p_jar.cacheFullSweep(1).
(The largest number of objects stored in a Zope database would be an interesting bit of trivia.)
Theoretically, 2^64. -Michel
Michel Pelletier writes:
Are you using substransactions? If you are using 2.0.x, then you might not notice any improvement with sub-transactions because of a deactivation bug that was fixed just after the release (sigh). If you
Trying the CVS version doesn't seem to help; I suspect that the problem is the growing in-memory index of FileStorage objects, and the fix is to use an alternate Storage class that scales better. -- A.M. Kuchling http://starship.python.net/crew/amk/ One form to rule them all, one form to find them, one form to bring them all and in the darkness rewrite the hell out of them. -- Digital Equipment Corporation, in a comment from SENDMAIL Ruleset 3
participants (2)
-
Andrew M. Kuchling -
Michel Pelletier