Leak, bug, improper configuration, what ever. ;-) With the onDelete in place for 1 week now I checked the temporary Cache Parameters as you requested. Since I put the onDelete on this server memory use has still grown, but at a much slower rate. I think it will go about 10 days like this instead of 1 before using up all system memory. For whatever reason we have 7 caches on this server. I do not see where this is configured. /Control_Panel/Database/temporary/manage_cacheParameters Total number of objects in the database 363761 Total number of objects in memory from all caches 34952 Target number of objects in memory per cache 5000 I flushed the cache and about 1 hour after flushing, memory use has not gone down at all. During that hour objects in all caches went from 0 to 15341. Objects in the transient object container are still holding steady at 2000-3000. Before the onDelete they grew without bound. /Control_Panel/Database/temporary/manage_cacheParameters Total number of objects in the database 369885 Total number of objects in memory from all caches 15341 Target number of objects in memory per cache 5000 It seems odd to me that the total objects in the db are growing so large. If I understand you correctly a workaround would be to change the number of objects per cache from 5000 to something more reasonable. Where is this configured and what is reasonable? 5, 50, 500. I do not see an item for this in the config file. Should there be any cache at all on an in-memory temp db? Andy Yates
-----Original Message----- From: Dieter Maurer [mailto:dieter@handshake.de] Sent: Friday, April 14, 2006 7:54 AM To: Andy Yates; zope@zope.org Subject: No memory leak! (was: Re: [Zope] Memory Leak in Session Data Container)
Dieter Maurer wrote at 2006-4-11 22:14 +0200:
Andy Yates wrote at 2006-4-10 12:43 -0500:
... Fortunately, I have an easy way to reproduce the problem and even better I've found a work around. My hope is that the work around will lead somebody more familiar with the innards of Zope to where the problem is located.
Congratulations!
I will look into this within the next days (in Zope 2.8.1, however).
I checked this:
There is no memory leak related to session handling in Zope 2.8.1!
... The ZODB cache may keep your session objects (and thereby its content) alive.
Almost surely, this is what happened in Andy's test.
Here is a description how I checked for the existence of a memory leak:
I instrumented the pickles in "tempstorage.TemporaryStorage.TemporaryStorage" such that I can find out the number of pickles stored (similar to the reference counts in "Control_Panel/DebugInfo").
I performed a mass test, creating thousands of sessions (similar to Andy's test).
The pickle number stored by "TemporaryStorage" grew as expected (as it stores the sessions).
I forced all session objects maintained by "/temp_folder/session_data" to be discarded.
Creating a single new session caused almost all "TemporaryStorage" pickles to be released.
However, the ZODB caches still contained about 20.000 session objects. It was very surprising (and contrary to my previous message) that the storage flushed most of its pickles while references to them were still in the ZODB caches. This looks like a bug (but not a memory leak).
I flushed the ZODB caches.
I repeated the test and verified that the number of pickles remaining in "TemporaryStorage" does not increase.
Results:
* no memory leak!
* maybe something that looks like a memory leak caused by the ZODB caches which may hold up to 20.000 (with standard configuration) no longer used objects
A more senseful configuration can work around this.
-- Dieter