RE: No memory leak! (was: Re: [Zope] Memory Leak in Session Data Container)
Leak, bug, improper configuration, what ever. ;-) With the onDelete in place for 1 week now I checked the temporary Cache Parameters as you requested. Since I put the onDelete on this server memory use has still grown, but at a much slower rate. I think it will go about 10 days like this instead of 1 before using up all system memory. For whatever reason we have 7 caches on this server. I do not see where this is configured. /Control_Panel/Database/temporary/manage_cacheParameters Total number of objects in the database 363761 Total number of objects in memory from all caches 34952 Target number of objects in memory per cache 5000 I flushed the cache and about 1 hour after flushing, memory use has not gone down at all. During that hour objects in all caches went from 0 to 15341. Objects in the transient object container are still holding steady at 2000-3000. Before the onDelete they grew without bound. /Control_Panel/Database/temporary/manage_cacheParameters Total number of objects in the database 369885 Total number of objects in memory from all caches 15341 Target number of objects in memory per cache 5000 It seems odd to me that the total objects in the db are growing so large. If I understand you correctly a workaround would be to change the number of objects per cache from 5000 to something more reasonable. Where is this configured and what is reasonable? 5, 50, 500. I do not see an item for this in the config file. Should there be any cache at all on an in-memory temp db? Andy Yates
-----Original Message----- From: Dieter Maurer [mailto:dieter@handshake.de] Sent: Friday, April 14, 2006 7:54 AM To: Andy Yates; zope@zope.org Subject: No memory leak! (was: Re: [Zope] Memory Leak in Session Data Container)
Dieter Maurer wrote at 2006-4-11 22:14 +0200:
Andy Yates wrote at 2006-4-10 12:43 -0500:
... Fortunately, I have an easy way to reproduce the problem and even better I've found a work around. My hope is that the work around will lead somebody more familiar with the innards of Zope to where the problem is located.
Congratulations!
I will look into this within the next days (in Zope 2.8.1, however).
I checked this:
There is no memory leak related to session handling in Zope 2.8.1!
... The ZODB cache may keep your session objects (and thereby its content) alive.
Almost surely, this is what happened in Andy's test.
Here is a description how I checked for the existence of a memory leak:
I instrumented the pickles in "tempstorage.TemporaryStorage.TemporaryStorage" such that I can find out the number of pickles stored (similar to the reference counts in "Control_Panel/DebugInfo").
I performed a mass test, creating thousands of sessions (similar to Andy's test).
The pickle number stored by "TemporaryStorage" grew as expected (as it stores the sessions).
I forced all session objects maintained by "/temp_folder/session_data" to be discarded.
Creating a single new session caused almost all "TemporaryStorage" pickles to be released.
However, the ZODB caches still contained about 20.000 session objects. It was very surprising (and contrary to my previous message) that the storage flushed most of its pickles while references to them were still in the ZODB caches. This looks like a bug (but not a memory leak).
I flushed the ZODB caches.
I repeated the test and verified that the number of pickles remaining in "TemporaryStorage" does not increase.
Results:
* no memory leak!
* maybe something that looks like a memory leak caused by the ZODB caches which may hold up to 20.000 (with standard configuration) no longer used objects
A more senseful configuration can work around this.
-- Dieter
Andy Yates wrote at 2006-4-17 16:35 -0500:
... /Control_Panel/Database/temporary/manage_cacheParameters Total number of objects in the database 369885 Total number of objects in memory from all caches 15341 Target number of objects in memory per cache 5000
It seems odd to me that the total objects in the db are growing so large.
This looks fishy, indeed! Somehow, the cleanup code of "tempstorage.TemporaryStorage.TemporaryStorage" does not seem to work in your installation while it does in mine (Zope 2.8.1 with some modifications).
If I understand you correctly a workaround would be to change the number of objects per cache from 5000 to something more reasonable.
Only to get down the "total number of objects in memory from all caches". As I know now (and contrary to my original expectation) it would not affect the "Total number of objects in the database". Even if my original expectation were right, the "number in the database" is far too high to be explained by the cached objects.
Where is this configured and what is reasonable?
It moved from the ZODB (configured via the ZMI) to the Zope configuration file (one of the keys of the "zope_db" section). I no longer know when this move happened. I think, it was Zope 2.7 (but may be wrong).
5, 50, 500. I do not see an item for this in the config file.
If you expect X sessions, then you may want to keep about the most recent X/4 sessions in the cache (rather than repickling them over and over again). The exact value depends on your preferences (your balance for speed versus memory consumption). To stress it again: your large "number of objects in the database" is not explained by the objects in the cache. Something else must be wrong in your installation. Limiting the ZODB cache size for the "temporary" storage will not drastically effect your memory consumption.
Should there be any cache at all on an in-memory temp db?
The cache does not save IO, of course, but unpickling time which may be considerably as well. A recent profile showed: IO 3.9 s Unpickle 1.3 s However, it was a singular measurement. You probably cannot generalize it. -- Dieter
participants (2)
-
Andy Yates -
Dieter Maurer