[Zope] Sessions and RAM cache managers causing memory leak

[Chris McDonough]

On Mon, 2004-05-17 at 05:34, Richard Ettema wrote:

> I created a test dtml method that creates a simple session (with a 
> single string stored), and also calls all the methods/scripts that are 
> cached. I then went about adding/removing these and running httperf to 
> narrow down the problem. This is what happened calling the test method 
> by httperf for approx 15 mins after restarting zope after each test 
> (initial memory usage was 22M before running httperf)...
> 
> Calling all methods/scripts being cached (no sessions created):
> 86M used at end of test.
> 
> Calling the sessions only, creating a session for each call (no caching):
> 60M used at end of test.
> 
> Calling cached methods/scripts and session creation together (this is 
> everything):
> 471M used at end of test (usually more).
> 
> Calling 15min cache objects and session creation (exclude 24hour cache 
> objects):
> 192M used at end of test.
> 
> Calling 24hour cache objects and session creation (exclude 15min cache 
> objects):
> 159M used at end of test.
> 
> I don't understand whats going on here. there seems no problem between 
> the 2 RAM caches when you run them together. No problem with the session 
> machinery when you create sessions by itself. but put everything 
> together and it all goes very wrong!
> I would have thought if the caching on its own ends up at 86M and the 
> sessions on their own end at 60M, then together i should expect around 
> 140-150M.
> 
> Any ideas of where I should be looking, something I am overlooking, or 
> any known problems with cmf,caching,sessions that I should be aware of?

Not really, unfortunately.  How are you determining memory usage?  Do
you have any "cache keys" for your cache managers (except
AUTHENTICATED_USER, which is the default)?  There is a rendering
potentially stored for each cache key when the cache key differs per
request. 

Maybe try storing your "transient object container" in the "main" ZODB
instead of storing it in /temp_folder/session_data (which is backed
entirely by RAM).  I don't recommend doing this long-term but it might
be helpful during diagnosis.

The only other thing I can suggest is to try to make a reproducible test
case and post the steps (and if possible a .zexp) in the Zope.org
collector.

- C