[Zope-dev] Memory 2.1.4-2.1.6 a.k.a. how to get objects out of the cache

[Eric Sattler]

All,

I also am having severe problems with memory creep.  Our problem
exhibits itself by quickly using gobs of memory and requiring a
zope restart after about a day and a half at about 100M resident.  

First our configuration:

Core Components:

FreeBSD 4.0/FreeBSD 3.4 - on separate machines of course
Postgres 6.5.3/Postgres 7.0(fixes some memory leaks)
Python 1.5.2
Zope 2.1.4/2.1.6 have tried both
pcgi with ZServer
Apache w/mod_ssl 1.3.9 and 1.3.12

Products:

ZPySQLDA
SQLSession
GenericUserFolder

I have looked at a lot of different things.  This includes:

Installing the gc(garbage collection) python package 
   - Neil Schemenauer's patches
Installing Postgres 7.0(fixes some connection memory leaks)
Tracking the cache cleanup inside zope
Tracking thread locking inside DB.py, Transaction.py

Results:

gc python didn't help alleviate the problem
Postgres 7.0 instead of 6.5.3 didn't help the problem

I do see GenericUserFolder and SQLSession objects with the
Control_Panel_Debug screen, and they do not seem to go away.  I wrote
a simple python script to do nothing more than authenticate (log in)
using the GenericUserFolder method docLogin.  The memory usage quickly
grows out of control.  After waiting 15 minutes(my cookie timeout),
no decrease in memory usage.  The objects are still in the cache also. 

Restarting zope clears everything up, and starts out nice and clean
again.
We have a cron job doing this right now.

Questions:

1.  How does a user's connection resources etc. get cleaned up
    by Zope after a cookie timeout?  Do I need to do this myself?

2.  How does a genericuserfolder's set of objects used by
an                       
    authenticated user get cleaned up.  I can't seem to make those
    objects get reclaimed by the system, even when the user logs
    off?

3.  I am not explicitly removing SQLSession objects.  Will my usage
    counts for these objects remain > 1, thereby never allowing them
    to get cleaned up, and thereby keeping my usage counts for 
    genericuserfolder's > 1?

4.  Since GenericUserFolder inherits from Folder objects, is it 
    possible that the reason my memory grows so fast is that Zope
    does not release the resources used by a user properly, during
    an abrupt disconnect?

I will assist/help trying different things to try to solve this problem!

thanks.

eric.