[Zope-dev] Memory 2.1.4-2.1.6 a.k.a. how to get objects out of the cache

[Jason Spisak]

Eric Sattler:

> Jason,
> 
> I don't have time right now to post this to the zope
> newsgroup, but we are bleeding profusely, and could
> use any insight you might have...
> 

Unfortunately just before recieving your message, I fired another one out
because I have run out of tests. :(

> 
> I also am having severe problems with memory creep.  Our problem
> exhibits itself by quickly using gobs of memory and eventually requiring
> a
> zope restart.
> 
> First our configuration:
> 
> Core Components:
> 
> FreeBSD 4.0/FreeBSD 3.4 - on separate machines of course
> Postgres 6.5.3/Postgres 7.0(fixes some memory leaks)
> Python 1.5.2
> Zope 2.1.4/2.1.6
> Apache w/mod_ssl 1.3.9 and 1.3.12
> 
> Products:
> 
> ZPySQLDA
> SQLSession
> GenericUserFolder
> 

There was recently a post that detained a leak in a database product.  I
can't remeber if it was SQLSession, or what.

> I have looked at a lot of different things.  This includes:
> 
> Installing the gc(garbage collection) python package
> Installing Postgres 7.0(fixes some connection memory leaks)
> Tracking the cache cleanup inside zope
> Tracking thread locking inside DB.py, Transaction.py
> 
> Results:
> 
> gc python didn't help alleviate the problem
> Postgres 7.0 instead of 6.5.3 didn't help the problem
> 
> I do see GenericUserFolder and SQLSession objects with the
> Control_Panel_Debug screen, and they do not seem to go away.  I wrote
> a simple python script to do nothing more than authenticate (log in)
> using the GenericUserFolder method docLogin.  The memory usage quickly
> grows out of control.  After waiting 15 minutes(my cookie timeout),
> no decrease in memory usage.  The objects are still in the cache also.

This is consistant with what I remember seeing on the list.  I don't use
any external databases, so my situation is really an argument for their
being something amiss in Zope or Python.

> 
> Questions:
> 
> 1.  How does a user's connection resources etc. get cleaned up
>     by Zope after a cookie timeout?  Do I need to do this myself?
> 

I don't use cookie here.

> 2.  How does a genericuserfolder's set of objects used by an
> 
>     authenticated user get cleaned up.  I can't seem to make those
>     objects get reclaimed by the system, even when the user logs
>     off?  is there a magic cache setting somewhere?
> 

Or GUF.

> 3.  I am not explicitly removing SQLSession objects.  Will my usage
>     counts for these objects remain > 1, thereby never allowing them
>     to get cleaned up, and thereby keeping my usage counts for
>     genericuserfolder's > 1?
> 
> 4.  Since GenericUserFolder inherits from Folder objects, is it
>     possible that the reason my memory grows so fast is that Zope
>     does not release the resources used by a user properly, during
>     an abrupt disconnect?

That's funny that I am experienceing the same symptoms as you, but my
installation is totally different. I'm not using GUF, or any DA's, and I'm
running on RH 6.1.  We really need to figure this out as a community.  I am
willing to try just about anything short of shipping my machine somewhere
off premisis. ;)

Bledding on the floor,

Jason Spisak
CIO
HireTechs.com
6151 West Century Boulevard
Suite 900
Los Angeles, CA 90045
P. 310.665.3444
F. 310.665.3544

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.