[Zope] Memory leak

Tue, 4 Feb 2003 13:01:15 -0500

Mark,

I've also been struggling with Memory Leaks in Zope, and not having much
luck with it either. The behavior was much the same as yours, only over a
longer period of time (My Zope will last up to 8 or 10 days).

I'm using 2.5.1 however.

First thing I would recommend would be to install the LeakFinder product.
It was instrumental in my finding more about my leak.  Looks like the
LeakFinder, somehow imporves the refcount, and after installing it, you may
get better refcounts that will give you a clue of what's going on.  In my
case, where the DateTime refcounts never were a problem before, with
LeakFinder, I learned that I was leaking DateTime objects through lost
references it seems.  I'm also pretty sure it's got something to do with the
ZCatalog, since it's the one that instantiates these most. You may have been
reading my previous e-mails.
In my case though I think it's got to do with the searching, not the
indexing.

As far the request.set, apparently that's an "old" issue that probably is no
longer, but I removed them as best I could from my code ... didn't change
much anyways. This was due to concerns with the fact that the request object
is so intensively used and passed around internally, keep track of
references may have been a problem at some point.

Right now I've got my eyes on this piece of code:

from random import choice

selecteditem = choice(context.portal_catalog.searchResults(
meta_type='Featured Item', includeinrotation = '1',
review_state='published'))

return selecteditem

Which I could see how it might loose references if not done right, I'll be
running tests on this soon.

This type of behavior seems to be experienced by a range of people with
various apparent causes ... It would be nice if we could put our heads
together on this.

Anyways, sorry, I got carried away :) First: Setup LeakFinder :)

Cheers,
J.F.

-----Original Message-----
From: Mark Thomas [mailto:markt@kooee.com.au]
Sent: Monday, February 03, 2003 8:43 PM
To: zope@zope.org
Subject: [Zope] Memory leak

I'm having some pretty severe problems with memory leaks, where the 
memory usage on my server is growing rapidly, then crashing Zope (every 
2 days or so).

When Zope crashes, it often restarts itself which isn't too bad if this 
was all that is required to get all services up and running again.  
However, I have to manually re-create a 'transient object container' in 
the temp_folder every time Zope is restarted.  Also, when Zope goes down 
(or the memory is all taken up) it often takes out other services which 
don't automatically restart, such as ssh, telnet, mail etc.  Obviously, 
the real answer is to stop the memory leak in the first place, so that 
Zope never has to be restarted.

I'm running Python 2.1.3 and Zope 2.6.0 and I've read and followed much 
of the advice given on this mailing list re: memory leaks.  For 
instance, I've set ZOPE_SECURITY_POLICY=3DPYTHON in my Zope startup 
script and checked the reference counts for various things, but can't 
spot anything that might be a problem.  I also saw a 'memory leak 
possible solved' message which talked about indexing catalogs as being a 
problem.  However, this didn't solve my problem.

In my python scripts I often use request.set and session.set commands, 
and I read that they may be causing problems.  I use these functions to 
pass variables between python scripts and dtml methods or other python 
scripts.  I sort of understand why using these functions may be a 
problem - because there is nothing to clean them up once sessions are 
closed?  What is the better way to pass variables around?

Any help would be greatly appreciated, the mailing list seems to be 
running out of ideas on this topic.
Thanks in advance.

_______________________________________________
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )