Mark, I've also been struggling with Memory Leaks in Zope, and not having much luck with it either. The behavior was much the same as yours, only over a longer period of time (My Zope will last up to 8 or 10 days). I'm using 2.5.1 however. First thing I would recommend would be to install the LeakFinder product. It was instrumental in my finding more about my leak. Looks like the LeakFinder, somehow imporves the refcount, and after installing it, you may get better refcounts that will give you a clue of what's going on. In my case, where the DateTime refcounts never were a problem before, with LeakFinder, I learned that I was leaking DateTime objects through lost references it seems. I'm also pretty sure it's got something to do with the ZCatalog, since it's the one that instantiates these most. You may have been reading my previous e-mails. In my case though I think it's got to do with the searching, not the indexing. As far the request.set, apparently that's an "old" issue that probably is no longer, but I removed them as best I could from my code ... didn't change much anyways. This was due to concerns with the fact that the request object is so intensively used and passed around internally, keep track of references may have been a problem at some point. Right now I've got my eyes on this piece of code: from random import choice selecteditem = choice(context.portal_catalog.searchResults( meta_type='Featured Item', includeinrotation = '1', review_state='published')) return selecteditem Which I could see how it might loose references if not done right, I'll be running tests on this soon. This type of behavior seems to be experienced by a range of people with various apparent causes ... It would be nice if we could put our heads together on this. Anyways, sorry, I got carried away :) First: Setup LeakFinder :) Cheers, J.F. -----Original Message----- From: Mark Thomas [mailto:markt@kooee.com.au] Sent: Monday, February 03, 2003 8:43 PM To: zope@zope.org Subject: [Zope] Memory leak I'm having some pretty severe problems with memory leaks, where the memory usage on my server is growing rapidly, then crashing Zope (every 2 days or so). When Zope crashes, it often restarts itself which isn't too bad if this was all that is required to get all services up and running again. However, I have to manually re-create a 'transient object container' in the temp_folder every time Zope is restarted. Also, when Zope goes down (or the memory is all taken up) it often takes out other services which don't automatically restart, such as ssh, telnet, mail etc. Obviously, the real answer is to stop the memory leak in the first place, so that Zope never has to be restarted. I'm running Python 2.1.3 and Zope 2.6.0 and I've read and followed much of the advice given on this mailing list re: memory leaks. For instance, I've set ZOPE_SECURITY_POLICY=3DPYTHON in my Zope startup script and checked the reference counts for various things, but can't spot anything that might be a problem. I also saw a 'memory leak possible solved' message which talked about indexing catalogs as being a problem. However, this didn't solve my problem. In my python scripts I often use request.set and session.set commands, and I read that they may be causing problems. I use these functions to pass variables between python scripts and dtml methods or other python scripts. I sort of understand why using these functions may be a problem - because there is nothing to clean them up once sessions are closed? What is the better way to pass variables around? Any help would be greatly appreciated, the mailing list seems to be running out of ideas on this topic. Thanks in advance. _______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Thanks J.F., for the quick and relevant response. I had seen something about LeakFinder but was avoiding it because I had seen reports that it had increased some problems. Your experiences seem to indicate that it may just have given the appearance of making matters worse, but in fact it was just showing more accurate ref counts. I'll install it and see what I come up with! Mark Thomas Jean-Francois.Doyon@CCRS.NRCan.gc.ca wrote:
Mark,
I've also been struggling with Memory Leaks in Zope, and not having much luck with it either. The behavior was much the same as yours, only over a longer period of time (My Zope will last up to 8 or 10 days).
I'm using 2.5.1 however.
First thing I would recommend would be to install the LeakFinder product. It was instrumental in my finding more about my leak. Looks like the LeakFinder, somehow imporves the refcount, and after installing it, you may get better refcounts that will give you a clue of what's going on. In my case, where the DateTime refcounts never were a problem before, with LeakFinder, I learned that I was leaking DateTime objects through lost references it seems. I'm also pretty sure it's got something to do with the ZCatalog, since it's the one that instantiates these most. You may have been reading my previous e-mails. In my case though I think it's got to do with the searching, not the indexing.
As far the request.set, apparently that's an "old" issue that probably is no longer, but I removed them as best I could from my code ... didn't change much anyways. This was due to concerns with the fact that the request object is so intensively used and passed around internally, keep track of references may have been a problem at some point.
Right now I've got my eyes on this piece of code:
from random import choice
selecteditem = choice(context.portal_catalog.searchResults( meta_type='Featured Item', includeinrotation = '1', review_state='published'))
return selecteditem
Which I could see how it might loose references if not done right, I'll be running tests on this soon.
This type of behavior seems to be experienced by a range of people with various apparent causes ... It would be nice if we could put our heads together on this.
Anyways, sorry, I got carried away :) First: Setup LeakFinder :)
Cheers, J.F.
participants (2)
-
Jean-Francois.Doyon@CCRS.NRCan.gc.ca -
Mark Thomas