Chris -- Thanks for the assist. I think that I found the source of the problem last night and the the memory leak hypothesis was a cockpit error. The problem manifested itself when running a find script which was doing an edit cleanup of a lot of stuff (thousands of ZUBBB discussions and ZUBB postings). One of the things the script did was to reset all the ZUBB methods to the default by copying in the half-dozen or so methods present in every instance of the discussions. Zope apparently treats the whole edit as a single transaction and buffers up the changes until it comes time to commit. Rewriting the edit script to do things a bit differently and only make changes when they are needed dropped the memory growth and time by a factor of eight or so. While not really anything I'd want to run execpt for the nonce to maintain the system, it did solve my immediate problem. The huge memory that got allocated did get released upon successful completion of the script (and when the script was aborted as well). Is there some mechanism to force a ZODB commit explicitly? If there were, then the edit script would have the structure: find the thing to edit edit the thing commit the edits and the resouce demands would be under control. On Sun, 12 Sep 2004, Chris McDonough wrote:
I've noticed what I believe are "request leaks" if any variety of CMF is involved in a request though I haven't tracked this down. The symptoms are that every request causes the HTTPRequest class to have its reference count increased by one, plus a bunch of other objects are leaked because the request that was leaked apparently continues to hold on to references to them.
This may or may not be what you are seeing.
If you want to track something like this down, there is really no shortcut. My strategy (which I've never successfully used to actually fix a leak, mind you! ;-) is this:
- Confirm that bare Zope doesn't leak by testing refcounts before and after you load the quickstart page several times (remember to clear the ZODB cache before you start hitting the site whenever trying to test refcounts). Look at the number of references to all classes in the Debug tab. These references should not normally grow on every request.
- Do same for one of your "domain" objects.
If you see leakage when trying this against a method of one of your domain objects, a binary-exclusion method is usually the only way to pin down the leak. Prevent half of the code that gets run when your domain object method is called from running. If you still see the leak, the leak happens in the code you didn't prevent from running. If you did, the leak happens in the code that ran. And so forth. It's tedious and time-consuming, which is probably why I've never successfully used it to fix a leak. ;-)
- C
On Sun, 2004-09-12 at 01:33, Dennis Allison wrote:
Pound-current (V 1.7+) Zope 2.6.4 Python 2.3.4 RH 7.3 Dual Athalon processors
Occasionally, I see Zope gathering very large amounts of real memory. Normally buffering and the like seems to stabalize at around 500MB on a 4GB memory dual processor machine, but occasionally grows to 2.5GB or more. At the same time, CPU usage tends to fully occupy one of the processors for the threads holding all the memory. Clearly there is a problem. Any hints as to where to track down the problem?
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )