G'Day, OK, so I've tried to flush the cache, and look at the ref counts, and nothing jumped out at me other than the usual suspects. I often see refcounts to a variety of my own content types/classes, but they seem pretty stable. Since they represent a large portion of our content, I'm not worrying too much about them, though I probably should and soon will. Here's an interesting technique I'm using: First, in ZPublisher I patch HTTPRequest.__init__ to maintain a global list of weakref's to the instances created. This gives me a way to access existing HTTPRequest instances, notably the ones that are sticking around that shouldn't be. I'm also adding a unique id (In the form of a counter) to each request so I can uniquely identify them. Then, I write an external method and use the gc module to find out the referrers/referants. I'm hoping this will give me some clues as to what's holding the requests around. Right now I've managed to use gc.get_referants() with great success. I can dump (using pprint) the request object and read it's contents. Once I've got a few samples taken, I'll hopefully be able to track down whihc ones are actual leaked requests (by their id), and see if they have anything in common. I tried briefly gc.get_referrers() but I get ugly errors. Another idea I had was to through the frame stack upon __del__ of a req and log that info to see if there's a commonality to those also. One question: is this bad when applied to a persistent object?: setattr(self,'thumbnail', Image(id='thumbnail') ) (I've monkey patched CMFImage to have thumbnails) Thanks and cheers, J.F. -----Original Message----- From: Dieter Maurer [mailto:dieter@handshake.de] Sent: May 16, 2004 5:48 PM To: Jean-Francois.Doyon@CCRS.NRCan.gc.ca Cc: stefan@epy.co.at; zope@zope.org Subject: RE: Leaking HTTP requests (was: RE: [Zope] Leaking Acquisition.Im plic itAcquirerWrapper) Jean-Francois.Doyon@CCRS.NRCan.gc.ca wrote at 2004-5-14 17:23 -0400:

... I'll leave the iste running throught the week-end and we'll see what shows up. "high" refcounts is somewhat arbitrary I guess ... When you say high you mean high and stable or high and growing?

In fact, I mean unexpectedly high. The relevant references counts are not necessarily the highest ones. For example, when you make catalog searches, it is completely normal that you see lots of "DateTime" objects. Nevertheless, you can be sure that they are not the cause of memory leaks. The same applies for acquisition wrappers. You must look for usually rare objects that suddenly show up in large numbers. It may help very much to clear the ZODB cache ("Control_Panel" --> "Database management" --> <databases> --> "Flush Cache" --> "Minimize"). This should delete all objects from memory that are "normal" artefacts of Zope processing. Apart from a few objects only the leaked ones should remain. Be warned, there was a bug in "cPickleCache.c" that led "Minimize" enter an infinite loop. I am not sure whether this bug is fixed in the current Zope 2.7 release.

Right now, the only suspect class as far as refcounts (Other than the HTTP requests) is the DateTime one. I have some content ones that look high, but then they're highly used (CMFImage for example).

"DateTime" are normal, "HTTP requests" is strange....

Apart from using aq_explicit in a few places, I do also use __of__ in one case ... To do things like:

def getAbstract(self): try: pt = ZopePageTemplate( '', self.abstract, 'text/html').__of__(self) cookedbody = pt()

Harmless. The problem in my code has been that I stored an acquisition wrapped object in my object.

... How about this?

pt = ZopePageTemplate( '', self.aq_explicit.intropart1, 'text/html' ).__of__(self)

Could THAT be bad?

Unlikely (unless you store it as attribute of "self"). -- Dieter