[Zope-dev] debugging memory leaks

[Seb Bacon]

Seb Bacon wrote:
> So, say Foo is leaking because it is referenced from O which can't be 
> collected.  Given 100 things which refer to Foo, how do I identify which 
> one is O?  And of course, then O may be leaking because it is referenced 
> from P...
> 
> I sense this question is a bit like asking someone to explain how to 
> solve a Rubik's Cube in 3 words.

Well, I have come to some kind of resolution, though I am still slightly 
mystified.  Here's the sequence of events, in case they are of any help 
to others (doubtful...).

Although there probably is a memory leak in my application, the one I 
thought I was hunting wasn't what was causing my server collapse.

At first, I noticed that memory usage was increasing linearly over time 
until the server expired.  I examined reference counts for all the 
classes therein, mainly using Shane's LeakFinder product (I could have 
used the refcounts listing on the control panel, but I found the 
LeakFinder's reference count display tab nicer to use.)  I noticed that 
references to a particular Foo class were increasing in direct 
proportion to the memory usage, apparently without bound.  I also noted 
that Foos are involved in reference cycles.

I guess from this that maybe Foos were leaking somehow - which was 
incorrect.  There is nothing wrong with reference cycles *per se* (see 
earlier in this thread).

Then I looked at Bars,  which were referencing Foos.  Given the way that 
Foos are implemented (with mutual references to each other) and the fact 
that there may be several Foos stuck on a Bar, then a leak in a Bar 
could have a big knock-on effect of creating a whole ton of Foos.  The 
number of references to Bars was also increasing without bound.

This went on for ages.  Worth mentioning is Barry's cool reference 
visualisation tool (see earlier in this thread).

I had already tried my application using Zope 2.6.2 (it was on 2.6.1 
before) and noted reference counts also going up rapidly, so it wasn't 
that, I decided.

To cut a long story to a medium length, it *was* that.  When using 
2.6.2, I noticed that if I forced garbage collection, the refcounts went 
down.  Going over to the database connection caches, I noted that in 
Zope 2.6.1 the number of cache entries bore no relation to the target 
cache size.  In Zope 2.6.2, it did.

In other words, the way my application is implemented means that *lots* 
of references can accumulate in the space of a single request, and 
something about these references meant that they were never getting 
cleared out of the cache by Zope 2.6.1.  The cache was the culprit.

Moral: always keep on top of those Zope releases ;-)

What's puzzling me is that I can't see anything that changed between 
2.6.1 and 2.6.2 which might have fixed this behaviour.

Seb