[Zope-dev] debugging memory leaks

[Seb Bacon]

Tim Peters wrote:
> [Seb Bacon]
>>...but my (naive?) reading of the documentation was that reference
>>cycles are cleaned out by the garbage collector, *unless* they define
>>a __del__ (which is not the case here).  How am I wrong?
> 
> 
> You're reading the docs correctly.  It's not necessarily cycles directly
> involving Foo objects that causes Foo objects to leak, it can be instead
> that some other (non-Foo) objects in cycles can't be collected, from which
> the Foo objects are in turn reachable.  When an object O can't be collected,
> then neither can any object reachable from O.  gc.get_referrers() can be
> used to find objects that refer to a given Foo instance.  It's also possible
> that a something S refers to a Foo instance where S doesn't participate in
> cyclic gc.  Then any cycle containing S is immortal, regardless of whether
> __del__ methods are defined in the cycle, and also then gc.get_referrers()
> can't reveal S's existence.  Sometimes such an S is in the Python core, or
> in Zope's C code, although the more recent the release the less likely that
> is (more & more kinds of objects have been added to cyclic gc over time).
> Are you sure that *only* Foo objects are leaking?  It's pretty rare, when
> there's a leak, to see only one kind of object leaking.

You're right, there seem to be a few other things involved.  I think Foo 
comes out top simply because it is the most numerous instance involved 
in the leak.

So, say Foo is leaking because it is referenced from O which can't be 
collected.  Given 100 things which refer to Foo, how do I identify which 
one is O?  And of course, then O may be leaking because it is referenced 
from P...

I sense this question is a bit like asking someone to explain how to 
solve a Rubik's Cube in 3 words.  but FWIW, the kind of logic I'm using is:

   - run test case
   - notice that there are a lot of references to Foo
   - get an instance of Foo using sys.getobjects(0)
   - get referrers using gc.get_referrers(Foo)
   - run garbage collection using gc.collect()?
   - is Foo still there?  Which of its referrers are still there?

Incidentally, I've found some other bug.  I can get Zope to segfault by 
calling PickleCache.minimize(3), if a Bar object has been loaded which 
defines a __del__ method thus:

   def __del__(self):
      print "deleting", self.getId()

It couldn't be related, could it?

(it's borking at a point where it frees memory)

Cheers,

Seb