[Zope-dev] Very severe memory leak

[Shane Hathaway]

On 08/22/2003 05:38 PM, Leonardo Rochael Almeida wrote:
> In time, DateTime refcounts eventually dwarves the second place by an
> order of magnitude. I think this is related to the fact that DateTime
> instances are stored as metadata, even though the date indexes have been
> converted to DateTime indexes. The question is, why aren't those
> instances being released? What is holding on to them?

When you flush the cache, those DateTimes should disappear.  If they 
don't, the leak is keeping them.

> I tried installing the LeakFinder product but discovered it didn't work
> before stumbling in a message in the archives that told me exactly that
> :-) The RefCounts view in the LeakFinder object fails with the following
> traceback:

LeakFinder is an early attempt to share some of the techniques I use for 
finding leaks.  Unfortunately, those techniques aren't very useful until 
you've already searched for weeks, which means LeakFinder isn't very 
good for emergencies.

Here are the things you should look at first:

1) The ZODB cache size.  The meaning of this number changed dramatically 
in 2.6.  Before 2.6 it was a very vague number.  In 2.6 it's a target 
number of objects that Zope actually tries to maintain.  Before 2.6 it 
might have made sense to set the ZODB cache size to some arbitrarily 
high number like 100,000; in 2.6 you want to start at about 2000 and 
adjust from there.  There are tools in 2.6 for helping you adjust the 
number.

2) The number of ImplicitAcquisitionWrappers present in the system.  I 
have found it to be a reliable indicator of whether you have a leak or 
not.  Expect this number to stay under 400 or so.  If it grows 
gradually, there's a leak.  Watch the refcounts screen.

3) Is Python compiled with cyclic garbage collection enabled?  2.4 and 
above absolutely require cyclic garbage collection.

4) Don't use 2.6.1.  Use 2.6.2, which has fixes for known leaks.  It is 
actually already tagged in CVS as "Zope-2_6_2", and it's what zope.org 
is now running.  Various unrelated things prevented a formal release 
this week.

If all else fails, grep all Python modules for "sys._getframe()" and 
"sys.exc_info()".  These are the primary causes of memory leaks in 
Python 2.1 and below.  If you're brave, you can just run Zope under 
Python 2.2, which fixes those particular leaks AFAIK.

Finally, there's always hope. :-)  The latest thing I've been doing is 
running Zope in a debug build of Python.  A debug build makes a magical 
"sys._getobjects()" available, allowing you to inspect all live objects 
through a remote console.  Since debug builds aren't much slower than 
standard builds, you can even run a debug build in production for short 
periods of time.  I've been building a small library of functions for 
working in this mode, and if you need them, I'll pass them along.  I'd 
have to warn you that they are anything but intuitive in their purpose 
and use, though.

Shane