[Zope-Coders] BTrees memory leak?
Matthew T. Kromer
matt@zope.com
Wed, 31 Oct 2001 09:36:31 -0500
Jeremy Hylton wrote:
>If I run the BTrees test cases in a loop, they appear to leak ~18MB
>per run. This should be repeatable from a StandaloneZODB build:
>
>[do a checkout]
>cd StandaloneZODB
>python setup.py build
>python test.py -L BTree
>
>On Linux, it reports on memory usage each time through the loop.
>
>Jeremy
>
BTW, I instrumented the BTree code, replacing Py_INCREF and Py_DECREF
with versions that would log to stderr. After selecting grepping for
OOSets (which testrunner is leaking) and running it through a simple
filter program, it doesn't seem that I see any! This makes me highly
suspicious of the test suite itself.
Since ONLY the BTree modules were instrumented, I wouldn't see refcount
changes caused by modules.
Here's an example log file record:
djinn(177)$ more testout.run2.OOSet
INCREF OOSet @ 081cbca0 to 2 from BTrees/BTreeTemplate.c 340
INCREF OOSet @ 081cc568 to 2 from BTrees/BucketTemplate.c 435
INCREF OOSet @ 081cad50 to 2 from BTrees/BucketTemplate.c 435
and here's the log checker program, which just keeps stacks of INCREFs,
and tosses the entry when it sees a DECREF going to 0.
djinn(358)$ cat logcheck.py
#!/usr/bin/env python2.1
import xreadlines
import sys
import string
file = open(sys.argv[1])
objs = {}
for line in xreadlines.xreadlines(file):
(how, what, at, where, to, count, frm, file, lineno) =
string.split(line)
if how == "INCREF":
a = objs.get(where,[])
a.append(line)
objs[where] = a
elif how == "DECREF":
a = objs.get(where,None)
if count == '0' and a is not None: del objs[where]
print "INCREFS which did not get DECREF'd to 0:"
for k in objs.keys():
for l in objs[k]:
print l