[ZODB-Dev] [cPickleCache] leaks temporary connections --> severe memory leak

[Dieter Maurer]

ZODB 3.1, Python 2.1.3

Python 2.1.3 (#2, Mar 13 2003, 13:39:51) 
[GCC 3.2] on linux2
Type "copyright", "credits" or "license" for more information.
>>> from sys import getrefcount as gr
>>> from ZODB.DB import DB
>>> from ZEO.ClientStorage import ClientStorage
>>> from ZODB.Connection import Connection
>>> def step():
...   print gr(Connection)
...   db= DB(ClientStorage(('localhost',9888)))
...   c= db.open(temporary=1)
...   r= c.root()
...   print gr(r)
...   a= r['Application']
...   print gr(r)
...   c.close() # this calls "c.__dict__.clear()"
...   db.close()
...   print gr(Connection)
...
>>> step()
8
3
4
10
>>> step()
10
3
4
12

We leak 2 connections per step because access to "r" leaked a reference (once,
not repeated). This is quite serious as the lost root reference
keeps the complete pickle cache in memory -- we can loose arbitrary
amounts of memory for each step.


This is apparently caused by a "cPickleCache" bug.

  Accessing a ghost object (unghostifying it)
  apparently adds a reference for the object which is only
  released when the object is ghostified again.
  However, once the connection is closed, ghostifying will
  never take place.
-- 
Dieter