[ZODB-Dev] memory or cache leak?

[Michel Pelletier]

Jeremy Hylton wrote:
> 
> >>>>> "AD" == Andrew Dalke <adalke@mindspring.com> writes:
> 
>   AD> I'm not sure what to try next.  I'll try staring at C code, but
>   AD> barely have an idea of what's going on.  Does anyone here have
>   AD> any hints?
> 
> I can offer hints only, as I don't understand how the cache works.

Same here.
 
> I tried running your example code with StandaloneZODB.

I tried it with Zope's ZODB.

>  I made the
> following changes:
> 
>     - Commented out the Catalog-related stuff; it's not part of
>       StandaloneZODB and didn't appear to be used by the app.
> 
>     - Change the object size from 128*1024 to 32*1024, because I've
>       only got 256MB of memory on my machine.
> 
> When I run the test in that mode, its memory consumption does seem to
> grow quickly.  The cacheSize() returns a large number, but I'm not
> sure why.

I got the same result.  I tried:

  - commit subtransactions instead of transactions every % 1000.

  - Use BTrees instead of PMs

Memory still grew very fast.
 
> If I change the arguments to cacheMinimize() and cacheFullSweep() from
> 0 to 4, the memory usage seems to be capped at 37MB.

Hmm, that's interesting.  I just tried that too, and it works for me.

I think the problem is, when you pass 0 as a argument it runs in the
following code in cPickleCache.c:

  if (dt < 0) dt=0;
  else dt /= 3;

which of course sets dt to 0.  This value is then passed into gc_item
for each object in the cache which does nothing if dt is logicaly false
(see line 155 in cPickleCache.c).  Any positive value for dt will result
in gc_item actually cleaning up objects.

-Michel



>  The cache size
> still seems to grow frequently, but it occasionally goes down.
> 
> Jeremy
> 
> _______________________________________________
> For more information about ZODB, see the ZODB Wiki:
> http://www.zope.org/Wikis/ZODB/
> 
> ZODB-Dev mailing list  -  ZODB-Dev@zope.org
> http://lists.zope.org/mailman/listinfo/zodb-dev