[Zope] Memory Use problems

[Douwe Osinga]

Hi Zopers,

The memory use of my Zope instance (2.6.2 with python 2.1 on FreeBSD4)
increases over time until the whole thing hits the limit (at 512Mb) and
restarts. I have installed LeakFinder, but there doesn't seem to be an
obvious memory leak.

>From what I've learned from LeakFinder, it seems that the Catalog is the
problem, or rather, my use of the catalog. We have 50.000. objects in the
Catalog, which are instances of ZClasses. These ZClasses have sometimes
rather large string/text properties and some DateTime() properties. When
memory use approaches the 500Mbyte, I typically see around 100.000.
instances of BTrees._IOBTree.IOBucket, which seems like a lot considering
that we have 50.000. pageviews a day and this usually occurs after an hour
or two. There'll be around 50.000. DateTime() instances and also quite some
OFS.Image.Image (maybe 15.000.). And yes, our pages do use the Catalog
exensively, but this seems hard to avoid.

How do I find where this memory goes? The pages do not take a long time to
lot (2 seconds max) and the cpu usage is not that high that I'd expect some
kind of deadlocks there. Also, Zope never seems to release memory, but just
takes more and more, even if I flush the cache and/or check the perform
garbage collection box in LeakFinder.

Douwe Osinga
http://www.world66.com