I use a catalog-intensive setup, and my memory footprint for Zope seems to slowly increase over several days. Might this be a leak? On my 2 ZEO identical clients, I get about an average of 50-60 catalog queries per minute each, and load is distributed equally (round-robin) over the 2 nodes by my proxy server. Both nodes were restarted at the same time yesterday, yet the one node that performed a re-indexing of about 16,000 objects early this morning has an RSS of 242 MB vs. 219 MB on the other node. The RSS on both nodes had grown to over 450 MB earlier yesterday before I restarted Zope, so I wonder if that is a leak, esp. in regard to re-indexing a catalog, or something else.
I know one can do some bad coding that leads into memory leaks. The first one is when you perform a query in a zcatalog. Its very common to put results in the REQUEST using REQUEST.set. This procedure creates a potencial risk for memory leaks. I actually tracked one down: I had a heavy query to a zcatalog that each time it was executed the python footprint increased in 5 Mb when query started and decrease 3Mb as query ends "leaking" 2Mb. I was using the REQUEST.set code and after converting this into a set of dtml-let and dtml-return the "leak" from executing the mentioned query was no more, ceased to be. Another thing I done was to track down code which create objects and dont implement nothing to delete them from memory. The tipical error is, for instance, when you in a external method do something like: ---------------- file = open('BIG','r') buffer = file.read(4000000) [... execute some code with buffer ...] del buffer return something ---------------- Most people forget the "del buffer" part. Another possibility I ran was concerned with the new BTree implementation since zope2-I-dont-remember-version. With this new BTrees one can use a manage_convertBTrees to convert the existing catalogs to the new implementation. I converted all my zcatalogs to this new implementation and, I cant put my hands on fire for this but, I think the leaks started to increase. And this rumour gain another strength when I saw on Top of RefCounts this BTree class, which is a C extension and probably impossible to patch with the leakFinder, just like Shane said. I'll continue to look on zcatalogs because of this relation between the manage_convertBtrees and the finding of BTrees.IIBtree.IIBucket at the top of my servers refcount. Regards, Julio Dinis Silva _________________________________________________________________ Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp