Just in case anyone can shed anymore light on zope memory woes, here is where I've got to in my investigation: - Memory grows and mostly doesn't shrink. - Its not due to objects that can't be collected by the GC - Its not due to the ZODB cache That leaves the following causes I think - RAMCache etc and module/class variables. Either mine, plone's or zope's. - Pythons design of not releasing memory from small objects back to the OS http://www.sauria.com/~twl/conferences/pycon2005/20050325/Improving%20Python's%20Memory%20Allocator.html Has anyone tried to determine how much of an average catalog search is <256kb? I'm guessing a lot. - Some other kind of reference being help somewhere in the zope machinery. Any tips anyone?
"Peter Bengtsson" <mail@peterbe.com> wrote in message news:loom.20050426T185741-270@post.gmane.org...
Dylan Jay <me <at> dylanjay.com> writes:
If anyone has experience with finding these kind of rouge references I'd really appreciate any tips.
I have no idea how to patch tings with LeakFinder to find your problems because LeakFinder, albeit a great piece of code, attempts an incredibly difficult task. Most likely, your memory leaks is due to an assignment of really large lists on complex objects (eg. more complex than strings or numbers).
Can you expound on this a little as this sounds very interesting? Why do assignments to a list normally result in leaks? Do you mean lists of complex objects or complex objects that have lists?
Look for any constructs where you extract large lists and do operations on them. Once you have, if you ever, contain these, return to the mailing list and we can try to solve it together.
I certainly do operations on lists, partucularly LazyMap etc. I do operations with catalog results and map them to lazily to custom objects which are joined by aquisition to catalog brains.
I'm new to memory leak debugging in python but have done it before in java.
From what I understand garbage collection happens differently in java. In java you have to avoid circular references, is the same true in python?
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )