Tip: huge memory leak fixed (dtml and pythonscript)
Hello everybody, after much gnashing of teeth and pulling out of hair we finally seem to have found the error in our ways. What we were doing led to a massive memory leak and I thought I'd let everbody know so you know what to avoid. Our setup is something like <dtml-document or method> calls <pythonscript> and <pythonscript> calls <restrictedTraverse> for some given <path>. The resulting <object> is then used to render some content and the resulting data is returned. Here comes the warning: If you don't explicitly set the <object> obtained by <restrictedTraverse> to <None> (maybe you could <del> it instead just as well) this <object> will be leaked with any <subobjects> instantiated along the way, namely a <ImplicitAcquirerWrapper> and (in our case) a <DateTime>. Hope this helps someone somewhere. Jo. -- Internetmanufaktur Jo Meder ---------------------- Berlin, Germany http://www.meder.de/ ------------------- fon: ++49-30-417 17 63 33 Kollwitzstr. 75 ------------------------ fax: ++49-30-417 17 63 45 10435 Berlin --------------------------- mob: ++49-170- 2 98 89 97 Public GnuPG-Key ---------- http://www.meder.de/keys/jo-pubkey.txt
On Fri, 2 Aug 2002, Jo Meder wrote:
Hello everybody,
after much gnashing of teeth and pulling out of hair we finally seem to have found the error in our ways. What we were doing led to a massive memory leak and I thought I'd let everbody know so you know what to avoid.
Our setup is something like <dtml-document or method> calls <pythonscript> and <pythonscript> calls <restrictedTraverse> for some given <path>. The resulting <object> is then used to render some content and the resulting data is returned.
Here comes the warning: If you don't explicitly set the <object> obtained by <restrictedTraverse> to <None> (maybe you could <del> it instead just as well) this <object> will be leaked with any <subobjects> instantiated along the way, namely a <ImplicitAcquirerWrapper> and (in our case) a <DateTime>.
I wonder if this is true for python products also. I will certainly look into it and see if that is the case and if true that would certainly explain some strange behavior I have been seeing. I have a fair number of items that use restrictedTraverse and after zope has been running for a while it seems those objects refs never go down but other items do. Often my leaks do not seem as large but after long enough it gets fairly large. My zope process will grow from 70M or so to 500M in a few weeks and it seems to be all those objects refs which gets very high. Thanks a lot for the tip.
could you please add this to the zope collector at http://collector.zope.org/Zope -kapil On Friday 02 August 2002 02:50 am, Jo Meder wrote:
Hello everybody,
after much gnashing of teeth and pulling out of hair we finally seem to have found the error in our ways. What we were doing led to a massive memory leak and I thought I'd let everbody know so you know what to avoid.
Our setup is something like <dtml-document or method> calls <pythonscript> and <pythonscript> calls <restrictedTraverse> for some given <path>. The resulting <object> is then used to render some content and the resulting data is returned.
Here comes the warning: If you don't explicitly set the <object> obtained by <restrictedTraverse> to <None> (maybe you could <del> it instead just as well) this <object> will be leaked with any <subobjects> instantiated along the way, namely a <ImplicitAcquirerWrapper> and (in our case) a <DateTime>.
Hope this helps someone somewhere.
Jo.
Am 02.08.2002, 12:30 Uhr schrub kapil thangavelu <kthangavelu@earthlink.net>:
could you please add this to the zope collector at
Do you think this is a bug in Zope itself? I'm not sure. I'd rather suppose we brought this unfortunate situation upon ourselves by questionable application of whatever. Jo. -- Internetmanufaktur Jo Meder ---------------------- Berlin, Germany http://www.meder.de/ ------------------- fon: ++49-30-417 17 63 33 Kollwitzstr. 75 ------------------------ fax: ++49-30-417 17 63 45 10435 Berlin --------------------------- mob: ++49-170- 2 98 89 97 Public GnuPG-Key ---------- http://www.meder.de/keys/jo-pubkey.txt
On Friday 02 August 2002 11:03 am, Jo Meder wrote:
Am 02.08.2002, 12:30 Uhr
schrub kapil thangavelu <kthangavelu@earthlink.net>:
could you please add this to the zope collector at
Do you think this is a bug in Zope itself?
quite possibly, esp if the use case you give is repeatable generically. although its also likely/possible that if it was a zope bug it has been fixed in the 2.6 cvs as andy suggested. if you can test against that and still see the problem than i would file it in the collector. -kapil
I'm not sure. I'd rather suppose we brought this unfortunate situation upon ourselves by questionable application of whatever.
Jo.
This sounds like the leak in AccessControl that Matt fixed recently. We have been having similar Memory Leaks and apparently this is fixed in 2.6. What version are you using? -- Andy McKay @gmweb Consulting http://www.agmweb.ca ----- Original Message ----- From: "Jo Meder" <jo@meder.de> To: <zope@zope.org> Sent: Friday, August 02, 2002 2:50 AM Subject: [Zope] Tip: huge memory leak fixed (dtml and pythonscript)
Hello everybody,
after much gnashing of teeth and pulling out of hair we finally seem to have found the error in our ways. What we were doing led to a massive memory leak and I thought I'd let everbody know so you know what to avoid.
Our setup is something like <dtml-document or method> calls <pythonscript> and <pythonscript> calls <restrictedTraverse> for some given <path>. The resulting <object> is then used to render some content and the resulting data is returned.
Here comes the warning: If you don't explicitly set the <object> obtained by <restrictedTraverse> to <None> (maybe you could <del> it instead just as well) this <object> will be leaked with any <subobjects> instantiated along the way, namely a <ImplicitAcquirerWrapper> and (in our case) a <DateTime>.
Hope this helps someone somewhere.
Jo.
-- Internetmanufaktur Jo Meder ---------------------- Berlin, Germany http://www.meder.de/ ------------------- fon: ++49-30-417 17 63 33 Kollwitzstr. 75 ------------------------ fax: ++49-30-417 17 63 45 10435 Berlin --------------------------- mob: ++49-170- 2 98 89 97 Public GnuPG-Key ---------- http://www.meder.de/keys/jo-pubkey.txt
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
Am 02.08.2002, 20:02 Uhr schrub Andy McKay <andy@agmweb.ca>:
This sounds like the leak in AccessControl that Matt fixed recently. We have been having similar Memory Leaks and apparently this is fixed in 2.6. What version are you using?
Could be. We're using 2.5.1 with Python 2.1.3 Jo. -- Internetmanufaktur Jo Meder ---------------------- Berlin, Germany http://www.meder.de/ ------------------- fon: ++49-30-417 17 63 33 Kollwitzstr. 75 ------------------------ fax: ++49-30-417 17 63 45 10435 Berlin --------------------------- mob: ++49-170- 2 98 89 97 Public GnuPG-Key ---------- http://www.meder.de/keys/jo-pubkey.txt
participants (4)
-
Andy McKay -
Jo Meder -
kapil thangavelu -
kosh＠aesaeion.com