Hi there! I got another problem with my Zope server and that seems to be a memory hole.. I am using Zope 2.3.2 with Python 1.5.2 now on SuSE 7.0 (this seems to solve the hanging problem, which appeared on 7.1) and I use Python Script, ZSQL methods, ZMySQLDA and ZPatterns. Now using top I see over time how the memory consumption of the zope processes grows, which means I have to restart once a day as it then becomes too slow.. When looking at the refcount in the debug section I see the following: Acquisition.ImplicitAcquirerWrapper: 7509 DocumentTemplate.DT_Util.Eval: 719 Products.ZPatterns.Expressions.Expression: 716 ZPublisher.HTTPRequest.HTTPRequest: 457 ZServer.HTTPResponse.ZServerHTTPResponse: 450 ZPublisher.BaseRequest.RequestContainer: 447 So Acquisition.ImplicitAcquirerWrapper seems to be referenced quite often (this is in an early state..) So does anybody know what this might cause? Or maybe how to track that problem down? (seems to be a long way with that server to become it stable.. ;-) Thanks in advance! Christian -- COM.lounge http://comlounge.net/ communication & design info@comlounge.net
Hi Christian, It looks like you're leaking requests. This is often the consequence of setting an attribute on the REQUEST object which is aquisition wrapped. For instance: REQUEST.adtmldoc = REQUEST['PARENTS'][0] When requests leak, references to all the objects referenced within the request (acquisition wrappers, etc.) leak as well. This can be disastrous and gets worse when the site is busy (when there are more requests). It used to be the case that tracking this problem down was tediously difficult, and involved basically a binary search for leaks in all methods on the site. But now, finding the cause is likely a job for Shane's LeakFinder product: http://www.zope.org/Members/hathawsh/LeakFinder . Note that this is likely caused by some sort of brainbending circular reference problem that might be helped by Python 2.X's cyclic garbage collector. - C Christian Scholz wrote:
Hi there!
I got another problem with my Zope server and that seems to be a memory hole..
I am using Zope 2.3.2 with Python 1.5.2 now on SuSE 7.0 (this seems to solve the hanging problem, which appeared on 7.1) and I use Python Script, ZSQL methods, ZMySQLDA and ZPatterns.
Now using top I see over time how the memory consumption of the zope processes grows, which means I have to restart once a day as it then becomes too slow..
When looking at the refcount in the debug section I see the following:
Acquisition.ImplicitAcquirerWrapper: 7509 DocumentTemplate.DT_Util.Eval: 719 Products.ZPatterns.Expressions.Expression: 716 ZPublisher.HTTPRequest.HTTPRequest: 457 ZServer.HTTPResponse.ZServerHTTPResponse: 450 ZPublisher.BaseRequest.RequestContainer: 447
So Acquisition.ImplicitAcquirerWrapper seems to be referenced quite often (this is in an early state..)
So does anybody know what this might cause? Or maybe how to track that problem down?
(seems to be a long way with that server to become it stable.. ;-)
Thanks in advance!
Christian
-- COM.lounge http://comlounge.net/ communication & design info@comlounge.net
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Hi!
It looks like you're leaking requests. This is often the consequence of setting an attribute on the REQUEST object which is aquisition wrapped. For instance:
REQUEST.adtmldoc = REQUEST['PARENTS'][0]
When requests leak, references to all the objects referenced within the request (acquisition wrappers, etc.) leak as well. This can be disastrous and gets worse when the site is busy (when there are more requests).
ok, that's what I experienced.. the day the site got some more traffic it also suddenly grow memorywise. I also didn't know that doing above is a bad thing to do.. (though I don't actually understand why this is a problem and why it's not freed then.. but on the other hand I am not that much into the inner workings of acquisition ;-)
It used to be the case that tracking this problem down was tediously difficult, and involved basically a binary search for leaks in all methods on the site. But now, finding the cause is likely a job for Shane's LeakFinder product: http://www.zope.org/Members/hathawsh/LeakFinder .
Ah, will try that out :) And actually I directly found some method which fetches the actual user object from a ZPatterns rack and sets this in the REQUEST at every request. So maybe that's already the problem then.. will try to remove that..
Note that this is likely caused by some sort of brainbending circular reference problem that might be helped by Python 2.X's cyclic garbage collector.
Well, will also try out python 2.x then... (btw. are all 2.x python versions working without problems with Zope? Don't want to get a new problem when resolving the old one ;-) Thanks! Christian
Note that this is likely caused by some sort of brainbending circular reference problem that might be helped by Python 2.X's cyclic garbage collector.
Well, will also try out python 2.x then... (btw. are all 2.x python versions working without problems with Zope? Don't want to get a new problem when resolving the old one ;-)
Sort of in a bad state at the moment. 2.4 requires Python 2.1, but it's in alpha. 2.3.2 isn't guaranteed to work with Python 2.X. So you're sort of screwed if you want support. OTOH, 2.3.2 is reported to work fine with Python 2.X, although if you post Python release-related weirdnesses here with 2.3.2, they'll likely be ignored in favor of directing you to 2.4 when it's out. - C
Chris McDonough wrote:
Note that this is likely caused by some sort of brainbending circular reference problem that might be helped by Python 2.X's cyclic garbage collector.
Well, will also try out python 2.x then... (btw. are all 2.x python versions working without problems with Zope? Don't want to get a new problem when resolving the old one ;-)
Sort of in a bad state at the moment. 2.4 requires Python 2.1, but it's in alpha. 2.3.2 isn't guaranteed to work with Python 2.X. So you're sort of screwed if you want support. OTOH, 2.3.2 is reported to work fine with Python 2.X, although if you post Python release-related weirdnesses here with 2.3.2, they'll likely be ignored in favor of directing you to 2.4 when it's out.
We're currently using python 2.1 to run zope 2.3.2 with no ill effects. We have written our own product that uses some OFS stuff, Catalog and ZPT. Not much else... Richard -- Richard Jones richard@bizarsoftware.com.au Senior Software Developer, Bizar Software (www.bizarsoftware.com.au)
Note that this is likely caused by some sort of brainbending circular reference problem that might be helped by Python 2.X's cyclic garbage collector.
Well, will also try out python 2.x then... (btw. are all 2.x python versions working without problems with Zope? Don't want to get a new problem when resolving the old one ;-)
Sort of in a bad state at the moment. 2.4 requires Python 2.1, but it's in alpha. 2.3.2 isn't guaranteed to work with Python 2.X. So you're sort of screwed if you want support. OTOH, 2.3.2 is reported to work fine with Python 2.X, although if you post Python release-related weirdnesses here with 2.3.2, they'll likely be ignored in favor of directing you to 2.4 when it's out.
We're currently using python 2.1 to run zope 2.3.2 with no ill effects. We have written our own product that uses some OFS stuff, Catalog and ZPT. Not much else...
The most serious problem with Zope 2.3.x on Python 2.0 that I have discovered so far is that Zope's xml-formatted export doesnt handle unicode objects. If you find anything that doesnt work, it will have to be fixed for Zope 2.4, and it shouldnt be too hard to back-port that fix into 2.3.x. Having said that, Python's garbage collector *needs* support from container objects. Acquisition wrappers dont implement this support (yet), and therefore I suspect a Python upgrade will not help you with this memory leak problem. Toby Dickenson tdickenson@geminidataloggers.com
participants (4)
-
Chris McDonough -
cs＠comlounge.net -
richard＠bizarsoftware.com.au -
Toby Dickenson