Zope 2.7 memory behaviour
Hello, I´m trying to find a memory leak in a zope appliction and started to test how zope memory consume behaves basically under load. The current result is: Zope constantly consumes more memory. The testcase is: 12 http clients request constantly request the same page template (it does not contain any tal statements and only 10 lines of HTML) in a simple folder. The increase is about 50 - 100 kb / minute. I´m using Windows 2000 as server platform. The database cache and debug information on the control panel seem allright. Now I wonder if there´s basically a problem with Zope 2.7? Has anyone made the same experience? Or an explanation why this happens? Thanks, Arndt.
Arndt Droullier wrote:
Hello,
Hello Arndt
I'm trying to find a memory leak in a zope appliction and started to test how zope memory consume behaves basically under load.
I'm having exactly the same problem, but under linux using Python 2.3.3 and Zope 2.7.0. First time I noticed this behaving was with Zope 2.6.4 using Python 2.1.3. Than I recreated our internal zope package just updating Python from 2.1.3 to Python 2.3.3 but the problem remained.
The current result is: Zope constantly consumes more memory.
Jup, exactly, the memory consumption goes up and up and up till the machine starts swapping and gets deadly slow.
The testcase is: 12 http clients request constantly request the same page template (it does not contain any tal statements and only 10 lines of HTML) in a simple folder. The increase is about 50 - 100 kb / minute. I´m using Windows 2000 as server platform.
My test case is quiet similar. I wrote a Perl script which requests continuously the same page template (which contains some basic DTML stuff).
The database cache and debug information on the control panel seem allright.
The debug information looks rather strange to me, but I don't know anything about internals of Zope. Anyway I have here reference counts going up fast. Don't know whether this is normal or not. Class March 22, 2004 5:26 pm March 22, 2004 5:38 pm Delta Acquisition.ImplicitAcquirerWrapper 850 18136 +17286 ZPublisher.HTTPRequest.HTTPRequest 145 3026 +2881 ZServer.HTTPResponse.ZServerHTTPResponse 139 3019 +2880 ZPublisher.BaseRequest.RequestContainer 135 3015 +2880 DateTime.DateTime.DateTime 51 52 +1 I'll let you know, should I find some solution (apart from restarting :-)).
Now I wonder if there´s basically a problem with Zope 2.7? Has anyone made the same experience? Or an explanation why this happens?
Thanks, Arndt.
Have nice day. Stef
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
-- ------------------------------------------------------------- addr://Rathausgasse 31, CH-5001 Aarau fon://++41 62 823 9355 http://www.terreactive.com fax://++41 62 823 9356 ------------------------------------------------------------- terreActive AG Wir sichern Ihren Erfolg -------------------------------------------------------------
Stefan Doerig wrote at 2004-3-22 17:46 +0100:
... The debug information looks rather strange to me, but I don't know anything about internals of Zope. Anyway I have here reference counts going up fast. Don't know whether this is normal or not.
Class March 22, 2004 5:26 pm March 22, 2004 5:38 pm Delta Acquisition.ImplicitAcquirerWrapper 850 18136 +17286 ZPublisher.HTTPRequest.HTTPRequest 145 3026 +2881 ZServer.HTTPResponse.ZServerHTTPResponse 139 3019 +2880 ZPublisher.BaseRequest.RequestContainer 135 3015 +2880 DateTime.DateTime.DateTime 51 52 +1
Apparently, your application is leaking "HTTPRequest" instances. Are you storing acquisition wrapped objects in the REQUEST object? Do not do this! It forms cyclic garbarge that cannot be recycled by Python's cyclic garbage collector because the special Zope classes have not yet been made compatible with GC (this comes with Zope 2.8). -- Dieter
Dieter Maurer wrote:
Apparently, your application is leaking "HTTPRequest" instances.
Are you storing acquisition wrapped objects in the REQUEST object?
No, I don't think so, but I might be wrong. I mail you the test template I'm accessing. ### BEGIN mon-clear <dtml-var standard_html_header> <dtml-var REQUEST> <dtml-var standard_html_footer> ### END mon-clear ### BEGIN standard_html_header <html> <head> <title></title> </head> <body> standard_html_header ### END standard_html_header ### BEGIN standard_html_footer standard_html_footer </body> </html> ### END standard_html_footer That's all. The reference count looks like this after a while Class March 23, 2004 9:46 am March 23, 2004 10:24 am Delta Acquisition.ImplicitAcquirerWrapper 68 8793 +8725 ZPublisher.HTTPRequest.HTTPRequest 17 2198 +2181 ZServer.HTTPResponse.ZServerHTTPResponse 10 2191 +2181 ZPublisher.BaseRequest.RequestContainer 7 2188 +2181 BTW: Flushing the cache does not decrease the reference count. Stef -- ------------------------------------------------------------- addr://Rathausgasse 31, CH-5001 Aarau fon://++41 62 823 9355 http://www.terreactive.com fax://++41 62 823 9356 ------------------------------------------------------------- terreActive AG Wir sichern Ihren Erfolg -------------------------------------------------------------
Stefan Doerig wrote at 2004-3-23 10:31 +0100:
... No, I don't think so, but I might be wrong. I mail you the test template I'm accessing.
### BEGIN mon-clear <dtml-var standard_html_header> <dtml-var REQUEST> <dtml-var standard_html_footer> ### END mon-clear
### BEGIN standard_html_header <html> <head> <title></title> </head> <body> standard_html_header ### END standard_html_header
### BEGIN standard_html_footer standard_html_footer </body> </html> ### END standard_html_footer
That's all. The reference count looks like this after a while
Your test looks quite innocent. I tried to reproduce it (about 5.000 "ab" requests against something similar to your "mon-clear") and could not see any memory leak. Especially, no HTTPRequest or ImplicitAcquirerWrapper were leaked. However, you may have an "AccessRule" (or some other "__before_publishing_traverse__" or "__bobo_traverse__" hook) that leaks requests. -- Dieter
Dieter Maurer wrote:
Your test looks quite innocent.
I tried to reproduce it (about 5.000 "ab" requests against something similar to your "mon-clear") and could not see any memory leak. Especially, no HTTPRequest or ImplicitAcquirerWrapper were leaked.
However, you may have an "AccessRule" (or some other "__before_publishing_traverse__" or "__bobo_traverse__" hook) that leaks requests.
Thanks for the good hint, I could reproduce it by accessing my template and if I revoke the permission (on root level) View from the Role 'Anonymous' and grant it to 'Authenticated' then it leaks the objects - Acquisition.ImplicitAcquirerWrapper - ZPublisher.HTTPRequest.HTTPRequest - ZServer.HTTPResponse.ZServerHTTPResponse - ZPublisher.BaseRequest.RequestContainer When I granted 'View' to 'Anonymous' again leaking stopped. Am I using this facility wrongly? Stef -- ------------------------------------------------------------- addr://Rathausgasse 31, CH-5001 Aarau fon://++41 62 823 9355 http://www.terreactive.com fax://++41 62 823 9356 ------------------------------------------------------------- terreActive AG Wir sichern Ihren Erfolg -------------------------------------------------------------
Stefan Doerig wrote at 2004-3-25 11:10 +0100:
... Thanks for the good hint, I could reproduce it by accessing my template and if I revoke the permission (on root level) View from the Role 'Anonymous' and grant it to 'Authenticated' then it leaks the objects - Acquisition.ImplicitAcquirerWrapper - ZPublisher.HTTPRequest.HTTPRequest - ZServer.HTTPResponse.ZServerHTTPResponse - ZPublisher.BaseRequest.RequestContainer
When I granted 'View' to 'Anonymous' again leaking stopped. Am I using this facility wrongly?
I tried to reproduce it, but have not succeeded. Do you provide authentication information with your requests? I did not (such that each request resulted in a 401 response). -- Dieter
Dieter Maurer wrote:
I tried to reproduce it, but have not succeeded.
Do you provide authentication information with your requests? I did not (such that each request resulted in a 401 response).
Yes, I did provide authentication information with each of my requests, so each request resulted in a 200 response. Sorry for not mentioning that. Stef -- ------------------------------------------------------------- addr://Rathausgasse 31, CH-5001 Aarau fon://++41 62 823 9355 http://www.terreactive.com fax://++41 62 823 9356 ------------------------------------------------------------- terreActive AG Wir sichern Ihren Erfolg -------------------------------------------------------------
Stefan Doerig wrote at 2004-3-29 09:52 +0200:
Dieter Maurer wrote:
I tried to reproduce it, but have not succeeded.
Do you provide authentication information with your requests? I did not (such that each request resulted in a 401 response).
Yes, I did provide authentication information with each of my requests, so each request resulted in a 200 response. Sorry for not mentioning that.
I tried and lost a single "HTTPRequest" instance (and a few related objects). Still nothing that looks like a serious memory leak... -- Dieter
Your debug information looks like severel objects referenced by acquisition remain in memory. You can be sure this is the case if you in the management interface to Control Panel/Database Management/main/Flush Cache and remove all objects from cache. If the reference count doesn´t change then some objects are leaking. The reference count is the number in the table, or in other words you have 18136 Acquisition.ImplicitAcquirerWrapper objects in memory. This value should normally be much less after flushing the cache. BTW, the effect I described is no real problem. After some time the memory usage gets stable. Greetings, Arndt.
The debug information looks rather strange to me, but I don't know anything about internals of Zope. Anyway I have here reference counts going up fast. Don't know whether this is normal or not.
Class March 22, 2004 5:26 pm March 22, 2004 5:38 pm Delta Acquisition.ImplicitAcquirerWrapper 850 18136 +17286 ZPublisher.HTTPRequest.HTTPRequest 145 3026 +2881 ZServer.HTTPResponse.ZServerHTTPResponse 139 3019 +2880 ZPublisher.BaseRequest.RequestContainer 135 3015 +2880 DateTime.DateTime.DateTime 51 52 +1
I'll let you know, should I find some solution (apart from restarting :-)).
____________ Virus checked by G DATA AntiVirusKit Version: AVK 14.0.298 from 27.01.2004 Virus news: www.antiviruslab.com
participants (3)
-
Arndt Droullier -
Dieter Maurer -
Stefan Doerig