Re: [Zope-dev] Memory 2.1.4-2.1.6 a.k.a. how to get objects out of the cache
All, I also am having severe problems with memory creep. Our problem exhibits itself by quickly using gobs of memory and requiring a zope restart after about a day and a half at about 100M resident. First our configuration: Core Components: FreeBSD 4.0/FreeBSD 3.4 - on separate machines of course Postgres 6.5.3/Postgres 7.0(fixes some memory leaks) Python 1.5.2 Zope 2.1.4/2.1.6 have tried both pcgi with ZServer Apache w/mod_ssl 1.3.9 and 1.3.12 Products: ZPySQLDA SQLSession GenericUserFolder I have looked at a lot of different things. This includes: Installing the gc(garbage collection) python package - Neil Schemenauer's patches Installing Postgres 7.0(fixes some connection memory leaks) Tracking the cache cleanup inside zope Tracking thread locking inside DB.py, Transaction.py Results: gc python didn't help alleviate the problem Postgres 7.0 instead of 6.5.3 didn't help the problem I do see GenericUserFolder and SQLSession objects with the Control_Panel_Debug screen, and they do not seem to go away. I wrote a simple python script to do nothing more than authenticate (log in) using the GenericUserFolder method docLogin. The memory usage quickly grows out of control. After waiting 15 minutes(my cookie timeout), no decrease in memory usage. The objects are still in the cache also. Restarting zope clears everything up, and starts out nice and clean again. We have a cron job doing this right now. Questions: 1. How does a user's connection resources etc. get cleaned up by Zope after a cookie timeout? Do I need to do this myself? 2. How does a genericuserfolder's set of objects used by an authenticated user get cleaned up. I can't seem to make those objects get reclaimed by the system, even when the user logs off? 3. I am not explicitly removing SQLSession objects. Will my usage counts for these objects remain > 1, thereby never allowing them to get cleaned up, and thereby keeping my usage counts for genericuserfolder's > 1? 4. Since GenericUserFolder inherits from Folder objects, is it possible that the reason my memory grows so fast is that Zope does not release the resources used by a user properly, during an abrupt disconnect? I will assist/help trying different things to try to solve this problem! thanks. eric.
Eric Sattler wrote:
All,
I also am having severe problems with memory creep. Our problem exhibits itself by quickly using gobs of memory and requiring a zope restart after about a day and a half at about 100M resident.
[...]
I do see GenericUserFolder and SQLSession objects with the Control_Panel_Debug screen, and they do not seem to go away. I wrote a simple python script to do nothing more than authenticate (log in) using the GenericUserFolder method docLogin. The memory usage quickly grows out of control. After waiting 15 minutes(my cookie timeout), no decrease in memory usage. The objects are still in the cache also.
Restarting zope clears everything up, and starts out nice and clean again. We have a cron job doing this right now.
This seems to dovetail with the reports of SQLSession being a memory leak source. Could you drop the SQLSession, and repeat the test?
On Sat, 27 May 2000, Eric Sattler wrote:
I do see GenericUserFolder and SQLSession objects with the Control_Panel_Debug screen, and they do not seem to go away. I wrote a simple python script to do nothing more than authenticate (log in) using the GenericUserFolder method docLogin. The memory usage quickly grows out of control. After waiting 15 minutes(my cookie timeout), no decrease in memory usage. The objects are still in the cache also.
Phillip J. Eby identified a memory leak in GenericUserFolder over the weekend. I'm just downloading LoginManager now to see if I can steal their fix :-) (Oh... thats easy. Just search for 'Waaaaaaa!') There is also a good chance that this is also causing the SQLSession leak - GUF is maintaining a reference to REQUEST (in a nice circular way causing the memory leak), and REQUEST maintains a reference to the SESSION, so the SESSION won't be freed. I should have a patch available shortly. I would appreciate people who know how to drive the debuggers better than I confirming that the leak is gone. /me hops on the 'real garbage collection for Python' bandwagon -- Stuart Bishop Work: zen@cs.rmit.edu.au Senior Systems Alchemist Play: zen@shangri-la.dropbear.id.au Computer Science, RMIT University
Stuart 'Zen' Bishop wrote:
/me hops on the 'real garbage collection for Python' bandwagon
Have you looked at this? http://www.enme.ucalgary.ca/~nascheme/python/gc.html I'd like to know how successful this project is. I wonder whether it would have an issue with ExtensionClass instances (one of the foundations of Zope). Shane
Shane Hathaway wrote:
Stuart 'Zen' Bishop wrote:
/me hops on the 'real garbage collection for Python' bandwagon
Have you looked at this?
Yes, I actually built, installed and enabled it. It doesn't seem to have a major impact on my problems at least...neither does Zen's latest set of fixes. I enabled Neil Schemenauer's fixes, rebuilt python, and then tried tracking through chunks of the CPickleCache code. There did not seem to be any hanging python references. I believe the problem to be circular zope references on a higher level. I could be wrong though. The gc code for cPickleCache.c is not trivial, and I believe there is something wrong there, but have not proved it yet. This is the code that determines when things are removed from the zope cache. The wierd thing is that in my case, I am not really writing anything to the Zope db. Everything is stored externally in a postgres database. I am using GUF, and SQLSession though. Could it be that connection objects are not being cleaned up properly? As far as I understand it, connection objects are containers for zope objects for the duration of that request. At least in the case of postgres 6.5.3 there was some connection memory leaks, so I did try postgres 7.0, it solved some of postgres's problem but not zope's. sorry for rambling, just trying to get all of the tidbits that I know out there, maybe someone will see something. eric.
http://www.enme.ucalgary.ca/~nascheme/python/gc.html
I'd like to know how successful this project is. I wonder whether it would have an issue with ExtensionClass instances (one of the foundations of Zope).
CPickleCache
Shane
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
participants (4)
-
Bill Anderson -
Eric Sattler -
Shane Hathaway -
Stuart 'Zen' Bishop