Charlie, Thanks for pointing this out! I'm not sure though ... Now that you mention it though, when I look in the Debug page, the one object type that has a delta WAY above the others is the DateTime.DateTime.DateTime object ... I'll try to keep a look at the toprefcount to see if it grows abnormally ... And I will keep an eye out for the Acquisition.ImplicitAcquirerWrapper counts and delta's to see if there's anything weird going on there ... Heh, in the time I've been typing this, the Delta for the DateTime is 9014 !!! The top refcount for it is 44618 now ... I allways thought these high numbers were to be expected, since something like DateTime gets used a lot I presume. The top refcount for Acquisition.ImplicitAcquirerWrapper is 56 ... I'm starting to look at upgrading to 2.6, but we're not there just yet, I have to learn a couple more things about writing your own products and CMF portal types first :) Thanks for the tips! J.F. -----Original Message----- From: Charlie Reiman [mailto:creiman@kefta.com] Sent: Monday, November 25, 2002 2:05 PM To: Jean-Francois.Doyon@CCRS.NRCan.gc.ca; zope@zope.org Subject: RE: [Zope] My Zope is leaking memory ...
-----Original Message----- From: zope-admin@zope.org [mailto:zope-admin@zope.org]On Behalf Of Jean-Francois.Doyon@CCRS.NRCan.gc.ca Sent: Monday, November 25, 2002 10:57 AM To: zope@zope.org Subject: [Zope] My Zope is leaking memory ...
Hello,
Yup, that's right, my Zope is leaking memory, and so far, I've had no luck figuring out where it might come from. I run a Zope 2.5.0 on RedHat 7.3.
I started by looking into the Python/C type modules, which I fugred are most likely to be causing this. I upgraded Psycopg to the latest version, but it's still occuring. I also use the PIL, but it is seldom used, and could not be the cause of the kind of leakage I'm seeing. Then there's a much more obscure thing, which is a SWIG interface to PROJ.4 (A geographical projection library) ... I had the author look at it, and no leaks were detected in the C library, or using the SWIG interface (Using Valgrind I beleive). My Z SQL Methods do NOT cache results (Maximum time set 0).
I'm using a custom built (with default configuration) Python 2.1.3 ...
The symptom: I run Zope with -t 6 due to the site being very busy. There's 1GB of RAM. When I first start Zope, it the memory of the process running running the threads is around 10% (as shown in "ps"). Here, now after 4 days 19 hours up:
root 943 0.0 0.1 5024 1456 ? S Nov20 0:00 /usr/local/bin/python2.1 /usr/local/Zope-2.5.0/z2.py -a 132.156.10.87 -t 6 -w 8080 -p - -f - -D nobody 946 0.8 40.4 470040 417420 ? S Nov20 60:26 /usr/local/bin/python2.1 /usr/local/Zope-2.5.0/z2.py -a 132.156.10.87 -t 6 -w 8080 -p - -f - -D nobody 950 0.0 40.4 470040 417420 ? S Nov20 0:03 /usr/local/bin/python2.1 /usr/local/Zope-2.5.0/z2.py -a 132.156.10.87 -t 6 -w 8080 -p - -f - -D nobody 951 3.8 40.4 470040 417420 ? S Nov20 266:02 /usr/local/bin/python2.1 /usr/local/Zope-2.5.0/z2.py -a 132.156.10.87 -t 6 -w 8080 -p - -f - -D nobody 952 3.8 40.4 470040 417420 ? S Nov20 264:41 /usr/local/bin/python2.1 /usr/local/Zope-2.5.0/z2.py -a 132.156.10.87 -t 6 -w 8080 -p - -f - -D nobody 953 3.7 40.4 470040 417420 ? S Nov20 262:58 /usr/local/bin/python2.1 /usr/local/Zope-2.5.0/z2.py -a 132.156.10.87 -t 6 -w 8080 -p - -f - -D nobody 954 3.8 40.4 470040 417420 ? S Nov20 267:13 /usr/local/bin/python2.1 /usr/local/Zope-2.5.0/z2.py -a 132.156.10.87 -t 6 -w 8080 -p - -f - -D nobody 955 4.3 40.4 470040 417420 ? S Nov20 299:51 /usr/local/bin/python2.1 /usr/local/Zope-2.5.0/z2.py -a 132.156.10.87 -t 6 -w 8080 -p - -f - -D nobody 956 3.8 40.4 470040 417420 ? S Nov20 265:31 /usr/local/bin/python2.1 /usr/local/Zope-2.5.0/z2.py -a 132.156.10.87 -t 6 -w 8080 -p - -f - -D
Memory usage is at 40.4 percent ...
In another 4 days, it will have reached almost 60%, and I will restart it ... Basically, after a week, it gets to be too much.
As far as ZODB Caching, here are my numbers right now:
Total objects in DB: 44786 Total objects in all caches: 30290 (This is peak actually, it varies between 26K and 30K depending on time of day). Target size: 5000 Max time: 300
When packed my Data.fs is under 100Mb ...
The memory keeps growing regardless of the caching going on with the database, so that's not the issue.
The site takes millions of this ever week ...
So now I'm running out of ideas to try and track this down, I was hoping it would be the PROJ interface/library ... But apparently not (Even then the leakage would probably be much smaller than what we're seeing here).
Does anyone know how I might go about tracking this down further? I know of the Medusa monitor thing, but I wouldn't know the first thing on how to start using that to track memory usage, if it is even possible.
Right now I'm wondering about the PostgreSQL usage ... It is the only thing that has intensive enough usage to leak this much memory I think ...
Any ideas, comments, insights, tips would be MOST appreciated!
Thanks in advance,
Jean-François Doyon Internet Service Development and Systems Support GeoAccess Division Canada Center for Remote Sensing Natural Resources Canada http://atlas.gc.ca Phone: (613) 992-4902 Fax: (613) 947-2410
Look under Control Panel:Debug. There is a display of current object allocations. You can refresh to see deltas. There a good chance you are exercising a known Zope bug: http://collector.zope.org/Zope/421 This is fixed in 2.6 and there a patch for 2.5.1 on the collector issue.