[Zope] Zope Eating Memory for Breakfast

[Monty Taylor]

This is a multi-part message in MIME format.
--------------E60D40CC69627D2B6F979135
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Hung Jung Lu wrote:
> 
> --- In zope@egroups.com, Tony McDonald <tony.mcdonald@n...>  wrote:
> > >This is on Solaris 2.7 with Zope 2.1.4. I can provide any other info you
> > >like, but I'm not sure what would be useful to know.
> > >   PID USERNAME THR PRI NICE  SIZE   RES STATE   TIME    CPU COMMAND
> > >  5866 zope       8  10    0 2262M 2034M cpu2  149:09 24.70% python
> 
> Wow wow wow, sometimes my Zope eats up to 160 MB of RAM memory,
> and I thought that was bad. :)
> 
> There's gotta be a memory leak somewhere in Zope. And we'd
> better figure out where it is. I have already seen several
> messages complaining about Zope's memory usage.
> 
> -------------------------------------------------------------
> 
> OK, let us try to narrow down a bit.
> 
> (1) Version: I also use Zope 2.1.4. System is Linux.

      Zope version: Zope 2.1.4 (source release, python 1.5.2, linux2) 
      Python version: 1.5.2 (#1, Feb 14 2000, 18:27:27) [GCC 2.95.1
19990816 (release)] 
      System Platform: sunos5 
      Process ID: 2977 (5) 
      Running for: 7 hours 26 min 7 sec 

   That is Solaris, not linux. It was the source dist. Strange.
   PS. It's up to 28Meg now.

> (2) Webserver: ZServer with multiple threads.

      Apache. I can't remember which version...

> (3) Products: I use SybaseDAv2 for database adapter. I also
> use a bit of Gadfly (through SQLSession), this of course is
> a temporary feature because eventually I want to get rid of
> Gadfly.

      ZMySQLDA-1-1-3
      ZOracleDA
      PTK
      Confera
      ZPhotoAlbum

    But really I'm not using the DA's or PTK right now. They're
installed, but the code that uses them is still in test.

 
> (4) ZODB: I don't have anything that dynamically changes Zope's
> ZODB, that is, I don't upload files into ZODB (Do you?), I
> don't create more Zope users on the fly. In fact, I believe my
> site can run from a CD-Rom, if necessary.

      My site changes a bit more than that, but it's still fairly
static. My web-content manager may post a new org-chart or internal news
story, but we don't even have that many people connecting to read yet.

> (5) Size of ZODB: my Data.fs is about 9 MB.

     That would be 525Meg. My god, that's large. But I think that's
mainly pictures. 

> (6) Uptime pattern: the maximum time I have run Zope is less
> than a week. Usually I restart it a few times a week.

No clue, as I restart it every couple of days to get the size of that db
down.

> (7) Memory growth pattern: my Zope starts out with 7 MB in each
> of the 4 threads. It starts to climb up slowly, by the end of
> the day it reaches 18 MB per thread. After 3 days it climbs up
> to 38 MB. That's about 160 MB total. (No, I don't think the
> threads are sharing memory, because the Linux "top" command
> also tells me the percentage of memory usage.)

When I restart it, it starts out at about 18Meg. Then it gradually
grows, and I eventually notice that it's at 258M. I'll keep a better eye
on that.

> (8) Performance: I have severe problem with SQLSession and Gadfly,
> and also some problem with SybaseDAv2. I think the performance
> problem is related to the RAM memory consumption, because when
> I clean up Gadfly (deleting non-used table records), the Gadfly
> performance increased substantially.

     The only performance problems I've noticed are once the memory
consuption starts putting the box into swapping. 

> (9) Database Management cache parameters: (from Zope's control panel)
> 
>     Total number of objects in the database 4285
>     Total number of objects in all of the caches combined 1533
>     Target size 400
>     Target maximum time between accesses 60
> 
>     Flush Cache: Full Sweep and Minimize don't help to reduce the
>     memory consumption.

      Total number of objects in the database                11057
      Total number of objects in all of the caches combined       1502
      Target size 400
      Target maximum time between accesses 20

I also have Zope on another Solaris Box (which is considerably smaller)
and on a Linux machine. Neither present this problem. The other Solaris
box is our test machine, and no code goes on the Production Box that
hasn't gone on the test machine, so I'm even more confused. One of our
first thoughts concerned the compiler on the big boy, as it was put
there before we got here (old admins) but the same one would go on the
test as the
prod.                                                                       

Anyone else?

Monty
--------------E60D40CC69627D2B6F979135
Content-Type: text/x-vcard; charset=us-ascii;
 name="mtaylor.vcf"
Content-Transfer-Encoding: 7bit
Content-Description: Card for Monty Taylor
Content-Disposition: attachment;
 filename="mtaylor.vcf"

begin:vcard 
n:Taylor;Monty
tel;cell:+31 (0)6 200 17486
tel;fax:+31 (0)20 589 5566
tel;work:+31 (0)20 589 5517
x-mozilla-html:TRUE
url:http://www.goldridge.net
org:Information Innovation
adr:;;Amstelveenseweg 88-90;Amsterdam;;1075 XJ;The Netherlands
version:2.1
email;internet:mtaylor@goldridge.net
title:Information Artist
x-mozilla-cpt:;0
fn:Monty Taylor
end:vcard

--------------E60D40CC69627D2B6F979135--