[Zope] Zope Eating Memory for Breakfast

Bill Anderson bill@libc.org
Wed, 12 Apr 2000 16:49:16 -0600


Hung Jung Lu wrote:


Well, I am *not* having memory problems, but since my setup may be
useful in that regard:

 
> --- In zope@egroups.com, Tony McDonald <tony.mcdonald@n...>  wrote:
> > >This is on Solaris 2.7 with Zope 2.1.4. I can provide any other info you
> > >like, but I'm not sure what would be useful to know.
> > >   PID USERNAME THR PRI NICE  SIZE   RES STATE   TIME    CPU COMMAND
> > >  5866 zope       8  10    0 2262M 2034M cpu2  149:09 24.70% python
> 
> Wow wow wow, sometimes my Zope eats up to 160 MB of RAM memory,
> and I thought that was bad. :)
> 
> There's gotta be a memory leak somewhere in Zope. And we'd
> better figure out where it is. I have already seen several
> messages complaining about Zope's memory usage.
> 
> -------------------------------------------------------------
> 
> OK, let us try to narrow down a bit.
> 
> (1) Version: I also use Zope 2.1.4. System is Linux.


About 7 currently running Zopes at the moment. Ranging from 2.1.3 to
2.1.6, with the majority being 2.1.4

> (2) Webserver: ZServer with multiple threads.

ZServer, 4-6 threads, running behind Roxen (-1 on it's own)
 
> (3) Products: I use SybaseDAv2 for database adapter. I also
> use a bit of Gadfly (through SQLSession), this of course is
> a temporary feature because eventually I want to get rid of
> Gadfly.
> 
> (4) ZODB: I don't have anything that dynamically changes Zope's
> ZODB, that is, I don't upload files into ZODB (Do you?), I
> don't create more Zope users on the fly. In fact, I believe my
> site can run from a CD-Rom, if necessary.


I am using Sybase DA, PTK on four, and BannerClasses which modifies the
Data.fs _every_ page view.
 
> (5) Size of ZODB: my Data.fs is about 9 MB.

Sizes:
 from <1M to 50M at a given time.

 
> (6) Uptime pattern: the maximum time I have run Zope is less
> than a week. Usually I restart it a few times a week.

Mang up for weeks at atime, obvious exception would be the two test
servers.
 
> (7) Memory growth pattern: my Zope starts out with 7 MB in each
> of the 4 threads. It starts to climb up slowly, by the end of
> the day it reaches 18 MB per thread. After 3 days it climbs up
> to 38 MB. That's about 160 MB total. (No, I don't think the
> threads are sharing memory, because the Linux "top" command
> also tells me the percentage of memory usage.)

The Linux 'Top' command is a tricky beast.
How much memory is _in_use_? Can't tell that with Top (without a
callculator anyway ;).
Use 'free', it will report the amount currently in use, the amount in
buffer, cache, etc.
...and do the math for you.

If I were to count the memory for _each_ thread, it would allegedly be
on the order of 200-300MB easy. In reality, it isn't using anywhere near
that. The memory statistics per thread in Linux _is_ a 'shared' memory
amongst the threads (though not in the programmatic definition).
 

> (9) Database Management cache parameters: (from Zope's control panel)
> 
>     Total number of objects in the database 4285
>     Total number of objects in all of the caches combined 1533
>     Target size 400
>     Target maximum time between accesses 60
> 
>     Flush Cache: Full Sweep and Minimize don't help to reduce the
>     memory consumption.

Average object cache target: 	5000
Number in cache (average):	4000
Average max time: 		180 seconds.


Hope it helps in some way.
Bill

-- 
In flying I have learned that carelessness and overconfidence are 
usually far more dangerous than deliberately accepted risks. 
          -- Wilbur Wright in a letter to his father, September 1900