[Zope] Problems with restarts, memory usage, DB connections, FastCGI ?? Help!

Thu, 24 Oct 2002 11:55:32 -0400

Hello,

I'm having some problems with my main Zope installation that I'm hoping
someone could shed some light on.  My nowledge of Zope internals is too
limited for me to figure it out by myself right now :)

I have a Zope 2.5.0 (With various products) , running on a custom (But
default build) Python 2.1.3 on a RedHat 7.3 ... The machine has a 1GB =
Ram. I
use Apache+FastCGI+Squid.

The symptoms of the problem are as such:

Every once in a while, zope stops responding for a few minutes ... =
Sometimes
it comes back on it's own, sometimes it takes too long and I restart it
manually.

So far I've been unable to track down WHY.  One thing I finally =
noticed, is
that when Zope seems "hung", if I restart Apache, the number of  =
"httpd"
processes starts normal, but then keep growing steadily.  Once I =
restart
zope, things settle down.  So it would seem that the Apache<->Zope
connection is suffering of "blocking" one some level. So now I'm =
wondering
why this locking might happen.  First there's these 4 database =
connections.
What happens if these stay "full" for too long ? Could THAT be it ? I =
also
have an application inside Zope that has a user upload LARGE images =
(severa
megs at least , up to 12 or 13 megs) through Zope.   I f I understand =
things
correctly, the entire image is loaded into memory before being written =
to
disk.  Does this activity lock a database ? Could heavy usage of this
process be growing memory usage beyond anything reasonable ?

One thing that also happened just once was that the entire server ended =
up
running out of memory! Last week, I got kernel messages about the =
processes
being out of memory.  I couldn't even log-in to the console during this
time. After a while though the kernel was good enough to kill the =
processes,
but I had hit ctrl-alt-del, and then it went on to reboot before I had =
time
to look at anything :(

This site takes millions of hits per week (Squid took 3.75M last week, =
with
a 40% cache hit ratio).  Should I look at using more than 4 DB =
connections ?

I might also be open to using an entirely different setup, such as
Apache+ZServer with mod_rewrite and mod_proxy ... This setup would be
somewhat simpler and lighter ... would it also be more reliable ?

Any insight, opinions, coments, or tips would be greatly appreciated !

Thanks,

Jean-Fran=E7ois Doyon
Internet Service Development and Systems Support
GeoAccess Division
Canada Center for Remote Sensing
Natural Resources Canada
http://atlas.gc.ca
Phone: (613) 992-4902
Fax: (613) 947-2410