[Zope] ZServer stops responding !? Help !?

[Chris McDonough]

On Sat, 2004-04-24 at 13:53, Jean-Francois.Doyon at CCRS.NRCan.gc.ca wrote:
> G'day,
> 
> I've got a rather bizarre but catastrophic problem.
> 
> ZServer seems to stop responding.  Sometimes it does so after days of
> running, sometimes after a few seconds or minutes of uptime.
> 
> I know it's ZServer because I can talk to the monitoring port without
> problem.

That may be a bit of flawed logic, because ZServer also runs the monitor
port.

> Also, the apache processes just pile up up to the limit allowed,
> suggesting the proxying is not getting replies from the downstream
> server.
> 
> The strange thing is the cause seems to be occacional, or vary.  For
> hours on end I can sit there and restart it, and within minutes it stops
> responding ... Then suddenly the problem "disappears", I restart it, and
> I wait .... and nothing happens, it just keeps running.  Nothing else
> abnormal is going on the server so far as I can tell, there is very
> little memory swapped, and the CPU usage is not abnormally high.

It sounds as if Zope is doing something which blocks, consuming all
database threads.

> I used to have this problem very very rarely in the past, but since I
> upgraded to Zope 2.7, it seems to have gotten much worse :(
> 
> I tried accesing the DebugPanel from the monitor, but can't seem to get
> it to do anything useful ... I don't know where else to look to find the
> cause of this.
> 
> This causes serious uptime problem on our main, high traffic site, which
> is Very Bad.
> 
> I'm on RedHat 7.3 (fully patched)
> Python 2.3.3 (custom compiled)
> Zope 2.7
> CMF 1.4.x (I forget ... the latest!)
> Psycopg (Latest also)
> And a variety of other products.

I'd suggest using the "big M" or "trace" logging features along with the
requestprofiler script to find out where the problem might be.

- C