[Zope-dev] How to make Zope fail nicely under high load?

[Bjorn Stabell]

Hi,

We've run into an interesting problem when load-testing a Zope site
behind Apache, a problem that we've also encountered in real life.

Basically, when the load gets high, Zope has a huge backload of work
(several minutes of requests), making the average latency for each
request many minutes.  What are effective ways to do this kind of
overload management so that the backlog of work doesn't get that big?

The ideal would be for requests to fail immediately if the backlog of
work is more than a certain number of requests (or even better,
estimated time to process).


Here's what we've tried:

Naively, we thought we could just set the socket.listen() backlog in
Apache and Zope to a lower value, but in TCP connect()'s apparently
don't fail if the server's listen backlog is full; instead requests are
retried, resulting in a client side managed "listen backlog", also
giving the same long latency.  (If someone knows this stuff, please
confirm/deny these allegations against TCP :)

It appears the way to control it would for Apache or Zope to return "503
Service Unavailable" when the load is too high, but we haven't found a
good way to do this; Zope doesn't appear to have any mechanism for it,
and Apache's ProxyPass doesn't either.  I guess load balancers would,
but that's a bit overkill since we run the server on one machine.


Regards,
-- 
Bjorn Stabell bjorn at exoweb.net
Tel +86 (10) 65918490