[Zope-dev] How to make Zope fail nicely under high load?

[Jamie Heilman]

Bjorn Stabell wrote:
> Basically, when the load gets high, Zope has a huge backload of work
> (several minutes of requests), making the average latency for each
> request many minutes.  What are effective ways to do this kind of
> overload management so that the backlog of work doesn't get that big?

Make the app faster so that doesn't happen.

> The ideal would be for requests to fail immediately if the backlog of
> work is more than a certain number of requests (or even better,
> estimated time to process).

Kinda...

> It appears the way to control it would for Apache or Zope to return "503
> Service Unavailable" when the load is too high, but we haven't found a
> good way to do this; Zope doesn't appear to have any mechanism for it,
> and Apache's ProxyPass doesn't either.  I guess load balancers would,
> but that's a bit overkill since we run the server on one machine.

...return 503 to every new request probably isn't a good idea, 503 to
every new session is probably OK.  Maybe.  It depends on the workload.
If you have a user who sends 10 requests and 5 of them fail, they're
going to keep hitting reload to try to get them all to work, which
just compounds your problems.  The idea is that you want tweak it so
some of your users get everything, and some get nothing, instead of
everybody getting partials.  This means you have to track sessions
though... thats probably easier to do in ZServer than it is in Apache
1.3 just because of the process model, but that doesn't mean its easy.

By default, Apache doesn't track sessions and do that kind of planned
failure.  There are probably modules that can enable that behavior.

ZServer's multiplexed IO model just hides the accepting socket from
the poller if its concurrency level is reached, and yeah, then the
incoming connections end up in the backlog.  You'd have to change
ZServer so it handled those new requests instead, identified if they
were part of a session, queued them up if they were, or spat out a 503
if they weren't.

Frankly, I bet you'd have more fun just making your app faster.

-- 
Jamie Heilman                     http://audible.transient.net/~jamie/
"Paranoia is a disease unto itself, and may I add, the person standing
 next to you may not be who they appear to be, so take precaution."
						-Sathington Willoughby