[Zope-dev] How to make Zope fail nicely under high load?

[Casey Duncan]

On Wed, 11 Feb 2004 20:25:14 +0800
"Bjorn Stabell" <bjorn at exoweb.net> wrote:

> Toby wrote:
> [...]
> > Zope's ZServer manages a queue of requests that have been 
> > recieved over http, 
> > but not dispatched to the publisher. This is handled in 
> > PubCore/ZRendezvous.py. I suspect this queue will be holding 
> > your backlog. 
> > 
> > You might get some benefit from capping the length of that queue.
> 
> Thanks Toby.
> 
> It doesn't seem like that queue (the requests list) has a cap at all.
> Have been stresstesting Zope for a few minutes and the queue is up to
> 300+ requests in the backlog queue.  Not sure how to force a 503
> return from within ZRendezvous.handle(), though; that's ZServer medusa
> magic.
> 
> 300 requests at 1 request per 2 seconds yields a backlog of work of 10
> minutes.  This is before we've started optimizing.

What kinds of requests are these? Do they all require a dynamic output?
If not, then you should put better caching in front of Zope, or at a
minimum tweak you caching headers so that some could be served as 304s
for instance.

If they are all dynamic, how dynamic? How different is the page for one
session than another? If much of the page is the same then you might
benefit from an ESI approach where you cache parts of pages and assemble
them in the cache, serving only small pieces from Zope.

I'm a bit surprised that you have not (from what I can see), used the
most common Zope speedup (and often the cheapest overall): Buy more
hardware and use ZEO.

> I wonder if Zope also processes requests that have been closed from
> the client end?  I know Zope will continue processing a request even
> if it times out at least (so you can do long-running requests, like
> packing, through the web).  Many (all?) of these 300 requests would
> have been closed by the time Zope got to process them, and so Zope
> shouldn't bother with them.

Why would they be closed by the client?

-Casey