[Zope] Re: Preventing Zope Spinning?

[Tres Seaver]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Brian Brinegar wrote:
> The majority of the Purdue University Engineering web presence is
> provided via a cluster running ZEO. We offer hosting for every school,
> department, faculty, staff, and student in the College. Because of this
> we have a large number of content maintainers/developers on our system.
> We are running into problems with users writing bad code which spins or
> uploading huge files which seems to tie up the database for long periods
> of time.
> 
> We end up in a situation where something will spin a client and a user
> will repeatedly resubmit the request until all of our zeo clients are
> spinning. Our clients are eventually killed, usually manually, and
> quickly come back up. We drop the zeo cache on a restart to improve
> startup speed, so when the clients do come back up we have 100% cache
> misses and the zeo server gets pounded resulting in slow performance
> until the client caches repopulate.
> 
> Occasionally we can track down the offending URL and correct the
> problem, sometimes we cannot.
> 
> Perhaps these issues will be addressed in future versions of Zope, we
> are currently running Zope 2.6.4
> 
> What I would like is some sort of timeout for requests, however I do not
> want to punish users with slow connections. Perhaps a way to kill off a
> specific request that is consuming excessive resources, without killing
> the entire client.
> 
> Below is some information on our setup:
> 
> 1 zeo server (Solaris)
>   - 82 gig datafs
>   - transaction time out of 120 seconds
> 
> 2 load balanced zeoclients (Linux)
>   - 2 gig zeo cache
>   - Database Cache 30000 objects
>   - 4 threads
> 
> 2 failover apaches (Linux)
>   - using pydirector for load balancing
> 
> We are receiving appoximately 1 million hits per day, which from what
> I've read is not all that much. We probably have a higher number of DB
> writes than usual because of the number of developers/maintainers. Can
> anyone make suggestions for providing a more stable environment?

Enabling the "trace log" would help to find the errant requests.  In
Zope 2.6, I think that involved adding '-M <logfilename>' to the z2.py
command line.  There is a zope.conf stanza for it in Zope 2.7 and later.
There is a 'requestprofiler.py' script which is useful for analyzing the
log.

I would strongly recommend upgrading to a more recent version of Zope
and ZODB, as there have been a large number of stability-related fixes
and features added since Zope 2.6 / ZODB 3.1, neither of which is being
maintained at this point.


Tres.
- --
===================================================================
Tres Seaver          +1 202-558-7113          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFEdeIQ+gerLs4ltQ4RAqb3AJ4viBPiuD4kOmxvf9780zhRihKnLgCgu+4m
OpMkS1V+6ZZoOynQDPTwYTI=
=sMfD
-----END PGP SIGNATURE-----