Jonathan, We have debated this, however the time necessary to revalidate the cache can be several minutes (maybe more) at startup for a 2 gig cache. If only one client was going down at a time this would not be a problem, however in the event that someone clicks reload until all of the clients are dead it would cause complete down time while the clients startup. We made the decision that having the server be slow was better than having it be completely unavailable for an extended period of time. I would like to prevent the clients from ever needing to be restarted. Brian Brinegar Web Systems Developer Engineering Computer Network Jonathan wrote:
How about setting up persistent caches for your zeo clients, so that when the zeo server goes down you save some cache-reloading time when it restarts?
Jonathan
----- Original Message ----- From: "Brian Brinegar" <brinegar@ecn.purdue.edu> To: <zope@zope.org> Cc: "David S Carmichael" <carmicha@ecn.purdue.edu>; "Andrew T Sydelko" <sydelko@ecn.purdue.edu>; "Christopher N Deckard" <cnd@ecn.purdue.edu> Sent: Thursday, May 25, 2006 10:49 AM Subject: [Zope] Preventing Zope Spinning?
The majority of the Purdue University Engineering web presence is provided via a cluster running ZEO. We offer hosting for every school, department, faculty, staff, and student in the College. Because of this we have a large number of content maintainers/developers on our system. We are running into problems with users writing bad code which spins or uploading huge files which seems to tie up the database for long periods of time.
We end up in a situation where something will spin a client and a user will repeatedly resubmit the request until all of our zeo clients are spinning. Our clients are eventually killed, usually manually, and quickly come back up. We drop the zeo cache on a restart to improve startup speed, so when the clients do come back up we have 100% cache misses and the zeo server gets pounded resulting in slow performance until the client caches repopulate.
Occasionally we can track down the offending URL and correct the problem, sometimes we cannot.
Perhaps these issues will be addressed in future versions of Zope, we are currently running Zope 2.6.4
What I would like is some sort of timeout for requests, however I do not want to punish users with slow connections. Perhaps a way to kill off a specific request that is consuming excessive resources, without killing the entire client.
Below is some information on our setup:
1 zeo server (Solaris) - 82 gig datafs - transaction time out of 120 seconds
2 load balanced zeoclients (Linux) - 2 gig zeo cache - Database Cache 30000 objects - 4 threads
2 failover apaches (Linux) - using pydirector for load balancing
We are receiving appoximately 1 million hits per day, which from what I've read is not all that much. We probably have a higher number of DB writes than usual because of the number of developers/maintainers. Can anyone make suggestions for providing a more stable environment?
Thank you, Brian Brinegar Web Systems Developer Engineering Computer Network _______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )