[Zope] Preventing Zope Spinning?

[Brian Brinegar]

Jonathan,

We have debated this, however the time necessary to revalidate the cache
can be several minutes (maybe more) at startup for a 2 gig cache. If
only one client was going down at a time this would not be a problem,
however in the event that someone clicks reload until all of the clients
are dead it would cause complete down time while the clients startup. We
made the decision that having the server be slow was better than having
it be completely unavailable for an extended period of time.

I would like to prevent the clients from ever needing to be restarted.

Brian Brinegar
Web Systems Developer
Engineering Computer Network

Jonathan wrote:
> How about setting up persistent caches for your zeo clients, so that
> when the zeo server goes down you save some cache-reloading time when it
> restarts?
> 
> 
> Jonathan
> 
> 
> ----- Original Message ----- From: "Brian Brinegar"
> <brinegar at ecn.purdue.edu>
> To: <zope at zope.org>
> Cc: "David S Carmichael" <carmicha at ecn.purdue.edu>; "Andrew T Sydelko"
> <sydelko at ecn.purdue.edu>; "Christopher N Deckard" <cnd at ecn.purdue.edu>
> Sent: Thursday, May 25, 2006 10:49 AM
> Subject: [Zope] Preventing Zope Spinning?
> 
> 
>> The majority of the Purdue University Engineering web presence is
>> provided via a cluster running ZEO. We offer hosting for every school,
>> department, faculty, staff, and student in the College. Because of this
>> we have a large number of content maintainers/developers on our system.
>> We are running into problems with users writing bad code which spins or
>> uploading huge files which seems to tie up the database for long periods
>> of time.
>>
>> We end up in a situation where something will spin a client and a user
>> will repeatedly resubmit the request until all of our zeo clients are
>> spinning. Our clients are eventually killed, usually manually, and
>> quickly come back up. We drop the zeo cache on a restart to improve
>> startup speed, so when the clients do come back up we have 100% cache
>> misses and the zeo server gets pounded resulting in slow performance
>> until the client caches repopulate.
>>
>> Occasionally we can track down the offending URL and correct the
>> problem, sometimes we cannot.
>>
>> Perhaps these issues will be addressed in future versions of Zope, we
>> are currently running Zope 2.6.4
>>
>> What I would like is some sort of timeout for requests, however I do not
>> want to punish users with slow connections. Perhaps a way to kill off a
>> specific request that is consuming excessive resources, without killing
>> the entire client.
>>
>> Below is some information on our setup:
>>
>> 1 zeo server (Solaris)
>>  - 82 gig datafs
>>  - transaction time out of 120 seconds
>>
>> 2 load balanced zeoclients (Linux)
>>  - 2 gig zeo cache
>>  - Database Cache 30000 objects
>>  - 4 threads
>>
>> 2 failover apaches (Linux)
>>  - using pydirector for load balancing
>>
>> We are receiving appoximately 1 million hits per day, which from what
>> I've read is not all that much. We probably have a higher number of DB
>> writes than usual because of the number of developers/maintainers. Can
>> anyone make suggestions for providing a more stable environment?
>>
>> Thank you,
>> Brian Brinegar
>> Web Systems Developer
>> Engineering Computer Network
>> _______________________________________________
>> Zope maillist  -  Zope at zope.org
>> http://mail.zope.org/mailman/listinfo/zope
>> **   No cross posts or HTML encoding!  **
>> (Related lists -
>> http://mail.zope.org/mailman/listinfo/zope-announce
>> http://mail.zope.org/mailman/listinfo/zope-dev )
>>