[Zope] Urgent help needed: Zope falls over under moderate load
Chris McDonough
chrism@zope.com
Tue, 20 Nov 2001 14:48:59 -0500
This might be due to a configuration incompatibility with the Cache
Manager stuff (for example, ZClass methods can't be cached). As a
result, you may need to cache the pages manually by using a reverse
caching proxy and Expires/Last-Modified headers as I described.
I realize that needing to go in and tweak your configuration is not
something you want to have to do while your site is getting hammered,
but there's probably not much of a choice. That said, it's likely
that the emergency will be over with before you get a working
configuration up, which is of course frustrating.
Hopefully, at some point we'll have a pretty and comprehensive set of
step-by-step documents describing specifically how to set Zope up for
high load. In the meantime, the rule of thumb is "always cache as
much as possible"; the corollary is "always test your cache hit rate
before putting something into production". You'll probably want to do
both before you get Slashdotted the next time.
----- Original Message -----
From: "Michael Fraase" <mfraase@farces.com>
To: "'Chris McDonough'" <chrism@zope.com>; <zope@zope.org>
Sent: Tuesday, November 20, 2001 2:22 PM
Subject: RE: [Zope] Urgent help needed: Zope falls over under moderate
load
> They just don't show up in the list of cacheable items.
>
> --
> Michael Fraase
> ARTS & FARCES LLC
> mfraase@farces.com
> www.farces.com
> PGP Fingerprint:
> 3D85 F3F4 9E65 4949 176A 260C CB47 190D C864 9A96
>
> > -----Original Message-----
> > From: Chris McDonough [mailto:chrism@zope.com]
> > Sent: Tuesday, November 20, 2001 12:45 PM
> > To: mfraase@farces.com; zope@zope.org
> > Subject: Re: [Zope] Urgent help needed: Zope falls over under
> > moderate load
> >
> >
> > Are the pages uncacheable because they're restricted by
> > security or for another reason?
> >
> > ----- Original Message -----
> > From: "Michael Fraase" <mfraase@farces.com>
> > To: "'Chris McDonough'" <chrism@zope.com>; <zope@zope.org>
> > Sent: Tuesday, November 20, 2001 1:14 PM
> > Subject: RE: [Zope] Urgent help needed: Zope falls over under
> > moderate load
> >
> >
> > > Thanks Chris,
> > >
> > > I've cached what I can find. The only page that's cacheable is
> > > index_html; the pages being hit are another layer deeper in the
> > > hierarchy. I've also cached all of the page elements (images,
> > header,
> > > footer) I could find.
> > >
> > > --
> > > Michael Fraase
> > > ARTS & FARCES LLC
> > > mfraase@farces.com
> > > www.farces.com
> > > PGP Fingerprint:
> > > 3D85 F3F4 9E65 4949 176A 260C CB47 190D C864 9A96
> > >
> > > > -----Original Message-----
> > > > From: Chris McDonough [mailto:chrism@zope.com]
> > > > Sent: Tuesday, November 20, 2001 12:11 PM
> > > > To: mfraase@farces.com; zope@zope.org
> > > > Subject: Re: [Zope] Urgent help needed: Zope falls over under
> > > > moderate load
> > > >
> > > >
> > > > Your clients maybe *should* laugh at you if you're not doing
any
> > > > caching. ;-)
> > > >
> > > > To cache, you have two options.
> > > >
> > > > Use Zope's RAM Cache Manager to do caching. As a result,
> > the pages
> > > > you choose to cache will be cached in RAM and won't be
> > regenerated
> > > > on each request.
> > > >
> > > > - OR -
> > > >
> > > > Use a caching web server in front of Zope to bear the
> > brunt of the
> > > > load in conjunction with (perhaps) the HTTP Accelerated Cache
> > > > Manager stuff in Zope. In this case, your web server (or
proxy)
> > > > keeps a copy of a pregenerated page and serves it out to
> > requestors
> > > > instead of continually asking Zope to regenerate it.
> > > >
> > > > One of these is going to be necessary for any sort of
> > high-traffic
> > > > site. Zope can't keep up when it needs to dynamically
generate
> > > > content; you need to help it out by doing some kind of
caching.
> > > >
> > > > If you don't or can't use Cache Managers, but you still
> > want to use
> > > > an HTTP caching proxy in front of your Zope site, you'll need
to
> > > > make each of the pages you want to cache has an "Expires"
> > and "Last
> > > > Modified" header. You may do this by using RESPONSE.setHeader
if
> > > > you're not using a cache manager. If you're behind
> > Apache, you can
> > > > use mod_proxy with caching turned on.
> > > >
> > > > See http://www.zope.org/SiteIndex/search?text_content=caching
> > > > for more information.
> > > >
> > > > I've also found the document about mod_perl and headers at
> > > > http://perl.apache.org/guide/correct_headers.html
> > helpful... there
> > > > are lots of other nuggets in that document as well.
> > > >
> > > > HTH,
> > > >
> > > > - C
> > > >
> > > >
> > > >
> > > > ----- Original Message -----
> > > > From: "Michael Fraase" <mfraase@farces.com>
> > > > To: <zope@zope.org>
> > > > Sent: Tuesday, November 20, 2001 12:39 PM
> > > > Subject: [Zope] Urgent help needed: Zope falls over under
> > moderate
> > > > load
> > > >
> > > >
> > > > > So yesterday my site got slashdotted. Zope fell over within
> > minutes.
> > > > A
> > > > > day later it's still falling over within an hour or so.
> > > > >
> > > > > I have 512Kb DSL (the outbound channel is 512Kb; the inbound
> > channel
> > > > is
> > > > > 768Kb). I figured that bandwidth would be saturated or that
> > Windows
> > > > 2000
> > > > > Pro would melt long before Zope would fail, but that has
proven
> > not
> > > > to
> > > > > be the case.
> > > > >
> > > > > Here's the particulars of my installation:
> > > > >
> > > > > Pentium III 850MHz with 768Mb RAM
> > > > > MS Windows 2000 Pro with current patches and service packs
> > > > Zope 2.4.3
> > > > > running as a service Squishdot 1.3.0
> > > > > 512Kb bandwidth
> > > > >
> > > > > Here's the registry start parameters:
> > > > >
> > > > > "C:\PROGRA~1\FARCES~1\bin\python.exe"
> > > > "C:\PROGRA~1\FARCES~1\z2.py" -S
> > > > > -w80 -W9800
> > > > >
> > > > > I don't know what the -S is; -w80 sets the port Zope's web
> > server
> > > > > listens on; and -W9800 sets the WebDAV port.
> > > > >
> > > > > Zope just shuts down and closes port 80 on what seems
> > to be very
> > > > > moderate load. We're talking a peak of less than 60,000
hits, a
> > peak
> > > > of
> > > > > 125Mb bandwidth usage, just over 58,000 page views, and a
peak
> > of
> > > > 4013
> > > > > visitors.
> > > > >
> > > > > What I've been doing is to manually stop the Zope service
and
> > > > restart
> > > > > it, but that's ridiculous. At this rate, clients are going
to
> > laugh
> > > > at
> > > > > me when I suggest using Zope for a web project.
> > > > >
> > > > > Anyone have any ideas? I'm digestified, so please cc me in
any
> > > > response.
> > > > >
> > > > > Thanks kindly.
> > > > >
> > > > > --
> > > > > Michael Fraase
> > > > > ARTS & FARCES LLC
> > > > > mfraase@farces.com
> > > > > www.farces.com
> > > > > PGP Fingerprint:
> > > > > 3D85 F3F4 9E65 4949 176A 260C CB47 190D C864 9A96
> > > > >
> > > > >
> > > > > _______________________________________________
> > > > > Zope maillist - Zope@zope.org
> > > > > http://lists.zope.org/mailman/listinfo/zope
> > > > > ** No cross posts or HTML encoding! **
> > > > > (Related lists -
> > > > http://lists.zope.org/mailman/listinfo/zope-announce
> > > > >
> > > > http://lists.zope.org/mailman/listinfo/zope-dev )
> > > > >
> > > >
> > > >
> > >
> > >
> > > _______________________________________________
> > > Zope maillist - Zope@zope.org
> > > http://lists.zope.org/mailman/listinfo/zope
> > > ** No cross posts or HTML encoding! **
> > > (Related lists -
> > http://lists.zope.org/mailman/listinfo/zope-announce
> > >
> > http://lists.zope.org/mailman/listinfo/zope-dev )
> > >
> >
> >
>
>
> _______________________________________________
> Zope maillist - Zope@zope.org
> http://lists.zope.org/mailman/listinfo/zope
> ** No cross posts or HTML encoding! **
> (Related lists -
> http://lists.zope.org/mailman/listinfo/zope-announce
> http://lists.zope.org/mailman/listinfo/zope-dev )
>