[Zope] Zope scalabilty and problems

[Chris McDonough]

What you've got now is a pretty reasonable setup.  Moving to 2.7.7 will
likely not get you any "free" performance increases and it doesn't
appear that there are any "critical" ZODB bugfixes relating to your
problems since 2.7.3.  FWIW, also, I'd venture a guess that moving "off
Data.fs" (as you mentioned in your original email) may be quite costly,
requiring a lot of application recoding.  It also might not help solve
the scaling problem you're having.  It's a "high risk, potentially low
reward" solution.  You need something that is low-risk and potentially
high reward, I'd suspect.

Others have said this, but it bears repeating.  The biggest
"administrative" win you can get for the cheapest cost is to add an HTTP
cache (like Squid or even Apache's mod_proxy) in front of your ZEO
clients and to cause your Zope application to set the proper response
headers which allow the cache server to cache (and thus serve) those
pages on subsequent requests.

This can be tricky when most of your traffic is authenticated because
there is usually very little win and a lot of danger to serving up
cached pages when a page in the cache represents a view that a
privileged user sees when visiting the site.  You typically don't want
other people to see the same thing he does.  As a result, typically
people will say "pages that require authentication can't be cached"
which isn't entirely true (you can cache them but it's often useless).

However, even serving up fully static content that doesn't require any
authentication like CSS, images, and so on can be a win.  You might
start there.

HTH,

- C


On Thu, 2005-09-01 at 11:54 -0500, Kennamore, Matthew G [NTK] wrote:
> Zope version 2.7.3 (planning to goto 2.7.7 soon with ZODB 3.2.9)
> Pyhton is 2.3.4
> Apache 1.3 
> 
> We have 943 users as of this minute with a bout 1000 objects being created a day (Lots of creates)
> 
> -----Original Message-----
> From: Chris McDonough [mailto:chrism at plope.com]
> Sent: Thursday, September 01, 2005 12:51 PM
> To: Kennamore, Matthew G [NTK]
> Cc: zope at zope.org
> Subject: Re: [Zope] Zope scalabilty and problems
> 
> Your "tranaaction blocked" error messages seem to imply that either or
> both of the following is true:
> 
> - some transactions are taking a "long time"
>   (any more than, say, 200 milliseconds is a long time)
> 
> - you have a very high transaction volume.
> 
> That said, these messages are informational rather than signifying an
> error condition.  Some transaction blockage is expected since the ZEO
> server can only deal with one transaction at a time.
> 
> What version of Zope are you using?
> 
> Is this an "intranet" application?  Are there lots of users creating
> content?  Can you venture a guess as to how many users are "creating"
> content vs. "viewing" content at any given time?  Are the users that are
> creating content logged in?  The users that are viewing content?
> 
> On Wed, 2005-08-31 at 11:03 -0500, Kennamore, Matthew G [NTK] wrote:
> > Here is our situation, we originally launched zope with a small user
> > base in mind and like most IT implementations it got away from us
> > quickly and we have over 1000 users now.  We are experiencing huge
> > slowdowns and lots of complaints of performance.
> >
> > Let me outline our system:
> >
> > I have a load balancer that round robins incoming request to 3 zeo
> > servers
> >
> > I have 1 zope server on the back en that severs the data, this is a
> > large single processor machine with 4 gigs of ram
> >
> > Each zeo server is 2 gigs of ram, single processor 3.2 gighz
> >
> > The throughput doesn't appear to be the problem as ram/processors are.
> >
> > Short term I plan to change all zeo's to 4 gigs ram and add 3
> > additional servers.
> >
> >
> > The Technologies we are using are Zope, Zeo, plone and so forth.  Why
> > are we having these large issues?  I see sites with hundreds of
> > thousands of users and they don't seem to have these issues?  We plan
> > to recode our site to get away from the Data.fs as soon as we can but
> > that is a ways off yet.  What am I doing wrong can anyone help?
> >
> > _______________________________________________
> > Zope maillist  -  Zope at zope.org
> > http://mail.zope.org/mailman/listinfo/zope
> > **   No cross posts or HTML encoding!  **
> > (Related lists -
> >  http://mail.zope.org/mailman/listinfo/zope-announce
> >  http://mail.zope.org/mailman/listinfo/zope-dev )
> 
>