[Zope] Zope scalabilty and problems

Andrew Sawyers andrew at zope.com
Thu Sep 1 13:30:28 EDT 2005 

Sorry, 
I missed the beginning of this thread, I've got some questions that I didn't
see answered or asked:  
Did Matt indicate if he was running multiple zeo app servers?  It might help
to be spreading the load.  1,0000 writes a day is not outrageous....so it's
possible that really what you need to spread the load across a load balanced
pool of zeo app servers.  

How many users are authenticated at any given time actually using the system
for writing?  

Are you running a persistent or non-persistent cache?  If so, what size it
is?  

Check the control panel, under database management and choose your database,
then look at the activity tab.  Do you have a high volume of object loads?
What is the volume of object stores?  

There is also an in-memory zeo object cache - click the cache parameters tab
and see the number of objects in your in-memory cache.  If your zeo
client(s) have lots of ram (they definitely should), you can crank this up
as well.

What is the 'horespower' of your app server(s)?

Andrew
--
Zope Managed Hosting
Software Engineer
Zope Corporation
(540) 361-1700 

> -----Original Message-----
> From: zope-bounces at zope.org [mailto:zope-bounces at zope.org] On Behalf Of
> Kennamore, Matthew G [NTK]
> Sent: Thursday, September 01, 2005 1:19 PM
> To: Chris McDonough
> Cc: zope at zope.org
> Subject: RE: [Zope] Zope scalabilty and problems
> 
> Thanks everyone for their suggestions, bought a squid book today and I
> know what I'll be doing all weekend <grin>
> 
>  -----Original Message-----
> From: 	Chris McDonough [mailto:chrism at plope.com]
> Sent:	Thursday, September 01, 2005 1:13 PM
> To:	Kennamore, Matthew G [NTK]
> Cc:	zope at zope.org
> Subject:	RE: [Zope] Zope scalabilty and problems
> 
> What you've got now is a pretty reasonable setup.  Moving to 2.7.7 will
> likely not get you any "free" performance increases and it doesn't
> appear that there are any "critical" ZODB bugfixes relating to your
> problems since 2.7.3.  FWIW, also, I'd venture a guess that moving "off
> Data.fs" (as you mentioned in your original email) may be quite costly,
> requiring a lot of application recoding.  It also might not help solve
> the scaling problem you're having.  It's a "high risk, potentially low
> reward" solution.  You need something that is low-risk and potentially
> high reward, I'd suspect.
> 
> Others have said this, but it bears repeating.  The biggest
> "administrative" win you can get for the cheapest cost is to add an HTTP
> cache (like Squid or even Apache's mod_proxy) in front of your ZEO
> clients and to cause your Zope application to set the proper response
> headers which allow the cache server to cache (and thus serve) those
> pages on subsequent requests.
> 
> This can be tricky when most of your traffic is authenticated because
> there is usually very little win and a lot of danger to serving up
> cached pages when a page in the cache represents a view that a
> privileged user sees when visiting the site.  You typically don't want
> other people to see the same thing he does.  As a result, typically
> people will say "pages that require authentication can't be cached"
> which isn't entirely true (you can cache them but it's often useless).
> 
> However, even serving up fully static content that doesn't require any
> authentication like CSS, images, and so on can be a win.  You might
> start there.
> 
> HTH,
> 
> - C
> 
> 
> On Thu, 2005-09-01 at 11:54 -0500, Kennamore, Matthew G [NTK] wrote:
> > Zope version 2.7.3 (planning to goto 2.7.7 soon with ZODB 3.2.9)
> > Pyhton is 2.3.4
> > Apache 1.3
> >
> > We have 943 users as of this minute with a bout 1000 objects being
> created a day (Lots of creates)
> >
> > -----Original Message-----
> > From: Chris McDonough [mailto:chrism at plope.com]
> > Sent: Thursday, September 01, 2005 12:51 PM
> > To: Kennamore, Matthew G [NTK]
> > Cc: zope at zope.org
> > Subject: Re: [Zope] Zope scalabilty and problems
> >
> > Your "tranaaction blocked" error messages seem to imply that either or
> > both of the following is true:
> >
> > - some transactions are taking a "long time"
> >   (any more than, say, 200 milliseconds is a long time)
> >
> > - you have a very high transaction volume.
> >
> > That said, these messages are informational rather than signifying an
> > error condition.  Some transaction blockage is expected since the ZEO
> > server can only deal with one transaction at a time.
> >
> > What version of Zope are you using?
> >
> > Is this an "intranet" application?  Are there lots of users creating
> > content?  Can you venture a guess as to how many users are "creating"
> > content vs. "viewing" content at any given time?  Are the users that are
> > creating content logged in?  The users that are viewing content?
> >
> > On Wed, 2005-08-31 at 11:03 -0500, Kennamore, Matthew G [NTK] wrote:
> > > Here is our situation, we originally launched zope with a small user
> > > base in mind and like most IT implementations it got away from us
> > > quickly and we have over 1000 users now.  We are experiencing huge
> > > slowdowns and lots of complaints of performance.
> > >
> > > Let me outline our system:
> > >
> > > I have a load balancer that round robins incoming request to 3 zeo
> > > servers
> > >
> > > I have 1 zope server on the back en that severs the data, this is a
> > > large single processor machine with 4 gigs of ram
> > >
> > > Each zeo server is 2 gigs of ram, single processor 3.2 gighz
> > >
> > > The throughput doesn't appear to be the problem as ram/processors are.
> > >
> > > Short term I plan to change all zeo's to 4 gigs ram and add 3
> > > additional servers.
> > >
> > >
> > > The Technologies we are using are Zope, Zeo, plone and so forth.  Why
> > > are we having these large issues?  I see sites with hundreds of
> > > thousands of users and they don't seem to have these issues?  We plan
> > > to recode our site to get away from the Data.fs as soon as we can but
> > > that is a ways off yet.  What am I doing wrong can anyone help?
> > >
> > > _______________________________________________
> > > Zope maillist  -  Zope at zope.org
> > > http://mail.zope.org/mailman/listinfo/zope
> > > **   No cross posts or HTML encoding!  **
> > > (Related lists -
> > >  http://mail.zope.org/mailman/listinfo/zope-announce
> > >  http://mail.zope.org/mailman/listinfo/zope-dev )
> >
> >
> 
> 
> 
> _______________________________________________
> Zope maillist  -  Zope at zope.org
> http://mail.zope.org/mailman/listinfo/zope
> **   No cross posts or HTML encoding!  **
> (Related lists -
>  http://mail.zope.org/mailman/listinfo/zope-announce
>  http://mail.zope.org/mailman/listinfo/zope-dev )

More information about the Zope mailing list