RE: [Zope] Zope scalabilty and problems
Zope version 2.7.3 (planning to goto 2.7.7 soon with ZODB 3.2.9) Pyhton is 2.3.4 Apache 1.3 We have 943 users as of this minute with a bout 1000 objects being created a day (Lots of creates) -----Original Message----- From: Chris McDonough [mailto:chrism@plope.com] Sent: Thursday, September 01, 2005 12:51 PM To: Kennamore, Matthew G [NTK] Cc: zope@zope.org Subject: Re: [Zope] Zope scalabilty and problems Your "tranaaction blocked" error messages seem to imply that either or both of the following is true: - some transactions are taking a "long time" (any more than, say, 200 milliseconds is a long time) - you have a very high transaction volume. That said, these messages are informational rather than signifying an error condition. Some transaction blockage is expected since the ZEO server can only deal with one transaction at a time. What version of Zope are you using? Is this an "intranet" application? Are there lots of users creating content? Can you venture a guess as to how many users are "creating" content vs. "viewing" content at any given time? Are the users that are creating content logged in? The users that are viewing content? On Wed, 2005-08-31 at 11:03 -0500, Kennamore, Matthew G [NTK] wrote:
Here is our situation, we originally launched zope with a small user base in mind and like most IT implementations it got away from us quickly and we have over 1000 users now. We are experiencing huge slowdowns and lots of complaints of performance.
Let me outline our system:
I have a load balancer that round robins incoming request to 3 zeo servers
I have 1 zope server on the back en that severs the data, this is a large single processor machine with 4 gigs of ram
Each zeo server is 2 gigs of ram, single processor 3.2 gighz
The throughput doesn't appear to be the problem as ram/processors are.
Short term I plan to change all zeo's to 4 gigs ram and add 3 additional servers.
The Technologies we are using are Zope, Zeo, plone and so forth. Why are we having these large issues? I see sites with hundreds of thousands of users and they don't seem to have these issues? We plan to recode our site to get away from the Data.fs as soon as we can but that is a ways off yet. What am I doing wrong can anyone help?
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
What you've got now is a pretty reasonable setup. Moving to 2.7.7 will likely not get you any "free" performance increases and it doesn't appear that there are any "critical" ZODB bugfixes relating to your problems since 2.7.3. FWIW, also, I'd venture a guess that moving "off Data.fs" (as you mentioned in your original email) may be quite costly, requiring a lot of application recoding. It also might not help solve the scaling problem you're having. It's a "high risk, potentially low reward" solution. You need something that is low-risk and potentially high reward, I'd suspect. Others have said this, but it bears repeating. The biggest "administrative" win you can get for the cheapest cost is to add an HTTP cache (like Squid or even Apache's mod_proxy) in front of your ZEO clients and to cause your Zope application to set the proper response headers which allow the cache server to cache (and thus serve) those pages on subsequent requests. This can be tricky when most of your traffic is authenticated because there is usually very little win and a lot of danger to serving up cached pages when a page in the cache represents a view that a privileged user sees when visiting the site. You typically don't want other people to see the same thing he does. As a result, typically people will say "pages that require authentication can't be cached" which isn't entirely true (you can cache them but it's often useless). However, even serving up fully static content that doesn't require any authentication like CSS, images, and so on can be a win. You might start there. HTH, - C On Thu, 2005-09-01 at 11:54 -0500, Kennamore, Matthew G [NTK] wrote:
Zope version 2.7.3 (planning to goto 2.7.7 soon with ZODB 3.2.9) Pyhton is 2.3.4 Apache 1.3
We have 943 users as of this minute with a bout 1000 objects being created a day (Lots of creates)
-----Original Message----- From: Chris McDonough [mailto:chrism@plope.com] Sent: Thursday, September 01, 2005 12:51 PM To: Kennamore, Matthew G [NTK] Cc: zope@zope.org Subject: Re: [Zope] Zope scalabilty and problems
Your "tranaaction blocked" error messages seem to imply that either or both of the following is true:
- some transactions are taking a "long time" (any more than, say, 200 milliseconds is a long time)
- you have a very high transaction volume.
That said, these messages are informational rather than signifying an error condition. Some transaction blockage is expected since the ZEO server can only deal with one transaction at a time.
What version of Zope are you using?
Is this an "intranet" application? Are there lots of users creating content? Can you venture a guess as to how many users are "creating" content vs. "viewing" content at any given time? Are the users that are creating content logged in? The users that are viewing content?
On Wed, 2005-08-31 at 11:03 -0500, Kennamore, Matthew G [NTK] wrote:
Here is our situation, we originally launched zope with a small user base in mind and like most IT implementations it got away from us quickly and we have over 1000 users now. We are experiencing huge slowdowns and lots of complaints of performance.
Let me outline our system:
I have a load balancer that round robins incoming request to 3 zeo servers
I have 1 zope server on the back en that severs the data, this is a large single processor machine with 4 gigs of ram
Each zeo server is 2 gigs of ram, single processor 3.2 gighz
The throughput doesn't appear to be the problem as ram/processors are.
Short term I plan to change all zeo's to 4 gigs ram and add 3 additional servers.
The Technologies we are using are Zope, Zeo, plone and so forth. Why are we having these large issues? I see sites with hundreds of thousands of users and they don't seem to have these issues? We plan to recode our site to get away from the Data.fs as soon as we can but that is a ways off yet. What am I doing wrong can anyone help?
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
On 9/1/05, Kennamore, Matthew G [NTK] <Matthew.G.Kennamore@sprint.com> wrote:
Zope version 2.7.3 (planning to goto 2.7.7 soon with ZODB 3.2.9) Pyhton is 2.3.4 Apache 1.3
We have 943 users as of this minute with a bout 1000 objects being created a day (Lots of creates)
Interesting. Somebody have a couple of values for heavy load sites (Zope only and with Plone) ? - hit per hour & day - visitor per hour & day - Gb of the ZODB Regards. -- Sébastien Douche <sdouche@gmail.com>
participants (3)
-
Chris McDonough -
Kennamore, Matthew G [NTK] -
Sebastien Douche