- Zope - Zope lists

Zope scalabilty and problems
by Kennamore, Matthew G [NTK] 01 Sep '05

01 Sep '05

Here is our situation, we originally launched zope with a small user base in mind and like most IT implementations it got away from us quickly and we have over 1000 users now. We are experiencing huge slowdowns and lots of complaints of performance. Let me outline our system: I have a load balancer that round robins incoming request to 3 zeo servers I have 1 zope server on the back en that severs the data, this is a large single processor machine with 4 gigs of ram Each zeo server is 2 gigs of ram, single processor 3.2 gighz The throughput doesn't appear to be the problem as ram/processors are. Short term I plan to change all zeo's to 4 gigs ram and add 3 additional servers. The Technologies we are using are Zope, Zeo, plone and so forth. Why are we having these large issues? I see sites with hundreds of thousands of users and they don't seem to have these issues? We plan to recode our site to get away from the Data.fs as soon as we can but that is a ways off yet. What am I doing wrong can anyone help?

8 7

RE: [Zope] Zope scalabilty and problems
by Kennamore, Matthew G [NTK] 01 Sep '05

01 Sep '05

I am seeing a large amounts of these 2005-08-31T08:40:18 INFO(0) ZSS:528/10.110.110.12:50602 Transaction blocked waiting for storage. Clients waiting: 1. 2005-08-31T08:40:18 INFO(0) ZSS:528/10.110.110.23:49161 Blocked transaction restarted. 2005-08-31T09:05:21 INFO(0) ZSS:528/10.110.110.23:49161 Transaction blocked waiting for storage. Clients waiting: 1. 2005-08-31T09:05:22 INFO(0) ZSS:528/10.110.110.12:50602 Blocked transaction restarted. 2005-08-31T10:18:26 INFO(0) ZSS:528/10.110.110.23:49161 Transaction blocked waiting for storage. Clients waiting: 1. 2005-08-31T10:18:28 INFO(0) ZSS:528/10.110.110.12:50602 Blocked transaction restarted. 2005-08-31T10:59:08 INFO(0) ZSS:528/10.110.110.23:49161 Transaction blocked waiting for storage. Clients waiting: 1. 2005-08-31T10:59:17 INFO(0) ZSS:528/10.110.110.12:50602 Blocked transaction restarted. 2005-08-31T10:59:31 INFO(0) ZSS:528/10.110.110.22:49161 Transaction blocked waiting for storage. Clients waiting: 1. 2005-08-31T11:00:13 INFO(0) ZSS:528/10.110.110.12:50602 Blocked transaction restarted. 2005-08-31T11:00:15 INFO(0) ZSS:528/10.110.110.23:49161 Transaction blocked waiting for storage. Clients waiting: 1. 2005-08-31T11:00:16 INFO(0) ZSS:528/10.110.110.22:49161 Transaction blocked waiting for storage. Clients waiting: 2. -----Original Message----- From: Paul Winkler [mailto:pw_lists@slinkp.com] Sent: Wednesday, August 31, 2005 1:33 PM To: Kennamore, Matthew G [NTK] Cc: zope(a)zope.org Subject: Re: [Zope] Zope scalabilty and problems > Here is our situation, we originally launched zope with a small user > base in mind and like most IT implementations it got away from us > quickly and we have over 1000 users now. We are experiencing huge > slowdowns and lots of complaints of performance. > > Let me outline our system: > > I have a load balancer that round robins incoming request to 3 zeo > servers Just to get the terminology correct: These are Zope servers. They are ZEO *clients*. > I have 1 zope server on the back en that severs the data, this is a > large single processor machine with 4 gigs of ram I think you mean ZEO server, not Zope server. > Each zeo server is 2 gigs of ram, single processor 3.2 gighz That is plenty of hardware for most sites... > Short term I plan to change all zeo's to 4 gigs ram and add 3 additional > servers. That might help if your users' typical usage patterns are unavoidably dynamic: e.g. lots of unique catalog queries, lots of personalization, and/or lots of writes. But it's more typical to have quite a lot of reads of rarely-changing data. In which case, the usual low-cost answer to performance issues is: cache, cache, and more cache. There are several levels of caching, which vary in difficulty of setup and potential performance impact. See below... > The Technologies we are using are Zope, Zeo, plone and so forth. Why > are we having these large issues? I see sites with hundreds of > thousands of users and they don't seem to have these issues? We plan to > recode our site to get away from the Data.fs as soon as we can but that > is a ways off yet. What am I doing wrong can anyone help? Plone is very highly configurable, and every one of those runtime options adds a performance cost. You certainly can run high-performance plone sites on your hardware, you just need to tune it properly. The general steps are: 1) Test your performance. See http://plope.com/Members/chrism/loadtesting-methodology/view?searchterm=per… 2) Profile to see what's slow. 3) Increase caching and/or optimize slow code identified by profiling. Caching is generally a bigger win than optimizing, for less effort. (I have been doing full-time zope work for about 4 years now and I have yet to ever consciously optimize any code ... I try to have efficiency in mind while writing it but that's it... so far there's always been a simple solution via caching. But I don't work on mega-heavy-load sites.) 4) Repeat until satisfied. The best docs I've found on this are: "Scaling Zope", Chris McDonough's presentation from 2003: http://plope.com/misc/szweb Read that one from start to finish at least once. There is also a good chapter on maintaining & optimizing plone in Andy McKay's "Plone book". Another good one is http://web.archive.org/web/20030723200058/http://matthardy.us/mywiki/HighPe… but the original has gone missing so the charts aren't there. -PW

1 0

RE: Zope Digest, Vol 16, Issue 2
by Kennamore, Matthew G [NTK] 01 Sep '05

01 Sep '05

I am seeing a large amounts of these 2005-08-31T08:40:18 INFO(0) ZSS:528/10.110.110.12:50602 Transaction blocked waiting for storage. Clients waiting: 1. 2005-08-31T08:40:18 INFO(0) ZSS:528/10.110.110.23:49161 Blocked transaction restarted. 2005-08-31T09:05:21 INFO(0) ZSS:528/10.110.110.23:49161 Transaction blocked waiting for storage. Clients waiting: 1. 2005-08-31T09:05:22 INFO(0) ZSS:528/10.110.110.12:50602 Blocked transaction restarted. 2005-08-31T10:18:26 INFO(0) ZSS:528/10.110.110.23:49161 Transaction blocked waiting for storage. Clients waiting: 1. 2005-08-31T10:18:28 INFO(0) ZSS:528/10.110.110.12:50602 Blocked transaction restarted. 2005-08-31T10:59:08 INFO(0) ZSS:528/10.110.110.23:49161 Transaction blocked waiting for storage. Clients waiting: 1. 2005-08-31T10:59:17 INFO(0) ZSS:528/10.110.110.12:50602 Blocked transaction restarted. 2005-08-31T10:59:31 INFO(0) ZSS:528/10.110.110.22:49161 Transaction blocked waiting for storage. Clients waiting: 1. 2005-08-31T11:00:13 INFO(0) ZSS:528/10.110.110.12:50602 Blocked transaction restarted. 2005-08-31T11:00:15 INFO(0) ZSS:528/10.110.110.23:49161 Transaction blocked waiting for storage. Clients waiting: 1. 2005-08-31T11:00:16 INFO(0) ZSS:528/10.110.110.22:49161 Transaction blocked waiting for storage. Clients waiting: 2. -----Original Message----- From: zope-bounces(a)zope.org [mailto:zope-bounces@zope.org] On Behalf Of zope-request(a)zope.org Sent: Thursday, September 01, 2005 12:00 PM To: zope(a)zope.org Subject: Zope Digest, Vol 16, Issue 2 Send Zope mailing list submissions to zope(a)zope.org To subscribe or unsubscribe via the World Wide Web, visit http://mail.zope.org/mailman/listinfo/zope or, via email, send a message with subject or body 'help' to zope-request(a)zope.org You can reach the person managing the list at zope-owner(a)zope.org When replying, please edit your Subject line so it is more specific than "Re: Contents of Zope digest..." Today's Topics: 1. Re: Zope scalabilty and problems (Sascha Ottolski) ---------------------------------------------------------------------- Message: 1 Date: Thu, 1 Sep 2005 16:45:11 +0200 From: Sascha Ottolski <sascha.ottolski(a)gallileus.de> Subject: Re: [Zope] Zope scalabilty and problems To: zope(a)zope.org Message-ID: <200509011645.12601.sascha.ottolski(a)gallileus.de> Content-Type: text/plain; charset="windows-1252" Am Mittwoch, 31. August 2005 18:50 schrieb J Cameron Cooper: > There are lots of ways to improve performance just administratively. You > can improve various zope.conf settings, and set up caching (either RAM > or HTTP Accel.) in Plone. You can also put a caching proxy server (like > Squid) in front of your Zopes. not to forget that playing with the zeo and/or zope database caches might help; if you see lots of conflict errors in you logs, may be an upgrade to 2.8 with MVCC does a benefit. and, of course, look carefully if any other tasks on all of your servers compete with your cpu and io resources. Cheers, Sascha P.S. just in case, debug mode or profiling and such are better to be disabled on your live system...:-) ------------------------------ _______________________________________________ Zope maillist - Zope(a)zope.org End of Zope Digest, Vol 16, Issue 2 ***********************************

1 0

Visibility and Copy&Paste support
by Peter Bengtsson 01 Sep '05

01 Sep '05

I've understood that in product development I have to register all my classes with registerClass() that I want to, as instanciated objects, copy/cut & paste. Some of these objects I don't want to encourage people to create via the ZMI and I don't want to clutter up the "Add new object drop down" with stuff that can't be added. If I register the class, but set 'visibility=None' I still can't copy/cut & paste the instanciated objects. Why? What's the point of the 'visibility=None' parameter? -- Peter Bengtsson, work www.fry-it.com home www.peterbe.com hobby www.issuetrackerproduct.com

3 3

Sending Fax through MailHost
by Chris Larsen 01 Sep '05

01 Sep '05

Hi, I'm trying to put a fax number into the "To:" field of the mailhost header and am not having much luck. Obviously the standard for this is to address the email to [FAX:18479131145] but when I do [FAX:1<dtml-var faxnum>] it just tells me that there are no recipients designated. Any ideas? Thanks, chris

2 1

RE: [Zope] Excel spreadsheet attachment in dtml-sendmail
by Chris Larsen 01 Sep '05

01 Sep '05

Never mind.. the dtml-boundary looks like this now.. I guess that zopelabs.com how-to needs to be updated.. oh well. <dtml-boundary disposition=attachment type=application/vend.ms-excel encode=base64 filename=ics.xls><dtml-var expr="ics.data"> _____ From: Chris Larsen Sent: Wednesday, August 31, 2005 3:10 PM To: zope(a)zope.org Subject: [Zope] Excel spreadsheet attachment in dtml-sendmail Hi again, Now on to my next problem.. I have tried the below code to send off a spreadsheet as an attachment and am getting a dat file and a 232B file that can't be read. Here's the code from zopelabs.com appropriated to my own info: <dtml-sendmail mailhost=MailHost> To: chris(a)hssc.com From: chris(a)hssc.com Subject: some subject <dtml-mime type=application/msexcel encode=base64> Content-Disposition: inline; filename="ics.xls" <dtml-boundary name=ics.xls type=application/msexcel disposition=attachment encode=base64> </dtml-mime> </dtml-sendmail> I thought they type was supposed to be application/vend.msexcel I've tried both ways tho but to no avail. Thanks in advance, Chris Larsen

2 1

RE: [Zope] Sending Fax through MailHost
by Chris Larsen 31 Aug '05

31 Aug '05

thanks -----Original Message----- From: Jens Vagelpohl [mailto:jens@dataflake.org] Sent: Wednesday, August 31, 2005 4:57 PM To: zope user list Subject: Re: [Zope] Sending Fax through MailHost On 31 Aug 2005, at 22:29, Chris Larsen wrote: > By the way, I did notice that in my address in outlook it's > specified as the "email-type" "FAX" and when I put a normal email > address in it is "email type" "SMTP". I guess when you do [fax: ] > it changes the email-type to FAX and uses the number as the address. > > > I am thinking there would have to be an additional header line > similar to content-type but I don't know where this would go or > what it would be. Is this even supported (or supportable??) What you are trying to do is not supported because your FAX addresses are not valid and legal email addresses. MailHost will simply ignore them. You would have to write your own implementation that does something intelligent with those odd addresses. jens _______________________________________________ Zope maillist - Zope(a)zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )

1 0

RE: [Zope] Sending Fax through MailHost
by Chris Larsen 31 Aug '05

31 Aug '05

By the way, I did notice that in my address in outlook it's specified as the "email-type" "FAX" and when I put a normal email address in it is "email type" "SMTP". I guess when you do [fax: ] it changes the email-type to FAX and uses the number as the address. I am thinking there would have to be an additional header line similar to content-type but I don't know where this would go or what it would be. Is this even supported (or supportable??) Thanks, chris _____ From: Chris Larsen Sent: Wednesday, August 31, 2005 1:19 PM To: zope(a)zope.org Subject: [Zope] Sending Fax through MailHost Hi, I'm trying to put a fax number into the "To:" field of the mailhost header and am not having much luck. Obviously the standard for this is to address the email to [FAX:18479131145] but when I do [FAX:1<dtml-var faxnum>] it just tells me that there are no recipients designated. Any ideas? Thanks, chris

2 1

RE: [Zope] Sending Fax through MailHost
by Chris Larsen 31 Aug '05

31 Aug '05

Sorry for the slim amount of information.. here's what my dtml-sendmail header looks like: <dtml-sendmail mailhost="MailHost"> To: [Fax:18478767346] From: chris(a)hssc.com Subject: <dtml-in select_message><dtml-var subject> <dtml-mime type=text/html disposition=inline encode=7bit> _____ From: Chris Larsen Sent: Wednesday, August 31, 2005 1:19 PM To: zope(a)zope.org Subject: [Zope] Sending Fax through MailHost Hi, I'm trying to put a fax number into the "To:" field of the mailhost header and am not having much luck. Obviously the standard for this is to address the email to [FAX:18479131145] but when I do [FAX:1<dtml-var faxnum>] it just tells me that there are no recipients designated. Any ideas? Thanks, chris

1 0

Excel spreadsheet attachment in dtml-sendmail
by Chris Larsen 31 Aug '05

31 Aug '05

Hi again, Now on to my next problem.. I have tried the below code to send off a spreadsheet as an attachment and am getting a dat file and a 232B file that can't be read. Here's the code from zopelabs.com appropriated to my own info: <dtml-sendmail mailhost=MailHost> To: chris(a)hssc.com From: chris(a)hssc.com Subject: some subject <dtml-mime type=application/msexcel encode=base64> Content-Disposition: inline; filename="ics.xls" <dtml-boundary name=ics.xls type=application/msexcel disposition=attachment encode=base64> </dtml-mime> </dtml-sendmail> I thought they type was supposed to be application/vend.msexcel I've tried both ways tho but to no avail. Thanks in advance, Chris Larsen

1 0