here you are! (see end of post)(i hope you dont mind my adressing the list). btw. does anybody know of other benchmarking software? (i once tried to grab httperf but the ftp site holding the source is down...) ru, peter. On Wed, 22 Mar 2000, J. Atwood wrote: :Peter, : :Have you ever done these test against just the ZServer? : :JMA : :> From: Peter Sabaini <sabaini@niil.at> :> Date: Wed, 22 Mar 2000 13:36:17 +0100 (CET) :> To: Stephan Richter <srichter@cbu.edu> :> Cc: zope@zope.org :> Subject: RE: [Zope] Server Sizing :> :> On Wed, 22 Mar 2000, Stephan Richter wrote: :> :> :>i agree on apache increasing your performance when serving static :> :>files (for instance images), but wouldn't url rewriting and the proxy :> :>pass module of apache be an even better solution? i just switched from :> :>PCGI to the Rewrite/Proxypass combo because of the additional overhead :> :>of PCGI. :> : :> :Mmmhh, I have to have a look. Okay, I just read the the little tutorial. :> :Does it really speed things up? :> :I guess SSL should not be hard using this solution. I will be thinking :> :about it. :> : :> :Anyone else? :> : :> :> some testing with apache benchmark: :> :> the setup is an apache 1.3.12 sitting on port 1988. :> :> /tstfolder/bmeth is a very simple dtml method ("i am bmeth in :> /tstfolder" <dtml-call "REQUEST.set('testvar', 'hm...')"> :> &dtml-testvar;). :> :> i do 1000 requests at 50 concurrent users, keepalive-connection via :> apache benchmark: :> :> first the result of doing a proxypass to zserver on port 8080: :> :> ------ :> :> /root# /usr/local/apache/bin/ab -n 1000 -c 50 -k -v 2 :> http://druck.kleinezeitung.at:1988/tstfolder/bmeth :> This is ApacheBench, Version 1.3c <$Revision: 1.38 $> apache-1.3 :> Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, :> http://www.zeustech.net/ :> Copyright (c) 1998-1999 The Apache Group, http://www.apache.org/ :> :> Benchmarking druck.kleinezeitung.at (be patient)...INFO: POST header :> == :> --- :> GET /tstfolder/bmeth HTTP/1.0 :> User-Agent: ApacheBench/1.3c :> Connection: Keep-Alive :> Host: druck.kleinezeitung.at :> Accept: */* :> :> :> --- :> Server Software: Zope/Zope :> Server Hostname: druck.kleinezeitung.at :> Server Port: 1988 :> :> Document Path: /tstfolder/bmeth :> Document Length: 25 bytes :> :> Concurrency Level: 50 :> Time taken for tests: 16.817 seconds :> Complete requests: 1000 :> Failed requests: 0 :> Keep-Alive requests: 0 :> Total transferred: 243000 bytes :> HTML transferred: 25000 bytes :> Requests per second: 59.46 :> Transfer rate: 14.45 kb/s received :> :> Connnection Times (ms) :> min avg max :> Connect: 0 1 38 :> Processing: 99 818 877 :> Total: 99 819 915 :> :> ---------------------- :> :> then i modify httpd.conf to get apache to talk to zope via :> PCGI / Zope.cgi (otherwise unchanged): :> :> ----------------------- :> :> /home/Zope# /usr/local/apache/bin/ab -n 1000 -c 50 -k -v 2 :> http://druck.kleinezeitung.at:1988/tstfolder/bmeth :> This is ApacheBench, Version 1.3c <$Revision: 1.38 $> apache-1.3 :> Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, :> http://www.zeustech.net/ :> Copyright (c) 1998-1999 The Apache Group, http://www.apache.org/ :> :> Benchmarking druck.kleinezeitung.at (be patient)...INFO: POST header :> == :> --- :> GET /tstfolder/bmeth HTTP/1.0 :> User-Agent: ApacheBench/1.3c :> Connection: Keep-Alive :> Host: druck.kleinezeitung.at :> Accept: */* :> :> :> --- :> Server Software: Apache/1.3.12 :> Server Hostname: druck.kleinezeitung.at :> Server Port: 1988 :> :> Document Path: /tstfolder/bmeth :> Document Length: 25 bytes :> :> Concurrency Level: 50 :> Time taken for tests: 21.984 seconds :> Complete requests: 1000 :> Failed requests: 0 :> Keep-Alive requests: 1000 :> Total transferred: 273050 bytes :> HTML transferred: 25000 bytes :> Requests per second: 45.49 :> Transfer rate: 12.42 kb/s received :> :> Connnection Times (ms) :> min avg max :> Connect: 0 15 355 :> Processing: 41 1058 6039 :> Total: 41 1073 6394 :> :> -------------------- :> :> :> conclusion: with proxy pass i get around 30% more requests per second :> and around 25% less connection time (and im not able to use keep-alive :> as it seems). :> :> please note that this is only a quick check and not scientific in any :> way, and naturally the benefits will diminish when more complicated :> pages are going to be served. :> :> if there are any other test results please let me know... :> :> peter. ps.: same test to zserver directly (shows further increase in thoughput): /root# /usr/local/apache/bin/ab -n 1000 -c 50 -k -v 2 http://druck.kleinezeitung.at:8080/tstfolder/bmeth This is ApacheBench, Version 1.3c <$Revision: 1.38 $> apache-1.3 Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Copyright (c) 1998-1999 The Apache Group, http://www.apache.org/ Benchmarking druck.kleinezeitung.at (be patient)...INFO: POST header == --- GET /tstfolder/bmeth HTTP/1.0 User-Agent: ApacheBench/1.3c Connection: Keep-Alive Host: druck.kleinezeitung.at Accept: */* --- Server Software: Zope/Zope Server Hostname: druck.kleinezeitung.at Server Port: 8080 Document Path: /tstfolder/bmeth Document Length: 25 bytes Concurrency Level: 50 Time taken for tests: 14.821 seconds Complete requests: 1000 Failed requests: 0 Keep-Alive requests: 0 Total transferred: 224000 bytes HTML transferred: 25000 bytes Requests per second: 67.47 Transfer rate: 15.11 kb/s received Connnection Times (ms) min avg max Connect: 0 0 31 Processing: 209 722 734 Total: 209 722 765 -- _________________________________________________ peter sabaini, mailto: sabaini@niil.at -------------------------------------------------
Thanks Peter, that at the very least lives up to what people are saying (ZServer alone is your fastest option) Here is another Benchmarking program. http://stein.cshl.org/~lstein/torture/ JMA
From: Peter Sabaini <sabaini@niil.at> Date: Wed, 22 Mar 2000 15:18:59 +0100 (CET) To: zope@zope.org Cc: "J. Atwood" <Jatwood@bwanazulia.com> Subject: Re: [Zope] Server Sizing
here you are! (see end of post)(i hope you dont mind my adressing the list).
btw. does anybody know of other benchmarking software? (i once tried to grab httperf but the ftp site holding the source is down...)
ru, peter.
On Wed, 22 Mar 2000, J. Atwood wrote:
:Peter, : :Have you ever done these test against just the ZServer? : :JMA : :> From: Peter Sabaini <sabaini@niil.at> :> Date: Wed, 22 Mar 2000 13:36:17 +0100 (CET) :> To: Stephan Richter <srichter@cbu.edu> :> Cc: zope@zope.org :> Subject: RE: [Zope] Server Sizing :> :> On Wed, 22 Mar 2000, Stephan Richter wrote: :> :> :>i agree on apache increasing your performance when serving static :> :>files (for instance images), but wouldn't url rewriting and the proxy :> :>pass module of apache be an even better solution? i just switched from :> :>PCGI to the Rewrite/Proxypass combo because of the additional overhead :> :>of PCGI. :> : :> :Mmmhh, I have to have a look. Okay, I just read the the little tutorial. :> :Does it really speed things up? :> :I guess SSL should not be hard using this solution. I will be thinking :> :about it. :> : :> :Anyone else? :> : :> :> some testing with apache benchmark: :> :> the setup is an apache 1.3.12 sitting on port 1988. :> :> /tstfolder/bmeth is a very simple dtml method ("i am bmeth in :> /tstfolder" <dtml-call "REQUEST.set('testvar', 'hm...')"> :> &dtml-testvar;). :> :> i do 1000 requests at 50 concurrent users, keepalive-connection via :> apache benchmark: :> :> first the result of doing a proxypass to zserver on port 8080: :> :> ------ :> :> /root# /usr/local/apache/bin/ab -n 1000 -c 50 -k -v 2 :> http://druck.kleinezeitung.at:1988/tstfolder/bmeth :> This is ApacheBench, Version 1.3c <$Revision: 1.38 $> apache-1.3 :> Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, :> http://www.zeustech.net/ :> Copyright (c) 1998-1999 The Apache Group, http://www.apache.org/ :> :> Benchmarking druck.kleinezeitung.at (be patient)...INFO: POST header :> == :> --- :> GET /tstfolder/bmeth HTTP/1.0 :> User-Agent: ApacheBench/1.3c :> Connection: Keep-Alive :> Host: druck.kleinezeitung.at :> Accept: */* :> :> :> --- :> Server Software: Zope/Zope :> Server Hostname: druck.kleinezeitung.at :> Server Port: 1988 :> :> Document Path: /tstfolder/bmeth :> Document Length: 25 bytes :> :> Concurrency Level: 50 :> Time taken for tests: 16.817 seconds :> Complete requests: 1000 :> Failed requests: 0 :> Keep-Alive requests: 0 :> Total transferred: 243000 bytes :> HTML transferred: 25000 bytes :> Requests per second: 59.46 :> Transfer rate: 14.45 kb/s received :> :> Connnection Times (ms) :> min avg max :> Connect: 0 1 38 :> Processing: 99 818 877 :> Total: 99 819 915 :> :> ---------------------- :> :> then i modify httpd.conf to get apache to talk to zope via :> PCGI / Zope.cgi (otherwise unchanged): :> :> ----------------------- :> :> /home/Zope# /usr/local/apache/bin/ab -n 1000 -c 50 -k -v 2 :> http://druck.kleinezeitung.at:1988/tstfolder/bmeth :> This is ApacheBench, Version 1.3c <$Revision: 1.38 $> apache-1.3 :> Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, :> http://www.zeustech.net/ :> Copyright (c) 1998-1999 The Apache Group, http://www.apache.org/ :> :> Benchmarking druck.kleinezeitung.at (be patient)...INFO: POST header :> == :> --- :> GET /tstfolder/bmeth HTTP/1.0 :> User-Agent: ApacheBench/1.3c :> Connection: Keep-Alive :> Host: druck.kleinezeitung.at :> Accept: */* :> :> :> --- :> Server Software: Apache/1.3.12 :> Server Hostname: druck.kleinezeitung.at :> Server Port: 1988 :> :> Document Path: /tstfolder/bmeth :> Document Length: 25 bytes :> :> Concurrency Level: 50 :> Time taken for tests: 21.984 seconds :> Complete requests: 1000 :> Failed requests: 0 :> Keep-Alive requests: 1000 :> Total transferred: 273050 bytes :> HTML transferred: 25000 bytes :> Requests per second: 45.49 :> Transfer rate: 12.42 kb/s received :> :> Connnection Times (ms) :> min avg max :> Connect: 0 15 355 :> Processing: 41 1058 6039 :> Total: 41 1073 6394 :> :> -------------------- :> :> :> conclusion: with proxy pass i get around 30% more requests per second :> and around 25% less connection time (and im not able to use keep-alive :> as it seems). :> :> please note that this is only a quick check and not scientific in any :> way, and naturally the benefits will diminish when more complicated :> pages are going to be served. :> :> if there are any other test results please let me know... :> :> peter.
ps.: same test to zserver directly (shows further increase in thoughput):
/root# /usr/local/apache/bin/ab -n 1000 -c 50 -k -v 2 http://druck.kleinezeitung.at:8080/tstfolder/bmeth This is ApacheBench, Version 1.3c <$Revision: 1.38 $> apache-1.3 Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Copyright (c) 1998-1999 The Apache Group, http://www.apache.org/
Benchmarking druck.kleinezeitung.at (be patient)...INFO: POST header == --- GET /tstfolder/bmeth HTTP/1.0 User-Agent: ApacheBench/1.3c Connection: Keep-Alive Host: druck.kleinezeitung.at Accept: */*
--- Server Software: Zope/Zope Server Hostname: druck.kleinezeitung.at Server Port: 8080
Document Path: /tstfolder/bmeth Document Length: 25 bytes
Concurrency Level: 50 Time taken for tests: 14.821 seconds Complete requests: 1000 Failed requests: 0 Keep-Alive requests: 0 Total transferred: 224000 bytes HTML transferred: 25000 bytes Requests per second: 67.47 Transfer rate: 15.11 kb/s received
Connnection Times (ms) min avg max Connect: 0 0 31 Processing: 209 722 734 Total: 209 722 765
--
_________________________________________________ peter sabaini, mailto: sabaini@niil.at -------------------------------------------------
Just wondering if anyone has thought about personalization services based on Zope ? The most obvious/traditional approach seems to be : 1. Acquire user information (via forms/surveys/transaction-histories etc) and store in RDB. 2. When the user comes to the site (and either logs in or is identified by IP or cookie), query the RDB for customized content or appropriate products. OK, no magic there. But it's not very scalable in terms of : a) Load. We can keep shoving Zope servers on the front end but the RDB is going to get hammered if this has to occur for each and every request. b) Development. A more object-oriented approach would be nicer. eg. Personalities with behaviours and properties. Could a pool of persistent personalities be left floating around ? (No, this is not an early April's fool) A while back (1997) I saw Broadvision and didn't really appreciate what they were doing. It seems that they've really improved a lot since then but I can't get my hands on their software (not for free anyway). So, I'm wondering if anybody has experience with it and can enlighten a heathen on key features we might wish to, er, 'appropriate' in Zope systems ? Or is it, like Vignette and Netscape ECXpert, nothing special and you're basically paying for the 'knowhow' ? Thank you, chas
On Fri, 24 Mar 2000, chas wrote:
Just wondering if anyone has thought about personalization services based on Zope ?
The most obvious/traditional approach seems to be : 1. Acquire user information (via forms/surveys/transaction-histories etc) and store in RDB. 2. When the user comes to the site (and either logs in or is identified by IP or cookie), query the RDB for customized content or appropriate products.
OK, no magic there. But it's not very scalable in terms of : a) Load. We can keep shoving Zope servers on the front end but the RDB is going to get hammered if this has to occur for each and every request.
I was thinking of a general simple caching product for Zope, to use in conjuction with FSSession for high load sites. A simple implementation is to use an in memory python dictionary to hold marshalled or cPickled data. Since the access times are fast we could use the asyncore classes that come with python to achieve really high rates. The cache contents can be served from a socket or a named pipe. A more difficult but more efficient way is to use shared memory for IPC. As I don't expect two clients to connect simultaneously with the same user id, write conflicts will be few (I suppose writes will be few too). In our environment I have found FSSession to be very fast, probably because the OS caches files. If someone complains about FSSession's speed I will code the caching product. Pavlos
participants (4)
-
chas -
J. Atwood -
Pavlos Christoforou -
Peter Sabaini