Zope's performance on xBSD
Hello, I'm using Zope on FreeBSD and OpenBSD and I've noticed some serious performance issues. Since I hadn't experienced any problems at the past with Linux, I did some benchmarks. I've written a small program that connects to Zope and tries to retrieve a non-existant URL (source code appended). While Linux performed almost linearly depending on the number of hits/sec, FreeBSD and OpenBSD performed exponential. I did the following experiment: I installed a binary distribution of python-1.5.2 (pkg_add or apt-get) and I compiled Zope-2.3.3+HotFix from sources (with wo_pcgi.py). I entered Zope's management screen and expanded all folders. I ran the previous program and after about 1000 hits I reloaded the page and measured the time it gets to finish. The results are below: |---------------------+------------+---------------+-------------+ | OS | Hit Rate* | Real hit rate | Reload time | | | (hits/sec) | (hits/sec) | (sec) | |---------------------+------------+---------------+-------------+ | | 30 | 20 | 13 | | Linux debian 2.2.19 | 50 | 33 | 16 | | | 100 | 38 | 16 | |---------------------+------------+---------------+-------------+ | | 30 | 20 | 10 | | FreeBSD 4.3-Release | 50 | 33 | 23 | | | 100 | 40 | 100 | |---------------------+------------+---------------+-------------+ | | 30 | 20 | 10 | | OpenBSD 2.9-Release | 50 | 33 | 15 | | | 100 | 50 | 285 | |---------------------+------------+---------------+-------------+ *The hit rate is the one passed as an argument to the program, while real hit rate was calculated according to Zope's log file. All tests were ran on the same computer, a 600MHz Celeron with 192MB RAM These tests were ran using the default Data.fs. When I use a real Data.fs + my products I even get time outs when I try to access the content. Why is there such a difference between the OSs? Do the BSD users of this list experience similar problems? Giorgos Verigakis ------------------ 8< ------------------ #include <stdio.h> #include <stdlib.h> #include <unistd.h> #include <sys/types.h> #include <sys/socket.h> #include <netinet/in.h> #include <netdb.h> int main(int argc, char *argv[]) { struct sockaddr_in sa; struct hostent *hp; int s; int i, t, len; unsigned long sleeptime; char request[128]; char *url="/foo"; if (argc != 4) { printf("Usage: %s <host> <port> <rate>\n", argv[0]); printf("rate in hits/sec\n"); exit(1); } if ((hp = gethostbyname(argv[1])) == NULL) { printf("error looking up host\n"); exit(1); } sprintf(request, "GET %s HTTP/1.0\015\012\015\012", url); len = strlen(request); sleeptime = (1 / atof(argv[3])) * 1000000; bzero(&sa, sizeof(sa)); bcopy(hp->h_addr, (char *)&sa.sin_addr, hp->h_length); sa.sin_family = hp->h_addrtype; sa.sin_port = htons(atoi(argv[2])); for (i = 0;; i++) { if (( s= socket(hp->h_addrtype, SOCK_STREAM, 0)) < 0) { printf("Socket error\n"); continue; } if (connect(s, (struct sockaddr *) &sa, sizeof sa) < 0) { printf("Connection error\n"); close(s); continue; } t = write(s, request, len); printf("i=%d: %d bytes\n", i, t); usleep(sleeptime); close(s); } } ------------------ 8< ------------------
Hi Giorgios, Let me see if I understand the testing methodology and results... You generate load using your program, and while the program is running, you try to reload the page in a browser.. right? Furthermore, your test results show that on Linux, the time it takes to reload the page under any of the tested loads is virtually the same, while on BSD, the time that it takes to reload the page appears to be related to the load in an O(n^2) way. I don't know the answer, but I just want to restate the findings in another way so that you can tell me if I'm misunderstanding them. - C Giorgos Verigakis wrote:
Hello, I'm using Zope on FreeBSD and OpenBSD and I've noticed some serious performance issues. Since I hadn't experienced any problems at the past with Linux, I did some benchmarks.
I've written a small program that connects to Zope and tries to retrieve a non-existant URL (source code appended). While Linux performed almost linearly depending on the number of hits/sec, FreeBSD and OpenBSD performed exponential.
I did the following experiment: I installed a binary distribution of python-1.5.2 (pkg_add or apt-get) and I compiled Zope-2.3.3+HotFix from sources (with wo_pcgi.py). I entered Zope's management screen and expanded all folders. I ran the previous program and after about 1000 hits I reloaded the page and measured the time it gets to finish. The results are below:
|---------------------+------------+---------------+-------------+ | OS | Hit Rate* | Real hit rate | Reload time | | | (hits/sec) | (hits/sec) | (sec) | |---------------------+------------+---------------+-------------+ | | 30 | 20 | 13 | | Linux debian 2.2.19 | 50 | 33 | 16 | | | 100 | 38 | 16 | |---------------------+------------+---------------+-------------+ | | 30 | 20 | 10 | | FreeBSD 4.3-Release | 50 | 33 | 23 | | | 100 | 40 | 100 | |---------------------+------------+---------------+-------------+ | | 30 | 20 | 10 | | OpenBSD 2.9-Release | 50 | 33 | 15 | | | 100 | 50 | 285 | |---------------------+------------+---------------+-------------+ *The hit rate is the one passed as an argument to the program, while real hit rate was calculated according to Zope's log file.
All tests were ran on the same computer, a 600MHz Celeron with 192MB RAM
These tests were ran using the default Data.fs. When I use a real Data.fs + my products I even get time outs when I try to access the content. Why is there such a difference between the OSs? Do the BSD users of this list experience similar problems?
Giorgos Verigakis
------------------ 8< ------------------ #include <stdio.h> #include <stdlib.h> #include <unistd.h> #include <sys/types.h> #include <sys/socket.h> #include <netinet/in.h> #include <netdb.h>
int main(int argc, char *argv[]) { struct sockaddr_in sa; struct hostent *hp; int s; int i, t, len; unsigned long sleeptime; char request[128]; char *url="/foo";
if (argc != 4) { printf("Usage: %s <host> <port> <rate>\n", argv[0]); printf("rate in hits/sec\n"); exit(1); }
if ((hp = gethostbyname(argv[1])) == NULL) { printf("error looking up host\n"); exit(1); }
sprintf(request, "GET %s HTTP/1.0\015\012\015\012", url); len = strlen(request); sleeptime = (1 / atof(argv[3])) * 1000000;
bzero(&sa, sizeof(sa)); bcopy(hp->h_addr, (char *)&sa.sin_addr, hp->h_length); sa.sin_family = hp->h_addrtype; sa.sin_port = htons(atoi(argv[2]));
for (i = 0;; i++) { if (( s= socket(hp->h_addrtype, SOCK_STREAM, 0)) < 0) { printf("Socket error\n"); continue; }
if (connect(s, (struct sockaddr *) &sa, sizeof sa) < 0) { printf("Connection error\n"); close(s); continue; }
t = write(s, request, len); printf("i=%d: %d bytes\n", i, t);
usleep(sleeptime);
close(s); } } ------------------ 8< ------------------
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
-- Chris McDonough Zope Corporation http://www.zope.org http://www.zope.com """ Killing hundreds of birds with thousands of stones """
On Fri, 17 Aug 2001, Chris McDonough wrote:
Hi Giorgios,
Let me see if I understand the testing methodology and results... You generate load using your program, and while the program is running, you try to reload the page in a browser.. right?
Yes, that's right
Furthermore, your test results show that on Linux, the time it takes to reload the page under any of the tested loads is virtually the same, while on BSD, the time that it takes to reload the page appears to be related to the load in an O(n^2) way.
That's right again. I haven't produced any graphs to know if it is O(n^2) but it sure is non-linear.
I don't know the answer, but I just want to restate the findings in another way so that you can tell me if I'm misunderstanding them.
Well, apologies if I wasn't very clear...
- C
Giorgos Verigakis wrote:
Hello, I'm using Zope on FreeBSD and OpenBSD and I've noticed some serious performance issues. Since I hadn't experienced any problems at the past with Linux, I did some benchmarks.
I've written a small program that connects to Zope and tries to retrieve a non-existant URL (source code appended). While Linux performed almost linearly depending on the number of hits/sec, FreeBSD and OpenBSD performed exponential.
I did the following experiment: I installed a binary distribution of python-1.5.2 (pkg_add or apt-get) and I compiled Zope-2.3.3+HotFix from sources (with wo_pcgi.py). I entered Zope's management screen and expanded all folders. I ran the previous program and after about 1000 hits I reloaded the page and measured the time it gets to finish. The results are below:
|---------------------+------------+---------------+-------------+ | OS | Hit Rate* | Real hit rate | Reload time | | | (hits/sec) | (hits/sec) | (sec) | |---------------------+------------+---------------+-------------+ | | 30 | 20 | 13 | | Linux debian 2.2.19 | 50 | 33 | 16 | | | 100 | 38 | 16 | |---------------------+------------+---------------+-------------+ | | 30 | 20 | 10 | | FreeBSD 4.3-Release | 50 | 33 | 23 | | | 100 | 40 | 100 | |---------------------+------------+---------------+-------------+ | | 30 | 20 | 10 | | OpenBSD 2.9-Release | 50 | 33 | 15 | | | 100 | 50 | 285 | |---------------------+------------+---------------+-------------+ *The hit rate is the one passed as an argument to the program, while real hit rate was calculated according to Zope's log file.
All tests were ran on the same computer, a 600MHz Celeron with 192MB RAM
These tests were ran using the default Data.fs. When I use a real Data.fs + my products I even get time outs when I try to access the content. Why is there such a difference between the OSs? Do the BSD users of this list experience similar problems?
Giorgos Verigakis
------------------ 8< ------------------ #include <stdio.h> #include <stdlib.h> #include <unistd.h> #include <sys/types.h> #include <sys/socket.h> #include <netinet/in.h> #include <netdb.h>
int main(int argc, char *argv[]) { struct sockaddr_in sa; struct hostent *hp; int s; int i, t, len; unsigned long sleeptime; char request[128]; char *url="/foo";
if (argc != 4) { printf("Usage: %s <host> <port> <rate>\n", argv[0]); printf("rate in hits/sec\n"); exit(1); }
if ((hp = gethostbyname(argv[1])) == NULL) { printf("error looking up host\n"); exit(1); }
sprintf(request, "GET %s HTTP/1.0\015\012\015\012", url); len = strlen(request); sleeptime = (1 / atof(argv[3])) * 1000000;
bzero(&sa, sizeof(sa)); bcopy(hp->h_addr, (char *)&sa.sin_addr, hp->h_length); sa.sin_family = hp->h_addrtype; sa.sin_port = htons(atoi(argv[2]));
for (i = 0;; i++) { if (( s= socket(hp->h_addrtype, SOCK_STREAM, 0)) < 0) { printf("Socket error\n"); continue; }
if (connect(s, (struct sockaddr *) &sa, sizeof sa) < 0) { printf("Connection error\n"); close(s); continue; }
t = write(s, request, len); printf("i=%d: %d bytes\n", i, t);
usleep(sleeptime);
close(s); } } ------------------ 8< ------------------
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
Giorgos Verigakis wrote:
I don't know the answer, but I just want to restate the findings in another way so that you can tell me if I'm misunderstanding them.
Well, apologies if I wasn't very clear...
Actually, as it turns out, you were very clear (I even understood it!) ;-) Hopefully someone with more bsd experience can speak up...
On Fri, 17 Aug 2001, Giorgos Verigakis wrote:
Hello, I'm using Zope on FreeBSD and OpenBSD and I've noticed some serious performance issues. Since I hadn't experienced any problems at the past with Linux, I did some benchmarks.
Tune your BSD: http://www.sysadminmag.com/articles/2001/0108/0108q/0108q.htm Oleg. ---- Oleg Broytmann http://www.zope.org/Members/phd/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.
I've read this article, as well as http://www.daemonnews.org/200108/benchmark.html and FreeBSD's tuning man page http://www.freebsd.org/cgi/man.cgi?query=tuning&manpath=FreeBSD+5.0-current In my case howver they did not affect much the performance On Fri, 17 Aug 2001, Oleg Broytmann wrote:
On Fri, 17 Aug 2001, Giorgos Verigakis wrote:
Hello, I'm using Zope on FreeBSD and OpenBSD and I've noticed some serious performance issues. Since I hadn't experienced any problems at the past with Linux, I did some benchmarks.
Tune your BSD: http://www.sysadminmag.com/articles/2001/0108/0108q/0108q.htm
Oleg. ---- Oleg Broytmann http://www.zope.org/Members/phd/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.
Hi Giorgos, I'm not sure I understand your process here. Could you explain exactly what you mean by:
I ran the previous program and after about 1000 hits I reloaded the page and measured the time it gets to finish.
??? For what it's worth, I've not seen any performance problems on FreeBSD. -steve On Friday, August 17, 2001, at 09:25 AM, Giorgos Verigakis wrote:
I did the following experiment: I installed a binary distribution of python-1.5.2 (pkg_add or apt-get) and I compiled Zope-2.3.3+HotFix from sources (with wo_pcgi.py). I entered Zope's management screen and expanded all folders. I ran the previous program and after about 1000 hits I reloaded the page and measured the time it gets to finish. The results are below:
|---------------------+------------+---------------+-------------+ | OS | Hit Rate* | Real hit rate | Reload time | | | (hits/sec) | (hits/sec) | (sec) | |---------------------+------------+---------------+-------------+ | | 30 | 20 | 13 | | Linux debian 2.2.19 | 50 | 33 | 16 | | | 100 | 38 | 16 | |---------------------+------------+---------------+-------------+ | | 30 | 20 | 10 | | FreeBSD 4.3-Release | 50 | 33 | 23 | | | 100 | 40 | 100 | |---------------------+------------+---------------+-------------+ | | 30 | 20 | 10 | | OpenBSD 2.9-Release | 50 | 33 | 15 | | | 100 | 50 | 285 | |---------------------+------------+---------------+-------------+ *The hit rate is the one passed as an argument to the program, while real hit rate was calculated according to Zope's log file.
On Fri, 17 Aug 2001, Steve Spicklemire wrote:
Hi Giorgos,
I'm not sure I understand your process here. Could you explain exactly what you mean by:
I ran the previous program and after about 1000 hits I reloaded the page and measured the time it gets to finish.
???
Hi, The small program I wrote continusly requests a URL from a web server. I let it run until it has made about 1000 requests (so that the server will get loaded) and then I try to load a not-too small page (while the program continues to run). The reason I used the management screens with all the folders expanded was so that it will be bigger in size and would be easy to be reproduced by anyone in this list.
For what it's worth, I've not seen any performance problems on FreeBSD.
Have you tested it on heavy load? If it's not a big problem, maybe you can run my program and tell me if you experience similar performance issues.
-steve
On Friday, August 17, 2001, at 09:25 AM, Giorgos Verigakis wrote:
I did the following experiment: I installed a binary distribution of python-1.5.2 (pkg_add or apt-get) and I compiled Zope-2.3.3+HotFix from sources (with wo_pcgi.py). I entered Zope's management screen and expanded all folders. I ran the previous program and after about 1000 hits I reloaded the page and measured the time it gets to finish. The results are below:
|---------------------+------------+---------------+-------------+ | OS | Hit Rate* | Real hit rate | Reload time | | | (hits/sec) | (hits/sec) | (sec) | |---------------------+------------+---------------+-------------+ | | 30 | 20 | 13 | | Linux debian 2.2.19 | 50 | 33 | 16 | | | 100 | 38 | 16 | |---------------------+------------+---------------+-------------+ | | 30 | 20 | 10 | | FreeBSD 4.3-Release | 50 | 33 | 23 | | | 100 | 40 | 100 | |---------------------+------------+---------------+-------------+ | | 30 | 20 | 10 | | OpenBSD 2.9-Release | 50 | 33 | 15 | | | 100 | 50 | 285 | |---------------------+------------+---------------+-------------+ *The hit rate is the one passed as an argument to the program, while real hit rate was calculated according to Zope's log file.
Hi Giorgos, On Friday, August 17, 2001, at 10:19 AM, Giorgos Verigakis wrote:
Have you tested it on heavy load? If it's not a big problem, maybe you can run my program and tell me if you experience similar performance issues.
-steve
On Friday, August 17, 2001, at 09:25 AM, Giorgos Verigakis wrote:
Yes.. I think I probably can, but maybe not today. One thing I do find interesting about your numbers is that FreeBSD and OpenBSD both have somewhat *higher* real hit rates under load than Linux, but the response time for an individual request (esp your big request) is longer. This suggests to me that your "big" request is being sacrificed at the alter of all these other hits. (it's especially interesting that OpenBSD appears to have 25% higher "real hit rate", and at the same time seems to be unable to deliver your browser a complex page in less than about 5 minutes!) I also think that seeing a complex page rendered may be bit too subjective a measure for my taste. It might be more instructive to run two concurrent benchmarking programs, such as 'ab'. One fetching simple urls at a high rate another fetching some highly complex dynamic page. It seems to me that these two kinds of pages stress different parts of Zope. The simple pages stress ZServer and the I/O bottleneck, the complex processes just take a lot of processing internally before ZServer gets a crack at delivering them to the "browser". It would also be interested to adjust the number of Zope threads. Anyway.. random mumblings. I will take a look at this! Thanks for bringing it up. -steve
I did the following experiment: I installed a binary distribution of python-1.5.2 (pkg_add or apt-get) and I compiled Zope-2.3.3+HotFix from sources (with wo_pcgi.py). I entered Zope's management screen and expanded all folders. I ran the previous program and after about 1000 hits I reloaded the page and measured the time it gets to finish. The results are below:
|---------------------+------------+---------------+-------------+ | OS | Hit Rate* | Real hit rate | Reload time | | | (hits/sec) | (hits/sec) | (sec) | |---------------------+------------+---------------+-------------+ | | 30 | 20 | 13 | | Linux debian 2.2.19 | 50 | 33 | 16 | | | 100 | 38 | 16 | |---------------------+------------+---------------+-------------+ | | 30 | 20 | 10 | | FreeBSD 4.3-Release | 50 | 33 | 23 | | | 100 | 40 | 100 | |---------------------+------------+---------------+-------------+ | | 30 | 20 | 10 | | OpenBSD 2.9-Release | 50 | 33 | 15 | | | 100 | 50 | 285 | |---------------------+------------+---------------+-------------+ *The hit rate is the one passed as an argument to the program, while real hit rate was calculated according to Zope's log file.
participants (4)
-
Chris McDonough -
Giorgos Verigakis -
Oleg Broytmann -
Steve Spicklemire