RE: [Zope-dev] ZServer speed enhancement (I hope)
I got to poking around inside medusa's guts and said, "Hey, asyncore.poll() is the innermost loop, why not rewrite it in C?" So I did.
<snip> I hope that for heavily loaded sites with many connections that it will produce a noticable performance improvement. But then, you probably shouldn't try this on a heavily-loaded site yet, unless you can tolerate some downtime... If you want to make this a standard piece of Zope (or medusa), that's okay by me, just let me know.
Andy - Do have any sense yet of the speedup? Just some basic load tests using ab against a specific set of URLs with and without the patch would provide some not-very-scientific-but-still-useful feel for how big the win is... Brian Lloyd brian@digicool.com Software Engineer 540.371.6909 Digital Creations http://www.digicool.com
On Fri, 31 Mar 2000, Brian Lloyd wrote:
I got to poking around inside medusa's guts and said, "Hey, asyncore.poll() is the innermost loop, why not rewrite it in C?" So I did.
<snip> I hope that for heavily loaded sites with many connections that it will produce a noticable performance improvement. But then, you probably shouldn't try this on a heavily-loaded site yet, unless you can tolerate some downtime... If you want to make this a standard piece of Zope (or medusa), that's okay by me, just let me know.
Andy -
Do have any sense yet of the speedup? Just some basic load tests using ab against a specific set of URLs with and without the patch would provide some not-very-scientific-but-still-useful feel for how big the win is...
To be honest, no idea whatsoever. I simply do not have a heavily-loaded ZServer to try it on. But I may have some additional improvements later today. -- andy dustman | programmer/analyst | comstar.net, inc. telephone: 770.485.6025 / 706.549.7689 | icq: 32922760 | pgp: 0xc72f3f1d "Therefore, sweet knights, if you may doubt your strength or courage, come no further, for death awaits you all, with nasty, big, pointy teeth!"
Do have any sense yet of the speedup? Just some basic load tests using ab against a specific set of URLs with and without the patch would provide some not-very-scientific-but-still-useful feel for how big the win is...
To be honest, no idea whatsoever. I simply do not have a heavily-loaded ZServer to try it on. But I may have some additional improvements later today.
If you have apache installe then you probably have apache-bench (/usr/sbin/ab) which should be quite enough for rough performance tests. ----------- Hannu
I've whipped up another version: ftp://starship.python.net/pub/crew/adustman/_asyncore-0.0.1.tar.gz This is not a bug fix; no bugs found yet. It should offer slightly more speed improvement. This version depends on the fact (or at least, I hope it's a fact) that fileno() on an object in the socket_map does not change over time (modifies asyncore.py to store fileno() as the value in socket_map; previously always stored 1 as a placeholder). You will need to undo the patch to asyncore.py from the previous one to apply the new patch (another couple lines, pitiful, really). -- andy dustman | programmer/analyst | comstar.net, inc. telephone: 770.485.6025 / 706.549.7689 | icq: 32922760 | pgp: 0xc72f3f1d "Therefore, sweet knights, if you may doubt your strength or courage, come no further, for death awaits you all, with nasty, big, pointy teeth!"
I've whipped up another version:
ftp://starship.python.net/pub/crew/adustman/_asyncore-0.0.1.tar.gz
This is not a bug fix; no bugs found yet. It should offer slightly more speed improvement. This version depends on the fact (or at least, I hope it's a fact) that fileno() on an object in the socket_map does not change over time (modifies asyncore.py to store fileno() as the value in socket_map; previously always stored 1 as a placeholder).
You will need to undo the patch to asyncore.py from the previous one to apply the new patch (another couple lines, pitiful, really).
I was just trying to install this patch on my system (Debian Potato, Zope 2.1.6 from src) and ran into a couple of problems. Firstly it failed to import the library due to an fprint statement which I think is meant to be fprintf. I changed this and then the module loaded correctly however it then seemed to just die with no error or anything. After starting ope in read only mode (-r) it would start and be fine until you access it and then it would segfault. I really don't know where to start in debugging this and unfortunately don't have time at the moment :(. Just thought I would let you know about the problem. Cheers, Benno
On Sun, 2 Apr 2000, Ben Leslie wrote:
I've whipped up another version:
ftp://starship.python.net/pub/crew/adustman/_asyncore-0.0.1.tar.gz
This is not a bug fix; no bugs found yet. It should offer slightly more speed improvement. This version depends on the fact (or at least, I hope it's a fact) that fileno() on an object in the socket_map does not change over time (modifies asyncore.py to store fileno() as the value in socket_map; previously always stored 1 as a placeholder).
You will need to undo the patch to asyncore.py from the previous one to apply the new patch (another couple lines, pitiful, really).
I was just trying to install this patch on my system (Debian Potato, Zope 2.1.6 from src) and ran into a couple of problems.
Firstly it failed to import the library due to an fprint statement which I think is meant to be fprintf. I changed this and then the module loaded correctly however it then seemed to just die with no error or anything. After starting ope in read only mode (-r) it would start and be fine until you access it and then it would segfault.
I really don't know where to start in debugging this and unfortunately don't have time at the moment :(.
Jeez, you're right. I thought all the debugging crap was out of there. Just delete that statement. I'm amazed the stupid thing compiled. Look for the new version (soon) at: ftp://starship.python.net/pub/crew/adustman/_asyncore-0.0.2.tar.gz -- andy dustman | programmer/analyst | comstar.net, inc. telephone: 770.485.6025 / 706.549.7689 | icq: 32922760 | pgp: 0xc72f3f1d "Therefore, sweet knights, if you may doubt your strength or courage, come no further, for death awaits you all, with nasty, big, pointy teeth!"
Andy Dustman wrote:
Jeez, you're right. I thought all the debugging crap was out of there. Just delete that statement. I'm amazed the stupid thing compiled. Look for the new version (soon) at:
ftp://starship.python.net/pub/crew/adustman/_asyncore-0.0.2.tar.gz
Have you done any performance test yet ? ---------------- Hannu
participants (4)
-
Andy Dustman -
Ben Leslie -
Brian Lloyd -
Hannu Krosing