How to best use Zope on SMP machines?
Hi :) I am to use Zope on a SMP machine which has 2 Xeon HT cpus (so 4 logical processors). As far as I know Python is not multithreaded, so the full potential of the server will not be used... How can I change that? I was thinking about running a couple of identical instances connected with ZEO and then using some load balancing between them. Do you think this is a good idea? How many instances should I use on a machine I have described, four? Are there any cons of such a configuration? I am open to any suggestions on how to use the machine's power best :-) Best regards, Maciej
--On 12. Juli 2006 15:54:29 +0000 krokodylek@tenbit.pl wrote:
Hi :)
I am to use Zope on a SMP machine which has 2 Xeon HT cpus (so 4 logical processors). As far as I know Python is not multithreaded, so the full potential of the server will not be used... How can I change that?
There is nothing you can change. One Zope instance or ZEO instance per CPU.
I was thinking about running a couple of identical instances connected with ZEO and then using some load balancing between them.
THat's the standard way
Do you think this is a good idea?
Why shouldn't that be a good idea?
How many instances should I use on a machine I have described, four?
Since hyperthreaded CPUs share one real CPU you won't give much performance over running four processes on two real CPUs...I haven't much performance improvements in using hyperthreading over using real CPUs. -aj
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Andreas Jung wrote:
--On 12. Juli 2006 15:54:29 +0000 krokodylek@tenbit.pl wrote:
Hi :)
I am to use Zope on a SMP machine which has 2 Xeon HT cpus (so 4 logical processors). As far as I know Python is not multithreaded, so the full potential of the server will not be used... How can I change that?
There is nothing you can change. One Zope instance or ZEO instance per CPU.
I was thinking about running a couple of identical instances connected with ZEO and then using some load balancing between them.
THat's the standard way
Do you think this is a good idea?
Why shouldn't that be a good idea?
How many instances should I use on a machine I have described, four?
YMMV. E.g.: - Try loading up four appserver instances, and compare throughput with two instances. - Over time, the SMP kernel should migrate each appserver process to its own CPU, once it notices thatt threads within each process end up contending for a per-process resource (the Python GIL). - Benchmark your stress test (against the load-balanced IP), keeping an eye on memory consumption. I would recommend giving such a box at least 4 Gb of RAM (assuming four appservers running). - Play around with the zope.conf setting for Python's sys.setcheckinterval value (I don't recall the spelling right off). - Tweak the number of appserver threads, as well (e.g., maybe two instead of 4, or maybe 6). You may need to dial up or down the ZODB connection pool size as well (It should be at least one or two connections larger than the number of appserver threads).
Since hyperthreaded CPUs share one real CPU you won't give much performance over running four processes on two real CPUs...I haven't much performance improvements in using hyperthreading over using real CPUs.
I have heard anecdotal evidence that running 4 appserver instances on a dual-core dual-CPU box worked best in one application under load. I don't recall what number of threads they ended up using per appserver instance, nor what checkinterval setting they settled on. Tres. - -- =================================================================== Tres Seaver +1 202-558-7113 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFEtRtk+gerLs4ltQ4RAvzgAJ9sLxMgZB5r9E9fK1Q1sxIEvgstGgCgv49z T/ugMuJrTbbOR3Y+6ggxOts= =5Qs7 -----END PGP SIGNATURE-----
On 7/12/06, krokodylek@tenbit.pl <krokodylek@tenbit.pl> wrote:
I am to use Zope on a SMP machine which has 2 Xeon HT cpus (so 4 logical processors). As far as I know Python is not multithreaded, so the full potential of the server will not be used... How can I change that?
I was thinking about running a couple of identical instances connected with ZEO and then using some load balancing between them. Do you think this is a good idea? How many instances should I use on a machine I have described, four? Are there any cons of such a configuration?
I am open to any suggestions on how to use the machine's power best :-)
KISS. Just use Zope as anyone would use it. Hyperthreading is not that much of a gain, and actual sometimes it decreases overall performance. You would need a Hyperthreading-aware OS (don't know how far Linux is in this, but it is _not_ sufficient for the OS to schedule its processes/threads on the four "visible" processors). If you have one processor running Zope and one processor running the system and the rest, you will be pretty happy I think. Depending on your application you could also run two zope instances doing different things Start doing the fancy stuff if you happen to experience actual performance problems. Find the bottleneck. Do you know that the bottleneck will be on CPU scheduling? What happens, if your application on this server with this OS is i/o bound? You wasted your valuable time on premature optimization. With a simple application architecture you'll also be more efficient with troubleshooting, if any problems arise. --knitti
participants (4)
-
Andreas Jung -
knitti -
krokodylek＠tenbit.pl -
Tres Seaver