[Zope] Re: How to best use Zope on SMP machines?

[Tres Seaver]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Andreas Jung wrote:
> 
> 
> --On 12. Juli 2006 15:54:29 +0000
> krokodylek at tenbit.pl wrote:
> 
>>
>> Hi :)
>>
>> I am to use Zope on a SMP machine which has 2 Xeon HT cpus (so 4 logical
>> processors). As far as I know Python is not multithreaded, so the full
>> potential of the server will not be used... How can I change that?
> 
> There is nothing you can change. One Zope instance or ZEO instance per CPU.
> 
>>
>> I was thinking about running a couple of identical instances connected
>> with ZEO and then using some load balancing between them.
> 
> THat's the standard way
> 
>> Do you think
>> this is a good idea?
> 
> Why shouldn't that be a good idea?
> 
>> How many instances should I use on a machine I have
>> described, four?

YMMV.  E.g.:

 - Try loading up four appserver instances, and compare throughput with
   two instances.

 - Over time, the SMP kernel should migrate each appserver process to
   its own CPU, once it notices thatt threads within each process end up
   contending for a per-process resource (the Python GIL).

 - Benchmark your stress test (against the load-balanced IP), keeping an
   eye on memory consumption.  I would recommend giving such a box at
   least 4 Gb of RAM (assuming four appservers running).

  - Play around with the zope.conf setting for Python's
    sys.setcheckinterval value (I don't recall the spelling right off).

  - Tweak the number of appserver threads, as well (e.g., maybe two
    instead of 4, or maybe 6).  You may need to dial up or down the
    ZODB connection pool size as well (It should be at least one or two
    connections larger than the number of appserver threads).

> Since hyperthreaded CPUs share one real CPU you won't give much performance
> over running four processes on two real CPUs...I haven't much
> performance improvements in using hyperthreading over  using real CPUs.

I have heard anecdotal evidence that running 4 appserver instances on a
dual-core dual-CPU box worked best in one application under load.  I
don't recall what number of threads they ended up using per appserver
instance, nor what checkinterval setting they settled on.



Tres.
- --
===================================================================
Tres Seaver          +1 202-558-7113          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.2 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFEtRtk+gerLs4ltQ4RAvzgAJ9sLxMgZB5r9E9fK1Q1sxIEvgstGgCgv49z
T/ugMuJrTbbOR3Y+6ggxOts=
=5Qs7
-----END PGP SIGNATURE-----