[Zope] SMP and Zope.
dario at ita.chalmers.se
Sat Mar 12 04:47:05 EST 2005
Jean-Francois.Doyon at CCRS.NRCan.gc.ca wrote:
> You don't HAVE to, but the optimal configuration is to have one interpreter
> per CPU, yes.
In my experience, you don't HAVE to in the same way that you don't HAVE
to sleep at all. However after a while, you start to experience very
very erratic behaviour, and things will start to work and not work
intermittently. In the end, you will end up going mad :-)
Waiting a while, like Tres suggested, and letting things fix themselves
by letting the CPU scheduler migrate processes to different CPU's will
not work consistently. Such a solution depends hihgly on the kind of CPU
load you have. High CPU load -> processes do not spread out over
multiple CPUs, no problem. Low CPU load -> processes may spread over
multiple CPUs, problem
There was a large disussion some years ago, actually more or less
exaclty 3 years ago, before Linux was as widespread as it is now, and
the issue was on Solaris machines, but eventually was identified as
relevant to all non-single-cpu architectures.
The second link contains a nice explanation of the problem by Matt Kromer.
What happens is that Zope sometimes feels like it's hung, however after
a while it starts repsonding again. This behaviour is intermittent. Let
be for a while and it starts to hang again and is suddenly responsive
again, and so on.
The issue is unavoidable in the standard case for normal multithreaded
zope-instances, since on SMP systems all processes are started unbound
to any cpu, and will intermitently be distributed across more than one
CPU depending on load, etc.
There are workarounds for linux/solaris, in solaris there is pbind, on
linux there is taskset (taskset is from the taskutils package,
http://directory.fsf.org/sysadmin/misc/schedutils.html for more info,
the source site is http://rlove.org/schedutils/).
Taskset is included in for instance Red Hat AS 3.0 (we use RHAS), but
requires kernel 2.5.8 or later, but there are patches for earlier
kernels). Chekc your distro, if it is recent enough, it may be allready
The advantage of using schedutils is that they are userspace tools, and
thus can be executed as part of your start script.
I whish that zopectl was able to be smart enough and have taskset or
> So if you start an instance, you should bind it to a specific CPU.
> It'll work fine even if you don't do it, but performance won't be optimal.
No it won't, but you *will* go crazy and not understand at all the
sloppy performance you get from your nice shiny, multiCPU, probably
really expensive machine.
> I've moved away from multi-cpu machine for my Zope because of this, makes
> things much easier.
it sure makes life easier :-). But if you have a multicpu machine, use
Dario Lopez-Kästen, IT Systems & Services Chalmers University of Tech.
"...and click? damn, I need to kill -9 Word again..." - b using macosx
More information about the Zope