Python on multi-processor machines (Was Re: [Zope] Re: Windows vs. Linux)

[Dennis Allison]

I have been running Zope successfully on a SMP Athlon system having
ignored Matt's advice given below.  There do not appear to be correctness
issues in a SMP environment, only potential performance issues with
Python/Zope.  I wanted the multiple CPUs available for non-Python tasks
and was willing to accept the performance hit.

I do think that there is a need for a SMP friendly Python runtime.  The 
cost/performance ratios for small SMPs make them very attractive
platforms.

On Thu, 29 Aug 2002, Guido van Rossum wrote:

> > Paul Browning wrote:
> > > 
> > > I don't think you are missing anything. The Python GIL is a bit of
> > > a show-stopper - I've been suprised that this isn't more widely
> > > known. Hoping I'm wrong nonetheless ....
> 
> The GIL *is* widely known, and there's nothing that can be done about
> it (without redesigning all of Python's runtime from scratch, anyway).
> 
> To use Python on multiple processors, the best thing to do is to
> run multiple processes, rather than multiple threads.
> 
> --Guido van Rossum (home page: http://www.python.org/~guido/)
> 

> On Tue, 28 May 2002, Matthew T. Kromer wrote:
>
> I do *not* recommend running Zope on multiprocessor machines without
an ability to restrict Zope to execution on a single CPU. The reason for
this is that the Python Global Interpreter Lock is shared inside a Zope
process. However, threads in Python are backed by underlying OS threads.
Thus, Zope will create multiple threads, and each thread is likely to be
assigned to a different CPU by the OS scheduler. However, all CPUs but one
which are dispatching any given Zope process will have to then wait and
attempt to acquire the GIL; this process introduces significant latency
into Python and thus into Zope.

> Linux has no native mechanism for processor binding. In fact, there is
a CPU dispatch mask for processes, but there is no facility to set the
mask that I know of. Solaris can use a system command like 'pbind'
to bind a process to a particular CPU.