Python on multi-processor machines (Was Re: [Zope] Re: Windows vs. Linux)

[Dieter Maurer]

Guido van Rossum writes:
 > > I do think that there is a need for a SMP friendly Python runtime.
 > 
 > Feel free to submit patches to Python.  This was tried before, making
 > many of the internal data structures thread-safe and adding
 > fine-grained locks where necessary.  The net effect was a 50%
 > slow-down on uniprocessor machines running Linux.  On Windows it was a
 > bit better (Windows has more efficient low-level locks than Linux) but
 > still a significant slowdown.
 > 
 > So whether there's a need or not, I believe we'll all have to cope.
 > The multi-process approach works well.  For certain specialized
 > applications, it also works to write an extension module in C that
 > releases the GIL around CPU intensive calculations (as long as those
 > calculations don't touch any Python objects).
I think, there are two issues:

  It is well known that the GIL prevents a multi-threaded application
  to use the full potential of a multi-processor architecture.
  A Multi-process architecture should be used instead.

  However, what seem to be new: Even a multi-process architecture
  is far from optimal, unless the processes are explicitly bound
  to a single processor. Rumours say that otherwise the process
  is (unnecessarily) move to and fro the different processors
  which significantly reduce performance.
  Personnaly, I do not believe this out of hand. But, apparently,
  others have strong indications to this effect.

Dieter