Guido van Rossum writes:
I do think that there is a need for a SMP friendly Python runtime.
Feel free to submit patches to Python. This was tried before, making many of the internal data structures thread-safe and adding fine-grained locks where necessary. The net effect was a 50% slow-down on uniprocessor machines running Linux. On Windows it was a bit better (Windows has more efficient low-level locks than Linux) but still a significant slowdown.
So whether there's a need or not, I believe we'll all have to cope. The multi-process approach works well. For certain specialized applications, it also works to write an extension module in C that releases the GIL around CPU intensive calculations (as long as those calculations don't touch any Python objects). I think, there are two issues:
It is well known that the GIL prevents a multi-threaded application to use the full potential of a multi-processor architecture. A Multi-process architecture should be used instead. However, what seem to be new: Even a multi-process architecture is far from optimal, unless the processes are explicitly bound to a single processor. Rumours say that otherwise the process is (unnecessarily) move to and fro the different processors which significantly reduce performance. Personnaly, I do not believe this out of hand. But, apparently, others have strong indications to this effect. Dieter