RE: CPU Affinity, was RE: [Zope] Linux vs. UNIX vs. BSD
Hey Matt, while we are on the subject, I just wanted to make sure I had something right (and figured you might know): I know that affinity helps a single Zope instance on an SMP box, but how about 2? With Python's global interpreter lock (or whatever makes Python/Zope best suited to bind to a given CPU) - am I still going to run into problems if I am using the same Python to run 2 Zope instances in different software/instance homes, with all respective processes for each given Zope bound (via affinity) to respective CPUs? Or is this okay? I have some existing 2P AthlonMP and Xeon boxes, and I'd like to get the most zoom out of their horsepower as possible... Any thoughts are greatly appreciated. Thanks, Sean -----Original Message----- From: Matthew T. Kromer [mailto:matt@zope.com] Sent: Wednesday, August 21, 2002 6:48 AM To: zope@zope.org Subject: Re: CPU Affinity, was RE: [Zope] Linux vs. UNIX vs. BSD Tim Hoffman wrote:
Thats what Solaris is for ;-) Processor affinity, processor sets, fair share scheduler etc....
Rgds
Tim
The downside of Solaris is that Python executes more or less at the same speed on all CPUs, primarily determined by clock speed. Thus, a 2 Ghz Pentium IV or Athlon blows away expensive SPARC CPUs for running Zope. -- Matt Kromer Zope Corporation http://www.zope.com/ _______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
sean.upton@uniontrib.com wrote:
Hey Matt, while we are on the subject, I just wanted to make sure I had something right (and figured you might know):
I know that affinity helps a single Zope instance on an SMP box, but how about 2? With Python's global interpreter lock (or whatever makes Python/Zope best suited to bind to a given CPU) - am I still going to run into problems if I am using the same Python to run 2 Zope instances in different software/instance homes, with all respective processes for each given Zope bound (via affinity) to respective CPUs? Or is this okay? I have some existing 2P AthlonMP and Xeon boxes, and I'd like to get the most zoom out of their horsepower as possible...
Any thoughts are greatly appreciated.
Thanks, Sean
As long as *each* Python process (and thus each GIL) is processor bound, you'll be OK. Any python that isnt pbound (on Solaris) or have its processor dispatch mask set (on Linux) on a multiprocessor machine will end up slowing down when the machine is partially idle. A machine that's going full-throttle isn't as bad, curiously enough -- because the other CPU's are busy doing real work, the GIL doesn't have as much opportunity to get shuffled between CPUs. On a MP box it's very important to set sys.setcheckinterval() up to a fairly large number, I recommend pystones / 50 or so. You can make the number higher still without much concern. There ought to be a practical upper limit for sys.setcheckinterval() (because eventually you'll enter a routine that yeilds the lock anyway) but I dont know what that is. I want to say the minimum bytecode path thru Zope is about 24,000 bytecodes, but that's just a WAG -- but that would represent the "insanely high" outer bound for sys.setcheckinterval(). Why mention the check interval? On a busy enough system, increasing sys.setcheckinterval() decreases the amount the GIL is released, and as such, helps so that you can run more than one Zope on a multiprocessor machine *without* processor binding. I found, when I did tests about 2 years ago (wow, long time ago!) that I got diminishing returns above pystones/50, but below that I got good results. You may discover that you can get performance close enough to processor binding that you chose not to bind. -- Matt Kromer Zope Corporation http://www.zope.com/
On Wednesday 21 Aug 2002 3:55 pm, Matthew T. Kromer wrote:
There ought to be a practical upper limit for sys.setcheckinterval() (because eventually you'll enter a routine that yeilds the lock anyway)
It would be interesting to instrument Python to measure the distribution of inter-yield bytecode counts.
participants (3)
-
Matthew T. Kromer -
sean.upton@uniontrib.com -
Toby Dickenson