pbind - maximising performance on a multiprocessor Solaris box (Was: ZMySQLDA Help )
Sorry. Only just caught up with this:
Tony McDonald wrote:
(replies to list as some of this might be interesting to them)
Agreed. I imagine quite alot of people missed this next bit because of the subject line.
Loads of Zope work, upgrading servers to 2.5.1/python 2.1.3, and therein lies a tale. You might have seen the interchanges on the list between fearless Paul Browning, magic Matt Hamilton
Overdoing the Newcastle Brown again?
and I about threading on Solaris
http://www.zope.org/Members/glpb/news2
Upshot there is that Python is not very happy on multiprocessors
For those of you that didn't catch it: -------------------- On Tue, 28 May 2002, Matthew T. Kromer wrote: I do *not* recommend running Zope on multiprocessor machines without an ability to restrict Zope to execution on a single CPU. The reason for this is that the Python Global Interpreter Lock is shared inside a Zope process. However, threads in Python are backed by underlying OS threads. Thus, Zope will create multiple threads, and each thread is likely to be assigned to a different CPU by the OS scheduler. However, all CPUs but one which are dispatching any given Zope process will have to then wait and attempt to acquire the GIL; this process introduces significant latency into Python and thus into Zope. Linux has no native mechanism for processor binding. In fact, there is a CPU dispatch mask for processes, but there is no facility to set the mask that I know of. Solaris can use a system command like 'pbind' to bind a process to a particular CPU. Full post at <http://zope.nipltd.com/public/lists/zope-archive.nsf/ByKey/381AE53C5F57B42 6> --------------------
hence I'm using the 'pbind' (solaris) command to allocate a processor to a particular Zope process. This has seriously made me look at Apple Xserve boxes (you know I have a penchant for the kind of stuff they make :) and the anguish from Bristol (PB's lair - a big solaris shop) has been heard all the way to the northeast!
Positive aspect of 2.5.1/2.1.3 is that our systems are much more stable, but I have a weird problem re:sessions that even has ChrisMcD baffled! (puffs out chest).
Tone: So how do you use pbind? Can you share a script or two? Thanks, Paul -- The Library, Tyndall Avenue, Univ. of Bristol, Bristol, BS8 1TJ, UK E-mail: paul.browning@bristol.ac.uk URL: http://www.bris.ac.uk/
On 5/7/02 4:05 pm, "Paul Browning" <paul.browning@bristol.ac.uk> wrote:
hence I'm using the 'pbind' (solaris) command to allocate a processor to a particular Zope process. This has seriously made me look at Apple Xserve boxes (you know I have a penchant for the kind of stuff they make :) and the anguish from Bristol (PB's lair - a big solaris shop) has been heard all the way to the northeast!
Tone:
So how do you use pbind? Can you share a script or two?
Thanks,
Paul
SYNOPSIS pbind -b processor_id pid ... pbind -u pid ... pbind [ -q ] [ pid ... ] DESCRIPTION pbind controls and queries bindings of processes to proces- sors. pbind binds all the LWPs (lightweight processes) of a process to a processor, or removes or displays the bindings. This may mean that you need to make sure the lwp libraries are used, ie lines like LD_LIBRARY_PATH='/usr/lib/lwp:/lib:/usr/local/lib:/usr/local/mysql/lib' export LD_LIBRARY_PATH Should be included in the 'start' script. The problem seems to be that the 'start' script starts off zope using an 'exec' command. This replaces the current process (a shell) with the zope process. Thus adding commands at the end of the script to grab the process id and 'pbind' it will be pointless as those commands wont get run... At the moment, I'm 'pbind-ing' manually :( Tone -- Dr Tony McDonald, Assistant Director, FMCC, http://www.fmcc.org.uk/ The Medical School, Newcastle University Tel: +44 191 243 6140 A Zope list for UK HE/FE http://www.fmcc.org.uk/mailman/listinfo/zope
--On 05 July 2002 16:39 +0100 Tony McDonald <tony.mcdonald@ncl.ac.uk> wrote: [snip] Yeah, I read the man page too. What I couldn't figure was how to automate the pbinding ....
The problem seems to be that the 'start' script starts off zope using an 'exec' command. This replaces the current process (a shell) with the zope process. Thus adding commands at the end of the script to grab the process id and 'pbind' it will be pointless as those commands wont get run...
At the moment, I'm 'pbind-ing' manually :(
.... snap! Do we need some sort of cron job that looks for the right sort of process and then pbinds it if it isn't pbound? Making-it-up-as-I-go-along-as-usually yours, Paul -- The Library, Tyndall Avenue, Univ. of Bristol, Bristol, BS8 1TJ, UK E-mail: paul.browning@bristol.ac.uk URL: http://www.bris.ac.uk/
There is no need to use 'exec'. If you start Zope like /usr/bin/env /usr/bin/python \ $ZOPE_HOME/z2.py \ >> $INSTANCE_HOME/var/Zope.log 2>&1 & the script continues and you can pbind later on. HTH, Stefan --On Freitag, 05. Juli 2002 16:39 +0100 Tony McDonald <tony.mcdonald@ncl.ac.uk> wrote:
The problem seems to be that the 'start' script starts off zope using an 'exec' command. This replaces the current process (a shell) with the zope process. Thus adding commands at the end of the script to grab the process id and 'pbind' it will be pointless as those commands wont get run...
At the moment, I'm 'pbind-ing' manually :(
-- BLOWFISH, n. - Preference for beef.
participants (3)
-
Paul Browning -
Stefan H. Holek -
Tony McDonald