Unfair Linux Threads And Python Oh My
I have added my observation to the ZWiki devoted to Zope in production environments. http://www.zope.org/Members/tseaver/Projects/HighlyAvailableZope/UnfairLinux... I don't know if this addresses all our troubles but it might be worth investigating. I was able to verify, with the example python code, that this thread thing is an issue on two of our three server; the two that exhibit unexplained restarts. The third server which does not restart also does not suffer from this thread problem. Does anyone know how to change the scheduling policy for zope/python? In the quote from the LinuxThreads FAQ there appears to be an indication that SCHED_FIFO or SCHED_RR would more precisely define the scheduling behavior. Anyone know how to do that? Would it help? There is also some indication that this issue is addressed in pthreads 0.8. I've upgraded a workstation with glibc 2.1.3, which I believe should have ptheads 0.8, and the problem still exists. Any ideas? -- ------------------------------- tonyr@ep.newtimes.com Director of Web Technology New Times, Inc. -------------------------------
----- Original Message ----- From: Tony Rossignol <tonyr@ep.newtimes.com>
I have added my observation to the ZWiki devoted to Zope in production environments.
http://www.zope.org/Members/tseaver/Projects/HighlyAvailableZope/UnfairLinux ThreadsAndPythonOhMy What I infer from this that a single spinning thread can totally stop Zope on some machines. Couple this with the bug Jason Spisak reported today, which caused some DTML Methods to spin very tightly, and we may have the reason for some lockups. It would be *really* nice if Python had a mechanism to put restrictions on threads' use of CPU resources. Cheers, Evan @ digicool & 4-am
Can a few folks with BSD/Solaris/HPUX/whateverotherthanLinux boxes try the script that Tony found and see what it does on their machines? Evan Simpson wrote:
----- Original Message ----- From: Tony Rossignol <tonyr@ep.newtimes.com>
I have added my observation to the ZWiki devoted to Zope in production environments.
http://www.zope.org/Members/tseaver/Projects/HighlyAvailableZope/UnfairLinux ThreadsAndPythonOhMy
What I infer from this that a single spinning thread can totally stop Zope on some machines. Couple this with the bug Jason Spisak reported today, which caused some DTML Methods to spin very tightly, and we may have the reason for some lockups.
It would be *really* nice if Python had a mechanism to put restrictions on threads' use of CPU resources.
Cheers,
Evan @ digicool & 4-am
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
-- Chris McDonough Digital Creations Publishers of Zope - http://www.zope.org
Chris McDonough wrote:
Can a few folks with BSD/Solaris/HPUX/whateverotherthanLinux boxes try the script that Tony found and see what it does on their machines?
I'll give it a whack on HPUX tomorrow... -- In flying I have learned that carelessness and overconfidence are usually far more dangerous than deliberately accepted risks. -- Wilbur Wright in a letter to his father, September 1900
On Wed, 26 Apr 2000, Chris McDonough wrote: :Can a few folks with BSD/Solaris/HPUX/whateverotherthanLinux boxes try :the script that Tony found and see what it does on their machines? here on an old sun:
uname -a SunOS archiv 5.4 Generic_101945-32 sun4m sparc
tonys script prints one star, after a few (5-10) seconds top output looks like this: ---- last pid: 15721; load averages: 2.37, 1.18, 0.90 10:30:39 91 processes: 87 sleeping, 2 running, 2 on cpu Cpu states: 0.0% idle, 93.1% user, 6.9% kernel, 0.0% iowait, 0.0% swap Memory: 244M real, 528K free, 242M swap, 85M free swap PID USERNAME PRI NICE SIZE RES STATE TIME WCPU CPU COMMAND 15700 styria -5 0 2152K 1088K run 2:46 40.09% 31.36% python 15721 styria -15 0 2160K 924K run 0:25 13.47% 27.95% python 15720 styria -4 0 2160K 1488K cpu 0:21 11.63% 23.29% python 15713 styria -7 0 2184K 1284K cpu 0:15 3.80% 3.82% top ---- i then had to stop the experiment; we need that machine and it is overloaded anyway ;) on my linux box the test script printed 131 stars (54 in a second run) then stops with two python threads eating all the cpu (x86 linux, kernel 2.2.14) we have some hpux machines here also but getting python w/ threading to compile there seems a little difficult. peter. -- _________________________________________________ peter sabaini, mailto: sabaini@niil.at -------------------------------------------------
At 11:10 am +0200 26/4/00, Peter Sabaini wrote:
On Wed, 26 Apr 2000, Chris McDonough wrote:
:Can a few folks with BSD/Solaris/HPUX/whateverotherthanLinux boxes try :the script that Tony found and see what it does on their machines?
Ok. Firstly, on our main site; 252 % uname -a SunOS caddy 5.6 Generic_105181-17 sun4u sparc 253 % python thread.py (nothing, no stars) after a while, 'top' on another terminal gives; PID USERNAME THR PRI NICE SIZE RES STATE TIME CPU COMMAND 13601 bing 3 -15 0 2696K 2032K cpu/0 2:56 24.99% python then, on another sun 102 % uname -a SunOS tot 5.7 Generic sun4u sparc 103 % python thread.py *********** .... ********** (ie line after line of them) 'top' on another terminal gives; PID USERNAME THR PRI NICE SIZE RES STATE TIME CPU COMMAND 283 daemon 6 16 0 9208K 7928K run 257.6H 55.78% dwhttpd 15201 bing 4 26 0 4856K 2744K run 2:20 37.47% python Does this imply I should upgrade our main server to 5.7 (or take the plunge and go to Solaris 8?) TIA Tone. ------ Dr Tony McDonald, FMCC, Networked Learning Environments Project http://nle.ncl.ac.uk/ The Medical School, Newcastle University Tel: +44 191 222 5888 Fingerprint: 3450 876D FA41 B926 D3DD F8C3 F2D0 C3B9 8B38 18A2
participants (6)
-
Bill Anderson -
Chris McDonough -
Evan Simpson -
Peter Sabaini -
Tony McDonald -
Tony Rossignol