[Zope-dev] Where to put time.sleep(0.01) to get around solaris threading issue?

[Tony McDonald]

(this is a bit long - delete it if not interested in Solaris, at least it's
not HTML mail! :)

Hi,
I'm still trying to get our solaris box to work with more than one thread.

Briefly,
Put the following into a pythonscript;

## Script (Python) "sleeper"
##bind container=container
##bind context=context
##bind namespace=
##bind script=script
##bind subpath=traverse_subpath
##parameters=
##title=
##
## Warnings:
##  Prints, but never reads 'printed' variable.
##
# Example code:

# Import a standard function, and get the HTML request and response objects.
from Products.PythonScripts.standard import html_quote
from time import time

e= time() + float(15)
while time() < e: pass
print 'active sleep finished'
return 'slept'

Click 'test'. Try another request on that server. It will not run until the
pythonscript has finished. Replace 'sleeper' with a longish running request
that does real work and you've suddenly got a single-threaded server.

Ok, now try this python script on a non-solaris box (ie one that allows more
than a single thread to run, I'm using MacOS-X; (this script is from
/usr/local/lib/python2.1/test/test_thread.py)

# Very rudimentary test of thread module
# Create a bunch of threads, let each do some work, wait until all are done
#from test_support import verbose
verbose = 1
import random
import thread
import time
import sys

mutex = thread.allocate_lock()
rmutex = thread.allocate_lock() # for calls to random
running = 0
done = thread.allocate_lock()
done.acquire()

numtasks = 10

def task(ident):
    global running
    rmutex.acquire()
    delay = random.random() * numtasks
    rmutex.release()
    if verbose:
        print 'task', ident, 'will run for', round(delay, 1), 'sec'
    time.sleep(delay)
    if verbose:
        print 'task', ident, 'done'
    mutex.acquire()
    running = running - 1
    if running == 0:
        done.release()
    mutex.release()

next_ident = 0
def newtask():
    global next_ident, running
    mutex.acquire()
    next_ident = next_ident + 1
    if verbose:
        print 'creating task', next_ident
    thread.start_new_thread(task, (next_ident,))
    running = running + 1
    mutex.release()

for i in range(numtasks):
    newtask()

print 'waiting for all tasks to complete'
done.acquire()
print 'all tasks done'

sys.exit()

% [MacOS-X] python2.1 test_thread.py
reating task 1
task 1 will run for 7.8 sec
creating task 2
task 2 will run for 5.6 sec
creating task 3
task 3 will run for 6.3 sec
creating task 4
task 4 will run for 4.2 sec
creating task 5
task 5 will run for 9.8 sec
creating task 6
task 6 will run for 2.8 sec
creating task 7
task 7 will run for 0.2 sec
creating task 8
task 8 will run for 3.6 sec
creating task 9
task 9 will run for 5.1 sec
creating task 10
task 10 will run for 3.0 sec
waiting for all tasks to complete
task 7 done
task 6 done
task 10 done
task 8 done
task 4 done
task 9 done
task 2 done
task 3 done
task 1 done
task 5 done
all tasks done

Do the same on solaris and you get this;
creating task 1
creating task 2
creating task 3
creating task 4
creating task 5
creating task 6
creating task 7
creating task 8
creating task 9
creating task 10
waiting for all tasks to complete
task 1 will run for 3.3 sec
task 2 will run for 3.5 sec
task 3 will run for 3.7 sec
task 4 will run for 5.8 sec
task 5 will run for 3.3 sec
task 6 will run for 0.0 sec
task 7 will run for 3.9 sec
task 8 will run for 6.6 sec
task 9 will run for 1.4 sec
task 10 will run for 7.9 sec
task 6 done
task 9 done
task 1 done
task 5 done
task 2 done
task 3 done
task 7 done
task 4 done
task 8 done
task 10 done
all tasks done

Now add the line 
time.sleep(0.01) before the thread_start.new_thread(task, (next_ident, ))
line in the newtask method above.
    if verbose:
        print 'creating task', next_ident
    time.sleep(0.01)
    thread.start_new_thread(task, (next_ident,))

And you get this on Solaris...
creating task 1
task 1 will run for 6.2 sec
creating task 2
creating task 3
task 2 will run for 5.9 sec
creating task 4
task 3 will run for 3.6 sec
creating task 5
task 4 will run for 1.8 sec
creating task 6
task 5 will run for 7.3 sec
creating task 7
task 6 will run for 3.9 sec
creating task 8
task 7 will run for 2.4 sec
creating task 9
task 8 will run for 7.5 sec
creating task 10
task 9 will run for 8.7 sec
waiting for all tasks to complete
task 10 will run for 1.7 sec
task 10 done
task 4 done
task 7 done
task 3 done
task 6 done
task 2 done
task 1 done
task 5 done
task 8 done
task 9 done
all tasks done

Ie pretty similar.

With this in mind, I've added time.sleep(0.01) lines to the following files
in a Zope 2.4.1 source tree.

304 [12:08] % find . -name "*.py" -print | xargs grep -i 'start_new'
./ZServer/PubCore/ZRendezvous.py:
thread.start_new_thread(ZServerPublisher,
./ZServer/medusa/contrib/bobo_handler.py:
thread.start_new_thread(self._continue_request,(sin,request))
./ZServer/medusa/monitor_client_win32.py:    thread.start_new_thread
(reader, (l, s, p))
./ZServer/medusa/thread/select_trigger.py:
thread.start_new_thread (thread_function, (tf, self.count, n))
./ZServer/medusa/thread/thread_channel.py:        thread.start_new_thread (
./lib/python/ZODB/bpthread.py:    start_new_thread=apply

And pointed a zope Data.fs that has the test methods at this new
distribution.

No joy. That is, testing the 'sleeper' script still blocks other requests
until it is finished, when those requests are then serviced.

Can anyone see what I'm doing wrong here?

Tone.
-- 
Dr Tony McDonald,  Assistant Director, FMCC, http://www.fmcc.org.uk/
The Medical School, Newcastle University Tel: +44 191 243 6140
A Zope list for UK HE/FE  http://www.fmcc.org.uk/mailman/listinfo/zope