Multithreading issue - python, zope, opensbd, et al
Hi all, Have scanned lots of places to find a clear answer, so as a last resort, am asking the group.... First assumption - if multithreading was enabled, we would see a number of python/zope tasks running (ps, top, etc) when numerous simultaneous requests come in... right? (as we do with apache - lots of httpd processes). We are using python/zope on both Solaris and openBSD, and in looking at the detail log of Python configure on either, I see compile and then loader failures (different details for each OS though). Meanwhile the python build process goes merrily along and creates a python evironment, unbeknown to us at the time. (yes I know one shouls scour the logs, but time is always an issue...). So with openBSD, there seems a requirement for both pthread.h (found) and thread.h (not found). I assume with these that if they wre both found we might expect multithreading to work. I've also noted in passing: - similar problems have arisen with MySQL/Python environments... - something about locking in python that might prevent multithreading anyway (maybe a red herring). Versions in use: openBSD 3.0, 3.1, Solaris 8 python 2.1.3 Zope 2.5.0, 2.5.1 I note that for openBSD 3.0 there is a thread package for python 2.1.1 but I assume that the crash issues with Zope from a few months ago preclude me from using this. Any advice gratefully accepted - like many people I don't know everything. :) cheers Graham
Graham, I'm no expert, by any means, but I was involved in a thread last year re: Solaris/threading w/ Zope. Firstly...
First assumption - if multithreading was enabled, we would see a number of python/zope tasks running (ps, top, etc) when numerous simultaneous requests come in... right? (as we do with apache - lots of httpd processes).
Processes != threads One of the reasons Apache 1 has so many processes running is because it is actually fairly poor at multithreading... Ideally, you'd like to have fewer processes than threads because there is usually more overhead associated with creating a process than a thread. Improved multithreading (the developers are using phrases like "vastly improved") is one of the main benefits of Apache 2. So, by extension, the number of processes you see running is not indicative of Zope/python threading ... IIRC, the way we tested Zope threading in the earlier discussion was by having Zope run a Python method which is guaranteed to hog cycles and check to see if the site still responded... As luck would have it, I deleted the test script about one week ago... Luckily, the Internet never forgets and the entire exchange is still here: http://zope.nipltd.com/public/lists/zope-archive.nsf/Main/95DE5A56E970374480 256B14006FAC96?OpenDocument&mode=subject&topic=[Zope]BuildingZopew/SunForte( ratherthangcc) Another - probably better - test is to use the Python-supplied threading test... You should find this in /path/to/python/lib/python2.1/test/test_thread.py Run the test simply by doing "python /path/to/python/lib/python2.1/test/test_thread.py". You should see output like this: creating task 1 creating task 2 ... creating task 10 waiting for all tasks to complete task 1 will run for 2.1 sec task 2 will run for 3.7 sec ... task 10 will run for 7.1 sec task 7 done task 1 done ... task 5 done all tasks done If you see the "task # done"'s come up out of numerical order (they should complete in chronological order), this is a pretty good indication that python threading is working fine. Also, it's been noted that Solaris threads work differently from Linux threads... Linux will start threads almost immediately, while Solaris seems to queue them up and start them at some convenient time...
We are using python/zope on both Solaris and openBSD, and in looking at the detail log of Python configure on either, I see compile and then loader failures (different details for each OS though). Meanwhile the python build process goes merrily along and creates a python evironment, unbeknown to us at the time. (yes I know one shouls scour the logs, but time is always an issue...).
I can't speak for OpenBSD, but I have had a good experience using the binary build of Zope for Solaris (it is built for 2.6, but I use it with 8 just fine). This comes with a pre-built/included python install - so you don't have to mess with installing python yourself.
So with openBSD, there seems a requirement for both pthread.h (found) and thread.h (not found). I assume with these that if they wre both found we might expect multithreading to work.
For OpenBSD, are you building Python from scratch? Have you tried a port/package? These will usually address major problems like threads, if any. I would try to install an OpenBSD package version of Python (or ports, though IIRC, OpenBSD doesn't have quite the ports system of the other BSDs), run the threads test, then tackle a source build of Zope. Hope this helps... Paul.
participants (2)
-
Graham King -
Paul Horbal