[Zope] Re: Threading, dtml and performance?!
Tony McDonald
tony.mcdonald@ncl.ac.uk
Fri, 26 Oct 2001 13:13:20 +0100
On 26/10/01 8:42 am, "Oliver Erlewein" <oerlewein@gmx.de> wrote:
> Well I've tested your script with an old backup of Zope 2.3.2 on an Win N=
T
> SP 5 machiene. There are no problems. The script runs in one window and i=
n the
> other I can surf the management screen. My Solaris can't do that.
>=20
Neither can mine.
> On Solaris and on NT I could see your second problem. When I press cancel=
on
> the Browser the process/thread doesn't stop. Thats really a problem. I
> started your script twice and the machiene (NT) is nearly not usable anym=
ore.
> But
> thats better than Solaris - there I can't start the process twice!
>=20
Same situation here.
> For me I've prooven that there is something VERY WEIRD with the Solaris
> Zope. OK it might not be Zope but Python. As for me I need an answer /
> solution
> desperately!=20
I'm not going to guess whether it's a combination of Zope/Python/Solaris.
I'm not a python-guru. What I am going to do is go over what I've tried.
Warning: I don't have a solution :(
I've scoured the python lists and found that there does seem to be a proble=
m
with threads where Python and Solaris are involved. This is present in *all=
*
versions of python up until version 2.2b1
This is a test on my MacOS-X python2.1 installation (only the first bit is
shown);
162[12:49]tonymcd@orion ~/zope % python
/usr/local/lib/python2.1/test/test_thread.py
creating task 1
task 1 will run for 9.2 sec
creating task 2
task 2 will run for 0.5 sec
creating task 3
task 3 will run for 1.7 sec
creating task 4
task 4 will run for 7.5 sec
creating task 5
task 5 will run for 0.8 sec
creating task 6
task 6 will run for 9.0 sec
creating task 7
task 7 will run for 4.7 sec
creating task 8
task 8 will run for 3.1 sec
creating task 9
task 9 will run for 6.6 sec
creating task 10
task 10 will run for 0.1 sec
waiting for all tasks to complete
task 10 done
task 2 done
task 5 done
task 3 done
task 8 done
task 7 done
task 9 done
task 4 done
task 6 done
task 1 done
all tasks done
Now try that under Solaris;
201 [12:52] % python2.1 /usr/local/lib/python2.1/test/test_thread.py
creating task 1
creating task 2
creating task 3
creating task 4
creating task 5
creating task 6
creating task 7
creating task 8
creating task 9
creating task 10
waiting for all tasks to complete
task 1 will run for 5.4 sec
task 2 will run for 3.0 sec
task 3 will run for 9.3 sec
task 4 will run for 8.2 sec
task 5 will run for 8.9 sec
task 6 will run for 0.0 sec
task 7 will run for 6.0 sec
task 8 will run for 2.5 sec
task 9 will run for 8.7 sec
task 10 will run for 6.6 sec
task 6 done
task 8 done
task 2 done
task 1 done
task 7 done
task 10 done
task 4 done
task 9 done
task 5 done
task 3 done
all tasks done
(It would be interesting to see what happens under Linux or a Windows-OS I
think).
Ok, so 2.2b1 should be better...
the relevant section from;
http://sourceforge.net/project/shownotes.php?group_id=3D5470&release_id=3D57697
Is
- Thread scheduling on Solaris should be improved; it is no longer
necessary to insert a small sleep at the start of a thread in order
to let other runnable threads be scheduled.
So I downloaded the 2.2b1 source distribution and compiled it up under
Solaris.
I did the same test as above;
202 [12:55] % python2.2 /usr/local/lib/python2.2/test/test_thread.py
creating task 1
creating task 2
creating task 3
creating task 4
task 1 will run for 5.7 sec
task 4 will run for 1.7 sec
creating task 5
creating task 6
creating task 7
task 2 will run for 7.5 sec
task 3 will run for 4.2 sec
task 5 will run for 1.2 sec
creating task 8
task 6 will run for 3.9 sec
task 8 will run for 2.5 sec
creating task 9
creating task 10
waiting for all tasks to complete
task 7 will run for 2.4 sec
task 9 will run for 8.3 sec
task 10 will run for 3.2 sec
task 5 done
task 4 done
task 7 done
task 8 done
task 10 done
task 6 done
task 3 done
task 1 done
task 2 done
task 9 done
all tasks done
Encouraging, but not fantastic (I think - my ignorance of threading is
showing here).
Ok, the acid test - I got a clean Zope 2.4.1 installation and did a;
% python2.2 wo_pcgi.py
installation. I then started Zope up (obviously with python 2.2) and got
this;
208 [12:58] % ./start
/usr/local/lib/python2.2/FCNTL.py:7: DeprecationWarning: the FCNTL module i=
s
deprecated; please use fcntl
DeprecationWarning)
/usr/local/lib/python2.2/FCNTL.py:7: DeprecationWarning: the FCNTL module i=
s
deprecated; please use fcntl
DeprecationWarning)
Etc.
I also got a core dump from python2.2
Thinking it might be a 2.4.2 issue, I built that from scratch as well (usin=
g
python 2.2). I got the same result.
> Any ideas anybody? My primary guess is, that Solaris doesn't
> relese the threads after using them. I've tried a threaded Python script =
on my
> Solaris machiene and that had no problems. But I must confess I have no c=
lue
> on
> how Zope Threading works. Therefore my test doesn't proove anything reall=
y.
>=20
I have no idea what might be causing this. I'm not a Solaris kernel hacker
or anything like that.
It seems that zope and python2.2 don't want to play anyhow (that's a little
unfair - python 2.2 is a beta at the moment).
So where am I?
1) I can run Zope 2.4.1 and python 2.1 happily - as long as I understand
it's effectively single-threaded.
2) python 2.2 seems to have the fixes for threading problems under Solaris
3) I can't run Zope 2.4.1/2.4.2 and python 2.2b1 under Solaris at the momen=
t
...and that's it.
I don't expect Zope to be python 2.2 compatible until sometime after 2.2
final is released. If that sorts out my threading problem, I'll be a happy
camper. The thing is *I don=B9t know* if that will fix the problems I'm
seeing.
> By the way...it also shows how fast Zope really is because if I run
> "singlethreaded" and Zope can still serve a few thousand requests a day t=
hats
> really
> not bad.
>=20
Yeah, this is ok, but one of our main sites is used by 50 students at a tim=
e
- this is where the real single-threaded nature of Zope/Solaris is *painful=
*
to watch.
There must be *someone* out there who's using a Solaris Zope-based site tha=
t
has got this fixed? Mustn't there? Anyone? (gad!, I sound like a lonely
hearts ad.)
> Oliver Erlewein
This is really getting me down :(
Tone.
--=20
Dr Tony McDonald, Assistant Director, FMCC, http://www.fmcc.org.uk/
The Medical School, Newcastle University Tel: +44 191 243 6140
A Zope list for UK HE/FE http://www.fmcc.org.uk/mailman/listinfo/zope