Re: [Zope] Re: Threading, dtml and performance?!
Hi Tony No, I havn't solved the problem but after restarting production (after 19 days) I noticed that the funny way the threadcount reacted. I'd say, that the threads don't get killed after they have finished. I also noticed that the memory use went down from 344Mb to about 26Mb. Now I've put a restart into my nightly cronjob. (Are there some reasons why Zope should not be restarted dayly?????) I've just tested the whole thing on a WinNT 4.0 PC with Zope 2.4.2 (binary distribution). No problems! The PC gets slower but can answer two browser instances parallel. If I do the same thing on Solaris (intel & sun servers!) it is impossible! Normally you have sortof a break in a loop where control is given to the other parallel threads. This seems not to happen on my source compile with Solaris. I'm not so firm on where to address this problem (or if the problem was already adressed elswhere e.g. python maillists). Therefore I'd really apreciate some help! ;-) I don't want to start thinking about a migration Linux or even (puke) Windows. On Solaris I'm still using 2.4.0. Does it make sense to upgrade? I've seen that in Zope 2.4.3b1 there was some issue with dtml-recursion? Greetings from a rany Cologne! Oliver -- GMX - Die Kommunikationsplattform im Internet. http://www.gmx.net
On 7/11/01 2:20 pm, "Oliver Erlewein" <oerlewein@gmx.de> wrote:
Hi Tony
No, I havn't solved the problem but after restarting production (after 19 days) I noticed that the funny way the threadcount reacted. I'd say, that the threads don't get killed after they have finished. I also noticed that the memory use went down from 344Mb to about 26Mb. Now I've put a restart into my nightly cronjob. (Are there some reasons why Zope should not be restarted dayly?????)
Not that I'm aware of. I know some people do restart their servers daily. I had thought about doing that when I had *massive* memory bloat on a machine with a small amount of memory. However, we now have a machine with more memory, but the problem disappeared with a new zope release (could have been as early as the 2.2 series I think, I can't be sure though).
I've just tested the whole thing on a WinNT 4.0 PC with Zope 2.4.2 (binary distribution). No problems! The PC gets slower but can answer two browser instances parallel. If I do the same thing on Solaris (intel & sun servers!) it is impossible!
I've not tried on Intel Solaris, just Sparc. You're absolutely right though, nothing I have tried so far has enabled me to get more than one thread at a time serviced under a Solaris server.
Normally you have sortof a break in a loop where control is given to the other parallel threads. This seems not to happen on my source compile with Solaris. I'm not so firm on where to address this problem (or if the problem was already adressed elswhere e.g. python maillists).
I've brought it up on the main python list. Some people responded with suggestions and one person said they would try to get some threading fixes into a python 2.1.2 release. However, I grabbed some of the source code from the python 2.2b1 release (which gives test_thread.py output similar to that on my MacOS-X machine - which *does* service more than one thread) and compiled up a 2.1 hybrid python that gave some similar output to that of my MacOS-X and Solaris 2.2b1 python. I then built a 2.4.1 release from source, using this hybrid python and *still* found that I could only service one thread at a time. I don't know if I'm doing anything wrong or what, my knowledge of python innards is virtually nil. I don't know if I've linked against the right thread libraries or what.
Therefore I'd really apreciate some help! ;-) I don't want to start thinking about a migration Linux or even (puke) Windows.
Same here... :(
On Solaris I'm still using 2.4.0. Does it make sense to upgrade? I've seen that in Zope 2.4.3b1 there was some issue with dtml-recursion?
I've had some core dumps from 2.4.0, but not had time to do a 'big M' on them to see what's happening. We are also running 2.4.1, I've no plans to upgrade any further for the time being. The thread issue does not go away with Zope 2.4.1 though :(
Greetings from a rany Cologne! Oliver
And a cold Newcassle! Tone. -- Dr Tony McDonald, Assistant Director, FMCC, http://www.fmcc.org.uk/ The Medical School, Newcastle University Tel: +44 191 243 6140 A Zope list for UK HE/FE http://www.fmcc.org.uk/mailman/listinfo/zope
Hi all, As a suggestion for Solaris users (Im on linux) please be sure to check that your PYTHON thread is the culprit and NOT the database thread. They are indeed two separate issues. Why I say this is because I don't know much about Solaris, but on Redhat linux (kernel 2.4+) a ps -ef command returns the status of the database thread. What we experienced months ago was the very same issue. (as per this thread in its history). We had thought that python was the culprit (we used to be on Irix 6.5 which doesn't feedbak thread status). What it turned out to be was UPDATE WAITING or COMMIT WAITING was appended to the thread. Not python at all, but the database driver. When a stalled thread ocurrs, zope stalls because its waiting for the thread to close. This action is required because it supports the required sql locking schema. Otherwise you get ancient problems of updates impinging on updates causing bad data integrity. We had never experienced crashes, related directly to this, but we have had crashes. That at first made me think of this same problem, but then watching activity in the manage_interface/control_panel/debug/refresh+1second showed me a python process in a loop that caused the crash. That culprit was entirely bad programming practice on my part. I don't know if this blurb helps anyone, but I hope I can cause some ideas to occur per narrowing down the possibilities. It may also be worth noting that we still run zope 2.3.3 to this date because python2 just seems simply to be slower. Perhaps this weekend I'll take a second stab at building a test server on python2 and the latest zope. If I learn anything more, I'll update... Cheers, Paul Zwarts -----Original Message----- From: zope-admin@zope.org [mailto:zope-admin@zope.org] On Behalf Of Tony McDonald Sent: Wednesday, November 07, 2001 3:48 PM To: Oliver Erlewein Cc: zope@zope.org Subject: Re: [Zope] Re: Threading, dtml and performance?! On 7/11/01 2:20 pm, "Oliver Erlewein" <oerlewein@gmx.de> wrote:
Hi Tony
No, I havn't solved the problem but after restarting production (after 19 days) I noticed that the funny way the threadcount reacted. I'd say, that the threads don't get killed after they have finished. I also noticed that the memory use went down from 344Mb to about 26Mb. Now I've put a restart into my nightly cronjob. (Are there some reasons why Zope should not be restarted dayly?????)
Not that I'm aware of. I know some people do restart their servers daily. I had thought about doing that when I had *massive* memory bloat on a machine with a small amount of memory. However, we now have a machine with more memory, but the problem disappeared with a new zope release (could have been as early as the 2.2 series I think, I can't be sure though).
I've just tested the whole thing on a WinNT 4.0 PC with Zope 2.4.2 (binary distribution). No problems! The PC gets slower but can answer two browser instances parallel. If I do the same thing on Solaris (intel & sun servers!) it is impossible!
I've not tried on Intel Solaris, just Sparc. You're absolutely right though, nothing I have tried so far has enabled me to get more than one thread at a time serviced under a Solaris server.
Normally you have sortof a break in a loop where control is given to the other parallel threads. This seems not to happen on my source compile with Solaris. I'm not so firm on where to address this problem (or if the problem was already adressed elswhere e.g. python maillists).
I've brought it up on the main python list. Some people responded with suggestions and one person said they would try to get some threading fixes into a python 2.1.2 release. However, I grabbed some of the source code from the python 2.2b1 release (which gives test_thread.py output similar to that on my MacOS-X machine - which *does* service more than one thread) and compiled up a 2.1 hybrid python that gave some similar output to that of my MacOS-X and Solaris 2.2b1 python. I then built a 2.4.1 release from source, using this hybrid python and *still* found that I could only service one thread at a time. I don't know if I'm doing anything wrong or what, my knowledge of python innards is virtually nil. I don't know if I've linked against the right thread libraries or what.
Therefore I'd really apreciate some help! ;-) I don't want to start thinking about a migration Linux or even (puke) Windows.
Same here... :(
On Solaris I'm still using 2.4.0. Does it make sense to upgrade? I've
seen
that in Zope 2.4.3b1 there was some issue with dtml-recursion?
I've had some core dumps from 2.4.0, but not had time to do a 'big M' on them to see what's happening. We are also running 2.4.1, I've no plans to upgrade any further for the time being. The thread issue does not go away with Zope 2.4.1 though :(
Greetings from a rany Cologne! Oliver
And a cold Newcassle! Tone. -- Dr Tony McDonald, Assistant Director, FMCC, http://www.fmcc.org.uk/ The Medical School, Newcastle University Tel: +44 191 243 6140 A Zope list for UK HE/FE http://www.fmcc.org.uk/mailman/listinfo/zope _______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
On 7/11/01 3:02 pm, "Paul Zwarts" <paz@oratrix.com> wrote:
Hi all,
As a suggestion for Solaris users (Im on linux) please be sure to check that your PYTHON thread is the culprit and NOT the database thread. They are indeed two separate issues. Why I say this is because I don't know much about Solaris, but on Redhat linux (kernel 2.4+) a ps -ef command returns the status of the database thread.
Good point Paul - thing is, this threading issue occurs without using a database adaptor. The original post used some standard DTML that did a Catalog search to demonstrate the problem. A different version of the DTML method used a ._range() construct to do much the same thing.
What we experienced months ago was the very same issue. (as per this thread in its history). We had thought that python was the culprit (we used to be on Irix 6.5 which doesn't feedbak thread status). What it turned out to be was UPDATE WAITING or COMMIT WAITING was appended to the thread. Not python at all, but the database driver. When a stalled thread ocurrs, zope stalls because its waiting for the thread to close. This action is required because it supports the required sql locking schema. Otherwise you get ancient problems of updates impinging on updates causing bad data integrity.
That would certainly make sense.
We had never experienced crashes, related directly to this, but we have had crashes. That at first made me think of this same problem, but then watching activity in the manage_interface/control_panel/debug/refresh+1second showed me a python process in a loop that caused the crash. That culprit was entirely bad programming practice on my part.
Thing is, if you try that on Solaris, you get nowhere, as the python process in a loop grabs the only thread you've got! :)
I don't know if this blurb helps anyone, but I hope I can cause some ideas to occur per narrowing down the possibilities.
I think that ideas are about the only thing that is driving this forward at the moment, well, everything *I've* tried has got me nowhere so far. I'm getting totally exasperated with this, is this the right list to be even discussing this I wonder, zope-dev maybe?
It may also be worth noting that we still run zope 2.3.3 to this date because python2 just seems simply to be slower. Perhaps this weekend I'll take a second stab at building a test server on python2 and the latest zope. If I learn anything more, I'll update...
Cheers, Paul Zwarts
That'd be great. Thanks for the thoughts, Tone. -- Dr Tony McDonald, Assistant Director, FMCC, http://www.fmcc.org.uk/ The Medical School, Newcastle University Tel: +44 191 243 6140 A Zope list for UK HE/FE http://www.fmcc.org.uk/mailman/listinfo/zope
participants (3)
-
Oliver Erlewein -
Paul Zwarts -
Tony McDonald