Hi Jim, first of all, thanks a lot for your quick response. On Thursday 06 December 2007, Jim Fulton wrote:
- You should avoid runaway processes. :) I'm actually quite serious.
Yes, we are trying hard. :-) We are currently using lovely.remotetask to export those calls even to a different server, where we run the code in forked subprocesses. After having played for a day with this problem, I came to the following conclusion: Python assumes that all used C-libraries are wrapped in a way that they are non-blocking and cannot lock up. When forking, you better know what you are doing. Based on your comment, I think you agree. :-)
- You can run multiple processes and monitor their progress -- killing processes that are stuck.
I think this is a really good idea that requires little software and not much setup either.
zc.z3monitor provides some output that makes this pretty straightforward.
Wow, zc.z3monitor -- and zc.ngi which I looked at too -- are very cool. How useful! I have been following the checkins and had an idea of what it was about, but it is definitely cooler than I thought. :-) However, some code calls a blocking operation, zc.z3monitor will be locked up as well. I guess then I only have to check whether I get connectivity or not. ;-)
- It might be interesting to see if Java or .Net give better control over threads. If they do, then this might make Zope ports to Jython or IronPython more interesting. (People who get upset by the Python GIL should already find these platforms interesting.)
Java has a lot more control over threads, but I still have found complaints that they left out some features for portability reasons. I could not immediately find an answer on whether blocking, run-away C-calls are handled correctly. I talked to Roy Mathew, a once at a time Java guru, and he said that Java has a time-limit it gives each thread for doing some work. (I am not supposed to quote him on that. ;-) So if that is true, then Java does not have the problem. Roy did mention, though, that debugging locked threads in Java is a common skill after you reached a certain level of Java Zen. I looked yesterday quiet a bit at Win32 threads. That part of the Win32 kernel seems pretty well thought out and from what I can tell you can have quiet a bit of control over the threads. Again, I am not sure how hanging threads are handled. BTW, there is a nice article on Python threads here (probably nothing new for you, Jim): http://linuxgazette.net/107/pai.html Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training