Multiple threads using Apache Rewrite
We have a machine (FreeBSD) set up running Apache to dish up 'normal' websites on port 80, and we have been developing a couple of Zope based Intranet type sites, so we have ZServer running on its default port of 8080. We're using Apache mod-rewrite to transaprently proxy Zope request to the ZServer. All is sweet. That is until a request that ties ZServer up for a period of time is closely followed by a second request - the result from Apache is a 'proxy error' - i.e. there was no server visible to respond to the second proxy request. Running ps -aux only ever shows two Zope processes, no matter how hard one is working. Isn't there supposed to be a monitor process running which forks new processes when existing ones get busy, like Apache does? Both Apache and Zope were installed from FreeBSD Ports using default startup scripts. Advice gratefully received. Jim Whiteside.
On Thu, 2004-01-08 at 13:59, jim@jaydublu.com wrote:
Running ps -aux only ever shows two Zope processes, no matter how hard one is working. Isn't there supposed to be a monitor process running which forks new processes when existing ones get busy, like Apache does?
Jim, No the monitor process does not spawn additional processes/threads, it just makes sure that Zope is still running. Zope internally uses python threads (mixed with an event loop in asyncore.py) to handle requests. We also run FreeBSD and have noticed some threading issues recently. I did a study about a year ago into Solaris threading issues an Zope and was comparing it to FreeBSD. At the time the threading was definately working on FreeBSD. Since then though we have upgraded Zope (2.3.x -> 2.6.2), python (1.5.2 -> 2.1.3), and FreeBSD (4.x to 4.9-STABLE) and added ZEO and DirectoryStorage into the mix, so I don't know when and what has broken it! :( I have tried replicating the setup, but with python 2.3.3 and Zope 2.7.x and at first experienced the same problems, but on a second try they seemed to go away. The main symptom was if you did a long running I/O intensive task (reads, not writes) such as doing a Zope Find on a large ZODB then all other threads would hang. In the end I have worked around it by recompiling python 2.1.3 with LinuxThreads support (look at the README in the LinuxThreads port) and it has been running fine now for a few months. -Matt -- Matt Hamilton matth@netsight.co.uk Netsight Internet Solutions, Ltd. Business Vision on the Internet http://www.netsight.co.uk +44 (0)117 9090901 Web Hosting | Web Design | Domain Names | Co-location | DB Integration
Jim, Matt, Did you use python port to install the python interpreter? If not, did you apply the patch that fix the thread stack size issue? BTW: I'm quite fed up with 4.X FreeBSD userthread implementation + python. I've messed with a fork/wait python program until I resolved the strange hangs recompiling python without threads support. Ciao Paolo Matt Hamilton wrote:
In the end I have worked around it by recompiling python 2.1.3 with LinuxThreads support (look at the README in the LinuxThreads port) and it has been running fine now for a few months.
On Thu, Jan 08, 2004 at 03:23:13PM +0100, Paolo Losi wrote:
Jim, Matt, Did you use python port to install the python interpreter? If not, did you apply the patch that fix the thread stack size issue? I think Paolo talks about the BSD-patches discussed at http://www.zwiki.org/PatchPython213
Note that http://www.zettai.net/Members/george/python213FreeBSDPlone.blog/view has a comment listing: FreeBSD's Python ports now includes a Makefile option called "WANT_HUGE_STACK_SIZE", which if you set will you give you a really big stack size (~1MB). I have not extensively tested with this option but Fabiano Weimar dos Santos of x3ng.com.br reports a lot of success with it. Thanks Fabiano! PieterB -- http://zwiki.org/PieterB
jim@jaydublu.com on 1/8/04 wrote:
Running ps -aux only ever shows two Zope processes, no matter how hard one is working. Isn't there supposed to be a monitor process running which forks new processes when existing ones get busy, like Apache does?
Freebsd shows processes differently. zope could be running 8 threads but that will only show as one process on freebsd. you can attach gdb to the zope proc to see the threads. The second smaller (wrt RAM) process that you see is the watchdog daemon. <--> george donnelly ~ http://www.zettai.net/ ~ "Quality Zope Hosting" Shared and Dedicated Zope Hosting ~ Zope Servers ~ Zope Websites Yahoo, AIM: zettainet ~ ICQ: 51907738 ~ Sales (USA): 1-866-967-3669
participants (5)
-
george donnelly -
jim＠jaydublu.com -
Matt Hamilton -
Paolo Losi -
PieterB