Hi I have a problem with a site running on Zope 2.7.5. After a restart the site serves request happily for a while and then hangs, meaning: - The server processes are still alive (I can see them with ps) - New attempts of page views do not show in the Z2.log - Browser just reports time out (not that the site is not available) - DeadlockDebugger does not respond either There is no recognisable pattern looking at the last requests in the Z2.log before the site hangs. The last requests are mostly just view requests of pages, which display fine after a restart. I've tried packing the ZODB (which worked fine, but did not resolve the issue). I've run the fsrecover.py, but that didn't seem to find anything. I have moved the Data.fs to another machine where I have a Zope 2.7.8 and tried some tests there, but again couldn't find anything pointing towards a possible source of the problem. I have now enabled the event.log and got this entry on the last "hangup": 2006-09-06T19:44:34 ERROR(200) ZServer uncaptured python exception, closing channel <ZServer.HTTPServer.zhttp_channel connected 212.96.102.29:3207 at 0x9c0326c channel#: 149 requests:> (exceptions.ValueError:HTTP headers invalid (too long) (got: 8760 bytes, allowed 8192 bytes [/sw1/packages/python235/lib/python2.3/ asyncore.py|read|69] [/sw1/packages/python235/lib/python2.3/ asyncore.py|handle_read_event|390] [/sw1/packages/python235/lib/ python2.3/asynchat.py|handle_read|148] [/sw0/sbzope_3_2/lib/python/ ZServer/HTTPServer.py|collect_incoming_data|378]) But I don't really understand what it means. Any help or suggestions welcomed! Thanks DR
David wrote at 2006-9-7 12:35 +0100:
I have a problem with a site running on Zope 2.7.5.
After a restart the site serves request happily for a while and then hangs, meaning:
- The server processes are still alive (I can see them with ps) - New attempts of page views do not show in the Z2.log - Browser just reports time out (not that the site is not available) - DeadlockDebugger does not respond either
There is no recognisable pattern looking at the last requests in the Z2.log before the site hangs. The last requests are mostly just view requests of pages, which display fine after a restart.
I have seen behaviour like this with older Python versions. In addition, the Zope requests could only be killed with "kill -9". The behaviour I observed was caused by a Python bug, fixed in Python 2.3.5 and Python 2.4.1. A fatal signal triggers this bug on some operating systems (e.g. Linux 2.4 with "linuxthread"). -- Dieter
David wrote:
- Browser just reports time out (not that the site is not available)
Too many negatives inside your brackets, please clarify...
- DeadlockDebugger does not respond either
I find that hard to believe...
I've tried packing the ZODB (which worked fine, but did not resolve the issue).
No, can't see that helping here.
I've run the fsrecover.py, but that didn't seem to find anything.
Again, irrelevent here.
I have now enabled the event.log
?! You should ALWAYS be writing an event log... and got this entry on the last "hangup":
2006-09-06T19:44:34 ERROR(200) ZServer uncaptured python exception, closing channel <ZServer.HTTPServer.zhttp_channel connected 212.96.102.29:3207 at 0x9c0326c channel#: 149 requests:> (exceptions.ValueError:HTTP headers invalid (too long) (got: 8760 bytes, allowed 8192 bytes [/sw1/packages/python235/lib/python2.3/asyncore.py|read|69] [/sw1/packages/python235/lib/python2.3/asyncore.py|handle_read_event|390] [/sw1/packages/python235/lib/python2.3/asynchat.py|handle_read|148] [/sw0/sbzope_3_2/lib/python/ZServer/HTTPServer.py|collect_incoming_data|378])
Ah, so you're exposing Zope to the big wide internet without a protective proxy like Apache in from? d'oh. Read the error message. People are trying to hack your server. Zope tries to prevent against it, but I'd still stick a defensive proxying configuration of Apache in front and see if the problem goes away.. Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
Chris Withers wrote at 2006-9-8 07:46 +0100:
... and got this entry on the last "hangup":
2006-09-06T19:44:34 ERROR(200) ZServer uncaptured python exception, closing channel <ZServer.HTTPServer.zhttp_channel connected 212.96.102.29:3207 at 0x9c0326c channel#: 149 requests:> (exceptions.ValueError:HTTP headers invalid (too long) (got: 8760 bytes, allowed 8192 bytes [/sw1/packages/python235/lib/python2.3/asyncore.py|read|69] [/sw1/packages/python235/lib/python2.3/asyncore.py|handle_read_event|390] [/sw1/packages/python235/lib/python2.3/asynchat.py|handle_read|148] [/sw0/sbzope_3_2/lib/python/ZServer/HTTPServer.py|collect_incoming_data|378])
Ah, so you're exposing Zope to the big wide internet without a protective proxy like Apache in from?
I have seen this kind of error *ALTHOUGH* we do have Apache in front of Zope -- probably, because Apache is less stupid about the header size than Zope. -- Dieter
FWIW, this message is raised when ZServer detects that the headers are too long. It is intentional and can be controlled up or down using the "http-header-max-length" directive in zope.conf. - C On Sep 8, 2006, at 3:44 PM, Dieter Maurer wrote:
Chris Withers wrote at 2006-9-8 07:46 +0100:
... and got this entry on the last "hangup":
2006-09-06T19:44:34 ERROR(200) ZServer uncaptured python exception, closing channel <ZServer.HTTPServer.zhttp_channel connected 212.96.102.29:3207 at 0x9c0326c channel#: 149 requests:> (exceptions.ValueError:HTTP headers invalid (too long) (got: 8760 bytes, allowed 8192 bytes [/sw1/packages/python235/lib/python2.3/asyncore.py|read|69] [/sw1/packages/python235/lib/python2.3/asyncore.py| handle_read_event|390] [/sw1/packages/python235/lib/python2.3/asynchat.py|handle_read|148] [/sw0/sbzope_3_2/lib/python/ZServer/HTTPServer.py| collect_incoming_data|378])
Ah, so you're exposing Zope to the big wide internet without a protective proxy like Apache in from?
I have seen this kind of error *ALTHOUGH* we do have Apache in front of Zope -- probably, because Apache is less stupid about the header size than Zope.
-- Dieter _______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
participants (4)
-
Chris McDonough -
Chris Withers -
David -
Dieter Maurer