A glitch :( It happens in zope 2.3.3 (python 1.5.2) and zope 2.4.0 (python 2.1.1) under OpenBSD 2.7. After a day or two of normal work zope simply stops responding to HTTP requests. When I telnet to 80 port it connects, but nothing else happens. What to do? How to know where it handgs and why? Is it possible to attach to zope process and see what's going there?.. -- Roman V. Isaev http://www.aroma.ru System Administrator P.S. I've set up a script that kills zope and restarts it at 7am, just before a workday begins, but it's not a solution %(
The thing to do here is to turn on "big M" logging (use the -M switch to z2.py)... this details all the transactions between clients and Zope. Then use the "requestprofiler.py" script that ships with Zope 2.4 (in the "utilities" directory) to give you a profile of the big M logs for the right before a hang... do this for a few days to determine if you can spot a pattern to the hangs. Roman V. Isaev wrote:
A glitch :( It happens in zope 2.3.3 (python 1.5.2) and zope 2.4.0 (python 2.1.1) under OpenBSD 2.7. After a day or two of normal work zope simply stops responding to HTTP requests. When I telnet to 80 port it connects, but nothing else happens. What to do? How to know where it handgs and why? Is it possible to attach to zope process and see what's going there?..
-- Chris McDonough Zope Corporation http://www.zope.org http://www.zope.com "Killing hundreds of birds with thousands of stones"
When zope hangs, -M file stalls too -- no changes after sending GET request. But in my nohup.out I found about 40 megs of these errors: 2001-08-20T09:53:57 PROBLEM(100) ZServer warning: server accept() threw an exception ------ 2001-08-20T09:53:57 PROBLEM(100) ZServer warning: server accept() threw an exception ------ 2001-08-20T09:53:57 PROBLEM(100) ZServer warning: server accept() threw an exception ------ 2001-08-20T09:53:57 PROBLEM(100) ZServer warning: server accept() threw an exception Next step? On 08/18, Chris McDonough wrote:
The thing to do here is to turn on "big M" logging (use the -M switch to z2.py)... this details all the transactions between clients and Zope. Then use the "requestprofiler.py" script that ships with Zope 2.4 (in the "utilities" directory) to give you a profile of the big M logs for the right before a hang... do this for a few days to determine if you can spot a pattern to the hangs.
Roman V. Isaev wrote:
A glitch :( It happens in zope 2.3.3 (python 1.5.2) and zope 2.4.0 (python 2.1.1) under OpenBSD 2.7. After a day or two of normal work zope simply stops responding to HTTP requests. When I telnet to 80 port it connects, but nothing else happens. What to do? How to know where it handgs and why? Is it possible to attach to zope process and see what's going there?..
-- Chris McDonough Zope Corporation http://www.zope.org http://www.zope.com "Killing hundreds of birds with thousands of stones"
-- Roman V. Isaev http://www.aroma.ru System Administrator
Hi Roman, This is a log message which is recorded in ZServer/medusa/http_server.py meaning that the system accept() call threw an error. This appears to happen at the OS layer, so I'm not entirely certain what the issue is. You might try instrumenting the place in the code where it throws the error with debugging output to get a better understanding of what the error actually is. HTH, - C "Roman V. Isaev" wrote:
When zope hangs, -M file stalls too -- no changes after sending GET request. But in my nohup.out I found about 40 megs of these errors:
2001-08-20T09:53:57 PROBLEM(100) ZServer warning: server accept() threw an exception ------ 2001-08-20T09:53:57 PROBLEM(100) ZServer warning: server accept() threw an exception ------ 2001-08-20T09:53:57 PROBLEM(100) ZServer warning: server accept() threw an exception ------ 2001-08-20T09:53:57 PROBLEM(100) ZServer warning: server accept() threw an exception
Next step?
On 08/18, Chris McDonough wrote:
The thing to do here is to turn on "big M" logging (use the -M switch to z2.py)... this details all the transactions between clients and Zope. Then use the "requestprofiler.py" script that ships with Zope 2.4 (in the "utilities" directory) to give you a profile of the big M logs for the right before a hang... do this for a few days to determine if you can spot a pattern to the hangs.
Roman V. Isaev wrote:
A glitch :( It happens in zope 2.3.3 (python 1.5.2) and zope 2.4.0 (python 2.1.1) under OpenBSD 2.7. After a day or two of normal work zope simply stops responding to HTTP requests. When I telnet to 80 port it connects, but nothing else happens. What to do? How to know where it handgs and why? Is it possible to attach to zope process and see what's going there?..
-- Chris McDonough Zope Corporation http://www.zope.org http://www.zope.com "Killing hundreds of birds with thousands of stones"
-- Roman V. Isaev http://www.aroma.ru System Administrator
-- Chris McDonough Zope Corporation http://www.zope.org http://www.zope.com """ Killing hundreds of birds with thousands of stones """
Sorry, I'm not very familiar with python. It looks like this place: try: conn, addr = self.accept() except socket.error: # linux: on rare occasions we get a bogus socket back from # accept. socketmodule.c:makesockaddr complains that the # address family is unknown. We don't want the whole server # to shut down because of this. self.log_info ('warning: server accept() threw an exception', 'warning') return How can I get accept() error codes like these on accept(2) man page? And how to make a temp fix -- if socket.error happens more than N times, kill this process and let zope restart (uhhh)? On 08/20, Chris McDonough wrote:
Hi Roman,
This is a log message which is recorded in ZServer/medusa/http_server.py meaning that the system accept() call threw an error. This appears to happen at the OS layer, so I'm not entirely certain what the issue is. You might try instrumenting the place in the code where it throws the error with debugging output to get a better understanding of what the error actually is.
HTH,
- C
"Roman V. Isaev" wrote:
When zope hangs, -M file stalls too -- no changes after sending GET request. But in my nohup.out I found about 40 megs of these errors:
2001-08-20T09:53:57 PROBLEM(100) ZServer warning: server accept() threw an exception ------ 2001-08-20T09:53:57 PROBLEM(100) ZServer warning: server accept() threw an exception ------ 2001-08-20T09:53:57 PROBLEM(100) ZServer warning: server accept() threw an exception ------ 2001-08-20T09:53:57 PROBLEM(100) ZServer warning: server accept() threw an exception
Next step?
On 08/18, Chris McDonough wrote:
The thing to do here is to turn on "big M" logging (use the -M switch to z2.py)... this details all the transactions between clients and Zope. Then use the "requestprofiler.py" script that ships with Zope 2.4 (in the "utilities" directory) to give you a profile of the big M logs for the right before a hang... do this for a few days to determine if you can spot a pattern to the hangs.
Roman V. Isaev wrote:
A glitch :( It happens in zope 2.3.3 (python 1.5.2) and zope 2.4.0 (python 2.1.1) under OpenBSD 2.7. After a day or two of normal work zope simply stops responding to HTTP requests. When I telnet to 80 port it connects, but nothing else happens. What to do? How to know where it handgs and why? Is it possible to attach to zope process and see what's going there?..
-- Chris McDonough Zope Corporation http://www.zope.org http://www.zope.com "Killing hundreds of birds with thousands of stones"
-- Roman V. Isaev http://www.aroma.ru System Administrator
-- Chris McDonough Zope Corporation http://www.zope.org http://www.zope.com
""" Killing hundreds of birds with thousands of stones """
-- Roman V. Isaev http://www.aroma.ru System Administrator
Something like: import traceback traceback.print_exc() .. after the self.log_info call.. .. this should print the actual exception out (which is caught by the except above) to sys.stdout... With this info, you might be able to understand what the OS is complaining about. "Roman V. Isaev" wrote:
Sorry, I'm not very familiar with python. It looks like this place:
try: conn, addr = self.accept() except socket.error: # linux: on rare occasions we get a bogus socket back from # accept. socketmodule.c:makesockaddr complains that the # address family is unknown. We don't want the whole server # to shut down because of this. self.log_info ('warning: server accept() threw an exception', 'warning') return
How can I get accept() error codes like these on accept(2) man page?
And how to make a temp fix -- if socket.error happens more than N times, kill this process and let zope restart (uhhh)?
On 08/20, Chris McDonough wrote:
Hi Roman,
This is a log message which is recorded in ZServer/medusa/http_server.py meaning that the system accept() call threw an error. This appears to happen at the OS layer, so I'm not entirely certain what the issue is. You might try instrumenting the place in the code where it throws the error with debugging output to get a better understanding of what the error actually is.
HTH,
- C
"Roman V. Isaev" wrote:
When zope hangs, -M file stalls too -- no changes after sending GET request. But in my nohup.out I found about 40 megs of these errors:
2001-08-20T09:53:57 PROBLEM(100) ZServer warning: server accept() threw an exception ------ 2001-08-20T09:53:57 PROBLEM(100) ZServer warning: server accept() threw an exception ------ 2001-08-20T09:53:57 PROBLEM(100) ZServer warning: server accept() threw an exception ------ 2001-08-20T09:53:57 PROBLEM(100) ZServer warning: server accept() threw an exception
Next step?
On 08/18, Chris McDonough wrote:
The thing to do here is to turn on "big M" logging (use the -M switch to z2.py)... this details all the transactions between clients and Zope. Then use the "requestprofiler.py" script that ships with Zope 2.4 (in the "utilities" directory) to give you a profile of the big M logs for the right before a hang... do this for a few days to determine if you can spot a pattern to the hangs.
Roman V. Isaev wrote:
A glitch :( It happens in zope 2.3.3 (python 1.5.2) and zope 2.4.0 (python 2.1.1) under OpenBSD 2.7. After a day or two of normal work zope simply stops responding to HTTP requests. When I telnet to 80 port it connects, but nothing else happens. What to do? How to know where it handgs and why? Is it possible to attach to zope process and see what's going there?..
-- Chris McDonough Zope Corporation http://www.zope.org http://www.zope.com "Killing hundreds of birds with thousands of stones"
-- Roman V. Isaev http://www.aroma.ru System Administrator
-- Chris McDonough Zope Corporation http://www.zope.org http://www.zope.com
""" Killing hundreds of birds with thousands of stones """
-- Roman V. Isaev http://www.aroma.ru System Administrator
-- Chris McDonough Zope Corporation http://www.zope.org http://www.zope.com """ Killing hundreds of birds with thousands of stones """
participants (3)
-
Chris McDonough -
Chris McDonough -
Roman V. Isaev