Zope=ZEO connection
Zope 2.9.0 Python 2.4.2 Amd dual Opteron Linux I am seeing occasional (several per day) Zope crashes under load. The event.log immediately before and after the crash looks like: 2006-02-06T14:07:20 INFO ZPublisher.Conflict ConflictError at <some path>: database conflict error (oid 0x086e, class BTrees._OOBTree.OOBTree, serial this txn started with 0x03633ca95f75e900 2006-02-06 22:01:22.373575, serial currently committed 0x03633caf59114244 2006-02-06 22:07:20.875176) (463 conflicts (0 unresolved) since startup at Mon Feb 6 08:21:26 2006) ------ 2006-02-06T14:09:09 INFO ZServer HTTP server started at Mon Feb 6 14:09:09 2006 Hostname: x-harper Port: 8081 which also triggers a connection drop and restarBt of ZEO. The failure happens silently even with <eventlog> level ALL <logfile> path $INSTANCE/log/event.log level info </logfile> </eventlog> When this happens zope is running under zopectl and zeo isi running under zeoctl. What sort of Zope failure can cause this sort of behavior? What's the best approach to get more information to localize the failure.
On Tue, Feb 07, 2006 at 06:18:49PM -0800, Dennis Allison wrote:
Zope 2.9.0 Python 2.4.2 Amd dual Opteron Linux
I am seeing occasional (several per day) Zope crashes under load. The event.log immediately before and after the crash looks like:
2006-02-06T14:07:20 INFO ZPublisher.Conflict ConflictError at <some path>: database conflict error (oid 0x086e, class BTrees._OOBTree.OOBTree, serial this txn started with 0x03633ca95f75e900 2006-02-06 22:01:22.373575, serial currently committed 0x03633caf59114244 2006-02-06 22:07:20.875176) (463 conflicts (0 unresolved) since startup at Mon Feb 6 08:21:26 2006) ------ 2006-02-06T14:09:09 INFO ZServer HTTP server started at Mon Feb 6 14:09:09 2006 Hostname: x-harper Port: 8081
which also triggers a connection drop and restarBt of ZEO.
The failure happens silently even with
<eventlog> level ALL <logfile> path $INSTANCE/log/event.log level info
^^^^^^^^^^ fwiw, you're still only logging at INFO.
What sort of Zope failure can cause this sort of behavior? What's the best approach to get more information to localize the failure.
I really don't know. But I'd have a look at the trace log, if you've got that configured (I routinely do). Maybe you could spot some patterns right before the crashes? *shrug* -- Paul Winkler http://www.slinkp.com
Arggggh... I forgot that internal logging levels override. Still, you'd think something that triggers a crash would have a level higher than info. Careful examination of the trace log shows nothing. I'll fix the log level & see if that turns anything up. On Tue, 7 Feb 2006, Paul Winkler wrote:
On Tue, Feb 07, 2006 at 06:18:49PM -0800, Dennis Allison wrote:
Zope 2.9.0 Python 2.4.2 Amd dual Opteron Linux
I am seeing occasional (several per day) Zope crashes under load. The event.log immediately before and after the crash looks like:
2006-02-06T14:07:20 INFO ZPublisher.Conflict ConflictError at <some path>: database conflict error (oid 0x086e, class BTrees._OOBTree.OOBTree, serial this txn started with 0x03633ca95f75e900 2006-02-06 22:01:22.373575, serial currently committed 0x03633caf59114244 2006-02-06 22:07:20.875176) (463 conflicts (0 unresolved) since startup at Mon Feb 6 08:21:26 2006) ------ 2006-02-06T14:09:09 INFO ZServer HTTP server started at Mon Feb 6 14:09:09 2006 Hostname: x-harper Port: 8081
which also triggers a connection drop and restarBt of ZEO.
The failure happens silently even with
<eventlog> level ALL <logfile> path $INSTANCE/log/event.log level info
^^^^^^^^^^
fwiw, you're still only logging at INFO.
What sort of Zope failure can cause this sort of behavior? What's the best approach to get more information to localize the failure.
I really don't know. But I'd have a look at the trace log, if you've got that configured (I routinely do). Maybe you could spot some patterns right before the crashes? *shrug*
--
Dennis Allison wrote at 2006-2-7 18:18 -0800:
... What sort of Zope failure can cause this sort of behavior? What's the best approach to get more information to localize the failure.
A crash presented to Zope as a fatal signal (usually "SIGSEGV" or "SIGBUS"). Reconfigure your Linux account running Zope to write core files ("ulimit -Sc 100000"). With a bit of luck you will get a core file written in case of such crashes. Look into the core with GDB. Note, that the value "100000" might not be big enough to get a complete core file. GDB may have problems to really understand incomplete ones. -- Dieter
Dennis Allison wrote:
2006-02-06T14:07:20 INFO ZPublisher.Conflict ConflictError at <some path>: database conflict error (oid 0x086e, class BTrees._OOBTree.OOBTree, serial this txn started with 0x03633ca95f75e900 2006-02-06 22:01:22.373575, serial currently committed 0x03633caf59114244 2006-02-06 22:07:20.875176) (463 conflicts (0 unresolved) since startup at Mon Feb 6 08:21:26 2006)
Well, at least the new conflict error logging stuff works :-)
------ 2006-02-06T14:09:09 INFO ZServer HTTP server started at Mon Feb 6 14:09:09 2006 Hostname: x-harper Port: 8081
which also triggers a connection drop and restarBt of ZEO.
Odd... not seen this at all, what OS you running on? cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
participants (4)
-
Chris Withers -
Dennis Allison -
Dieter Maurer -
Paul Winkler