[Zope-dev] Hmmmm.... what do *you* do when Zope is 'stuck'.

[Steve Spicklemire]

Hi Folks, 

Ok.. this is the wierdest thing I've run into yet. Any ideas how
to diagnose this would be most appreciated. It's a little bit like
quantum mechanics. When you try to measure it... it changes. ;-)

I have a Data.fs that has somehow gotten into a state where 
Zope just hangs there. I can attach to the python process with
the debugger when it's running and dump the stack trace. It
looks like this:

#0  0x281af6c0 in _thread_sys_poll () from /usr/lib/libc_r.so.4
#1  0x281a806f in _thread_kern_sched_state_unlock () from /usr/lib/libc_r.so.4
#2  0x281a775a in _thread_kern_sched () from /usr/lib/libc_r.so.4
#3  0x281a7b7e in _thread_kern_sched_state () from /usr/lib/libc_r.so.4
#4  0x281944c1 in wait4 () from /usr/lib/libc_r.so.4
#5  0x281865c7 in __waitpid () from /usr/lib/libc_r.so.4
#6  0x2816bba2 in waitpid () from /usr/lib/libc_r.so.4
#7  0x80890cd in pcre_exec ()
#8  0x8055069 in PyEval_CallObjectWithKeywords ()
#9  0x8054f73 in PyEval_CallObjectWithKeywords ()
#10 0x8054028 in PyEval_EvalCode ()
#11 0x8053f10 in PyEval_EvalCode ()
#12 0x805210d in PyEval_EvalCode ()
#13 0x80629a3 in PyRun_File ()
#14 0x806295c in PyRun_File ()
#15 0x806293c in PyRun_File ()
#16 0x8062046 in PyRun_SimpleFile ()
#17 0x8061cba in PyRun_AnyFile ()
#18 0x8050935 in Py_Main ()
#19 0x8050420 in main ()
#20 0x80503a5 in _start ()

The problem is somehow related to timing and threads. If I start Zope
with only one thread.... no hang. If I start Zope with 'PROFILE_PUBLISHER'
defined... no hang. I've added some debugger hooks at various places (e.g.,
ZPublisher, ZServer, etc... ) in an attempt to 'step' through the problem,
but when I do.. it works! Ahhhh!!! I'm about to add some 'print' statements
around but I'm fearing that if I add some print statements it will disturb
the timing enough to destroy the electron interference patterns... er...
I mean mask the problem. Has anyone got any ideas on how to track something
like this down? 

The only other clue I have is that, at first, when I saw the problem,
whenever I attached to the process it would be in the 'PoPymodule', so
I naturally suspected PoPy. However, I've *removed* the ZPoPyDA
Product from the Products folder (on the filesystem) , so that
presumably PoPymodule.so will/should not be loaded or
accessed. Anyway.... popymodule is no longer showing up in the stack
trace, but still Zope is hanging.... All this started showing up by the
way not long after we started experimenting with ZPoPyDA. But.. now that
ZPoPyDA is gone... is there any way my Data.fs could be affected in 
a way that could cause this behavior?

thanks for any insight.... 

-steve