[Zope] Zope hangs in FreeBSD with plenty of memory and CPU resources

[Jean Lagarde]

All,

As a coincidence, that hang of Zope I mentioned in a message I posted
yesterday has happened again today (second time), after a planned
reboot. So here is a completely different issue from the memory one
we've been discussing today.

There was other weirdness (for whatever reason _mysql could not find
libmysqlclient_r.so.15 on some restart attempts), but here is
something that seemed to occur consistently while Zope was
unresponsive:

On FreeBSD 5.4, with zserver-threads set to the default 4, there were
seven Zope threads with the following states (CPU use is 0% and memory
use is well below the maximum, pretty much where it is at after a
normal fresh Zope restart):

select
ksesig
kserel
kqread
kqread
kqread
kqread

When Zope is operating normally and idle, I generally only see the
three top threads, and when serving pages, I rarely see more than
three threads; one actually often disappears and I only see two, one
in the RUN state.

I know that the kqread state has to do with the kqueue or kevent
system call, and assume that Zope being in that waiting state for a
long time is abnormal. Asking here in case someone has seen this
behavior before and knows what might be the cause. As for the last and
only other time this happened, Zope regained normal behavior by itself
after a little while (in the order of half an hour). I don't see
anything abnormal in the event log (other than the _mysql issue I
mentioned above), and while stuck, requests do not register in the
Z2.log. Browsers do get a connection because they state "waiting
for..." but they do not receive anything, not even headers. After a
fresh restart, I could get Zope to respond to a relatively benign
request (the top level ZMI page), while there were only three Zope
threads, but if asking for more, Zope would quickly get to the
unresponsive state described above.

We're thinking that we may be having underlying OS issues and will
probably ask on a FreeBSD list and/or upgrade FreebBSD and hope that
helps, but I'm asking here first just in case that this pattern of
behavior was observed and resolved before.

Cheers,

Jean