[ZODB-Dev] Re: ZEO Client deadlocking in asyncore.poll - how to I debug

[Anton Stonor]

Last month I ask for help about hanging zeo clients. I promised to 
follow up.

Dieter Maurer wrote:
> Anton Stonor wrote at 2008-4-7 16:16 +0200:
>> We have a setup with a ZEO server and 4 ZEO clients.
>>
>> During the last weeks we have seen almost daily deadlocks in some of the 
>> ZEO clients. I've tried to wait for up to 30 minutes before restarting a 
>> client.
>>
>> I could need an advice on how to debug this.
>>
>> With DeadlockDebugger I see the same pattern each time:
>>
>> One thread is hanging:
>> ...
>>     r_flags, r_args = self.wait(msgid)
>>   File "/usr/local/www/zope-2.9.6/lib/python/ZEO/zrpc/connection.py", 
>> line 638, in wait
> 
> This means that the client is waiting for a reply -- which apparently
> does not come.
> 
> Maybe some router or firewall sometimes drops packages and connections?

It was a hardware issue. /var/log/messages was full of random "link 
state changed to DOWN". The unstable net connection was caused by a 
unstable switch.

/Anton