[Zope-dev] TCP CLOSE_WAIT leaks

[Alan Milligan]

I managed to get a DeadlockDebugger trace on this thing, it made very
interesting reading:

Thread -1578087504 (GET
/VirtualHostBase/http/www.last-bastion.net:80/BastionLedger/ledgerlist/Mail_1111338253076):
  File "/opt/zope2.8/lib/python/ZServer/PubCore/ZServerPublisher.py",
line 23, in __init__
    response=response)
  File "/opt/zope2.8/lib/python/ZPublisher/Publish.py", line 387, in
publish_module
    environ, debug, request, response)
  File "/opt/zope2.8/lib/python/ZPublisher/Publish.py", line 188, in
publish_module_standard
    response = publish(request, module_name, after_list, debug=debug)
  File
"/opt/zope2.8/lib/python/Products/PlacelessTranslationService/PatchStringIO.py",
line 51, in new_publish
    x = Publish.old_publish(request, module_name, after_list, debug)
  File "/opt/zope2.8/lib/python/ZPublisher/Publish.py", line 114, in publish
    request, bind=1)
  File "/opt/zope2.8/lib/python/ZPublisher/mapply.py", line 88, in mapply
    if debug is not None: return debug(object,args,context)
  File "/opt/zope2.8/lib/python/ZPublisher/Publish.py", line 40, in
call_object
    result=apply(object,args) # Type s<cr> to step into published object.
  File "/opt/zope2.8/lib/python/Shared/DC/Scripts/Bindings.py", line
311, in __call__
    return self._bindAndExec(args, kw, None)
  File "/opt/zope2.8/lib/python/Shared/DC/Scripts/Bindings.py", line
329, in _bindAndExec
    def _bindAndExec(self, args, kw, caller_namespace):
  File "/opt/zope2.8/lib/python/Shared/DC/Scripts/Bindings.py", line
348, in _bindAndExec
    return self._exec(bound_data, args, kw)
  File "/opt/zope2.8/lib/python/Products/CMFCore/FSPageTemplate.py",
line 195, in _exec
    result = self.pt_render(extra_context=bound_names)
  File "/opt/zope2.8/lib/python/Products/CMFCore/FSPageTemplate.py",
line 134, in pt_render
    result = FSPageTemplate.inheritedAttribute('pt_render')(
  File "/opt/zope2.8/lib/python/Products/PageTemplates/PageTemplate.py",
line 104, in pt_render
    tal=not source, strictinsert=0)()
  File "/opt/zope2.8/lib/python/TAL/TALInterpreter.py", line 202, in
__call__
    def __call__(self):
  File "/opt/zope2.8/lib/python/Products/PageTemplates/ZRPythonExpr.py",
line 47, in __call__
    return eval(code, g, {})
  File "Python expression "putils.createTopLevelTabs(actions)"", line 1,
in <expression>
  File "/opt/zope2.8/lib/python/Products/CMFPlone/PloneTool.py", line
701, in createTopLevelTabs
    default=data['name'])
  File
"/opt/zope2.8/lib/python/Products/CMFPlone/TranslationServiceTool.py",
line 35, in utranslate
    return utranslate(*args, **kw)
  File "/opt/zope2.8/lib/python/Products/CMFPlone/i18nl10n.py", line 53,
in utranslate
    return service.utranslate(*args, **kw)
  File
"/opt/zope2.8/lib/python/Products/PlacelessTranslationService/PlacelessTranslationService.py",
line 133, in utranslate
    if not service: return default
  File "/opt/zope2.8/lib/python/ZODB/Connection.py", line 704, in setstate
    self._setstate(obj)
  File "/opt/zope2.8/lib/python/ZODB/Connection.py", line 760, in _setstate
    self._reader.setGhostState(obj, p)
  File "/opt/zope2.8/lib/python/ZODB/serialize.py", line 495, in
setGhostState
    state = self.getState(pickle)
  File "/opt/zope2.8/lib/python/ZODB/serialize.py", line 488, in getState
    return unpickler.load()
  File "/opt/zope2.8/lib/python/ZODB/serialize.py", line 436, in
_persistent_load
    return self._conn.get(oid)
  File "/opt/zope2.8/lib/python/ZODB/Connection.py", line 216, in get
    return obj
  File "/opt/zope2.8/lib/python/ZEO/ClientStorage.py", line 746, in load
    return self.loadEx(oid, version)[:2]
  File "/opt/zope2.8/lib/python/ZEO/ClientStorage.py", line 781, in loadEx
    return data, tid, ver

*every* thread was block-waiting on zeo (from a wide range of different
Zope/Plone types)!  It looks to me like Apache has timed out, clearing
down it's end, Zope however is still having to wait for zeo which is
completely borked.

I've consequently ditched zeo and everything is again well-behaved.

Alan