[Zope] Zope hangs

Nico Grubert nicogrubert at gmail.com
Thu Oct 9 04:17:38 EDT 2008 

> If your tracebacks are reliable, then neither of these entries
> should cause "hanging" (apart from this: CPU usage of about 100 percent
> does not indicate hanging but some other problem).
> 
> Tracebacks do not need to be reliable. They can be unreliable when the
> the "*.pyc/*.pyo" files do not correspond to the source. Therefore,
> an initial step may be to delete all "*.pyc/*.pyo" files and try again.
> 
> There is one case when "hanging" could apparently happen
> in a "return": the operation before takes a long time and does not release
> the GIL. Then, the deadlock debugger will get active only when the operation
> finished -- and then sees an apparently harmless operation (such as a "return").
> 
> We have had severe garbage collection problems (GC, too, holds the GIL
> for a long time). To detect such problems, we have added a GIL monitor
> to our system. This is an additional thread running in the system
> which schedules itself at regular times and measures the difference
> between the expected scheduling time and the actual activation time.
> If the difference is too large, this is logged.
> 
> 
> If your system gets slow but still occasionally responds, you can
> use "ZopeProfiler" to find out, where the time is spend.
> Ensure to use "real time" as the timer (not cpu time).

Hi Dieter

thank you very much for the information.

This morning the server did not respond and all 8 threads in the 
DeadlockDebugger's output look the same:

Thread 1098918208
   File 
"/usr/local/Zope-2.10.5/lib/python/ZServer/PubCore/ZServerPublisher.py", 
line 25, in __init__
     response=b)
   File "/usr/local/Zope-2.10.5/lib/python/ZPublisher/Publish.py", line 
401, in publish_module
     environ, debug, request, response)
   File "/usr/local/Zope-2.10.5/lib/python/ZPublisher/Publish.py", line 
202, in publish_module_standard
     response = publish(request, module_name, after_list, debug=debug)
   File 
"/zope/intranet/Products/PlacelessTranslationService/PatchStringIO.py", 
line 51, in new_publish
     x = Publish.old_publish(request, module_name, after_list, debug)
   File "/usr/local/Zope-2.10.5/lib/python/ZPublisher/Publish.py", line 
119, in publish
     request, bind=1)
   File "/usr/local/Zope-2.10.5/lib/python/ZPublisher/mapply.py", line 
88, in mapply
     if debug is not None: return debug(object,args,context)
   File "/usr/local/Zope-2.10.5/lib/python/ZPublisher/Publish.py", line 
42, in call_object
     result=apply(object,args) # Type s<cr> to step into published object.
   File 
"/usr/local/Zope-2.10.5/lib/python/Shared/DC/Scripts/Bindings.py", line 
313, in __call__
     return self._bindAndExec(args, kw, None)
   File 
"/usr/local/Zope-2.10.5/lib/python/Shared/DC/Scripts/Bindings.py", line 
350, in _bindAndExec
     return self._exec(bound_data, args, kw)
   File 
"/usr/local/Zope-2.10.5/lib/python/Products/PythonScripts/PythonScript.py", 
line 327, in _exec
     result = f(*args, **kw)
   File "Script (Python)", line 20, in index_html
   File 
"/usr/local/Zope-2.10.5/lib/python/Shared/DC/Scripts/Bindings.py", line 
313, in __call__
     return self._bindAndExec(args, kw, None)
   File 
"/usr/local/Zope-2.10.5/lib/python/Shared/DC/Scripts/Bindings.py", line 
350, in _bindAndExec
     return self._exec(bound_data, args, kw)
   File 
"/usr/local/Zope-2.10.5/lib/python/Products/PageTemplates/ZopePageTemplate.py", 
line 330, in _exec
     result = self.pt_render(extra_context=bound_names)
   File 
"/usr/local/Zope-2.10.5/lib/python/Products/PageTemplates/ZopePageTemplate.py", 
line 426, in pt_render
     result = PageTemplate.pt_render(self, source, extra_context)
   File "/zope/wwnet/Products/PTProfiler/ProfilerPatch.py", line 66, in 
__patched_render__
     ret = self._patching_class._org_method(self, source, extra_context)
   File 
"/usr/local/Zope-2.10.5/lib/python/Products/PageTemplates/PageTemplate.py", 
line 89, in pt_render
     return super(PageTemplate, self).pt_render(c, source=source)
   File 
"/usr/local/Zope-2.10.5/lib/python/zope/pagetemplate/pagetemplate.py", 
line 117, in pt_render
     strictinsert=0, sourceAnnotations=sourceAnnotations)()
   File "/usr/local/Zope-2.10.5/lib/python/zope/tal/talinterpreter.py", 
line 271, in __call__
     self.interpret(self.program)
   File "/usr/local/Zope-2.10.5/lib/python/zope/tal/talinterpreter.py", 
line 346, in interpret
     handlers[opcode](self, args)
   File "/usr/local/Zope-2.10.5/lib/python/zope/tal/talinterpreter.py", 
line 891, in do_useMacro
     self.interpret(macro)
   File "/usr/local/Zope-2.10.5/lib/python/zope/tal/talinterpreter.py", 
line 346, in interpret
     handlers[opcode](self, args)
   File "/usr/local/Zope-2.10.5/lib/python/zope/tal/talinterpreter.py", 
line 949, in do_defineSlot
     self.interpret(slot)
   File "/usr/local/Zope-2.10.5/lib/python/zope/tal/talinterpreter.py", 
line 346, in interpret
     handlers[opcode](self, args)
   File "/usr/local/Zope-2.10.5/lib/python/zope/tal/talinterpreter.py", 
line 534, in do_optTag_tal
     self.no_tag(stuff[-2], stuff[-1])
   File "/usr/local/Zope-2.10.5/lib/python/zope/tal/talinterpreter.py", 
line 516, in no_tag
     self.interpret(program)
   File "/usr/local/Zope-2.10.5/lib/python/zope/tal/talinterpreter.py", 
line 346, in interpret
     handlers[opcode](self, args)
   File "/usr/local/Zope-2.10.5/lib/python/zope/tal/talinterpreter.py", 
line 855, in do_condition
     self.interpret(block)
   File "/usr/local/Zope-2.10.5/lib/python/zope/tal/talinterpreter.py", 
line 346, in interpret
     handlers[opcode](self, args)
   File "/usr/local/Zope-2.10.5/lib/python/zope/tal/talinterpreter.py", 
line 536, in do_optTag_tal
     self.do_optTag(stuff)
   File "/usr/local/Zope-2.10.5/lib/python/zope/tal/talinterpreter.py", 
line 521, in do_optTag
     return self.no_tag(start, program)
   File "/usr/local/Zope-2.10.5/lib/python/zope/tal/talinterpreter.py", 
line 516, in no_tag
     self.interpret(program)
   File "/usr/local/Zope-2.10.5/lib/python/zope/tal/talinterpreter.py", 
line 346, in interpret
     handlers[opcode](self, args)
   File "/usr/local/Zope-2.10.5/lib/python/zope/tal/talinterpreter.py", 
line 586, in do_setLocal_tal
     self.engine.setLocal(name, self.engine.evaluateValue(expr))
   File "/usr/local/Zope-2.10.5/lib/python/zope/tales/tales.py", line 
696, in evaluate
     return expression(self)
   File "/zope/wwnet/Products/PTProfiler/ProfilerPatch.py", line 25, in 
__patched_call__
     ret = self._patching_class._org_method(self, econtext)
   File 
"/usr/local/Zope-2.10.5/lib/python/Products/PageTemplates/ZRPythonExpr.py", 
line 49, in __call__
     return eval(self._code, vars, {})
   File "PythonExpr", line 1, in <expression>
   File 
"/usr/local/Zope-2.10.5/lib/python/Shared/DC/Scripts/Bindings.py", line 
313, in __call__
     return self._bindAndExec(args, kw, None)
   File 
"/usr/local/Zope-2.10.5/lib/python/Shared/DC/Scripts/Bindings.py", line 
350, in _bindAndExec
     return self._exec(bound_data, args, kw)
   File 
"/usr/local/Zope-2.10.5/lib/python/Products/PythonScripts/PythonScript.py", 
line 327, in _exec
     result = f(*args, **kw)
   File "Script (Python)", line 109, in get_navigation_items
   File 
"/usr/local/Zope-2.10.5/lib/python/Shared/DC/Scripts/Bindings.py", line 
313, in __call__
     return self._bindAndExec(args, kw, None)
   File 
"/usr/local/Zope-2.10.5/lib/python/Shared/DC/Scripts/Bindings.py", line 
350, in _bindAndExec
     return self._exec(bound_data, args, kw)
   File 
"/usr/local/Zope-2.10.5/lib/python/Products/PythonScripts/PythonScript.py", 
line 327, in _exec
     result = f(*args, **kw)
   File "Script (Python)", line 31, in getSubdocumentsIncludingGhosts
   File "/zope/intranet/Products/Silva/Folder.py", line 718, in 
get_public_tree
     self._get_public_tree_helper(l, 0, depth)
   File "/zope/intranet/Products/Silva/Folder.py", line 768, in 
_get_public_tree_helper
     if self.service_toc_filter.filter(item):
   File "/zope/intranet/Products/Silva/tocfilter.py", line 36, in filter
     return False
   File "/zope/intranet/Products/Silva/tocfilter.py", line 19, in 
hideFromTOC
     return context.service_metadata.getMetadataValue(
   File "/zope/intranet/Products/SilvaMetadata/MetadataTool.py", line 
180, in getMetadataValue
     return binding.get(set_id, element_id)
   File "/zope/intranet/Products/SilvaMetadata/Binding.py", line 301, in get
     return data.get(element_id)
   File "/zope/intranet/Products/SilvaMetadata/Binding.py", line 500, in 
_getData
     return data
   File "/zope/intranet/Products/SilvaMetadata/Element.py", line 207, in 
getDefault
     return self.field.get_value('default', content=content)
   File "/zope/intranet/Products/Formulator/Field.py", line 142, in 
get_value
     return value
   File "/zope/intranet/Products/Formulator/TALESField.py", line 44, in 
__call__
     return getEngine().getContext(kw).evaluate(expr)
   File "/usr/local/Zope-2.10.5/lib/python/zope/tales/tales.py", line 
696, in evaluate
     return expression(self)
   File "/zope/wwnet/Products/PTProfiler/ProfilerPatch.py", line 31, in 
__patched_call__
     return ret
   File 
"/usr/local/Zope-2.10.5/lib/python/Products/PageTemplates/ZRPythonExpr.py", 
line 49, in __call__
     return eval(self._code, vars, {})
   File "PythonExpr", line 1, in <expression>
   File "/zope/intranet/Products/Silva/Security.py", line 272, in 
sec_get_creator_info
     return self.sec_get_member(self.getOwner().getId())
   File "/zope/intranet/Products/Silva/Security.py", line 227, in 
sec_get_member
     return self.service_members.get_cached_member(userid)
   File "/zope/intranet/Products/Silva/SimpleMembership.py", line 193, 
in get_cached_member
     return cloneMember(self.get_member(userid)).__of__(self)
   File "/zope/intranet/Products/Silva/SimpleMembership.py", line 187, 
in get_member
     return member
   File "/zope/intranet/Products/Silva/SimpleMembership.py", line 174, 
in is_user
     return userid in self.get_valid_userids()
   File "/usr/local/Zope-2.10.5/lib/python/AccessControl/Role.py", line 
360, in get_valid_userids
     return tuple(keys)
   File "/usr/local/Zope-2.10.5/lib/python/AccessControl/User.py", line 
984, in user_names
     return self.getUserNames()
   File "/zope/intranet/Products/LDAPUserFolder/LDAPUserFolder.py", line 
576, in getUserNames
     return self._misc_cache().get('loginlist')
   File "/zope/intranet/Products/LDAPUserFolder/LDAPUserFolder.py", line 
1846, in _misc_cache
     return getResource('%s-misc_cache' % self._hash, SharedObject, ())
   File "/zope/intranet/Products/LDAPUserFolder/SharedResource.py", line 
50, in getResource
     _ResourceLock.release()


I am not sure if all the problems are caused by the LDAPUserfolder.

Regards
Nico

More information about the Zope mailing list