[Zope] Zope woes continue - server going down regularly.

[Jim Fulton]

chas wrote:
> 
> This morning : using Zope 2.01
> ------------------------------
> 
> >>> Alternatively, the server just hangs and doesn't respond
> >>> (but ps -aux shows the process is running).
> >>
> >>Is it spinning? (consuming 100% CPU resources?) or hung? (consuming
> >>none..)
> >
> >Sorry, you're right, it was 'spinning' - 100% CPU resources.
> >Does that suggest anything in particular ?
> 
> Ok, I feel like the Cold-Fusion-Man today, I have a window open
> on my desktop showing the 'top' output of my server. Whenever
> the CPU usage goes up to over 90%, I know that the PCGI process
> is no longer responding and I restart Zope. Restarting the
> server does not help any more than restarting Zope itself.
> Repeat ad nauseum every 30-45 minutes if I'm lucky.
> 
> This afternoon : using Zope 2.10
> --------------------------------

OK, let's focus on 2.1.

> I installed and set up Zope 2.10 (using PCGI behind Apache) and
> transfered my site over it. It *seemed* to be holding up OK.
> CPU never went up to 90%. But I noticed that several times I'd
> get the fd=3 error.
> 
> Then within 2 hours of use, I get this error when external
> users try to access the site :
> 
>     Zope Error
>     Zope has encountered an error while publishing this resource.
>     Error Type: NameError
>     Error Value: name_param

Could you provide a traceback?
 
> Even from the administration interface, I can't access anything.
> Everything returns :
> 
>     Zope Error
>     Zope has encountered an error while publishing this resource.
>     Error Type: NameError
>     Error Value: type

Again, a traceback would be helpful.
 
> And now CPU is at 0% - probably because nobody's any longer
> accessing our site :(
> 
> So, again I had to restart Zope and then the site returned to
> normal.  In a few hours, I'm going to have to (a) leave the
> computer and (b) sleep... and I'll be unable to do this manual
> monitoring/restarting of the server. So, I'm quite  desperate to
> seek a solution to this soon.
> 
> To this end, does anybody have any ideas :
> 1) where to get better debugging information to help identify the
>    source of the problem.

- Obviously something changed.  You were running fine for 2 months, 
  than started having problems.  You can work forward by analysing
  changes, or you could work backward by debugging.

- Tracebacks are helpful. If you aren't in debug mode, then 
  you'll need to view the document source to see them,

- The Zope event log can be very helpful. See doc/LOGGING.txt,

- Look at the "debug" screen at Control_Panel/manage_debug.
  This can be used to spot memory leaks and stuck database connections.
  

- If the above doesn't show anything, then a more detailed log may be created:

  Add this stanza to your z2.py start script. 

  # turn on debug logging
  from ZServer import DebugLogger
  logfile=os.path.join(INSTANCE_HOME,'var/debug.log')
  DebugLogger.log=DebugLogger.DebugLogger(logfile).log

  It should be insertted after Zope is imported, e.g. after this line:

  exec "import "+MODULE in {}

  You'll want to watch the debug log file since it gets large quickly. 
  You might also want to read the docstrings in ZServer/DebugLogger.py 
  for more information about the debug log format.
   
  In particular, we want to look for requests that don't complete
  or for apparent leaking requests.

> 2) better yet, any idea what the problem(s) could be ?

Not off hand.
 
Jim

--
Jim Fulton           mailto:jim@digicool.com
Technical Director   (888) 344-4332              Python Powered!
Digital Creations    http://www.digicool.com     http://www.python.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.