I have a fairly old Zope server that has slowed down to the point where loading the ZMI doesn't complete and some pages load in > 25 secs while other pages don't load at all. There has been recent cosmetic changes, but they worked for a day or so before the slowdown. There is nothing new or unusual in the event log and traffic to the server is minimal (probably because it has been unusable for a day). The Data.fs is about 500 mb. I am fairly certain that I have eliminated all operating system factors, and I am at a loss as to what I can do next without a ZMI. Any suggestions welcome. ____________________________________________________________ Receive Notifications of Incoming Messages Easily monitor multiple email accounts & access them with a click. Visit http://www.inbox.com/notifier and check it out!
On Nov 19, 2007 8:16 PM, Ben Bartrum <bba@inbox.com> wrote:
I have a fairly old Zope server that has slowed down to the point where loading the ZMI doesn't complete and some pages load in > 25 secs while other pages don't load at all. There has been recent cosmetic changes, but they worked for a day or so before the slowdown. There is nothing new or unusual in the event log and traffic to the server is minimal (probably because it has been unusable for a day). The Data.fs is about 500 mb.
I am fairly certain that I have eliminated all operating system factors, and I am at a loss as to what I can do next without a ZMI.
I found that "cosmetic changes" I made cause an unintential self induced, intermittent DOS attack with similar results. Maybe you are experiencing something along the same lines? The DeadlockDebugger product helped me figure out what was going on http://www.zope.org/Members/nuxeo/Products/DeadlockDebugger (in my case I had buggy Javascript that caused an infinite reload loop)
On 11/20/07, Ben Bartrum <bba@inbox.com> wrote:
I have a fairly old Zope server that has slowed down to the point where loading the ZMI doesn't complete and some pages load in > 25 secs while other pages don't load at all. There has been recent cosmetic changes, but they worked for a day or so before the slowdown. There is nothing new or unusual in the event log and traffic to the server is minimal (probably because it has been unusable for a day). The Data.fs is about 500 mb.
I am fairly certain that I have eliminated all operating system factors, and I am at a loss as to what I can do next without a ZMI.
Any suggestions welcome.
Have you found out why it is slowing? Has your traffic increased -> limit your traffic temporarily Perhaps its the Hardware? If you mean with 'old' P3/P4, perhaps the CPU cooler is dead and the CPU is in reduced performance mode? Perhaps some IO controller (disks?) is failing. Or your changes weren't cosmetic. Roll back to your last known good backup (you have one, right?) and apply cour changes a atomic as possible and test them --knitti
Have you found out why it is slowing? Has your traffic increased -> limit your traffic temporarily Perhaps its the Hardware? If you mean with 'old' P3/P4, perhaps the CPU cooler is dead and the CPU is in reduced performance mode? Perhaps some IO controller (disks?) is failing.
Or your changes weren't cosmetic. Roll back to your last known good backup (you have one, right?) and apply cour changes a atomic as possible and test them
Bryan Sullivan's suggestion of the DeadlockDebugger helped me to solve the problem. The DeadlockDebugger was crucial, and far easier to work with than I anticipated. It was quite a subtle problem - an earlier change worked fine at the time, but the deadlock started when the underlying SQL data changed. That is, an SQL query returned less results than the code was meant to work with, and there was no clause in a Python script to deal with this eventuality. So rolling back to dates when the site worked fine did not help until I understood the nature of the problem.
participants (3)
-
Ben Bartrum -
Brian Sullivan -
knitti