it looks like the site developers used relative links in their pages, which can lead to this problem. using absolute links prevents it. jens On Thursday, June 6, 2002, at 01:04 , Granzow, Doug (NCI) wrote:
Bear with me, I'm a sysadmin, not a developer.
I'm maintaining a Zope 2.5.1 server with Python 2.1.3 on Solaris 7. We recently upgraded from Zope 2.3.1. I'm seeing a problem where, occasionally, a bot will come along to index our site and suddenly Zope stops responding to all requests. It looks like what is happening is that the bot finds a loop in the content where each time you click on a link, you are returned to the same page, but the URL has one more level of depth. For example, I see requests in the log file for things like this:
siteroot/info/catalog/catalog/catalog/catalog/catalog/catalog/catalog/catalo g/catalog/catalog/catalog/catalog/catalog/catalog/catalog
When a bot hits this it keeps going deeper and deeper until we deny access to the bot. When I request a URL like that in my browser, even just one time, Python's CPU usage shoots to near 100%, and stays there for several seconds. If I keep clicking on the link to go deeper, it doesn't take long for it to effectively become a denial of service attack, because the processing time required to process each request seems to grow more and more as you go deeper.
Is there any way to stop this from happening? Obviously we want to remove loops like this from the content, but as I said, I'm the sysadmin, not the developer.
Any input is appreciated!