[Zope] Limiting access

[Andreas Jung]

--On 21. August 2006 13:17:03 +0100 David <davidr at talamh.org.uk> wrote:

> Hi
>
> We are running a site for a local government providing local
> information. Over the past few months, the site has been spidered  daily
> and the information appears to be used for unwanted marketing  campaigns.
>
> We have been asked if we could somehow block this access. We tried
> blocking the originating IP on the server (using route to blackhole),
> but the spider is rotating IPs within a day of us putting in a block.
>
> Is there some mechanism in Zope or can be added to a Zope instance  that
> would limit the number of requests that are served per IP  address within
> a certain period of time? We are aware that this would  also affect
> Google and other search engine spiders, but as they  usually come from a
> known IP range, it would be nice if we could  exempt these from the
> "throttling".
>

You should solve this inside your front-end Apache (hope you have one!).
You might check mod_throttle for Apache and check your robots.txt file.

-aj
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 186 bytes
Desc: not available
Url : http://mail.zope.org/pipermail/zope/attachments/20060821/2ffd19c4/attachment.bin