[ZWeb] google results duplication

[Shane Hathaway]

Karl Anderson wrote:
> 
> Searching zope.org with google gets lots of duplicate results due to
> multiple views of many objects that contain the normal view content as
> well.  This makes google less useful for searching.
> 
> Wiki pages are especially bad for this - the edit, history, backlinks,
> and advanced action views for each page end up in the google listing.
> Most of the other types have a prettyprinting page that shows up.
> 
> Is there an easy way to exclude these?  Will robots.txt work for page
> suffixes, or just subdirectories?

ZWiki just needs to be fixed to not generate infinite URLs based on
acquisition.  If you browse around a ZWiki long enough, you'll find that
path elements in the URL get repeated.  Or you can just browse the
Zope.org logs to find googlebot and its excessive URLs.

Shane