[Zope] Zope URLs and spiders' behavior

[Dieter Maurer]

Sean Lee wrote at 2003-12-3 22:03 +0800:
 > I wonder if anyone has experienced this:
 > 
 > a) Google displays links to Zope site with a space in them 
 > For example, the search result has the right Title (and click on it
 > works), but underneath the page summary the URL (some, not all) may be
 > displayed like this
 > http://domain.com/dir/ subdir/index_html

I do not know this...

 > b) Spiders download non-existing directories
 > For example, the site has two directories, site.com/dir1, site.com/dir2
 > The weird thing is that dir1 can be accessed in these (and more) ways:
 > http://site.com/dir2/dir1
 > http://site.com/dir1/dir2/dir1
 > Unnecessary to mention, the spider keeps downloading all the time.
 > The worst of all is, the directories really can be accessed that way.
 > Why is this possible?

This is acquisition at work.

It is caused by non-trivial relative URL references (relative
URL references (those not starting with a protocol nor with a '/')
which contain at least one "/").

Do not use non-trivial URL references (use absolute URLs instead
or explicitely use sufficiently many "../" in your relavtive URLs).

-- 
Dieter