[Zope] TIP: Robots.txt

[Lennart Regebro]

From: "BZ" <bz@bwanazulia.com>
> It is always the trade off when  you tell robots what to and what not
> to index. I will let Zope security keep up the bad robots, but I rather
> good robots not even try.

True, but how would a good robot even try /acl_users? It doesn't know it
exists, unless you link to it from your frontpage. :-)

Robots follow the links they can see. If you don't tell them about
acl_users, they won't look there.