[Project_owners] Bot ignoring robots.txt

Ben Bucksch ben.bucksch.news at beonex.com
Tue Oct 28 16:02:32 EST 2003


Pete Collins wrote:

> Here is an example of a bot ignoring robots.txt just today:

You said you have the robots.txt in the bugs/ dir. Does 
xprint.mozdev.org/robots.txt also disallow bugs/?



More information about the Project_owners mailing list