[Project_owners] Bot ignoring robots.txt

Pete Collins pete at mozdev.org
Wed Oct 29 10:01:37 EST 2003

massey wrote:
> Block those bot's IP's. Nothing else will work. If they are so rude as 
> to ignore your robots.txt you should just worry about mozdev. There must 
> be something about this kind of terrorism in the 'homeland security act' 

Tell me about it.

Blocking IP's is my last resort. It's like sticking fingers in a dam, 
eventually it gets unmanageable.

The box seems a lot more responsive. So I think the changes I made 
yesterday are having some affect.


Pete Collins

More information about the Project_owners mailing list