[Fasterfox] RE: Abusing sites with pre-fetching
ben at intensewd.com
Thu Dec 15 15:03:31 EST 2005
By only using robots.txt you limit the website to absolutely "allow or
disallow", with no middle ground for the people who are really on the
connections Fasterfox can help.
Please also consider 'throttling' Fasterfox so that at most it may use a
limited amount of bandwidth. I would suggest maxing it at 10kb/sec, and
limiting how many requests they make at a time. This would allow dial up
users to prefetch at their maximum capacity, and broadband users to prefetch
at a limited capacity. For dialup and 128 or 256 (even to some extent 512)
kb adsl connections this would be enough to improve their experience with
consideration to all parties involved.
I cannot speak for all website owners, but I do pay $400/month in hosting,
and I would be most open to leaving prefetching enabled in such a scenario.
I'm quite sure that many others would stop condemning or arguing against
prefetching with an implementation like that (and if they can opt out with
robots.txt then they *have* no arguments anyway).
Also I have a question - what happens when you link to a folder
(http://some-website.com/stuff, as opposed to
http://some-website.com/stuff/index.html) and the server's determining what
document to display, is that prefetched?
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Fasterfox