Robots 2005 Internet Archive __top__ Instant

IA_Support_Team Date: October 12, 2005 Applies to: Webmasters, site owners, and robot enthusiasts Hey everyone, By now, many of you have noticed our bot, ia_archiver , visiting your sites. We’re the Internet Archive (archive.org), and we’re building the Wayback Machine – a historical record of the web.

— The Internet Archive Crawler Team

Thanks for helping us build a web history – or for choosing to stay out of it. Either way, robots.txt is your voice. robots 2005 internet archive