# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file # robots.txt for http://www.spaarinformatie.nl/ User-agent: * Disallow: /javascripts/ Disallow: /stylesheets/ # Autodiscover sitemap by spiders Sitemap: http://www.spaarinformatie.nl/sitemap.xml User-agent: ia_archiver Disallow: /