# All robots will spider the domain User-agent: * Disallow: Disallow: /catalog/tmp Disallow: robots.txt Disallow: /catalog/download Disallow: /catalog/includes Disallow: /catalog/images