My robots.txt check passed for six days on an attacker's robots.txt

php dev.to

Every morning a script checks my site's health and emails me. One of its lines, for six days running: Fine. Sitemap index lists 54 child sitemaps holding 6,240 URLs in total, robots.txt points at it, and every page earning impressions answers 200. For those six days, this was my robots.txt: User-agent: * Allow: / Sitemap: https://utilorax.com/?sitemapindex.xml Sitemap: https://utilorax.com/?sitemap249.xml Sitemap: https://utilorax.com/?sitemap476.xml Sitemap: https://utilorax.com/?s

Read Full Tutorial open_in_new
arrow_back Back to Tutorials