|Sitemap URL||Status Code|
|Resource URL||Status Code||File Type||Result||Host||robots.txt||robots.txt URL|
Learn more about controlling crawling and
Robots.txt files help you guide how search engines crawl your site, and can be an integral part of your SEO strategy. Learn more about Robots.txt files and their effect on SEO.
Robots tags are directives that allow you to control how search engines crawl and index the content of your website. They are page-level signals and can be implemented in a meta tag or in an HTTP response header.