The robots.txt file is one of the most important technical SEO files on a website. It tells search engine crawlers…