How to setup your Robots TXT file
Login to the CMS > Go to Robots TXT in the node tree.
Enter the relevant information in the field 'Crawl Instructions' provided. If not set already, assign the Sitemap XML Location using the picker provided.
If you are using the Robots TXT node to control your robots.txt file you will need to add the following rewrite rule to your web.config file:
<rule name="Robots TXT" stopProcessing="true">
<match url="robots.txt" />
<action type="Rewrite" url="robots-txt" />
IIS URL Rewrite module must be installed on your webserver to use this feature.
uSkinned Site Builder
Get started — for FREE!
Join thousands of agencies, freelancers & businesses of all sizes worldwide who use uSkinned Site Builder for Umbraco to deliver results.