Skip to main content

How to setup your Robots TXT file

Step 1

Login to the CMS > Go to Robots TXT in the node tree.

Step 2

Enter the relevant information in the field 'Crawl Instructions' provided. If not set already, assign the Sitemap XML Location using the picker provided.

Step 3

If you are using the Robots TXT node to control your robots.txt file you will need to add the following rewrite rule to your web.config file:

<rule name="Robots TXT" stopProcessing="true">
<match url="robots.txt" />
<action type="Rewrite" url="robots-txt" />

IIS URL Rewrite module must be installed on your webserver to use this feature.