Skip to main content Skip to footer

How to setup your Robots file

Step 1

Login to the CMS > Go to Robots TXT in the node tree.

Step 2

Enter the relevant information in the field 'Crawl Instructions' provided. If not set already, assign the Sitemap XML Location using the picker provided.

Step 3

If you are using the Robots TXT node to control your robots.txt file you will need to add the following rewrite rule to your web.config file:

<rule name="Robots TXT" stopProcessing="true">
<match url="robots.txt" />
<action type="Rewrite" url="robots-txt" />

IIS URL Rewrite module must be installed on your webserver to use this feature.


Step 1

Sign up

Start a free trial of Site Builder.

Step 2


Tailor the look and feel with the Visual Designer to suit your brand.

Step 3


Add content with built-in eCommerce and third-party integrations.

Step 4


World-class and secure hosting backed by Microsoft Azure.