Login to the CMS > Go to Robots TXT in the node tree.
Enter the relevant information in the field 'Crawl Instructions' provided. If not set already, assign the Sitemap XML Location using the picker provided.
If you are using the Robots TXT node to control your robots.txt file you will need to add the following rewrite rule to your web.config file:
<rewrite> <rules> <rule name="Robots TXT" stopProcessing="true"> <match url="robots.txt" /> <action type="Rewrite" url="robots-txt" /> </rule> </rules></rewrite>
IIS URL Rewrite module must be installed on your webserver to use this feature.
Join thousands of agencies, freelancers, enterprises & start ups worldwide who use uSkinned Site Builder for Umbraco to deliver results.
Take the tour
NEW Site Builder!
We have launched a brand new, game-changing product for Umbraco CMS.