The robots.txt file is a rather obscure file that would normally only be fully appreciated by deep geeks, but its importance in search engine optimsation should not be under estimated.

Simply put, a contemporary robots.txt guides search engines to the site’s XML Sitemap; this is the website’s street map especially designed for search engine spiders. Read more about sitemaps

Historically the robots.txt file’s only role was to inform search engine spiders which website files and folders it may not index.

In April 2007, the Sitemap Protocol was implemented elevating robots.txt to main stream relevance .  Refer www.sitemaps.org/protocol.php

Read more about the role of the XML Sitemap and why it is so important for optimsing your website.

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.