Robots.txt | Now with added Sitemaps!
The robots.txt file is a rather obscure file that would normally only be fully appreciated by deep geeks, but its importance in search engine optimsation should not be under estimated. Simply put, a contemporary robots.txt guides search engines to the site’s XML Sitemap; this is the website’s street map especially designed for search engine spiders. Read more [...]