When it comes to Search Engine Optimisation or SEO, most people understand that a Web site must have quality content, “search engine friendly” site architecture, meta data (title tags, meta description, and meta keywords tags) and etc.

However, many so called ‘optimised’ websites that I’ve checked recently have omitted the robots.txt file. When optimising your website, do not belittle the power of this text file.

In a nutshell, if you go to www.yourdomain.com/robots.txt, you should see a list of directories that the site owner or the SEO agency is getting the search engines to skip or not to engage. However, if you’re not careful, you may be putting information in this file that could really damage your business leads.

iClick Media