What is robots.txt? Does it have any value for SEO?
2 posters
Author
Message
commodity
Posts : 63 Join date : 2012-01-03
Subject: What is robots.txt? Does it have any value for SEO? Wed May 23, 2012 4:05 am
it is used by Google to know what you do NOT want indexed. So if you have content on your website, you do NOT want crawled, indicate that in robots.txt. .
JamesWt
Posts : 28 Join date : 2012-05-19
Subject: Re: What is robots.txt? Does it have any value for SEO? Tue May 29, 2012 7:21 am
robots.txt is a file that allows or disallows the spiders to index the intended files or folders.
What is robots.txt? Does it have any value for SEO?