Robots.txt
January 7, 2008 – 11:32 amYesterday, while blog hopping, I came across a blog article, I don’t even remember where anymore, that the author said that we should create a robots.txt and upload it in the root directory of our server and this will cause the search engines to better crawl our blog and hence would not return as many errors as a blog without the robots.txt file.
Some people who already did this have reported an increase in blog traffic and one person even says that his traffic went up by more than 11%. That’s a lot of traffic which I could certainly do with.
As you can tell, yes, I already included the robots.txt file to all my WordPress blogs which I also added in my sitemap and then submitted the file to my Google webmaster account.
Since doing this, I did not notice any increase in traffic specifically from the search engines and I haven’t found the time to check the status of this from my Google Webmaster account.
Let’s give it a week and see what happens. I hope I don’t have to mess with anything technical and just blog but then there are times when there’s a call to get my hands dirty. Bah!!!
