Learn more about robots.txt – Often pages that are set on “disallowed in robots.txt” anyway appear in Google’s search results. In the following robots.txt Video Matt Cutts from Google explains why this happens.
Follow Matt Cutts >@mattcuttson Twitter.
1. “disallowed in robots.txt
A complaint heard often is: “I have blocked Google from crawling a page and robots.txt and Google clearly violated that robot.txt by crawling that page because it’s showing up in Google search results anyway.”
From GoogleWebmasterHelp -- Matt Cutts explains why a page that is disallowed in robots.txt may still appear in Google’s search results.
Welcome to my personal blog, where I share my Photos, my perspectives on SEO,
Social Media, Personal Branding, Online Marketing and Reputation Management. Read more >>
