Learn more about robots.txt – Often pages that are set on “disallowed in robots.txt” anyway appear in Google’s search results. In the following robots.txt Video Matt Cutts from Google explains why this happens. Follow Matt Cutts >@mattcuttson Twitter.
1. “disallowed in robots.txt
This post is also available in: German
Posted in 
Tags: 
