Published: 6. October 2009

robots.txt – uncrawled URLs in search results

Ortwin Oberhauser Avatar
By:
Ortwin Oberhauser

A complaint heard often is: “I have blocked Google from crawling a page and robots.txt and Google clearly violated that robot.txt by crawling that page because it’s showing up in Google search results anyway.”

From GoogleWebmasterHelp – Matt Cutts explains why a page that is disallowed in robots.txt may still appear in Google’s search results.

Author: Ortwin Oberhauser
INITIATOR OF SEOLOGY & WORLD’S FIRST SEOLOGE

BSc Applied Computer Science
SEM / SEO & Conversion Optimization Geek


Founder of Oberhauser.com
Co Founder of bobdo.com International Film & Digital Solution Agency
film production . web design . app & web development data driven performance search & social media marketing ai and search engine content & visibility optimization conversion optimizing . datacenter for web data analytics
Bahnhofstraße 10 . 6900 Bregenz . Austria

bobdo.com

Let’s discuss it

Leave a Reply

Your email address will not be published. Required fields are marked *