disallowed in robots.txt

Learn more about robots.txt – Often pages that are set on “disallowed in robots.txt” anyway appear in Google’s search results. In the following robots.txt Video Matt Cutts from Google explains why this happens. Follow Matt Cutts >@mattcuttson Twitter.

1. “disallowed in robots.txt

robots.txt – uncrawled URLs in search results

A complaint heard often is: “I have blocked Google from crawling a page and robots.txt and Google clearly violated that robot.txt by crawling that page because it’s showing up in Google search results anyway.”

From GoogleWebmasterHelp - Matt Cutts explains why a page that is disallowed in robots.txt may still appear in Google’s search results.


The Future of search

SEO vs Social Media Hi, I was asked in a SEO Workshop, “Why do you like this Social Media Communities so much?” all Links there are “nofollow”.

Is this really the Question nofollow or follow.

Isn’t Twitter the first real time search engine that delivers results from top actual user-generated content?

Aren’t there 8 Social Media Communities in the top 20 by alexa.com Germany and only 3 real search engines?

How can it be wrong for a SEO to be interested in Social Media?

Here is a very good Video to this Topic:



Google Caffeine

Matt has putted in context what will change with “Google Coffeine” he said that “Google Coffeine” is a rewrite of the Google Indexing System. A good search engine has to crawl really well, has to index really well and has to serve up, what mean has to return the most relevant pages for users very fast.

So as I understood Google has changed the way of crawling and indexing WebPages. So far this is nothing spectacular, because Google always does changes with crawling and indexing WebPages but Matt too said, that the changes that have been done infrastructure wise rufly compare to the “Big Daddy up date” that happened late 2005 to begin 2006, hi ;-) I think all SEOs who did already SEO in this time have some lasting memories.

Here is the first video from the SES in San Joes 2009 that I found to this topic, it’s an interview with Matt Cutts where he speaks about Google Caffeine. Follow > @mattcutts on Twitter



Too I found this video where Jim Stewart @jimboot the Australian Google Guru speaks about Google Coffeine. Follow > jimboot on Twitter



Google invites you to visit the web developer preview of Google’s new infrastructure at http://www2.sandbox.google.com/ you can try the new searches there.

Here more Info toGoogle Coffeine from the Google Webmaster Central Blog.

SEO Website Review by the Experts

SEO Website Review by the Experts
Google Search Quality Team - SEO Website Review by the Experts (Google I/O 2009) - This is a one hour SEO video, what you should definitely take the time to watch if you anyhow interest in SEO.

By reviewing different Websites the Google Search Quality Team gives you very good tips how to create a SEO optimized Website.

This is the Topics the Video is about:

  • Bolding text and keyword density
  • link description
  • Spam, shadow domains, door way pages
  • Keep it simple as simple as possible
  • Page Rank, no follow
  • non www or www, using redirects and splitting PR
  • Images - small or large?
  • Video Ranking
  • Factors for Google indexing and ranking a pages




My contact details: Ortwin Oberhauser . 6900 Bregenz . Austria. Mobile.: 0043 (0)664 7501 3618 . Email: ortwin@oberhauser.at