There is many ways to tell the google bot not to index pages: Using the Allow Directive in the robot.txtUser-Agent: gsa-crawlerDisallow: /folder1/Allow: /folder1/myfile.html Using Robots META Tags to Control Access to a Web PageUsing .htaccessUser-agent: *Disallow: /tralllala But how does it work, if you just want part from your pages not get craweled? Excluding Unwanted …
The Website Architecture – SEO folks know that once a Web portal get up to more than thousands ore even hundred thousands or million of pages it gets difficult to bulk of the juice to the most valuable pages of the project or most profitable product content pages. Here is some speech from Dr. Pete …
How can I make sure Google reaches my deeper pages?Pai from Portugal asks Matt Cutts (Google): “How can I make sure that Google reaches and indexes pages that are on a lower (deeper) level of a website?”Here Matt Cutts answer:
