cheers - as alwaysAsiaplay - just double checked and you are right (I didn't see it)

Disallow: /shop/*price=*
Disallow: /shop/*title=*

I have now included them , and will run another WSA2 check (I'll get them out of the Google cache first though)

The robots.txt has been the same for a while though (at least for the last 9 months)

UPDATE - Just had it confirmed that the following (which was already in my robots.txt file - for a long time) tells Google not index them

Disallow: /shop/*sort=*
Disallow: /shop/*sort_direction=*

So why I wonder does WSA2 see them as duplicated meta desc's - how long does Google cache them for

Brent,

Thanks for the useful observations.



Yes, it'll be possible to specify pages to exclude from the project.



This is also going to be fixed in one of the nearest updates.

Hi Asiaplay,



Right now it doesn't but this part of functionality is being developed now (almost ready in fact). Duplicate content won't be considered for pages disallowed for all web bots. However, it's not quite clear at this point how to treat pages restricted only for certain search engines. If you have any ideas on that, please feel free to share.

I ran a project on a small site with 12 pages. Every page loads fine in the browser yet WA tags every page with a 403 Forbidden.

As one example of why this would seem to be impossible we can check here: http://www.checkupdown.com/status/E403.html

And their sample Forbidden page here: http://www.checkupdown.com/accounts/grpb/B1394343/

If it's a 403 Forbidden then the page will not load, correct? Yet all of the client's pages load while WA indicates they are all 403's. And, yes, WA does provide other information in the report so it really is reading the page, it just seems to indicate in that one report section that it cannot read the page.

Again, client's pages all load fine in Firefox and IE.

From the HTML Validity section.
-------------------------------------
HTML validity of pages' code - detailed information
Below are the pages grouped by specific HTML validation results. Get the problematic spots corrected to encourage search engines to index these pages quickly and precisely.

With errors and warnings - 12 pages
---------------------------------------------
Then WA lists all of the pages with errors. Should there not be an option at that stage to click that will then either list all of the errors or at least redirect to something like http://validator.w3.org/?

I think I remember WA doing that at a different part of the report but it would also be a natural fit at this stage of the report.

Ran new website with expert options checked. Something I checked (maybe pagerank for pages) caused the following Captcha screen to show up:

==============

Enter Captchas

Google requires you to enter a captcha before proceeding. Please enter a captcha, otherwise you will not be able to gather data from this search engine.

http://www.yourdomain.com/yourpage.php

Please type in the text from the above image:

Captchas remaining: 5

=============

Main issue: that "Captchas remaining" number is very, very wrong. It kept changing each time to a new small number (3, 2, 5, 4, etc.) but there were really HUNDREDS more coming at me. If the number isn't right, best to not show it at all.

Another problem: No way to get a new captcha when the one shown is completely unreadable and many were. I realize that's probably out of your hands, but thought I'd just make note of it anyway.

Minor issue:

In the Website Report:

Section titled: "Other title usage issues"

The issue says:

"Titles longer than 65 symbols"

symbols? Odd wording. Should probably be changed to:

"Titles longer than 65 characters"

A few pages show broken links but I've been unable to find any. It would be helpful if I could drill down to see which links the app thinks are broken. I've also rebuilt the pages in question in case it was a temporary situation the first time it ran, but they all still show broken links, even though I cannot find any links on the page that are broken.

Right now it doesn't but this part of functionality is being developed now (almost ready in fact). Duplicate content won't be considered for pages disallowed for all web bots. However, it's not quite clear at this point how to treat pages restricted only for certain search engines. If you have any ideas on that, please feel free to share.[/quote]

From Voodoo1967 - For info I just had a response back re this
In addition to the robots.txt CDSEO also has a canonical link tag which tells search engines that when they access http://www.mydomain.co.uk/shop/silver-p ... irection=1 they should only index http://www.mydomain.co.uk/shop/silver-picture-frames/

Hopefully WSA2 will pick on this issue, I had a bit of a wild goose chase with it.

I run a WSA on a page on my store www.mydomain.co.uk/shop/signature_frames.html

WSA then goes off and retrievs all the information accross the domain. I then click on Webpage Report
And WSA2 says in its report title www.mydomain.co.uk/shop/signature_frames.html
Report Overview
This is your personal onpage optimization report created for the webpage http://www.mydomain.co.uk/page_video.html

why does think its reporting on 2 different pages ?