The Screaming Frog SEO Spider is small desktop program you can install on your PC which spiders Web site links, images, CSS, script and apps. It also fetches key onsite page elements for SEO, presents them in tabs by type and allows you to filter for common SEO issues, or slice and dice the data how you see fit by exporting and importing into Excel.
You can view, analyze and filter the information as its gathered and updated continuously in the programs user interface. The Screaming Frog SEO Spider allows you to quickly analyze or review a site from an onsite SEO perspective. Its particularly good for analyzing medium to large sites where manually checking every page would be extremely labor intensive and where you can easily miss a redirect, meta refresh or duplicate page issue. The spider allows you to export key onsite SEO elements (URL, page title, meta descriptions, headings) to Excel so it can easily be used as a base to make SEO recommendations from.
March 01, 2014
Version: Screaming Frog SEO Spider 2.22
This is one of the best tools in the market for conducting on-site website audits
None that I could think of or find
I dont know why the previous reviewer could not use the tool. From the message, it seems that his site probably had a meta-no-index in the robots or on the pages themselves. By default, the tool obeys all instructions in the robots file.
Overall, the tool is great and must for any SEO/webmaster. And all questions/tech support is answered promptly (within hours most of the time).