From Software Atelier:
SpidEye is like a spider's eye, showing what a Web Crawler would see while navigating the Web. From each page visited it collects and summaries relevant information that are meant to assist Web masters on the design of their Web pages for bots.
This freeware provides full HTML navigation, displays NoFrames section of a compound document (FrameSet). Validates documents (W3C) and shows source code in advanced multi syntax highlighter. Generates a unique word list (for spell check and word analyzes). Calculates file's hash, displays all FavIcon images, analyzes robots.txt. Uses no DLLs nor external dependencies. Version 1 includes unspecified updates.