This Web spider utility detects and displays detailed information for a user-selected Web page, and it offers other Web page tools.
BlackWidow's clean, logically tabbed interface is simple enough for intermediate users to follow but offers just enough under the hood to satisfy advanced users. Simply enter your URL of choice and press Go. BlackWidow uses multithreading to quickly download all files and test the links. The operation takes only a few minutes for small Web sites.
Webmasters will love the comprehensive list of broken links and error codes. Finding file locations is simple, since the utility displays the full Web site directory structure. A handy feature searches a Web site to find all properly formatted e-mail addresses, although a tool to find broken addresses would have been a nice addition. It's easy to track the spider's process in the Threads tab, and you'll find a powerful net sniffer in the Net Spy tab that can watch all traffic that moves through your network card. Advanced users will enjoy tools to create plug-ins and apply filters to the spidering process.
The program loads with tips and information and includes a demo. The information is helpful, but the demo is incomplete, leaving some users with only a partial idea of the program's flexibility. Advanced users will appreciate BlackWidow's numerous features and flexibility.
From SoftByte Labs:
BlackWidow is a multi-function internet and webmaster tool. It is a web site downloader, site mapping tool, a site ripper, a site mirroring tool, a website scanner, network spy, and a download manager. Use it to scan a site and create a complete profile of the site's structure, files, external links and even link errors. Then use it to download part or entire web site to your computer, with its structure and files intact, to use as a site mirror or for long-term reference. Or use it to scan for and download any selection of files: from JPG to CGI to HTM to MIME types, from small to large files, in part of a site or in a group of sites. These pre-scan filtering options can save you countless online hours of searching and sorting. BlackWidow will scan HTTP sites, SSL sites (HTTPS), FTP sites, proxy servers. It will accesses password protected sites, use threads, pull links from Java Scripts and Java Scripts files, will scan Adobe Acrobat (.pdf) files for links. BlackWidow will download all file types such as picture and image, from JPG to PNG, audio and MP3, video such as YouTube, MySpace, Google, AVI, document, ZIP, Flash, program, CSS, Macromedia Flash, .pdf , PHP, CGI, HTM to MIME types from any web sites. Will pull links from Java Scripts and Java Scripts files, will scan Adobe Acrobat (.pdf) files for links from any web site. This feature is especially useful to find hidden links to files, or to spot redirections to other URLs. Use it when loading a page to find out what files and images are being loaded, then download them, or even the links to popup windows. BlackWidow can be controlled programmatically using the built-in Script Interpreter. The "View Source" has been updated to view the source page of every Frames and IFrames.
What it promised was good but what I got was the directory list but no content from the website. The flash files and images were not collected in the folder as promised and it failed with flying colors upon test.
They don't get much worse than this. I am going to have to look for something better to pull content from websites.
Save your time. Save yourselves. Look elsewhere unless you are a sadist and like beating yourself over the head to make this work.
January 31, 2013
Version: BlackWidow 6.30
I haven't finished evaluating the program yet, but so far it seems very fast, flexible and full featured. Of all the things it says it does, I can vouch for all the ones that fall under the "Basic" tab. I will leave it to someone else to evaluate the Advanced features. If you have a use for this, it's going to save you some SERIOUS time and frustration!