Chilkat Spider ActiveX for Windows

Name: Chilkat Spider ActiveX
Rating: 1 (1 reviews)
Author: Chilkat Software

By Chilkat Software
Free
User Rating

Download now

Key Details of Chilkat Spider ActiveX

Cache, "avoid" patterns, handle HTTP URLs automatically.
Last updated on July 16, 2015
There have been 4 updates
Virus scan status:
Clean (it’s extremely likely that this software program is clean)

Developer’s Description

By Chilkat Software

Cache, "avoid" patterns, handle HTTP URLs automatically.

Crawl a Web site with this free ActiveX spidering component. Advanced features include caching, "avoid" patterns, and robots.txt compliance. Automatically handles HTTPS URLs. Includes safety measures for avoiding large / infinite size Web pages or infinitely growing-in-length URLs. Includes a wind-down features for crawling a maximum number of pages per site.