Chilkat Spider ActiveX for Windows

Name: Chilkat Spider ActiveX
Rating: 1 (1 reviews)
Author: Chilkat Software

By Chilkat Software
Free
User Rating

Download now

Used Chilkat Spider ActiveX for Windows? Share your experience and help other users.

Developer’s Description

By Chilkat Software

Cache, "avoid" patterns, handle HTTP URLs automatically.

Crawl a Web site with this free ActiveX spidering component. Advanced features include caching, "avoid" patterns, and robots.txt compliance. Automatically handles HTTPS URLs. Includes safety measures for avoiding large / infinite size Web pages or infinitely growing-in-length URLs. Includes a wind-down features for crawling a maximum number of pages per site.