Norconex HTTP Collector

Norconex HTTP Collector is a web spider or crawler designed to help you browse the Internet and extract the information that you need for your projects. It is a portable tool that can be used in command line mode or as a Java library in order to load a certain configuration. Use the Norconex HTTP Collector as a command-line application or java library. Since it created with Java, you can use this tool on multiple platforms and extend it by creating custom code.