Working in progress
Web scraping even easier with F#.
This project does not provide means of parsing and extracting information from HTML files. For this purpose look for community project such as FSharp.Data.
- Reading robot.txt files and provide a configuration to enforce it
- Post requests
- Custom pipelines: export to csv, json, logging, statistics
- Imagem downloader