Skip to content

Latest commit

 

History

History
25 lines (18 loc) · 695 Bytes

README.md

File metadata and controls

25 lines (18 loc) · 695 Bytes

Data Crawler

This is a simple data crawler that crawls data from a website and stores it in a database. Then you can download your data in your local storage. For now it only supports crawling data from houzz, amazon(only some products), google-images, youtube.

Installation

Go to the root of the project and run the following commands:

pip install -r requirements.txt
pip install -e .

Usage

crawler

Contributing

Pull requests are welcome. You can add your own website crawler and downloader to help the project grow.