Skip to content

a scrapping bot that generates internal links & open assessable API report for any website by crawling every page.

Notifications You must be signed in to change notification settings

BigVeezus/NodeCrawler

Repository files navigation

NODE-CRAWLER

An application that crawls and scraps all internal links of a website/url.

Installation

Install the project with npm install

  npm install 

Set-Up

To begin, in your command line in the project enter

node main.js <url>

Make sure you replace the "url" in the field with the url you want to scrape.

After the url has been scraped, a Report.txt file will be generated at the bottom of your directory.

Open and view your internal links, Thank you for using this project.

Authors

About

a scrapping bot that generates internal links & open assessable API report for any website by crawling every page.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published