crawler

Run the provided assembly

	./crawl <domain-name>

Run the tests

You may need to download SBT.

	sbt test

Compile an assembly

You may need to download SBT.

	sbt assembly

Trade-offs

• Poor error-handling strategy
• No retry strategy
• No graphical interface
• Not used the Actor model to encapsulate the mutable shared state in the parallel crawler
• Wrote in Scala and used a few libraries, ended up with an assembly of substantial size

Context

Job interview's exercise