Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

馃惓 Dockerizing the CLI #101

Open
mattigrthr opened this issue Feb 28, 2022 · 0 comments
Open

馃惓 Dockerizing the CLI #101

mattigrthr opened this issue Feb 28, 2022 · 0 comments
Labels
CLI Issues related to the CLI core Issues related to the core help wanted Extra attention is needed

Comments

@mattigrthr
Copy link
Contributor

Currently, the CLI is implemented as a local Python script. Dockerizing the CLI reduces the local setup time and lowers the chances of errors.

Previously, we have tried to run the CLI as a Docker and created a "Docker in Docker" scenario which didn't work because of faulty volume mounting (see #99).

Alternatively, we could try to use Airflow for orchestrating the pipeline runs.

This is the user story to provide some background for the purpose of the CLI:

User Story

As a user, I want a CLI to select all the pipelines I'd like to run. I can choose the geographical region for which I want to run the pipelines. In the case of the population data, I also want to select specific demographic groups I am interested in.
Once I have made all my selections, the CLI will run all the pipelines correctly (e.g., the google-poi pipeline depends on the osm-poi pipeline). Once the data pipelines ran successfully, all the data should be imported into the Postgres database.
When all the data has been imported, the Jupyter environment should be launched so I can start working with the data conveniently.
Next to running the individual data pipelines, I want to be able to download the demo data through the CLI. Once the demo data is downloaded, the database and Jupyter notebook with the popularity correlation should be launched.

@mattigrthr mattigrthr added help wanted Extra attention is needed core Issues related to the core CLI Issues related to the CLI labels Feb 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLI Issues related to the CLI core Issues related to the core help wanted Extra attention is needed
Projects
Status: Todo
Development

No branches or pull requests

1 participant