Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ELE-1628] Call dbt deps on Dockerfile instead of running it on the edr run. #1110

Open
elongl opened this issue Aug 23, 2023 · 2 comments
Open

[ELE-1628] Call dbt deps on Dockerfile instead of running it on the edr run. #1110

elongl opened this issue Aug 23, 2023 · 2 comments
Labels
Feature Request 💡 Good First Issue 🥇 Good for newcomers Open to contribution 🧡

Comments

@elongl
Copy link
Member

elongl commented Aug 23, 2023 •
edited

Is your feature request related to a problem? Please describe.
Running dbt deps on the container run creates network I/O which for some users is problematic and also slows down the execution time.

Describe the solution you'd like
Run dbt deps as part of the Dockerfile.

ELE-1628
@elongl elongl changed the title Call dbt deps on Dockerfile instead of running it on the edr run. [ELE-1628] Call dbt deps on Dockerfile instead of running it on the edr run. Aug 23, 2023
@elongl
Copy link
Member Author

elongl commented Aug 23, 2023

The solution should probably be:

ELEMENTARY_PKG_LOCATION=$(pip show elementary-data | grep -i location | awk '{print $2}')
ELEMENTARY_DBT_PROJECT_PATH="$ELEMENTARY_PKG_LOCATION/elementary/monitor/dbt_project"
dbt deps --project-dir "$ELEMENTARY_DBT_PROJECT_PATH"

Or reading this variable via Python code.

@yimingpeng yimingpeng mentioned this issue Nov 26, 2023
Closed
@yimingpeng
Copy link

heya @elongl, did a PR for this request, please see #1296, any feedback would be appreciated, thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Feature Request 💡 Good First Issue 🥇 Good for newcomers Open to contribution 🧡
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

ELE-1628 - Move dbt deps to docker building-time rather than run-time yimingpeng/elementary
2 participants
@yimingpeng @elongl