Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DBT Invocations: support task_id and try_number #1472

Open
kokorin opened this issue Mar 20, 2024 · 1 comment
Open

DBT Invocations: support task_id and try_number #1472

kokorin opened this issue Mar 20, 2024 · 1 comment

Comments

@kokorin
Copy link

kokorin commented Mar 20, 2024

Is your feature request related to a problem? Please describe.

We run DBT with Elementary package in Airflow. Airflow DAG contains several DBT build tasks: core models first, then integration models (based on core). We want to build separate Elementary reports for core module and for each integration module:

flowchart LR
  freshness --> build_core --> elementary_core
  build_core --> build_integration_1 --> elementary_integration_1
  build_core --> build_integration_2 --> elementary_integration_2

For that we need to get dbt_invocation UUID from DBT Invocations table and use it with edr report --select invocation_id:XXXXXXXXXXXXX.

It would be useful to have Airflow task_id and try_number in DBT Invocations table to simplify dbt_invocation lookup

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
It's possible to include task_id and try_number into job_id or job_run_id, but Elementary uses the latter vars to build job_url and job_run_url.

Additional context
N/A

Would you be willing to contribute this feature?
Yes, but I don't have DBT Cloud and have limited experience with Github Actions.

@haritamar
Copy link
Collaborator

Hi @kokorin !
Thanks for opening this issue, and sorry about the delayed response.

This is a cool use case! Makes sense to me overall to save this information. I'd use separate fields rather than incorporate it into existing ones.
I think also that since this information is passed as external variables to dbt, then the feature is generic and doesn't necessarily require explicit logic for dbt Cloud / github actions, so I think you should be fine there.

Feel free to open a PR that adds this capability, lmk if further guidance is needed!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants