New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add example DAG for databricks delta support #1585
base: main
Are you sure you want to change the base?
Conversation
Codecov ReportPatch coverage has no change and project coverage change:
Additional details and impacted files@@ Coverage Diff @@
## main #1585 +/- ##
==========================================
+ Coverage 91.69% 97.67% +5.98%
==========================================
Files 72 23 -49
Lines 4250 904 -3346
Branches 511 0 -511
==========================================
- Hits 3897 883 -3014
+ Misses 257 21 -236
+ Partials 96 0 -96
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
@dimberman Can you get this to the finish line please? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Happy for this PR to be merged once the checks pass
@dimberman, it would be great if we could address CI issue #1744. before merging this..! How is that going? |
waiting on #1744 |
Description
What is the current behavior?
The Astro SDK currently does not have an example for loading, transforming, and processing data using Databricks' delta tables.
What is the new behavior?
This PR adds an example to the Astro SDK for loading, transforming, and processing data using Databricks' delta tables. The example includes the following steps:
load_file
function.transforn
functionaql.dataframe
function.Does this introduce a breaking change?
No, this is a new feature and does not affect existing functionality.
Checklist