[Feature] Support dispatch
for materializations to use implementations defined in installed packages
#10090
Open
3 tasks done
Labels
Milestone
Is this your first time submitting a feature request?
Describe the feature
This is a way of preserving a more granular version of the previous behavior, which would implicitly (and somewhat surprisingly) use materializations from packages that override builtins (
view
,table
,incremental
,test
, etc).We are opting for this syntax:
Considerations
If not specified, the default would remain:
dbt-core
/ adaptersWhich is how it works for materializations with the flag, and for dispatched macros.
Implementation
The relevant method is
find_materialization_macro_by_name
.Michelle spiked a similar capability, though by pulling the "allowlist" from
Project.flags
rather thanProject.dispatch
: 99998a5Describe alternatives you've considered
(1) Not doing this
Users must reimplement materializations one-by-one, by defining them in their root project and calling the implementations in packages.
(2) Bundling with existing macro dispatch
Under the hood, these materializations are macros, and defined within the
'dbt'
namespace. But I like the idea of continuing to keep the two separate, for three reasons:(3) Different syntax that wouldn't require evolving the type of
dispatch
configEdge case: this would not play well with an installed package named
materialization
.Who will this benefit?
elementary
package (Instruct users to explicitly overrideview
andincremental
materializations elementary-data/dbt-data-reliability#703)Are you interested in contributing this feature?
Yes, with the help of the elementarians
Anything else?
We should do this for dbt Core v1.8.x only. No backports. While we introduced the deprecation warning in dbt-core v1.6 + v1.7, it's still the default behavior in those versions to use implementations defined in packages.
We will document in the v1.8 upgrade guide and "legacy behaviors" the recommended sequence of:
False
to preserve existing behavior (regardless of the version they're running on), and to unblock their upgrade to v1.8dispatch
, set the flag toTrue
/ remove the flag setting, and confirm that they're seeing the same behavior as beforeThe text was updated successfully, but these errors were encountered: