Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generated incremental SQL when table does not exist should not do a create or replace #1561

Open
lewish opened this issue Oct 31, 2023 · 1 comment · May be fixed by #1712
Open

Generated incremental SQL when table does not exist should not do a create or replace #1561

lewish opened this issue Oct 31, 2023 · 1 comment · May be fixed by #1712

Comments

@lewish
Copy link
Collaborator

lewish commented Oct 31, 2023

In the rare situation that seems to have happened to a customer recently - that BigQuery tells us a incremental table does not exist even though it does - we should not generate a create or replace statement to rebuild the table in case it does actually exist, and instead generate a create table if not exists statement in case.

@lewish
Copy link
Collaborator Author

lewish commented Oct 31, 2023

Relevant code: https://github.com/dataform-co/dataform/blob/b5a561643e5020401639ef3a73aab6991083eb16/core/adapters/bigquery.ts#L34C46-L34C46

Should be:

  • Full refresh requested and NOT protected: create or replace
  • Otherwise, if the table does not exist: create table if not exists

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants