Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Question: How can I use Dataform to replace data for a specific date? #1721

Open
yoshitaku-jp opened this issue Apr 20, 2024 · 0 comments
Open

Comments

@yoshitaku-jp
Copy link

Hello. I have a question about using Dataform.
Is there a way to replace data for a specific date using Dataform?

In SQL terms, here is the kind of operation I'm looking for:

-- Delete existing data for a specific date
DELETE FROM sink_table
WHERE date = '2024-04-01';

-- Fetch data for the specific date from source_table and insert into sink_table
INSERT INTO sink_table (date, user_id)
SELECT date, user_id
FROM source_table
WHERE date = '2024-04-01';

Currently, I achieve this using the following SQLX:

config {
  type: "incremental",
  schema: "sample_schema",
  name: "sink_table",
}

pre_operations {
    DELETE FROM ${self()} WHERE date = '2024-04-01'
}

SELECT 
  date,
  user_id,
FROM 'source_table'
WHERE date = '2024-04-01'

Is there a more streamlined way to accomplish this using Dataform features?

Best regards

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant