Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

truncate the staging dataset tables after load #1104

Closed
rudolfix opened this issue Mar 18, 2024 · 0 comments · Fixed by #1292
Closed

truncate the staging dataset tables after load #1104

rudolfix opened this issue Mar 18, 2024 · 0 comments · Fixed by #1292
Assignees
@IlyaFaer
Labels
community This issue came from slack community workspace

Comments

@rudolfix
Copy link
Collaborator

Feature description

I want to free space in the staging dataset and truncate the tables after load

Are you a dlt user?

Yes, I'm already a dlt user.

Use case

No response

Proposed solution

We can reuse the storage init method in the loader and run the truncation again after the load is fully completed. this is done on best effort basis and should not fail the overall load

Related issues

No response
@rudolfix rudolfix added the community This issue came from slack community workspace label Mar 18, 2024
@IlyaFaer IlyaFaer mentioned this issue Apr 26, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@IlyaFaer IlyaFaer

Labels
community This issue came from slack community workspace
Projects
dlt core library
Status: Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat(pipeline): add an ability to auto truncate dlt-hub/dlt
2 participants
@rudolfix @IlyaFaer