Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

how do you guys back up your iceberg table? #10299

Open
JunseoChoJJ opened this issue May 10, 2024 · 0 comments
Open

how do you guys back up your iceberg table? #10299

JunseoChoJJ opened this issue May 10, 2024 · 0 comments
Labels
question Further information is requested

Comments

@JunseoChoJJ
Copy link

JunseoChoJJ commented May 10, 2024

Query engine

No response

Question

my situation is on-prem on top of that using k8s on top of that using trino or spark + iceberg + object storage

I just want to ask you guys how to back up your iceberg table in obs when one obs is down, what is your execution plan?

I am not sure this is a right skill.
Dual Object Storage with Two Iceberg Connectors:
Set up two separate object storage systems.
Configure two Iceberg connectors in your Trino Helm chart, each pointing to one of the object storage systems.
When querying data, create two Iceberg tables for each object storage system.

second,
Single Object Storage with One Iceberg Connector:
Set up a single object storage system.
Configure one Iceberg connector in your Trino Helm chart to connect to this object storage.
When querying data, you obtain metadata and data files (.json, .avro) from the Iceberg table.
Copy these files to another object storage system manually or through automated scripts.

if there is better architecture, Could you please tell me?

@JunseoChoJJ JunseoChoJJ added the question Further information is requested label May 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

1 participant