Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: replace cubestore's GCP service account with WorkloadIdentity #8148

Open
wants to merge 9 commits into
base: master
Choose a base branch
from

Conversation

BenHaimItay
Copy link

Check List

  • Tests has been run in packages where changes made if available
  • Linter has been run for changed code
  • Tests for the changes have been added if not covered yet
  • Docs have been added / updated if required

Issue Reference this PR resolves
#4434

Description of Changes Made (if issue reference is not provided)

in cubestore/cubestore/Cargo.toml upgrade from cloud-storage 0.7.0 to the 0.11.1 fork by @dark0dave additionally make sure to lock gcp_auth to 0.7.4 and time to 0.3.17 otherwise it won't compile with the toolchain channel = "nightly-2022-11-03". The original cloud-storage does not seem to be actively maintained.
cloud-storage = { git = "https://github.com/dark0dave/cloud-storage-rs", branch = "master", features = ["global-client", "sync", "native-tls"] }
gcp_auth = { version = "=0.7.4" }
time = { version = "=0.3.17" }
From remotefs/gcs.rs remove ensure_credentials_init to remove the requirement on the environment variable.

Adapt remotefs/gcs.rs to cloud-storage 0.11.1 + updated list_with_metadata and imports.

@BenHaimItay BenHaimItay requested a review from a team as a code owner April 15, 2024 11:58
Copy link

vercel bot commented Apr 15, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

8 Ignored Deployments
Name Status Preview Comments Updated (UTC)
examples-angular-dashboard ⬜️ Ignored (Inspect) Visit Preview Apr 19, 2024 8:51am
examples-react-d3 ⬜️ Ignored (Inspect) Visit Preview Apr 19, 2024 8:51am
examples-react-dashboard ⬜️ Ignored (Inspect) Visit Preview Apr 19, 2024 8:51am
examples-react-data-table ⬜️ Ignored (Inspect) Visit Preview Apr 19, 2024 8:51am
examples-react-highcharts ⬜️ Ignored (Inspect) Visit Preview Apr 19, 2024 8:51am
examples-react-material-ui ⬜️ Ignored (Inspect) Visit Preview Apr 19, 2024 8:51am
examples-react-pivot-table ⬜️ Ignored (Inspect) Visit Preview Apr 19, 2024 8:51am
examples-vue-query-builder ⬜️ Ignored (Inspect) Visit Preview Apr 19, 2024 8:51am

@github-actions github-actions bot added the pr:community Contribution from Cube.js community members. label Apr 15, 2024
@BenHaimItay BenHaimItay changed the title Replace cubestore GCP service account with WorkloadIdentity feat: replace cubestore's GCP service account with WorkloadIdentity Apr 15, 2024
@paveltiunov
Copy link
Member

@BenHaimItay Thanks for submitting it! We're open to having identity support for GCS! For this PR, we'd need to

  1. Make sure existing functionality with env variables still works,
  2. The GCS library used is either official GCS or, if absolutely necessary, the fork should land under https://github.com/cube-js org.

@BenHaimItay
Copy link
Author

My pleasure :)

  1. This requires some more work since currently only Workload Identity is supported ( hope I can get this working during the weekend )
  2. I think I will have to ask you to fork cloud-storage-rs into cube's org :)

@BenHaimItay
Copy link
Author

Hi @paveltiunov,
I made a few modifications to support both SA and WI. however, I would appreciate if someone can go over it thoroughly.

In addition, the cloud-storage should be forked into your org account :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr:community Contribution from Cube.js community members.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants