Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ISSUE] cannot create grants: unexpected error handling request: invalid character '<' looking for beginning of value #752

Open
zaxier opened this issue Dec 28, 2023 · 7 comments
Open

[ISSUE] cannot create grants: unexpected error handling request: invalid character '<' looking for beginning of value #752

zaxier opened this issue Dec 28, 2023 · 7 comments
Assignees
@mgyucht

Comments

@zaxier
Copy link

zaxier commented Dec 28, 2023 •
edited by mgyucht

Description
Bug faced when using databricks_grants resource using databricks terraform provider.
Error: cannot create grants: unexpected error handling request: invalid character '<' looking for beginning of value. This is likely a bug in the Databricks SDK for Go or the underlying REST API. Please report this issue with the following debugging information to the SDK issue tracker at https://github.com/databricks/databricks-sdk-go/issues. Request log:

Reproduction

terraform apply
resource "databricks_metastore" "this" {
  name          = "metastore"
  owner         = var.account_admin_group_name
  region        = "ap-southeast-2"
  force_destroy = true
}

  
resource "databricks_grants" "this" {
  metastore = databricks_metastore.this.id
  grant {
    principal = var.account_admin_group_name
    privileges = ["CREATE_CATALOG", "CREATE_EXTERNAL_LOCATION"]
  }
  
  # grant {
  #   principal = var.workspace_user_group_name
  #   privileges = ["CREATE_SHARE", "SET_SHARE_PERMISSION", "USE_MARKETPLACE_ASSETS", "USE_CONNECTION", "USE_PROVIDER", "USE_RECIPIENT", "USE_SHARE"]
  # }

  # grant {
  #   principal = var.workspace_admin_group_name
  #   privileges = ["CREATE_EXTERNAL_LOCATION", "CREATE_CONNECTION", "CREATE_PROVIDER", "CREATE_RECIPIENT", "CREATE_SHARE", "CREATE_STORAGE_CREDENTIAL", "MANAGE_ALLOWLIST", "SET_SHARE_PERMISSION", "USE_MARKETPLACE_ASSETS", "USE_CONNECTION", "USE_PROVIDER", "USE_RECIPIENT", "USE_SHARE"]
  # }
}

Expected behavior
I expected it to deploy grants.

Is it a regression?
Unknown

Debug Logs

│ GET /login?next_url=/api/2.1/unity-catalog/permissions/metastore/927c05a7-42b6-460f-a4bc-913510b835bc
│ > * Host: 
│ > * Accept: application/json
│ > * Authorization: REDACTED
│ > * Referer: https://accounts.cloud.databricks.com/api/2.1/unity-catalog/permissions/metastore/927c05a7-42b6-460f-a4bc-913510b835bc
│ > * User-Agent: databricks-tf-provider/1.32.0 databricks-sdk-go/0.28.1 go/1.20.12 os/linux terraform/1.6.6 resource/grants auth/oauth-m2m
│ < HTTP/2.0 200 OK
│ < * Cache-Control: no-cache, no-store, must-revalidate
│ < * Content-Type: text/html; charset=utf-8
│ < * Date: Thu, 28 Dec 2023 01:24:26 GMT
│ < * Server: databricks
│ < * Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
│ < * X-Content-Type-Options: nosniff
│ < * X-Ui-Svc: true
│ < <!doctype html>
│ < <html lang="en">
│ <  <head>
│ <   <meta charset="utf-8">
│ <   <meta name="viewport" content="width=device-width,initial-scale=1">
│ <   <meta name="theme-color" content="#000000">
│ <   <meta name="description" content="Databricks Sign in">
│ <   <title>Databricks - Sign in</title>
│ <   <script id="Cookiebot" src="https://consent.cookiebot.com/uc.js" data-cbid="459f54ba-f28b-4a56-ab47-7af5ef8b04b4" data-blockingmode="auto" type="text/javascript" defer></script>
│ <   <script>window.__DATABRICKS_CONFIG__={"publicPath":{"accounts-console":"https://ui-assets.cloud.databricks.com/"}}</script>
│ <   <link rel="icon" href="https://ui-assets.cloud.databricks.com/favicon.ico">
│ <   <script defer src="https://ui-assets.cloud.databricks.com/static/js/5005.329e3591.js"></script>
│ <   <script defer src="https://ui-assets.cloud.databricks.com/static/js/4582.62ad0ae9.js"></script>
│ <   <script defer src="https://ui-assets.cloud.databricks.com/static/js/1721.f5335b21.js"></script>
│ <   <script defer src="https://ui-assets.cloud.databricks.com/static/js/9592.cd9e3474.js"></script>
│ <   <script defer src="https://ui-assets.cloud.databricks.com/static/js/7384.ee78b6b9.js"></script>
│ <   <script defer src="https://ui-assets.cloud.databricks.com/static/js/6871.cbb14b60.js"></script>
│ <   <script>
│ <   function setNoCdnAndReload() {
│ <       const secIn7Days = 60 * 60 * 24 * 7;
│ <       document.cookie = `x-databricks-cdn-inaccessible=true; path=/; max-age=${secIn7Days}`;
│ <       const metric = 'cdnFallbackOccurred';
│ <       const browserUserAgent = navigator.userAgent;
│ <       const browserTabId = window.browserTabId;
│ <       const performanceEntry = performance.getEntriesByType('resource').filter(e => e.initiatorType === 'script').slice(-1)[0]
│ <       sessionStorage.setItem('databricks-cdn-fallback-telemetry-key', JSON.stringify({ tags: { browserUserAgent, browserTabId }, performanceEntry}));
│ <       window.location.reload();
│ <   }
│ < </script>
│ <   <script defer src="https://ui-assets.cloud.databricks.com/static/js/login.a950c75d.js" onerror="setNoCdnAndReload()"></script>
│ <   <link href="https://ui-assets.cloud.databricks.com/static/css/4582.c2a40ece.css" rel="stylesheet">
│ <   <link href="https://ui-assets.cloud.databricks.com/static/css/1451.199d10d0.css" rel="stylesheet">
│ <   <link href="https://ui-assets.cloud.databricks.com/static/css/login.3ba2d2e0.css" rel="stylesheet">
│ <  </head>
│ <  <body>
│ <   <noscript>
│ <    You need to enable JavaScript to run this app.
│ <   </noscript>
│ <   <div id="login"></div>
│ <  </body>
│ < </html>

│
│ with module.uc_metastore.databricks_grants.this,
│ on modules/4_uc_metastore/_uc_metastore.tf line 9, in resource "databricks_grants" "this":
│ 9: resource "databricks_grants" "this" {
│

Other Information

  • OS: Ubuntu (Docker)
  • Version: terraform provider version 1.32.0

Additional context
Add any other context about the problem here.
@mgyucht mgyucht self-assigned this Jan 3, 2024
@mgyucht
Copy link
Contributor

mgyucht commented Jan 3, 2024

Thanks for reporting this. I've followed up with the underlying team to respond with an appropriate error message rather than redirecting to the login page.

However, there is still an underlying issue that will need to be resolved. Are you able to create other account-level resources with your provider?

@nkvuong
Copy link
Contributor

nkvuong commented Jan 3, 2024

@zaxier databricks_grants is only available as workspace-level resources

@jesinity
Copy link

jesinity commented Jan 4, 2024

I got pretty much the same error and @nkvuong databricks_grants according to documentation are used as well to grant privileges on metastore resources.

my code snippet is:

resource "databricks_grants" "dbrk_metastore_grant_we" {
  provider  = databricks.account
  
  metastore = databricks_metastore.we.id
  grant {
    principal  = "metastore admins"
    privileges = [
      "CREATE_EXTERNAL_LOCATION"
    ]
  }
}

@nkvuong
Copy link
Contributor

nkvuong commented Jan 5, 2024

@jesinity it is counter-intuitive, but you still needs to execute that via a workspace. Essentially, it will grant access to the metastore that is currently assigned to the workspace.

@mohitdungrani
Copy link

Hello @nkvuong, I'm currently experiencing a similar issue while attempting to provide access to the metastore via an account. Although I've understood a solution to grant access via a workspace, I'm curious to understand the reasons behind Terraform's inability to accomplish the same access grant directly through an account

@sohailshaikh1920
Copy link

Hello @mohitdungrani , can you help me understand how you authenticated from workspace while deploying unity cataloge with tf?

@mohitdungrani
Copy link

Hello @sohailshaikh1920

To perform authentication via workspace, we need to pass the workspace URL inside the host parameter.

For example:

provider "databricks" {
host = <databricks_workspace_url>
azure_client_id = var.CLIENT_ID
azure_tenant_id = var.TENANT_ID
azure_client_secret = var.CLIENT_SECRET
}

Reference: https://registry.terraform.io/providers/databricks/databricks/latest/docs#argument-reference

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mgyucht mgyucht

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@jesinity @mgyucht @zaxier @nkvuong @mohitdungrani @sohailshaikh1920