Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Macro databricks__get_columns_from_information_schema is not working as expected #1428

Open
HantonMing opened this issue Feb 22, 2024 · 1 comment

Comments

@HantonMing
Copy link

Describe the bug
Unity catalog is disabled for us and we have trouble with dbt run -s elementary. In the macro elementary/macros/edr/metadata_collection/get_columns_from_information_schema.sql, another macor is defined for databricks ,which is databricks__get_columns_from_information_schema. The if condition in this macro,

{% if target.catalog is none %}
        {# Information schema is only available when using Unity Catalog. #}
        {% do return(elementary.get_empty_columns_from_information_schema_table()) %}
    {% endif %}

is not working properly in that if condition target.catalog is none is never met. In other words, return(elementary.get_empty_columns_from_information_schema_table()) is never executed. I know this because I made changes to rest of the macro, and changes were reflected in compiled query.
To Reproduce
Steps to reproduce the behavior:

  1. Install elementary-data following instructions
  2. Run dbt run -s elementary
  3. Error message "Unity Catalog is not abled on this cluster"

Expected behavior
Disabled Unity Catalog should be correctly detected. And workaround macro is run.

Screenshots
image

Environment (please complete the following information):

  • edr Version: [0.14.0]
  • dbt core version: [1.7.8]
  • dbt databricks version: [1.7.7]

Additional context
Add any other context about the problem here.

@HantonMing HantonMing added Bug Something isn't working Triage 👀 labels Feb 22, 2024
@haritamar
Copy link
Collaborator

Hi @HantonMing ,
Thank you for opening this issue and sorry for such a long delay in our reply.

I'm not sure if this is still relevant to you, but actually would love to know - you mentioned the condition target.catalog is none is never met.
Do you manage to see what value does it contain? Does it contain a legitimate value even though you are not using Unity Catalog?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants