Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Concat should be replaced with concat_ws in sqlserver from dbtVault version 0.9.1 onwards (version 0.9.0 is working and older ones) #188

Open
koillinengit opened this issue Apr 11, 2023 · 2 comments
Assignees
Labels
bug Something isn't working sqlserver Issues specific to SQLServer

Comments

@koillinengit
Copy link

koillinengit commented Apr 11, 2023

Describe the bug
From version 0.9.1 concat_ws is replaced with concat in hashed views in sqlserver. It gives an error: [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]The concat function requires 2 to 254 arguments.

Environment

dbt version: dbt-core 1.3.3, dbt-sqlserver 1.3.1
dbtvault version: >= 0.9.1
Database/Platform: Microsoft SQL Server Standard (64-bit) 15.0.2101.7, Debian 5.10.127-2 (2022-07-23) x86_64, Python 3.9.2

To Reproduce
Steps to reproduce the behavior:

  1. Dbt run
  2. See error

Expected behavior
Concat should be replaced with concat_ws to support more than 254 concanated fields

Screenshots
image

Log files
In progress or failed ('42000', '[42000] [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]The concat function requires 2 to 254 arguments. (189) (SQLMoreResults)')

Additional context
https://github.com/Datavault-UK/dbtvault/blob/master/macros/internal/metadata_processing/concat_ws.sql

Concat_ws has been in SQL Server 2017 (14.x) and later versions
https://learn.microsoft.com/en-us/sql/t-sql/functions/concat-ws-transact-sql?view=sql-server-ver16

AB#5364

@koillinengit koillinengit added the bug Something isn't working label Apr 11, 2023
@koillinengit koillinengit changed the title [BUG] <my bug title> [BUG] Concat should be replaced with concat_ws in sqlserver from dbtVault version 0.9.1 onwards (version 0.9.0 is working and older ones) Apr 11, 2023
@DVAlexHiggs
Copy link
Member

DVAlexHiggs commented Apr 11, 2023

Hi! Thanks for this.

We originally implemented the concat macro using concant_ws, however we had issues with consistency across platforms, as not all platforms have concat_ws available.

Saying this, I completely agree it should still be used where supported, and the limitations on other platforms with the regular concat documented in our docs. The downside is inconsistency across platforms, however, I think this is better than the alternative of the dbtvault concat macro not working at all for large concatenations on some platforms.

We shall add this to the backlog and get it out in a release soon. Thank you

@DVAlexHiggs DVAlexHiggs added the sqlserver Issues specific to SQLServer label Feb 21, 2024
@koillinengit
Copy link
Author

koillinengit commented May 14, 2024

Is there anyway to get this implemented? I'm not able to upgrade to automaDV versions at all. I have to use dbtvault 0.9.0 with our sqlserver implementtion because of the limit of 254 catenation strings.

@DVAlexHiggs DVAlexHiggs added bug Something isn't working and removed bug Something isn't working labels May 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working sqlserver Issues specific to SQLServer
Projects
None yet
Development

No branches or pull requests

2 participants