Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Upsert] Fix issue with lambdas in DO UPDATE SET expressions #11866

Merged
merged 14 commits into from
May 31, 2024

Conversation

Tishj
Copy link
Contributor

@Tishj Tishj commented Apr 29, 2024

This PR fixes #11827

What caused this error is that to avoid ambiguity between the expressions referring to the new data and the existing data (excluded.<column_name>) we qualify all of the references implicitly.

Where this breaks is in lambdas, because as we have seen, lambdas explicitly check that the column reference expressions in it are unqualified.

The fix is to not qualify the expressions inside the lambda.

@duckdb-draftbot duckdb-draftbot marked this pull request as draft May 1, 2024 08:06
@Tishj Tishj marked this pull request as ready for review May 1, 2024 14:52
Copy link
Collaborator

@Mytherin Mytherin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fine with this solution, but maybe @taniabogatsch wants to have a look at it as well when she is back?

Copy link
Contributor

@taniabogatsch taniabogatsch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am okay with these changes, but I want to raise some concerns. 😅 If we merge this PR as-is, then we should maybe note in the ON CONFLICT DO UPDATE documentation that all parameters inside lambda expression must be qualified? Although the error message is helpful already.

INSERT INTO hello (SELECT 1, 72) ON CONFLICT DO UPDATE SET i = list_transform(list_value(j), x -> x + hello.i + j)[1]; 
Binder Error: Ambiguous reference to column name "j" (use: "excluded.j" or "hello.j")
# Fixed, j still unqualified outside of lambda
INSERT INTO hello (SELECT 1, 72) ON CONFLICT DO UPDATE SET j = list_transform(list_value(j), x -> x + hello.i + hello.j)[1];

This example shows that it is ambiguous for the user when to start qualifying parameters. The j we use outside of the lambda expression does not require qualifying, whereas inside the lambda, it suddenly does. I imagine that this becomes confusing for nested lambdas with other scalar functions, where suddenly everything must be qualified.

A cleaner solution would be to track/push all LHS lambda parameters before entering a RHS lambda expression, and only qualifying parameters that are not lambda parameters. When exiting the lambda expression, we pop them. Similar to the changes in this PR: https://github.com/duckdb/duckdb/pull/10150/files#diff-da6eafc29929607c29bd6b4393ee1c9b1505a77ca8e346e975bbeffa85ad80e8. See vector<unordered_set<string>> &lambda_params.

This approach might be more future-proof, but outside of the scope of this PR. If we stick to the current changes, let's add a comment to the code?

src/planner/binder/statement/bind_insert.cpp Outdated Show resolved Hide resolved
@duckdb-draftbot duckdb-draftbot marked this pull request as draft May 17, 2024 10:40
@Tishj Tishj marked this pull request as ready for review May 17, 2024 10:50
@duckdb-draftbot duckdb-draftbot marked this pull request as draft May 18, 2024 12:45
@Tishj Tishj marked this pull request as ready for review May 18, 2024 12:49
@Tishj Tishj marked this pull request as draft May 30, 2024 15:34
@Tishj Tishj marked this pull request as ready for review May 30, 2024 15:34
@Tishj Tishj changed the base branch from main to feature May 30, 2024 15:34
@Tishj Tishj marked this pull request as draft May 30, 2024 15:37
@Tishj Tishj marked this pull request as ready for review May 30, 2024 15:37
@duckdb-draftbot duckdb-draftbot marked this pull request as draft May 30, 2024 15:38
@Tishj Tishj marked this pull request as ready for review May 30, 2024 15:40
@Mytherin Mytherin merged commit 8339a9c into duckdb:feature May 31, 2024
40 checks passed
@Mytherin
Copy link
Collaborator

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Incorrect binder error when a lambda is used inside on conflict do update.
3 participants