You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Not sure if Bigquery-specific but currently, DF generates a hash value for the temporary procedure to construct the MERGE or INSERT DML for an incremental table. I have learned that this hash is based (exclusively ?) on the execution time, probably rounded to seconds at best.
This creates a problem if the same step from another workflow execution runs at the (exact) same time, as same name is used.
Results can be either wrong procedure is run or, "Procedure is not found" error if the "first" execution dropped the procedure before the "second" calls it.
Unless I am oversimplifying, this can be easily fixed by generating an UUID instead of deterministic hash for the temporary procedure name created by DF.
Raised through other channels to no avail so hoping for more traction here.
The text was updated successfully, but these errors were encountered:
Not sure if Bigquery-specific but currently, DF generates a hash value for the temporary procedure to construct the MERGE or INSERT DML for an incremental table. I have learned that this hash is based (exclusively ?) on the execution time, probably rounded to seconds at best.
This creates a problem if the same step from another workflow execution runs at the (exact) same time, as same name is used.
Results can be either wrong procedure is run or, "Procedure is not found" error if the "first" execution dropped the procedure before the "second" calls it.
Unless I am oversimplifying, this can be easily fixed by generating an UUID instead of deterministic hash for the temporary procedure name created by DF.
Raised through other channels to no avail so hoping for more traction here.
The text was updated successfully, but these errors were encountered: