New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Paste API #3222
base: develop
Are you sure you want to change the base?
Conversation
@dmos62 can you give an update one where you're at with this PR? How close to completion is it? What work is remaining? |
I'll update the top post with what the goal/spec of the PR is. As to the progress, here's the summary, but keep in mind that I might not recall some details. The PR seems 80% complete. The SQL logic and tests seem to be there. The API tests are there for the update aspect of pasting. The API tests for the insert aspect of pasting are missing. The API tests for the paste operation being all-or-nothing are missing as well. |
Fixes #2844
Implements paste API.
Initially we envisioned a generic bulk upsert API that could be used by the frontend paste functionality, hence the title of #2844. I explored that idea extensively, but opted for a non-generic API specifically for pasting.
In my estimate, the bulk upsert as expected by a user using paste in a spreadsheet is wildly different than the bulk upsert expected by database administrator doing
INSERT ON CONFLICT DO UPDATE
. Trying to accomodate both seemed like a good way to waste a lot of time and end up with bad UX for both.Instead of summarizing my concerns, I'll summarize the features of the proposed paste API:
Above is the basic outline of the API. The endpoint accepts (via
POST
) a dict, that has two fields: updates and inserts.It's assumed that frontend will always know whether a given row being pasted should result in an update or an insert. Making that explicit prevents surprises in case of relevant changes having been made that were not synced with the calling frontend.
Updates are a sequence of dicts: each dict has a
changes
key and aconditionals
key:changes
define the field-value pairs to change in the row that is satisfied by the equality comparisons defined by field-value pairs inconditionals
. Think howUPDATE
works in SQL. An update withchanges
being{'field1':'val1'}
andconditionals
being{'id':'val2'}
requests that for the single row whoseid
field equalsval2
, itsfield1
field's value is set toval1
.If more than one row would be updated by a given update, that's a fatal exception, because when pasting a row it should never result in more than one updated row. The paste API attempts to minimize surprises to the end user.
Inserts are a sequence of dicts: each dict defines the field-value pairs that should be set on the new resulting row.
An insert will always result in a new row or will panic if that's impossible for whatever reason.
Any panic encountered will rollback all changes made. Paste is meant to be an all-or-nothing operation.
Also, this API panics if any user-defined changes to
SERIAL
orIDENTITY
columns are requested. These column types need special handling when mutating their values, because there are sequences involved, and that seems hard to safely generalize (i.e. prevent surprises).All updates are performed before any inserts. All updates are performed in order requested; same for inserts.
Checklist
Update index.md
).develop
branch of the repositoryvisible errors.
Developer Certificate of Origin
Developer Certificate of Origin