Handling of redirects in pipelines

[zuiderkwast]

Pipelines (a list of commands executed as a batch) are assumed to apply to keys in a single slot and is currently handled like this:

1. The commands are sent to the node responsible for the slot

2. The client waits for all responses (the same number of responses, with some exceptions for pubsub commands)

3. The client scans the replies for -MOVED, -ASK, -TRYAGAIN and -CLUSTERDOWN messages. If none of this is present, the replies are returned to the caller. Otherwise, the first occurrence of any of these errors determines the action.

   Reply	Action
   -MOVED	The whole pipeline is resent to the new node responsible for the slot
   -ASK	Only the commands that returned -ASK are resent to the new node, with each command prepended by ASKING
   -TRYAGAIN	The whole pipeline is sent again, after a delay
   -CLUSTERDOWN	The replies are returned to the caller (and a slotmap update is triggered)

The motivation for the handling of MOVED is that if one slot is moved, all keys in this slot are moved, so it makes sense to resend all commands to this new node.

The motivation for the handling of ASK is that during a slot migration, not all keys in the same slot are moved, so if keys in the same slot are accessed (using tags, keys like {a}x and {a}y), a subset of them may have been moved.

Problems:

• Resending commands that haven't failed may result in the wrong result. Commands that are not idempotent (like INCR) give the wrong result if executed twice.
• Resending only a subset the commands of the commands (the ones that failed with -ASK) may give an unexpected result. This is especially important with transactions.

Note that if an error occurs in a transaction (like -ASK or -MOVED) the EXEC command fails with an error. Thus, it is safe to resend the whole transaction.

It can be argued that with automatic redirects, the user cannot expect that the commands in a pipeline are executed in the right order and on the same node. If atomicity is required, a transaction must be used. And we must handle transactions correctly as an atomic unit.