Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement pipefail and errexit #4351

Open
hexagonrecursion opened this issue Jul 1, 2021 · 2 comments · Fixed by #5380 · May be fixed by #5239
Open

Implement pipefail and errexit #4351

hexagonrecursion opened this issue Jul 1, 2021 · 2 comments · Fixed by #5380 · May be fixed by #5239

Comments

@hexagonrecursion
Copy link

I want to know when a command other then the last command in the pipeline fails for any reason. If a command in the middle of a pipeline fails, the last command simply sees a EOF in its stdin. In many cases this will result in the last command returning success. Silently ignoring such errors can lead to all kinds of data loss, data corruption, truncation and other badness. Ideally any unhandled errors should terminate the script.

bash has pipefail:

#!/usr/bin/env bash
set -o errexit -o pipefail
echo one
false | true
echo two

VVV

one

Note: the above example does not print "two"

#!/usr/bin/env xon.sh
$RAISE_SUBPROC_ERROR = True
echo one
false | true
echo two

VVV

one
two

For community

⬇️ Please click the 👍 reaction instead of leaving a +1 or 👍 comment

@hexagonrecursion
Copy link
Author

More context:
Executing an external command can fail for a million different reasons. Here are some examples:

  1. The external command is not installed
  2. It encounters an internal bug and fails fast and loud
  3. It receives bad input due to a bug in my script
  4. An admin kills it
  5. The OOM Killer kills it https://linuxwheel.com/oom-killer-explained/
  6. It tries to talk to another process possibly on a different machine and that fails for any reason
  7. It uses a temporary file for internal buffering and /tmp is full
  8. It tries to read or write a file and that fails for any reason

A robust script can not afford to ignore the possibility that any of the external commands it executes may fail. Often the right choice of action is to fail fast because the code that follows requires that the preceding commands executed successfully.

Unix-y pipelines have an issue:

foo | bar

If foo exits successfully bar sees EOF. If foo fails bar also sees EOF. bar has no way to know whether foo failed or not. In many cases bar will exit successfully even if foo fails. The script has to detect that and fail. This is why you often see set -o errexit -o pipefail in bash.

Unfortunately set -o errexit is very broken in bash which is why I'm looking for a different shell.

@anki-code anki-code removed the feature label Aug 3, 2022
@anki-code anki-code changed the title [feature] I don't want to silently ignore errors in pipelines I don't want to silently ignore errors in pipelines Aug 3, 2022
@anki-code anki-code changed the title I don't want to silently ignore errors in pipelines Do not ignore errors in pipelines Aug 3, 2022
@carhe
Copy link

carhe commented Sep 6, 2022

I also miss this feature, which is quite important in my opinion. Two use cases are

pv input | foo (pv = pipe viewer, it generates progress output at stderr)

and

foo | tee logfile

I guess these use cases are rather common, but the lack of some pipefail equivalent makes them painful with xonsh.

@anki-code anki-code changed the title Do not ignore errors in pipelines Implement pipefail and errexit May 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants