--- layout: ../layouts/Layout.astro title: set -euo pipefail fucking sucks --- # `set -euo pipefail` fucking sucks LLMs paste this at the top of every Bash script and call it error handling. It is not error handling. It is a sticker. ```bash set -euo pipefail ``` If you cannot say which character does what, you do not get to put the line in a script that other people have to debug at 3am. This is not a safety net. It is not a default. It is not "best practice." It is a lie you put on line 1 so you can skip reading the rest of the file. No `set` header should be the default. A `set` option is a switch for one problem. You turn it on when you have that problem, in the smallest scope that solves it, and you turn it off when you are done. Stamping `-euo pipefail` on every script is how you pretend Bash grew exceptions. It did not. It grew a pile of special cases, and then the internet started copy-pasting a header that ignores them. ## What the line actually does - `-e` (`errexit`): exit if a command returns non-zero. Except when it doesn't. Which is often. - `-u` (`nounset`): treat unset variables as an error. Except the ones you meant to be optional, which you will discover in `cron`. - `-o pipefail`: the pipeline's status is the last non-zero command, not the last command. None of these are "on unless you have a reason." Each one changes what failure *means*. Each one is wrong in ordinary scripts. If you needed a global policy, you would not be writing Bash. ## `set -e` is a liar The man page says the shell exits when a command fails. Then it spends a page listing the exceptions. People who paste `-e` have not read that page. The people who wrote Debian maintainer scripts, `git`, and every init script that actually has to come back up after a reboot *have*. They do not treat `-e` as a default. There is a reason. These do **not** kill a `set -e` script: ```bash set -euo pipefail false && echo still_running ## still running if false; then echo no fi ## still running false || true ## still running ``` A command in a conditional, or on the left of `&&` / `||`, is allowed to fail. That is how `if cmd; then` works. It is also how a `curl` dies quietly in a script that "has `errexit` on," because some genius put it on the left of `&&` and assumed the sticker would save them. The exceptions are not a trivia question. `-e` is ignored in: - the test of `if`, `while`, `until` - any command except the last in `&&` or `||` - a command prefixed with `!` - a pipeline, unless `pipefail` is on, in which case you get a different set of surprises and still do not get a policy Then there are the ones you only learn when the box is down: ```bash set -e fail() { false; echo never; } if fail; then echo no fi ## `fail` ran as the `if` test, so `-e` is off inside it. ## `false` does not exit. `never` prints. `fail` returns `0`. `no` prints too. ## you asked a function to fail. it printed success. twice. wrapper() { local x x=$(false) ## assignment: may exit, depending on Bash version. pick a box, pick a behavior. local y=$(false) ## `local` succeeded. `false` did not. function continues. `y` is empty. ## this is how half the world sets variables. `-e` does not care. } wrapper ``` `local var=$(cmd)`, `declare var=$(cmd)`, and `export var=$(cmd)` hide the failure of `cmd` because the builtin returns `0`. That is not an edge case. That is `local`. If your "error handling" cannot see `local`, it is not error handling. It is a setting you do not understand. Pipelines are worse without `pipefail`: ```bash set -e false | true echo survived ``` `false` failed. `true` is the last command. Status is `0`. Script continues. With `pipefail`, that pipeline is status `1` and `-e` will finally kill it. That is not a reason to make `pipefail` a file-level default. That is `-e` being useless around a pipe, which is most of the interesting lines in a shell script. `set -e` also changes behavior across Bash versions when the failing command is inside a function, a subshell, or a command substitution (`$(...)`). BashFAQ 105 exists because Chet and a few hundred angry people could not make this coherent. If you cannot predict whether a failure exits, you do not have error handling. You have a lottery. `-e` as a default is an anti-pattern because it *claims* a policy the shell will not enforce. Claiming a policy you cannot enforce is how you ship a script that works on your laptop and lies in production. ## `set -u` is a different footgun `nounset` is a decent way to catch a typo. It is a shit way to handle optional arguments: ```bash set -u echo "${1}" ## dies if you passed no args echo "${DEBUG_FLAGS}" ## dies if the env var is unset ``` So you add `-u`, the first optional thing explodes, and you crust every expansion with `${1-}` and `${FOO-}`. Including the ones that should have been required. You did not get safety. You got noisier syntax and the same bugs, plus a new class where `cron` dies because nobody exported `DEBUG_FLAGS`. More ways `-u` is not a default: ```bash set -u f() { echo "${2}"; } f only-one ## unbound `$2`. the caller passed what they meant to pass. ## the function is wrong, or the header is. the header will win at 3am. arr=() echo "${arr[@]}" ## empty array: dies on Bash 4.3, fine on 4.4+. same script, different day. ## if you still have 4.3 in the fleet, congratulations, your "best practice" is a reboot. echo "${arr[0]}" ## unset element. dies. iterate a sparse array without a wrapper and eat the outage. source ./optional-env.sh echo "${MAYBE_SET}" ## you just imported someone else's unset variables into your crash policy. ## good luck. that file is older than the intern who added `-u`. ``` Empty arrays (`"${arr[@]}"` with `set -u` on older Bash) are the famous one. If you learned this on Bash 4.3, the header still means something different on 4.4. `${1}` vs `${1-}` vs `${1:-}` is three different programs. `-u` makes you pick one for every expansion instead of checking the arguments you actually require at the top of the script, like an adult. `-u` is a linter you run as a runtime crash. A linter that kills a `cron` job because `DEBUG_FLAGS` was unset is not "safer." It is an option you turn on when every expansion in that file is required, on purpose, and you meant it. If you did not mean it, leave it off. ## `pipefail` is a pipeline option This is the part that is usually correct *for that pipeline*: ```bash set -o pipefail curl -fsS "$url" | grep -e pattern ``` Without `pipefail`, a failed `curl` is hidden if `grep` returns `0`. With it, the pipeline fails. Good. That is a reason to enable `pipefail` around *that* pipe, or — here is a thought — to stop using a pipe when you need to know which stage died. Then someone turns `-e` on too, and `grep` with no match (status `1`) aborts the whole script. Maybe you wanted that. Maybe you wanted "filter this log and keep going if nothing matched." `pipefail` plus `errexit` cannot tell the difference. You still have to handle the case. The header will not do it for you. It will just pick one and dare you to notice. Other pipeline gotchas people dump on the sticker: - `grep` / `grep -q`: match is `0`, no match is `1`, error is `2`. Under `pipefail` and `-e`, "not found" is fatal. That is not what `grep` is for. - `yes | head`: `head` closes the pipe, `yes` gets `SIGPIPE` and a non-zero status. With `pipefail`, a working pipeline looks like a failure. This has been true since forever. The header does not know. - `cmd | while read; do ...; done`: the `while` runs in a subshell. Failures inside the loop vanish even with `pipefail`. You have known this since the first time a `while read` loop did not update a variable in the parent. Do not pretend the sticker fixed it. `pipefail` is the least dishonest of the three. It is still not a default. Use it when you have a pipe whose failure you intend to mean "any stage failed," and you know what each stage returns. If you do not know that, you have no business enabling it for the whole file. ## The antipattern The antipattern is not "using `pipefail`" or even "using `set`." The antipattern is treating one `set` line as the default, and as a substitute for noticing when commands fail. That is laziness with extra flags. `set -euo pipefail` is an anti-pattern as a header because: - It asserts a global failure policy the shell does not have and will not have. - It changes the meaning of `if`, `&&`, functions, `local`, pipes, and unset vars without saying so at the call site. The next person to edit the file will not see the trap. - It fails closed on things that were optional, and fails open on things that were required, often in the same script. That is the opposite of engineering. - It is copy-pasted by people, and by LLMs, who cannot explain what `-e`, `-u`, and `-o pipefail` actually do. If the author cannot read the line, the line does not belong in the tree. What you actually wanted, and have always wanted: - Fail the script when a command you *needed* fails. - Keep going when a command is allowed to fail. - See the error, the command, and the line. Not a silent exit from a function that ran under `if`. `set -e` tries to infer the first two from syntax. It infers wrong. The scripts that have to work — Debian packaging, `git`, the init that actually brings the box back — either avoid `-e` or check the parts that matter. They did not arrive at that by reading a blog post in 2024. ## What skillful Bash looks like Check the commands that matter. Put the check on the command. Do not ask the shell to guess. ```bash curl -fsS "$url" -o "$tmp" || exit 1 grep -e pattern "$tmp" || true ``` Ugly. Obvious. The `|| exit` is on the command you cannot proceed without. The `|| true` is on the command whose failure is data. You can read it without a mental model of `errexit` exceptions, which is good, because that model is garbage. Required vs optional is a check at the top, not a global `-u`: ```bash usage() { printf 'usage: %s [flag]\n' "${0##*/}" >&2 exit 2 } host="${1-}" flag="${2-}" [ -n "${host}" ] || usage ``` `${host}` is required because you said so, in English, with an error a human can read. `${flag}` is optional because you said so. `-u` is not involved. If that looks too long, you are not ready to write the script. Functions return status. Callers notice. That is the whole interface: ```bash fetch() { curl -fsS "$1" -o "$2" } fetch "${url}" "${tmp}" || { printf 'fetch failed: %s\n' "${url}" >&2 exit 1 } ``` A pipe you care about can be split, or inspected, instead of blessed by a file-level `set`: ```bash curl -fsS "$url" -o "$tmp" || exit 1 grep -e pattern "$tmp" || true ## or, if you really want one pipe and any-stage failure: set -o pipefail curl -fsS "$url" | grep -e pattern || { printf 'pipe failed\n' >&2 exit 1 } set +o pipefail ``` Cleanup is a `trap`. That is not `set -e`. Mixing them up is how you delete the wrong file *and* miss the failed `curl`. ```bash tmp="$(mktemp)" || exit 1 trap 'rm -f "${tmp}"' EXIT ``` Readable Bash is small, quoted, explicit about required arguments, and loud at the command that cannot fail. It does not start by asking the shell to read your mind. If you need the shell to read your mind, you are writing the wrong language. **If the script is long enough that you are scared to miss a check, it is long enough to not be a Bash script.** ## When a `set` is actually necessary When you have a specific reason. Not as a template. Not because Copilot opened the file. Not because some styleguide, written by people who do not get paged at 3 a.m., told you always. - `set -o pipefail` around a pipeline whose contract is "any stage fails ⇒ the pipe fails," then `set +o pipefail` if the rest of the file does not want that. - `set -u` in a file where every expansion is required, including positional parameters (`$1`, `$2`, …), and you have already rejected optional env. On purpose. - `set -e` almost never, and only after you have read BashFAQ 105 and audited every `if`, `&&`, `||`, `|`, `local`, function, and `$(...)`. If you have not done that, you do not get to turn it on. Pasting `set -euo pipefail` at line 1 and moving on is how you get a script that dies in `cron` for a missing optional variable, or worse, a script that keeps going after `curl` failed because the failure was on the left of a `&&`. Both of those outages are your fault. The sticker will not be the thing that gets paged. You will.