Zsh Lesson 16: Pipes & Redirection — |, >, <, 2>&1, /dev/null

cmd1 | cmd2 pipes stdout. > file redirects stdout (truncate); >> append. 2> file redirects stderr. 2>&1 merges stderr into stdout. &> file redirects both. < file reads from. /dev/null is the bit bucket.

Pipes and redirection are how Unix composes small tools into large ones. Knowing the file-descriptor numbers makes everything clearer.

File descriptors

Every process has three streams open by default:

• fd 0 — stdin (input).
• fd 1 — stdout (normal output).
• fd 2 — stderr (errors and diagnostics).

Most commands print results to stdout and errors to stderr. You can redirect each independently.

Pipes: |

echo "apple banana cherry" | tr ' ' '\n'
# apple
# banana
# cherry

cmd1 | cmd2 connects cmd1's stdout to cmd2's stdin. Build pipelines:

ls -la | grep ".sh"
ls | wc -l
echo -e "a\nb\na\nc\nb" | sort | uniq -c | sort -rn

The output of each stage becomes input to the next. This is the core Unix philosophy.

Output redirection: > and >>

echo "hello" > file.txt          # truncate, write
echo "world" >> file.txt          # append

• > overwrites. The file is truncated before the command runs.
• >> appends. The file is created if missing.

To redirect to a file that doesn't exist yet, both work — > creates it.

/dev/null: discard

some_command > /dev/null         # discard stdout
some_command 2> /dev/null        # discard stderr only
some_command &> /dev/null        # discard both

/dev/null is a special "device" that accepts everything and outputs nothing. Use it to silence output you don't care about.

# Run silently — only check exit status
if grep -q "pattern" file.txt; then
  echo "found"
fi

grep -q (quiet) doesn't print matches; combined with the exit code, it's a pure boolean check.

Stderr redirection: 2>

ls /nonexistent 2> errors.txt
echo "Exit: $?"
cat errors.txt
# ls: /nonexistent: No such file or directory

2> file redirects stderr (fd 2). The number 2 is the file descriptor — followed immediately by > (no space).

Merge stderr into stdout: 2>&1

ls /nonexistent /tmp 2>&1 | head

2>&1 is "redirect fd 2 to wherever fd 1 currently points." Now stderr flows through the pipe like stdout.

The order matters:

# Both to /tmp/log
cmd > /tmp/log 2>&1

# WRONG — stderr to terminal, stdout to /tmp/log
cmd 2>&1 > /tmp/log

Read right-to-left: - First (> /tmp/log): fd 1 → /tmp/log. fd 2 still → terminal. - Then (2>&1): fd 2 → wherever fd 1 points = /tmp/log. ✓

If you flip: - First (2>&1): fd 2 → wherever fd 1 points = terminal. - Then (> /tmp/log): fd 1 → /tmp/log. fd 2 still → terminal.

Always > file 2>&1, not 2>&1 > file.

Shorthand: &>

cmd &> output.txt

&> is "both stdout and stderr to this file." Equivalent to > output.txt 2>&1. Cleaner; supported in Bash 4+ and Zsh.

Separate stdout and stderr

cmd > stdout.txt 2> stderr.txt

Two separate files for two streams.

Input redirection: <

sort < unsorted.txt
wc -l < bigfile.txt

< file makes file the command's stdin. Less common than pipes, but useful for tools that don't accept filename arguments.

Here-doc and here-string

Heredoc:

cat <<EOF > config.txt
host=localhost
port=8080
EOF

The body until EOF becomes stdin. Already covered in lesson 14.

Here-string:

grep "zsh" <<< "I love zsh"

<<< passes one line as stdin. Cleaner than echo "..." | grep.

Process substitution: <(cmd) and >(cmd)

diff <(grep ERROR log1) <(grep ERROR log2)

<(cmd) makes cmd's output look like a file. Useful for tools that take filename args but you have command output.

# Write to two destinations
tee >(gzip > out.gz) > out.txt < input

>(cmd) is the inverse — feeds to a command as if it were a file.

tee: split output

echo "log entry" | tee log.txt
# log entry        <- printed
# (also written to log.txt)

# Append mode
echo "more" | tee -a log.txt

# Multiple files
echo "x" | tee file1 file2 file3

Already shown in lesson 14. Useful for "I want to see this AND save it."

Discarding stderr while keeping stdout

find / -name "*.txt" 2>/dev/null

find complains about permission denied for many directories. 2>/dev/null silences those errors while keeping the matched files.

Capturing output

output=$(some_command 2>&1)              # both streams
output=$(some_command)                    # stdout only; stderr to terminal

For just stderr (rare):

errors=$(some_command 2>&1 >/dev/null)
# Note the order — discard stdout first, then capture stderr that's been merged

Or use process substitution:

errors=$( { some_command >/dev/null; } 2>&1 )

Capturing only stderr is awkward — there's no clean syntax.

Appending to stderr

echo "error message" >&2

>&2 writes to stderr. Use in scripts:

if [[ ! -f "$config" ]]; then
  echo "Error: $config not found" >&2
  exit 1
fi

Errors should always go to stderr. Then cmd > log.txt doesn't catch them, and the user can pipe stdout independently.

A real-world pipeline

# Top 5 most common words in a file
cat document.txt \
  | tr '[:upper:]' '[:lower:]' \
  | tr -s '[:space:]' '\n' \
  | grep -v '^$' \
  | sort \
  | uniq -c \
  | sort -rn \
  | head -5

Reads document → lowercase → split into words → drop blanks → sort → count uniques → sort by count → top 5.

Each stage is one tool. Together they're a word-frequency analyzer.

/dev/stdin, /dev/stdout, /dev/stderr

diff /dev/stdin file2 < file1

Some commands take filenames; passing - or /dev/stdin lets you use a pipe instead.

Common idioms

# Silence both streams
cmd &>/dev/null

# Run, check exit, ignore output
if cmd > /dev/null 2>&1; then ...

# Capture and log everything
cmd 2>&1 | tee output.log

# Save error context, drop on success
cmd 2> errors.tmp || cat errors.tmp >&2
rm errors.tmp

# Redirect script's entire output
exec > /tmp/log.txt 2>&1     # redirect from this point on

Common stumbles

Order of > and 2>&1. Always > file 2>&1. The reverse 2>&1 > file doesn't work.

Pipe in set -e. If only the last command in a pipe fails, set -e doesn't catch it. Use set -o pipefail to propagate failures.

> on read-only directory. "Permission denied." Check write access first.

> overwrite without confirm. Disable with setopt noclobber (then >| to force).

Capturing stderr. var=$(cmd) only catches stdout. Use var=$(cmd 2>&1) for both.

/dev/null for discarding. > /dev/null not > /dev/nul (Windows is nul; Unix is /dev/null).

tee not seeing output. tee -a for append; without -a it truncates each time.

Process substitution on macOS without bash 4+. Should work with Zsh and modern Bash. If you get "syntax error" on <(...), check shell.

What's next

Lesson 17: grep. Search files for patterns.

Recap

Pipes cmd1 | cmd2: stdout of left to stdin of right. > file truncate-write, >> file append. 2> file for stderr. 2>&1 merges stderr into stdout (must come AFTER >file). &> file for both. < file reads. /dev/null discards. Always send errors to >&2 in scripts. <(cmd) and >(cmd) for process substitution. Use set -o pipefail for pipeline error propagation.

Next lesson: grep.