Text Processing with awk in Zsh — Complete Guide for Beginners

Text Processing with awk in Zsh — Complete Guide for Beginners

Awk is a powerful text-processing tool that excels at extracting and manipulating data from structured text files or command output. This guide covers essential awk features in Zsh, including field extraction, pattern matching, formatted printing, and advanced techniques like associative arrays and custom reports.

Code

# Basic field extraction and printing columns 1 and 3 from a file
awk '{ print $1, $3 }' input.txt

# Print entire line using $0
awk '{ print $0 }' input.txt

# Use a custom field separator (comma) with -F
awk -F',' '{ print $2, $1 }' data.csv

# Formatted output with printf: print first field left-aligned in 10 chars, second field as integer
awk '{ printf "%-10s %d\n", $1, $2 }' numbers.txt

# Pattern matching: print lines where the third field matches "error"
awk '$3 ~ /error/' logs.txt

# BEGIN block to print a header before processing input
awk 'BEGIN { print "Name\tCount" } { print $1, $2 }' data.txt

# END block to print summary after processing all lines
awk '{ sum += $2 } END { print "Total:", sum }' data.txt

# Using NR (record number) and NF (number of fields)
awk '{ print NR, NF, $0 }' file.txt

# Ternary expression: print "High" if $2 > 50 else "Low"
awk '{ print $1, ($2 > 50 ? "High" : "Low") }' scores.txt

# Summing values in column 2
awk '{ total += $2 } END { print "Sum:", total }' data.txt

# Associative arrays for counting occurrences of first field
awk '{ count[$1]++ } END { for (word in count) print word, count[word] }' words.txt

# Filtering lines where field 2 equals "active"
awk '$2 == "active"' users.txt

# Piping output of ls to awk to print filenames and sizes
ls -l | awk 'NR>1 { print $9, $5 }'

# Sorting unique values: list unique users sorted alphabetically
awk '{ print $1 }' data.txt | sort | uniq

# Building a custom report with formatted output
awk 'BEGIN { printf "%-10s %10s\n", "User", "Count" }
     { count[$1]++ }
     END {
       for (u in count) printf "%-10s %10d\n", u, count[u]
     }' data.txt

Key Points

• Awk fields are accessed with $1, $2, ..., and the entire line with $0 for flexible text extraction.
• Use -F to specify custom field separators, enabling parsing of CSV or other delimited formats.
• BEGIN and END blocks allow running code before and after processing input lines, useful for headers and summaries.
• Associative arrays in awk enable counting and grouping data by keys without prior initialization.
• Combining awk with pipes and Unix tools like sort and uniq creates powerful data-processing pipelines.