Answer: Variables are assigned without spaces around = and referenced with $. Single quotes preserve everything literally; double quotes allow variable expansion and command substitution; backslash escapes single characters.
name="Alice"greeting='Hello, $name' # literal: Hello, $namedynamic="Hello, $name" # expands: Hello, Alice
path="/home/$name/docs"echo "Files: $(ls "$path")" # command substitution inside double quotes
# Avoid word splitting and globbing on expansionsfile="my file.txt"cat "$file" # correctcat $file # wrong — splits on spaceWhy this approach: Double-quoting all variable expansions prevents word-splitting and pathname expansion bugs that are a top source of shell script errors.
$@ and $*, and when does it matter?Answer: Both hold positional parameters, but differ when quoted. "$@" expands each argument as a separate quoted word; "$*" joins all arguments into one word separated by IFS.
show_args() { echo "Using \"\$@\":" for arg in "$@"; do echo " [$arg]" done
echo "Using \"\$*\":" for arg in "$*"; do echo " [$arg]" done}
show_args "hello world" "foo" "bar baz"# "$@" → [hello world] [foo] [bar baz] (3 items, spaces preserved)# "$*" → [hello world foo bar baz] (1 item, all joined)Why this approach: Always use "$@" when forwarding arguments to preserve each argument's boundaries exactly.
Answer: Arrays use parentheses for declaration, ${arr[i]} for indexing, and ${arr[@]} to expand all elements.
xxxxxxxxxxfruits=("apple" "banana" "cherry")
# Appendfruits+=("date")
# Lengthecho "Count: ${#fruits[@]}"
# Iterate safelyfor fruit in "${fruits[@]}"; do echo "$fruit"done
# Iterate with indexfor i in "${!fruits[@]}"; do echo "$i: ${fruits[$i]}"done
# Slice: elements 1 and 2echo "${fruits[@]:1:2}"
# Associative array (Bash 4+)declare -A colorscolors["apple"]="red"colors["banana"]="yellow"for key in "${!colors[@]}"; do echo "$key -> ${colors[$key]}"doneWhy this approach: Quoting "${arr[@]}" is essential — without quotes, elements with spaces break into separate words.
for, while, and until loops with examples.Answer: for iterates over a list or C-style range; while runs while a condition is true; until runs until a condition becomes true.
xxxxxxxxxx# for — list iterationfor f in /etc/*.conf; do echo "$f"done
# for — C-stylefor ((i=0; i<5; i++)); do echo "$i"done
# while — condition-basedcount=0while [[ $count -lt 3 ]]; do echo "count=$count" ((count++))done
# while read — process file line by linewhile IFS= read -r line; do echo "Line: $line"done < /etc/hosts
# until — opposite of whilen=5until [[ $n -le 0 ]]; do echo "$n" ((n--))doneWhy this approach: while IFS= read -r line is the canonical safe pattern for line-by-line file processing — IFS= preserves leading/trailing whitespace and -r prevents backslash interpretation.
[, [[, and (( ))?Answer: [ is POSIX test; [[ is Bash's extended keyword with safer string handling; (( )) is arithmetic evaluation.
xxxxxxxxxxx=5file="/etc/hosts"
# [ — POSIX, needs quoting, no regex[ "$x" -eq 5 ] && echo "equal"
# [[ — Bash, no word splitting, supports regex/glob[[ $x -eq 5 ]] && echo "equal"[[ $file == *.hosts ]] && echo "glob match"[[ $file =~ ^/etc/[a-z]+ ]] && echo "regex match"
# (( )) — arithmetic, returns 0 on nonzero result(( x > 3 )) && echo "greater"(( x++ )) # incrementresult=$(( x * 2 + 1 ))echo "$result"
# Common file tests[[ -f "$file" ]] && echo "regular file"[[ -d /tmp ]] && echo "directory"[[ -r "$file" ]] && echo "readable"[[ -z "$var" ]] && echo "empty string"[[ -n "$var" ]] && echo "non-empty string"Why this approach: Prefer [[ ]] in Bash scripts for its safety features; use (( )) for all arithmetic to avoid the $ noise and get natural math semantics.
Answer: Functions are declared with name() { } or function name { }. They return an exit status (0–255) via return; to return data, use stdout capture or a nameref variable.
xxxxxxxxxx
# Declarationgreet() { local name="$1" # local prevents leaking echo "Hello, $name"}
# Return a computed value via stdoutadd() { local a=$1 b=$2 echo $(( a + b ))}
result=$(add 3 7)echo "Sum: $result"
# Return status codesis_even() { (( $1 % 2 == 0 )) # (( )) exits 0 if true, 1 if false}
if is_even 4; then echo "even"fi
# Nameref output variable (Bash 4.3+)compute() { local -n _out=$1 # nameref _out=$(( $2 * $3 ))}
compute product 6 7echo "Product: $product"Why this approach: Using local for all function variables prevents accidentally clobbering globals; stdout capture is the cleanest way to return string/numeric data.
$??Answer: Every command exits with a code: 0 = success, 1–255 = error. $? holds the exit code of the most recently executed foreground command.
xxxxxxxxxx
# Basic checkls /nonexistent 2>/dev/nullecho "Exit code: $?" # 2
# Using in conditionalsif grep -q "root" /etc/passwd; then echo "root found"else echo "not found (exit $?)"fi
# Capture and testrun_task() { some_command local rc=$? if [[ $rc -ne 0 ]]; then echo "ERROR: some_command failed with $rc" >&2 return $rc fi}
# Common codes# 0 success# 1 general error# 2 misuse of shell builtins# 126 command found but not executable# 127 command not found# 128+N killed by signal N (e.g. 130 = Ctrl+C / SIGINT)
# Exit script with specific codecleanup_and_exit() { echo "Cleaning up..." exit "${1:-0}"}Why this approach: Always capture $? immediately after the command you care about — subsequent commands overwrite it.
Answer: A subshell is a child process copy of the shell. Variable changes inside a subshell do not propagate back to the parent.
xxxxxxxxxxx=10
# Parentheses create a subshell( x=99 echo "Inside subshell: x=$x" # 99)echo "Outside subshell: x=$x" # 10 — unchanged
# Pipes run in subshells (common gotcha)count=0echo -e "a\nb\nc" | while read line; do ((count++))doneecho "count=$count" # 0! — while ran in a subshell
# Fix with process substitution or lastpipeshopt -s lastpipeecho -e "a\nb\nc" | while read line; do ((count++))doneecho "count=$count" # 3 — lastpipe runs last segment in current shell
# Or use a here-string / process substitutionwhile IFS= read -r line; do ((count++))done < <(echo -e "a\nb\nc")echo "count=$count" # 3Why this approach: The pipe-subshell gotcha breaks counters and variable accumulation — process substitution < <(...) is the clean fix.
>, >>, 2>&1, and /dev/null.Answer: Redirections rewire file descriptors. FD 0=stdin, 1=stdout, 2=stderr.
xxxxxxxxxx# Overwrite stdoutecho "hello" > output.txt
# Append stdoutecho "world" >> output.txt
# Redirect stderr to stdout (merge both streams)command 2>&1 | tee combined.log
# Redirect both to file (Bash shorthand)command &> both.logcommand > both.log 2>&1 # equivalent, POSIX-compatible order matters
# Discard outputcommand > /dev/null # discard stdoutcommand 2> /dev/null # discard stderrcommand > /dev/null 2>&1 # discard both
# Read stdin from filecommand < input.txt
# Here-string (feed a string as stdin)grep "pattern" <<< "some string to search"
# Redirect stderr to a file, stdout to a different filecommand > stdout.log 2> stderr.log
# Order matters: 2>&1 must come AFTER >command > file 2>&1 # stderr goes to file (correct)command 2>&1 > file # stderr goes to terminal (wrong order)Why this approach: The ordering rule > file 2>&1 trips up many candidates — 2>&1 means "point FD2 to wherever FD1 currently points," so FD1 must be redirected first.
set -euo pipefail do and when should you use it?Answer: These are safety options for production scripts: -e exits on error, -u errors on undefined variables, -o pipefail makes pipelines fail if any command fails.
xxxxxxxxxxset -euo pipefail
# -e: exit immediately on non-zero exitfalse # script exits here without -e being offecho "never reached"
# -u: treat unset variables as errorsecho "$UNDEFINED_VAR" # exits: unbound variable
# -o pipefail: pipeline exit = exit of rightmost failing commandcat nonexistent.txt | sort # exits because cat failed (without pipefail, sort's 0 wins)
# Patterns to handle expected failuresif grep -q "pattern" file; then # if suppresses -e echo "found"fi
value=$(some_command) || true # || true suppresses -e for this line
# Temporarily disableset +erisky_commandrc=$?set -e
# Good header for production scriptsset -euo pipefailIFS=$'\n\t' # also restrict IFS for safetyWhy this approach: set -euo pipefail converts silent failures (the default) into loud, obvious script termination — essential for any script run in production or CI/CD.
$$, $!, $#, and $0?Answer: Bash provides several automatic variables that expose runtime information about the script and its execution.
xxxxxxxxxx
echo "Script name: $0" # path used to invoke scriptecho "Arg count: $#" # number of positional paramsecho "All args (@): $@" # all positional params (separate words)echo "All args (*): $*" # all positional params (one word)echo "Current PID: $$" # PID of the current shellecho "Last bg job PID: $!" # PID of most recent background jobecho "Last exit code: $?" # exit status of last commandecho "Shell options: $-" # current option flags (e.g. himBH)
# Practical use of $$: unique temp filesTMPFILE="/tmp/myscript-$$"trap 'rm -f "$TMPFILE"' EXITecho "data" > "$TMPFILE"
# Practical use of $!: wait on background joblong_running_task &bg_pid=$!echo "Started task with PID $bg_pid"wait $bg_pidecho "Task finished with exit $?"
# $# for argument validationif [[ $# -lt 2 ]]; then echo "Usage: $0 <src> <dst>" >&2 exit 1fiWhy this approach: Using $$ in temp file names prevents collisions when multiple script instances run concurrently.
getopts work for parsing command-line options?Answer: getopts is the POSIX-standard built-in for parsing short options (-f, -v, -o val). It handles option clustering and argument association automatically.
xxxxxxxxxx
usage() { echo "Usage: $0 [-v] [-f file] [-n count] args..." >&2 exit 1}
verbose=falsefile=""count=1
while getopts ":vf:n:" opt; do case $opt in v) verbose=true ;; f) file="$OPTARG" ;; n) count="$OPTARG" ;; :) echo "Option -$OPTARG requires an argument" >&2; usage ;; \?) echo "Unknown option: -$OPTARG" >&2; usage ;; esacdone
# Shift past the options to get positional argsshift $((OPTIND - 1))
$verbose && echo "Verbose mode on"echo "file=$file, count=$count"echo "Remaining args: $@"
# Example call: ./script.sh -v -f data.txt -n 5 arg1 arg2Why this approach: getopts is built-in (no external dependency), handles -vf data.txt clusters correctly, and the leading : in the option string enables silent error handling.
Answer: A heredoc (<<DELIMITER) feeds a multiline string as stdin, preserving newlines. <<- strips leading tabs; quoting the delimiter prevents variable expansion.
xxxxxxxxxx
name="Alice"host="db.example.com"
# Basic heredoc (variables expand)cat <<EOFHello, $nameConnecting to $hostToday: $(date)EOF
# Indented heredoc (strips leading tabs, not spaces)if true; then cat <<-EOF This is indented with tabs. Variable: $name EOFfi
# Quoted delimiter — no expansion (literal)cat <<'EOF'No $expansion here.Backslash \n is literal.EOF
# Feed heredoc to a commandmysql -u root -p"$PASS" <<EOFUSE mydb;SELECT COUNT(*) FROM users;EOF
# Assign to variablemessage=$(cat <<EOFLine one: $nameLine two: $(hostname)EOF)echo "$message"Why this approach: Heredocs keep multiline content readable inline without escaping every quote, and quoting the delimiter ('EOF') is the safest way to pass literal text to commands like ssh or psql.
Answer: Process substitution <(cmd) provides the output of a command as a named pipe (filename), allowing commands that require file arguments to consume command output. Command substitution $(cmd) captures output as a string.
xxxxxxxxxx
# Command substitution — captures output as stringtoday=$(date +%Y-%m-%d)files=$(find /tmp -name "*.log" -mtime -1)
# Process substitution — provides output as a file path# diff two command outputs without temp filesdiff <(ls /dir1) <(ls /dir2)
# Feed to while read without subshell (vs pipe)while IFS= read -r line; do echo "Got: $line"done < <(find /var/log -name "*.log" -newer /tmp/marker)
# Sort and compare two sorted listscomm <(sort list1.txt) <(sort list2.txt)
# paste two command outputs side-by-sidepaste <(seq 5) <(seq 5 | awk '{print $1*$1}')
# tee to multiple processestee >(gzip > out.gz) >(wc -l > count.txt) < input.txt > /dev/nullWhy this approach: Process substitution avoids temp files and — crucially — avoids the subshell variable-scope issue that plagues pipelines.
trap?Answer: trap registers a handler that runs when the shell receives a signal or a pseudo-signal like EXIT, ERR, or DEBUG.
xxxxxxxxxxset -euo pipefail
TMPDIR_WORK=$(mktemp -d)LOCKFILE="/var/run/myscript.lock"
cleanup() { echo "Cleaning up..." >&2 rm -rf "$TMPDIR_WORK" rm -f "$LOCKFILE"}
# EXIT fires on any exit (normal, error, or signal)trap cleanup EXIT
# Handle specific signalstrap 'echo "Interrupted!" >&2; exit 130' INTtrap 'echo "Terminated!" >&2; exit 143' TERM
# ERR fires when a command fails (useful with set -e)trap 'echo "Error on line $LINENO: $BASH_COMMAND" >&2' ERR
# Ignore a signaltrap '' HUP # ignore hangup
# Reset to defaulttrap - INT
# Chaining — save and restore previous trapold_trap=$(trap -p INT)trap 'custom_handler' INT# ... do work ...eval "$old_trap" # restore
echo "Working in $TMPDIR_WORK"# Cleanup runs automatically at exitWhy this approach: Registering cleanup on EXIT (rather than individual signals) is the most robust pattern — it fires on normal exit, errors, and caught signals alike.
&, wait, and job control.Answer: Appending & runs a command in the background. wait blocks until background jobs finish. jobs, fg, bg, kill manage job control.
xxxxxxxxxx
# Run in backgroundlong_task() { sleep "$1" echo "Task $2 done"}
# Launch multiple parallel jobslong_task 3 "A" &pid_a=$!long_task 2 "B" &pid_b=$!long_task 1 "C" &pid_c=$!
# Wait for allwaitecho "All tasks done"
# Wait for specific PID and capture exit statuslong_task 2 "X" &bg_pid=$!if wait $bg_pid; then echo "X succeeded"else echo "X failed with $?"fi
# Parallel with exit-code checkingpids=()for i in 1 2 3 4; do (sleep $i && echo "done $i") & pids+=($!)done
fail=0for pid in "${pids[@]}"; do wait "$pid" || ((fail++))doneecho "Failures: $fail"
# Job control (interactive)sleep 100 & # backgroundjobs # list jobsfg %1 # bring job 1 to foreground# Ctrl+Z to suspendbg %1 # resume in backgroundkill %1 # kill by job specWhy this approach: Capturing each PID immediately after & (before the next &) and storing in an array is the correct pattern for parallelism with proper error tracking.
mktemp and file locking with flock work?Answer: mktemp creates uniquely named temporary files/directories safely. flock applies advisory locks so concurrent script instances don't race.
xxxxxxxxxx
# Create temp file (deleted by trap)tmpfile=$(mktemp /tmp/process-XXXXXX)tmpdir=$(mktemp -d /tmp/workdir-XXXXXX)trap 'rm -rf "$tmpfile" "$tmpdir"' EXIT
echo "Working in: $tmpdir"
# flock — exclusive lock on a fileLOCKFILE="/var/run/myscript.lock"
# Method 1: flock with file descriptorexec 200>"$LOCKFILE"if ! flock -n 200; then echo "Another instance is running" >&2 exit 1fi# Lock held until script exits or FD 200 is closed
# Method 2: flock wrapping a commandflock /var/lock/myapp.lock -c "some_exclusive_command"
# Method 3: timeoutif flock -w 10 200; then echo "Got lock within 10 seconds"else echo "Timed out waiting for lock" >&2 exit 1fi
# Atomic file write (write to temp, then rename)tmpout=$(mktemp /etc/config-XXXXXX)generate_config > "$tmpout"mv -f "$tmpout" /etc/config # rename is atomic on same filesystemWhy this approach: mv for atomic writes and flock for concurrency control are the two primitives that make scripts safe to run in parallel or via cron without races.
Answer: Write to a temp file on the same filesystem, then mv (rename), which is atomic at the kernel level — readers never see a partial file.
xxxxxxxxxx
TARGET="/etc/app/config.json"TMPFILE=$(mktemp "$(dirname "$TARGET")/.config-XXXXXX.tmp")
# Ensure temp file is cleaned on failuretrap 'rm -f "$TMPFILE"' EXIT
# Generate content into temp filegenerate_config() { cat <<EOF{ "version": "2.0", "updated": "$(date -Iseconds)", "host": "$(hostname)"}EOF}
generate_config > "$TMPFILE"
# Validate before replacingif ! python3 -m json.tool "$TMPFILE" > /dev/null 2>&1; then echo "Invalid JSON generated, aborting" >&2 exit 1fi
# Set correct permissions before movingchmod 640 "$TMPFILE"
# Atomic replacemv -f "$TMPFILE" "$TARGET"echo "Config updated successfully"Why this approach: Writing to a temp file on the same filesystem ensures mv is a single rename(2) syscall — atomic by POSIX — so no process ever reads a half-written config.
Answer: getopts only handles short options. For long options, use getopt (external), or a manual while/case loop.
xxxxxxxxxx
usage() { cat <<EOFUsage: $0 [OPTIONS] <target> -v, --verbose Enable verbose output -f, --file FILE Input file -n, --count N Repeat count (default: 1) -h, --help Show this helpEOF exit "${1:-0}"}
verbose=falsefile=""count=1
# Manual long-option parsingwhile [[ $# -gt 0 ]]; do case $1 in -v|--verbose) verbose=true; shift ;; -f|--file) file="$2"; shift 2 ;; --file=*) file="${1#--file=}"; shift ;; -n|--count) count="$2"; shift 2 ;; -h|--help) usage 0 ;; --) shift; break ;; -*) echo "Unknown option: $1" >&2; usage 1 ;; *) break ;; esacdone
target="${1:-}"[[ -z "$target" ]] && { echo "Error: target required" >&2; usage 1; }
echo "verbose=$verbose file=$file count=$count target=$target"Why this approach: Manual case parsing is portable, readable, and handles --flag=value and --flag value syntaxes without depending on GNU getopt.
set -euo pipefail together with error handling patterns?Answer: Combining strict mode with trap ERR, conditional ||, and local set +e blocks lets you have both safety and controlled error handling.
xxxxxxxxxxset -euo pipefail
# Trap provides a stack-trace-like messageerror_handler() { local exit_code=$? echo "ERROR: line $1: command '$2' exited with $exit_code" >&2}trap 'error_handler $LINENO "$BASH_COMMAND"' ERR
# Allow specific failures with || trueping -c1 -W1 8.8.8.8 > /dev/null 2>&1 || true
# Allow failure blockcheck_optional_dep() { set +e command -v jq > /dev/null 2>&1 local rc=$? set -e return $rc}
if check_optional_dep; then echo "jq available"fi
# Safe default for possibly-unset variableport="${PORT:-8080}"host="${HOST:?HOST environment variable required}"
# Run risky subshell, capture its exitresult=0( risky_operation ) || result=$?if [[ $result -ne 0 ]]; then echo "risky_operation failed: $result" >&2fiWhy this approach: The combination of set -euo pipefail + trap ERR + targeted || true/set +e gives you both fail-fast safety and the flexibility to handle expected non-zero exits gracefully.
grep flags and when do you use them?Answer: grep searches for patterns; its flags control matching behavior, output format, and recursion.
xxxxxxxxxx# -i case-insensitivegrep -i "error" /var/log/syslog
# -r / -R recursive (R follows symlinks)grep -r "TODO" /src --include="*.py"
# -l print only filenames with matchesgrep -rl "deprecated" /codebase
# -n show line numbersgrep -n "FATAL" app.log
# -v invert match (non-matching lines)grep -v "^#" /etc/hosts # strip comments
# -c count matching linesgrep -c "ERROR" app.log
# -w whole word onlygrep -w "int" source.c # matches "int" not "integer"
# -A/-B/-C context lines (after/before/both)grep -A 3 -B 1 "Exception" app.log
# -E extended regex (egrep)grep -E "^(ERROR|WARN)" app.log
# -F fixed string, no regex (fgrep — fastest)grep -F "192.168.1.1" access.log
# -o print only the matching partgrep -oE '[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+' access.log # extract IPs
# -P Perl-compatible regex (GNU grep)grep -P '(?<=user=)\w+' auth.log
# --color, -q (quiet/silent for exit code only)grep -q "pattern" file && echo "found"Why this approach: -q with an if statement is the cleanest way to use grep as a boolean test without capturing output.
sed work for substitution, deletion, and in-place editing?Answer: sed is a stream editor that applies commands to each line. s/// substitutes, d deletes, -i edits in-place.
xxxxxxxxxx# Basic substitution (first occurrence per line)sed 's/foo/bar/' file.txt
# Global substitution (all occurrences)sed 's/foo/bar/g' file.txt
# Case-insensitive (GNU sed)sed 's/error/ERROR/gI' file.txt
# In-place edit (GNU: -i; BSD/macOS: -i '')sed -i 's/localhost/db.prod/g' config.inised -i.bak 's/old/new/g' file.txt # saves backup as file.txt.bak
# Delete lines matching patternsed '/^#/d' config.txt # delete comment linessed '/^$/d' file.txt # delete blank lines
# Delete line rangesed '5,10d' file.txtsed '/START/,/END/d' file.txt
# Print specific lines (-n suppresses default print)sed -n '10,20p' file.txtsed -n '/ERROR/p' app.log
# Insert/append linessed '/pattern/i\New line before' file.txtsed '/pattern/a\New line after' file.txt
# Multiple expressionssed -e 's/foo/bar/g' -e 's/baz/qux/g' file.txt
# Address ranges with regexsed '/BEGIN/,/END/ s/x/y/g' file.txt
# Extract value from configsed -n 's/^port\s*=\s*//p' config.iniWhy this approach: -i.bak is safer than plain -i in production — you always have a rollback copy before testing a new sed expression.
awk work? Explain fields, NR, NF, patterns, BEGIN/END.Answer: awk processes records (lines) splitting them into fields. NR=record number, NF=number of fields, $0=whole line, $1..$NF=fields.
xxxxxxxxxx# Print specific fieldsawk '{print $1, $3}' /etc/passwd # columns 1 and 3
# Field separator with -Fawk -F: '{print $1, $6}' /etc/passwd # user:home
# Pattern matchingawk '/ERROR/ {print $0}' app.logawk '$3 > 100 {print $1, $3}' data.txt # numeric comparison
# BEGIN and END blocksawk 'BEGIN {sum=0} {sum+=$2} END {print "Total:", sum}' data.txt
# NR and NFawk 'NR==5' file.txt # print 5th lineawk 'NR>=10 && NR<=20' file.txt # lines 10-20awk 'NF>3' file.txt # lines with >3 fieldsawk '{print NR": "$0}' file.txt # add line numbers
# Associative arraysawk '{count[$1]++} END {for (k in count) print k, count[k]}' access.log
# Sum by groupawk -F, '{sum[$1]+=$2} END {for (k in sum) print k, sum[k]}' data.csv
# Printf formattingawk '{printf "%-20s %5d\n", $1, $2}' data.txt
# Multiple field separatorsawk -F'[,;:]' '{print $2}' mixed.txt
# Change output separatorawk 'BEGIN{OFS=","} {print $1,$2,$3}' file.txtWhy this approach: awk associative arrays are the go-to for frequency counting and grouping tasks that would require multi-step pipelines in pure shell.
cut, sort, uniq together?Answer: cut extracts columns, sort orders lines, uniq deduplicates (adjacent) lines. They compose naturally in pipelines.
xxxxxxxxxx# cut — by delimiter and fieldcut -d: -f1 /etc/passwd # first field (username)cut -d, -f2,4 data.csv # fields 2 and 4cut -c1-10 file.txt # characters 1-10
# sort flagssort file.txt # lexicographicsort -n numbers.txt # numericsort -rn numbers.txt # reverse numericsort -k2,2n data.txt # sort by field 2 numericallysort -k1,1 -k2,2n data.txt # multi-key sortsort -t: -k3,3n /etc/passwd # sort passwd by UIDsort -u file.txt # sort + deduplicate
# uniq (requires sorted input)sort file.txt | uniq # deduplicatesort file.txt | uniq -c # count occurrencessort file.txt | uniq -d # show only duplicatessort file.txt | uniq -u # show only unique lines
# Classic top-N frequency patterncut -d' ' -f1 access.log | sort | uniq -c | sort -rn | head -10
# Count unique IPs in logawk '{print $1}' access.log | sort -u | wc -l
# Find duplicate linessort file.txt | uniq -dWhy this approach: The sort | uniq -c | sort -rn | head pipeline is a universal frequency analysis idiom every SRE should know by heart.
tr work and what are its common uses?Answer: tr translates or deletes characters from stdin. It operates on individual characters (not strings/regex like sed).
xxxxxxxxxx# Translate characters (lowercase to uppercase)echo "Hello World" | tr 'a-z' 'A-Z'echo "Hello World" | tr 'A-Z' 'a-z'
# Delete characters (-d)echo "hello 123 world" | tr -d '0-9' # remove digitsecho "line1\r\nline2" | tr -d '\r' # strip Windows CR
# Squeeze repeated characters (-s)echo "hello world" | tr -s ' ' # "hello world"echo "aabbcc" | tr -s 'a-z' # "abc"
# Complement (-c: use characters NOT in set)echo "hello 123!" | tr -dc '0-9' # keep only digits: 123echo "hello" | tr -c 'a-z' '*' # replace non-lowercase with *
# Convert newlines to spaces (join lines)tr '\n' ' ' < file.txt
# Replace colons with newlines (split PATH)echo "$PATH" | tr ':' '\n'
# Remove non-printable characterstr -dc '[:print:]\n' < binary_file.txt
# ROT13echo "Hello" | tr 'A-Za-z' 'N-ZA-Mn-za-m'Why this approach: tr -d '\r' is the fastest way to fix Windows line endings in a pipeline without touching the file, and tr -dc '0-9' is the simplest digit-extraction one-liner.
wc work and what are its flags?Answer: wc counts lines, words, and bytes/characters in files or stdin.
xxxxxxxxxx# Count lines, words, byteswc file.txt # lines words bytes filename
# Specific countswc -l file.txt # lines onlywc -w file.txt # words onlywc -c file.txt # byteswc -m file.txt # characters (multibyte-aware)
# Count lines in multiple files + totalwc -l *.log
# Common idiom: count output linesls /etc | wc -lgrep -c "ERROR" app.log # grep -c is faster than grep | wc -l
# Count files in a directoryfind /etc -maxdepth 1 -type f | wc -l
# Longest line lengthawk '{ print length }' file.txt | sort -n | tail -1# orwc -L file.txt # GNU: length of longest line
# Strip leading whitespace from wc outputn=$(wc -l < file.txt) # redirect stdin → no filename, no leading spaceecho "Lines: $n"Why this approach: wc -l < file (with input redirection) avoids the leading-space issue in $(wc -l file) output — the filename is omitted and the number is clean.
xargs work and when should you use it over a loop?Answer: xargs builds and executes commands from stdin, batching arguments efficiently. It's faster than a shell loop for large input sets.
xxxxxxxxxx# Basic: pass stdin as argumentsfind /tmp -name "*.tmp" | xargs rm -f
# -I{}: one argument at a time with placeholderfind . -name "*.log" | xargs -I{} gzip {}
# -P: parallel executionfind . -name "*.jpg" | xargs -P4 -I{} convert {} -resize 800x {}
# -n: max arguments per invocationecho "a b c d e" | xargs -n2 echo # echo a b; echo c d; echo e
# -d: custom delimiter (GNU xargs)echo "a:b:c" | xargs -d: echo # echo a b c
# Handle filenames with spaces/newlines: -0 with find -print0find . -name "*.txt" -print0 | xargs -0 grep -l "pattern"
# Confirm before executing (-p)find /tmp -name "*.bak" | xargs -p rm
# -t: print commands before executing (debug)find . -name "*.conf" | xargs -t grep "port"
# When NOT to use a loop# Slow (N fork+execs):for f in *.log; do gzip "$f"; done# Fast (batch):printf '%s\0' *.log | xargs -0 gzipWhy this approach: -print0 | xargs -0 is the only safe combination for filenames containing spaces, newlines, or special characters.
find -exec work and how does it compare to find | xargs?Answer: find -exec cmd {} \; runs one command per found file; -exec cmd {} + batches multiple files per invocation (like xargs); | xargs offers more flexibility.
xxxxxxxxxx# One file per exec (slow for large sets)find /tmp -name "*.tmp" -exec rm {} \;
# Batch (faster — multiple files per command invocation)find /tmp -name "*.tmp" -exec rm {} +
# Complex per-file logic (needs shell)find . -name "*.log" -exec sh -c ' echo "Processing: $1" gzip "$1"' _ {} \;
# find | xargs -0 (most flexible)find /var/log -name "*.log" -mtime +30 -print0 | xargs -0 -P4 gzip
# Conditional: only if file has contentfind . -name "*.txt" -exec grep -l "TODO" {} +
# find with -ok (interactive confirmation per file)find /home -name "core" -ok rm {} \;
# Delete empty directoriesfind /tmp -type d -empty -exec rmdir {} +
# Find and chmodfind /var/www -type f -name "*.php" -exec chmod 644 {} +
# Multiple actionsfind . -name "*.bak" -exec cp {} {}.orig \; -exec rm {} \;Why this approach: -exec {} + is preferred over \; for performance — it batches files into single command invocations exactly like xargs, but without the pipe.
paste to combine files and columns?Answer: paste merges files line by line (horizontally), placing fields side by side with a delimiter.
xxxxxxxxxx# Merge two files line by line (default delimiter: tab)paste file1.txt file2.txt
# Custom delimiterpaste -d, file1.txt file2.txt # CSVpaste -d'|' names.txt ages.txt
# Serialize: join all lines of a file with a delimiterpaste -sd, list.txt # "a,b,c,d"paste -sd'\n' file.txt # no-op (already newlines)
# Combine with process substitutionpaste <(cut -d: -f1 /etc/passwd) <(cut -d: -f3 /etc/passwd)
# Number lines (alternative to nl)paste - file.txt <<< "$(seq $(wc -l < file.txt))"
# Create CSV from multiple column filespaste -d, \ <(awk '{print $1}' data.txt) \ <(awk '{print $2}' data.txt) \ > output.csv
# Transpose rows to columnseq 5 | paste - - - # 3 columns from one columnWhy this approach: paste -sd, is the fastest one-liner to join a file's lines with a custom delimiter — cleaner than an awk loop for simple serialization.
Answer: Combine tr, sort, and uniq -c in a pipeline to tokenize, count, and rank.
xxxxxxxxxx# Classic word frequency pipelinetr -s '[:space:][:punct:]' '\n' < file.txt \ | tr 'A-Z' 'a-z' \ | sort \ | uniq -c \ | sort -rn \ | head -5
# With grep to skip empty tokenstr -s '[:space:][:punct:]' '\n' < file.txt \ | tr 'A-Z' 'a-z' \ | grep -v '^$' \ | sort \ | uniq -c \ | sort -rn \ | head -5
# awk version (single pass, more efficient)awk '{ gsub(/[^a-zA-Z]/, " ") for (i=1; i<=NF; i++) { word = tolower($i) if (length(word) > 0) count[word]++ }}END { for (w in count) print count[w], w}' file.txt | sort -rn | head -5Why this approach: The tr | sort | uniq -c | sort -rn | head pipeline is a universal pattern that works on any Unix system without scripting, but the awk single-pass version is more efficient for large files.
awk?Answer: Use awk with field separators matching the log format to extract, filter, and aggregate data.
xxxxxxxxxx# Apache/Nginx access log: "IP - - [date] "method URL" status bytes"# Extract status codes and countawk '{print $9}' access.log | sort | uniq -c | sort -rn
# Find all 5xx errors with their IPsawk '$9 ~ /^5/ {print $1, $9, $7}' access.log
# Calculate average response sizeawk '{sum+=$10; n++} END {print "avg:", sum/n}' access.log
# Requests per minute from timestampawk '{print $4}' access.log | cut -c2-18 | sort | uniq -c
# Syslog: "Month Day HH:MM:SS host process[pid]: message"awk '/error/i {print $3, $5, $0}' /var/log/syslog
# CSV with quoted fields (simple case)awk -F'"*,"*' '{print $1, $3}' data.csv
# Key=value log format# "2024-01-01 level=ERROR user=alice action=login"awk '{ for (i=1; i<=NF; i++) { split($i, kv, "=") if (kv[1] == "user") print kv[2] }}' app.log
# Top N IPs making requestsawk '{print $1}' access.log | sort | uniq -c | sort -rn | head -10Why this approach: awk with associative arrays handles the most common log aggregation tasks in a single pass without loading the entire file into memory.
sed or awk?Answer: sed can use address ranges (/start/,/end/); awk accumulates lines with a flag variable for multiline blocks.
xxxxxxxxxx# sed: print lines between two patterns (inclusive)sed -n '/BEGIN/,/END/p' file.txt
# sed: delete block between patternssed '/BEGIN/,/END/d' file.txt
# sed: replace within blocksed '/BEGIN/,/END/ s/old/new/g' file.txt
# awk: collect and process multiline blocksawk '/BEGIN/ { capture=1; block="" }capture { block = block $0 "\n" }/END/ { capture=0; print "BLOCK:\n" block }' file.txt
# awk: join continuation lines (lines ending with \)awk '/\\$/ { gsub(/\\$/,""); printf "%s", $0; next } { print }' file.txt
# awk: paragraph mode (blank-line separated records)awk 'BEGIN{RS=""} /pattern/ {print NR": "$0}' file.txt
# grep: context for multiline awarenessgrep -A5 "^function " script.sh # show function + 5 lines after
# Perl for complex multiline (when awk/sed aren't enough)perl -0777 -ne 'while (/BEGIN(.*?)END/sg) { print $1 }' file.txtWhy this approach: awk's RS="" paragraph mode is underused but extremely powerful — it treats blank-line-separated blocks as single records, making multiline field extraction trivial.
sed for in-place configuration file editing in scripts?Answer: Use sed -i with anchored patterns to safely modify specific config keys without disrupting surrounding content.
xxxxxxxxxx
CONFIG="/etc/myapp/config.ini"
# Change a value: key = oldval → key = newvalsed -i 's/^\(port\s*=\s*\).*/\18080/' "$CONFIG"
# GNU sed (simpler with \K lookahead not available, use groups)sed -i "s/^port=.*/port=8080/" "$CONFIG"
# Uncomment a linesed -i 's/^#\s*\(listen_address\)/\1/' "$CONFIG"
# Comment out a linesed -i 's/^\(debug_mode\)/#\1/' "$CONFIG"
# Add a line after a matchsed -i '/^\[server\]/a listen_port = 9090' "$CONFIG"
# Add a line before a matchsed -i '/^\[database\]/i # Database config below' "$CONFIG"
# Replace only between section markerssed -i '/^\[cache\]/,/^\[/ s/^size=.*/size=512/' "$CONFIG"
# Idempotent: only change if not already setif ! grep -q "^max_connections=200" "$CONFIG"; then sed -i 's/^max_connections=.*/max_connections=200/' "$CONFIG"fi
# Validate config after changegrep "^port=" "$CONFIG"Why this approach: Using capture groups in sed substitutions (\1) is safer than rewriting the full line, because it preserves surrounding formatting and reduces the regex surface area.
Answer: sort -u loses order. To preserve insertion order, use awk with an associative array as a seen-set.
xxxxxxxxxx# awk: first-seen wins (preserves order)awk '!seen[$0]++' file.txt
# How it works:# seen[$0] is 0 (false) for new lines → !0 = 1 → print# seen[$0] is 1+ for duplicates → !1 = 0 → skip# ++ increments AFTER the test
# Case-insensitive dedup while preserving original caseawk '!seen[tolower($0)]++' file.txt
# Dedup on specific field (field 1)awk '!seen[$1]++' file.txt
# Dedup CSV on second columnawk -F, '!seen[$2]++' data.csv
# Dedup and count removed linesawk ' !seen[$0]++ { print; next } { dupes++ } END { print dupes " duplicates removed" > "/dev/stderr" }' file.txt
# For very large files: sort preserves order with stable sort + dedup# (loses original order but stable within equal elements)sort -u file.txt > deduped.txtWhy this approach: awk '!seen[$0]++' is a canonical one-liner — the ! test and ++ post-increment combine into a single elegant gate that every senior engineer should recognize.
Answer: Bash only handles integers natively via $(( )) or let. For floating-point, use bc, awk, or Python.
xxxxxxxxxx# Integer arithmeticx=10y=3echo $(( x + y )) # 13echo $(( x / y )) # 3 (integer division)echo $(( x % y )) # 1 (modulo)echo $(( x ** 2 )) # 100 (exponentiation)(( x++ ))(( y *= 2 ))echo "$x $y" # 11 6
# bc for floating pointecho "scale=4; 10/3" | bc # 3.3333echo "scale=2; sqrt(2)" | bc -l # 1.41result=$(echo "scale=6; 22/7" | bc)
# awk for floating pointawk 'BEGIN { printf "%.4f\n", 10/3 }' # 3.3333awk 'BEGIN { print sqrt(2) }'
# Python one-linerpython3 -c "print(f'{10/3:.4f}')"python3 -c "import math; print(math.pi)"
# Comparing floats in scriptsthreshold=80.5actual=85.2# Use awk for comparisonif awk "BEGIN { exit !($actual > $threshold) }"; then echo "Above threshold"fiWhy this approach: Using awk 'BEGIN { exit !(expr) }' is the cleanest way to use a floating-point comparison as a shell conditional without spawning a Python process.
Answer: jq is the standard tool for JSON processing; python3 -m json.tool validates; awk/grep work for simple cases.
xxxxxxxxxx# Pretty-printjq '.' data.json
# Extract fieldjq '.name' data.jsonjq -r '.name' data.json # -r: raw string (no quotes)
# Nested fieldjq '.server.port' config.json
# Array elementjq '.[0]' array.jsonjq '.items[2].name' data.json
# Filter arrayjq '.users[] | select(.active == true)' users.json
# Map/transformjq '.users[] | {name: .name, email: .email}' users.json
# Aggregatejq '[.items[].price] | add' data.jsonjq '.items | length' data.json
# Construct new JSONjq -n --arg host "$HOSTNAME" --argjson port 8080 \ '{host: $host, port: $port}'
# From shell variableecho '{"key":"val"}' | jq -r '.key'
# Validate JSONpython3 -m json.tool data.json > /dev/null && echo "valid"
# jq in scriptsmapfile -t names < <(jq -r '.users[].name' users.json)for name in "${names[@]}"; do echo "User: $name"doneWhy this approach: jq -r (raw output) is essential when using jq results in shell variables — without it, string values are double-quoted and break downstream commands.
Answer: For simple CSVs use cut/awk; for files with quoted fields or embedded commas, use python3 or mlr (Miller).
xxxxxxxxxx# Simple CSV (no quoted fields)cut -d, -f1,3 data.csv # columns 1 and 3awk -F, '{print $2}' data.csv # column 2awk -F, 'NR>1 {sum+=$3} END{print sum}' data.csv # sum col 3, skip header
# Skip headertail -n +2 data.csv | cut -d, -f2
# Add header back after processing{ head -1 data.csv; tail -n +2 data.csv | sort -t, -k2; }
# Handle quoted fields with Pythonpython3 -c "import csv, syswith open('data.csv') as f: reader = csv.DictReader(f) for row in reader: print(row['name'], row['age'])"
# Convert CSV to TSVsed 's/,/\t/g' simple.csv
# Count rows (excluding header)awk 'NR>1' data.csv | wc -l
# Filter rows where column 3 > 100awk -F, 'NR==1 || $3>100' data.csv
# Sort by column 2 numerically(head -1 data.csv; tail -n +2 data.csv | sort -t, -k2,2n) > sorted.csvWhy this approach: For production scripts with CSVs that might contain quoted fields, always use Python's csv module — awk/cut fail silently on embedded commas and quotes.
tee for logging and splitting output?Answer: tee writes stdin to both a file and stdout simultaneously, enabling output logging without breaking pipelines.
xxxxxxxxxx# Log to file while displayingcommand | tee output.log
# Append to logcommand | tee -a output.log
# Log without displaying (redirect stdout to /dev/null)command | tee output.log > /dev/null
# Split to multiple destinationscommand | tee file1.txt file2.txt | grep "ERROR"
# Process substitution: send to multiple commands simultaneouslymake 2>&1 | tee >(grep -c ERROR > error_count.txt) \ >(grep "WARNING" > warnings.log) \ build.log
# Log entire script outputexec > >(tee -a /var/log/myscript.log) 2>&1echo "This goes to both console and log"
# With timestampscommand | tee >(ts '[%Y-%m-%d %H:%M:%S]' >> timestamped.log)
# Capture output AND check exit codeoutput=$(command | tee /dev/stderr) # shows on stderr, captures to var
# In CI: log to file while streaming./run_tests.sh 2>&1 | tee test_results.logecho "Tests exited: ${PIPESTATUS[0]}"Why this approach: exec > >(tee -a logfile) 2>&1 at the top of a script is a clean way to make the entire script's output self-logging without wrapping every command.
grep with regular expressions for log analysis?Answer: Combine grep's regex modes with context flags and output formatting for efficient log triage.
xxxxxxxxxx# Extract timestamps and error messagesgrep -E "^[0-9]{4}-[0-9]{2}-[0-9]{2}.*ERROR" app.log
# Find lines with IP addressesgrep -oE '([0-9]{1,3}\.){3}[0-9]{1,3}' access.log | sort -u
# Multi-pattern ORgrep -E "ERROR|WARN|CRITICAL" app.log
# Count by log levelgrep -cE "^.*ERROR" app.logfor level in DEBUG INFO WARN ERROR FATAL; do count=$(grep -c " $level " app.log 2>/dev/null || echo 0) printf "%-8s %d\n" "$level:" "$count"done
# Find exceptions with stack traces (context lines)grep -A10 "Exception" app.log | grep -v "^--$"
# Between timestampsgrep -E "^2024-01-15 1[0-9]:" app.log # 10:xx to 19:xx
# Negative lookahead equivalent (exclude healthcheck hits)grep "GET /api" access.log | grep -v "/healthcheck"
# Extract unique error messagesgrep "ERROR" app.log | sed 's/.*ERROR: //' | sort -u
# Real-time log following with filtertail -f /var/log/app.log | grep --line-buffered "ERROR"Why this approach: --line-buffered on grep in a tail -f pipeline is required to see results in real time — without it, grep buffers output and you get delayed or no output.
Answer: Chain awk, sort, uniq, and head to extract the key metrics an operator needs at a glance.
xxxxxxxxxx# Apache log format: IP - - [date] "METHOD /path HTTP/1.1" status bytes
LOG="${1:-/var/log/apache2/access.log}"
echo "=== Top 10 Client IPs ==="awk '{print $1}' "$LOG" | sort | uniq -c | sort -rn | head -10
echo ""echo "=== HTTP Status Code Distribution ==="awk '{print $9}' "$LOG" | sort | uniq -c | sort -rn
echo ""echo "=== Top 10 Requested URLs ==="awk '{print $7}' "$LOG" | sort | uniq -c | sort -rn | head -10
echo ""echo "=== 5xx Errors ==="awk '$9 ~ /^5/' "$LOG" | awk '{print $9, $7, $1}' | sort | uniq -c | sort -rn | head -20
echo ""echo "=== Total Requests and Bandwidth ==="awk ' {requests++; bytes += ($10 ~ /^[0-9]+$/) ? $10 : 0} END { printf "Requests: %d\n", requests printf "Bandwidth: %.2f MB\n", bytes/1024/1024 }' "$LOG"
echo ""echo "=== Requests per Hour ==="awk '{print $4}' "$LOG" | cut -c2-15 | cut -d: -f1-2 \ | sort | uniq -c | sort -k2Why this approach: Breaking each metric into its own pipeline stage makes the script easy to extend, debug, and reuse — each section is independently testable.
Answer: Use flags that prevent accidental overwrites, verbose output for logging, and always quote paths.
xxxxxxxxxx
SRC="/data/source"DST="/data/backup"
# cp: safe optionscp -v "$SRC/file.txt" "$DST/" # verbosecp -i "$SRC/file.txt" "$DST/" # interactive (prompt on overwrite)cp -n "$SRC/file.txt" "$DST/" # no-clobber (skip if exists)cp -p "$SRC/file.txt" "$DST/" # preserve timestamps/permissionscp -a "$SRC/" "$DST/" # archive: recursive, preserve all attrscp -r --backup=numbered "$SRC/" "$DST/" # number backups
# mv: atomic on same filesystemmv -v "$SRC/file.txt" "$DST/"mv -i "$SRC/file.txt" "$DST/" # prompt on overwritemv -n "$SRC/file.txt" "$DST/" # no-clobber
# rm: safe practicesrm -v file.txt # verboserm -i *.txt # interactive confirm eachrm -f file.txt # force (no error if missing)rm -rf /tmp/workdir-$$ # recursive force (use with care)
# Safe rm pattern: verify before removingtarget="/tmp/old_data"if [[ -d "$target" && "$target" == /tmp/* ]]; then rm -rf "$target"else echo "Refusing to rm: $target" >&2fi
# Trash instead of delete (if trash-cli available)# trash-put file.txt
# rsync for safe incremental copyrsync -av --checksum "$SRC/" "$DST/"rsync -av --delete "$SRC/" "$DST/" # mirror (deletes extras in dst)Why this approach: Anchoring rm -rf with a path prefix guard ($target == /tmp/*) is a critical safety check — a variable expansion to empty string would otherwise rm -rf /.
find for file cleanup tasks?Answer: find with -mtime, -size, -name, and -exec builds powerful cleanup pipelines.
xxxxxxxxxx
# Delete files older than 30 days in /tmpfind /tmp -type f -mtime +30 -delete
# Delete but log firstfind /tmp -type f -mtime +30 -print -delete
# Delete files larger than 100MB in /var/logfind /var/log -type f -size +100M -exec ls -lh {} +
# Dry run with size reportfind /var/log -type f -size +100M -exec du -sh {} +
# Remove empty directoriesfind /data -type d -empty -delete
# Remove files by extension older than 7 daysfind /var/log -name "*.gz" -mtime +7 -delete
# Remove files, but keep newest 5# (needs more logic — combine with ls and tail)ls -t /var/log/app/app.log.* | tail -n +6 | xargs rm -f
# Find and compress old logsfind /var/log/app -name "*.log" -mtime +1 -not -name "*.gz" \ -exec gzip {} +
# Cleanup with size limit: remove oldest until under 1GBcleanup_dir() { local dir="$1" local max_bytes=$(( 1024 * 1024 * 1024 )) while [[ $(du -sb "$dir" | cut -f1) -gt $max_bytes ]]; do oldest=$(find "$dir" -type f -printf '%T+ %p\n' | sort | head -1 | cut -d' ' -f2-) [[ -z "$oldest" ]] && break rm -v "$oldest" done}Why this approach: -delete is safer than -exec rm {} + because find won't delete a directory unless it's empty, and it's faster since no shell process is spawned.
inotifywait?Answer: inotifywait (from inotify-tools) blocks until a filesystem event occurs — ideal for trigger-based automation.
xxxxxxxxxx# Requires: apt install inotify-tools / yum install inotify-tools
WATCH_DIR="/var/spool/incoming"
# Wait for a single eventinotifywait -e close_write "$WATCH_DIR/trigger.flag"echo "Trigger file written, processing..."
# Watch continuously (loop)inotifywait -m -e close_write --format '%f' "$WATCH_DIR" |while IFS= read -r filename; do echo "New file: $filename" process_file "$WATCH_DIR/$filename"done
# Watch multiple eventsinotifywait -m -e create,close_write,delete \ --format '%e %f' "$WATCH_DIR" |while read -r event filename; do case "$event" in CREATE|CLOSE_WRITE) ingest "$filename" ;; DELETE) cleanup "$filename" ;; esacdone
# Recursive watchinotifywait -rm -e modify,create,delete /etc/myapp/ |while read -r dir event file; do echo "Config changed: $dir$file ($event)" systemctl reload myappdone
# With timeout (stop watching after 60s)inotifywait -t 60 -e modify /etc/hostsWhy this approach: Piping inotifywait -m output into a while read loop is the standard pattern for event-driven file processing — it's efficient because it uses kernel inotify events rather than polling.
lsof and fuser?Answer: lsof (list open files) and fuser identify which processes are using files, sockets, or devices.
xxxxxxxxxx# lsof — which process has a file openlsof /var/log/app.log
# Which process is listening on port 8080lsof -i :8080lsof -i tcp:443
# All network connections by a processlsof -i -p $(pgrep nginx)
# Files opened by a specific userlsof -u www-data
# Files opened by a specific PIDlsof -p 1234
# Find deleted files still held open (disk space recovery)lsof | grep deleted
# Check what process holds a directory (blocking umount)lsof +D /mnt/usb
# fuser — simpler, focused on files/socketsfuser /var/log/app.log # PIDs using the filefuser -v /var/log/app.log # verbose: PID, user, access typefuser -k /var/log/app.log # kill all processes using the filefuser 8080/tcp # process on TCP port 8080fuser -v -n tcp 443 # verbose network
# Kill process on port (combine lsof + kill)kill -9 $(lsof -t -i :8080)
# ss as modern alternativess -tlnp | grep :8080 # faster than lsof for port checksWhy this approach: lsof | grep deleted is an essential SRE technique — when a log file is deleted but a process still writes to it, disk space isn't freed until the process closes the FD.
Answer: Combine ps, pgrep, kill, and a loop to watch and restart processes.
xxxxxxxxxxset -euo pipefail
SERVICE="myapp"BINARY="/usr/local/bin/myapp"LOG="/var/log/myapp-monitor.log"CHECK_INTERVAL=30
log() { echo "[$(date '+%Y-%m-%d %H:%M:%S')] $*" | tee -a "$LOG"; }
is_running() { pgrep -x "$SERVICE" > /dev/null 2>&1}
start_service() { log "Starting $SERVICE..." "$BINARY" --config /etc/myapp/config.yaml & sleep 2 if is_running; then log "$SERVICE started (PID $(pgrep -x "$SERVICE"))" else log "ERROR: $SERVICE failed to start" return 1 fi}
# Check CPU/memorycheck_resources() { local pid pid=$(pgrep -x "$SERVICE" | head -1) local cpu mem cpu=$(ps -p "$pid" -o %cpu= | tr -d ' ') mem=$(ps -p "$pid" -o %mem= | tr -d ' ') log "Resources: CPU=${cpu}% MEM=${mem}%"
# Restart if CPU > 90% for extended period if awk "BEGIN{exit !($cpu > 90)}"; then log "WARNING: High CPU usage" fi}
# Main loopwhile true; do if ! is_running; then log "$SERVICE not running" start_service || log "Failed to restart" else check_resources fi sleep "$CHECK_INTERVAL"doneWhy this approach: pgrep -x matches the exact process name (not substring), preventing false positives when multiple services share a name prefix.
Answer: Wrap the command in a for or while loop with exponential backoff and a max-retry limit.
xxxxxxxxxx
# Simple retry with fixed delayretry() { local max_attempts="$1" local delay="$2" shift 2 local attempt=1
until "$@"; do if (( attempt >= max_attempts )); then echo "Command failed after $max_attempts attempts: $*" >&2 return 1 fi echo "Attempt $attempt failed. Retrying in ${delay}s..." >&2 sleep "$delay" (( attempt++ )) done}
# Usageretry 5 3 curl -sf https://api.example.com/health
# Exponential backoffretry_backoff() { local max_attempts="${1:-5}" shift local attempt=1 local delay=1
until "$@"; do if (( attempt >= max_attempts )); then echo "Failed after $max_attempts attempts: $*" >&2 return 1 fi echo "Attempt $attempt failed. Backing off ${delay}s..." >&2 sleep "$delay" delay=$(( delay * 2 )) (( attempt++ )) done}
retry_backoff 5 wget -q https://example.com/large-file.tar.gz
# Wait for a condition (e.g., service ready)wait_for_port() { local host=$1 port=$2 timeout=${3:-60} local elapsed=0 until nc -z "$host" "$port" 2>/dev/null; do (( elapsed++ )) (( elapsed >= timeout )) && { echo "Timeout waiting for $host:$port" >&2; return 1; } sleep 1 done echo "$host:$port is ready"}
wait_for_port db.local 5432 30Why this approach: Exponential backoff is essential for retrying distributed system calls — linear retries can amplify thundering-herd problems during outages.
nohup and disown to keep processes running after logout?Answer: nohup makes a command immune to SIGHUP (sent on terminal close); disown removes a job from the shell's job table after it's started.
xxxxxxxxxx# nohup: immune to hangup, stdout/stderr to nohup.outnohup long_running_task &nohup long_running_task > /var/log/task.log 2>&1 &echo "PID: $!"
# disown: remove from shell job table (already backgrounded job)long_running_task &disown %1 # by job numberdisown $! # by last background PIDdisown -h $! # keep in table but mark SIGHUP-immune
# Run at low prioritynice -n 19 nohup ./heavy_computation.sh > output.log 2>&1 &
# Redirect all streams explicitly (best practice)nohup bash -c 'long_cmd arg1 arg2' \ > /var/log/task.out \ 2> /var/log/task.err \ < /dev/null &echo "Started: $!"
# screen/tmux (better for interactive sessions)screen -S mysession -dm bash -c './task.sh > task.log 2>&1'tmux new-session -d -s mysession './task.sh'
# systemd (production-grade)systemd-run --unit=mytask --scope ./task.shWhy this approach: Redirecting stdin from /dev/null in nohup commands prevents the background process from blocking on terminal input, which would hang it silently.
Answer: Use trap EXIT ERR INT TERM with a comprehensive cleanup function to ensure resources are released regardless of how the script exits.
xxxxxxxxxxset -euo pipefail
# --- State tracking ---TMPDIR_WORK=""LOCKFD=""CHILD_PIDS=()
# --- Cleanup function ---cleanup() { local exit_code=$?
# Kill any spawned children for pid in "${CHILD_PIDS[@]}"; do kill "$pid" 2>/dev/null || true wait "$pid" 2>/dev/null || true done
# Release lock if [[ -n "$LOCKFD" ]]; then flock -u "$LOCKFD" 2>/dev/null || true eval "exec $LOCKFD>&-" 2>/dev/null || true fi
# Remove temp directory if [[ -n "$TMPDIR_WORK" && -d "$TMPDIR_WORK" ]]; then rm -rf "$TMPDIR_WORK" echo "Removed $TMPDIR_WORK" >&2 fi
echo "Script exiting with code $exit_code" >&2 exit "$exit_code"}
trap cleanup EXITtrap 'echo "Interrupted" >&2; exit 130' INTtrap 'echo "Terminated" >&2; exit 143' TERM
# --- Setup ---TMPDIR_WORK=$(mktemp -d)LOCKFD=200exec 200>/var/run/myscript.lockflock -n 200 || { echo "Already running" >&2; exit 1; }
# --- Work ---background_task &CHILD_PIDS+=($!)
echo "Working..."sleep 10
# Cleanup runs automatically at exitWhy this approach: Registering cleanup on EXIT (not just INT/TERM) ensures it runs even on exit calls from set -e, making the cleanup unconditional.
Answer: Background jobs with & + wait, xargs -P, or GNU parallel provide different levels of parallelism control.
xxxxxxxxxx
# Method 1: background jobs with PID trackingprocess_file() { local f="$1" gzip "$f" echo "Compressed: $f"}
MAX_JOBS=4pids=()
for f in /var/log/*.log; do # Throttle: wait if at max jobs while (( ${#pids[@]} >= MAX_JOBS )); do for i in "${!pids[@]}"; do if ! kill -0 "${pids[$i]}" 2>/dev/null; then unset 'pids[$i]' fi done pids=("${pids[@]}") # repack array (( ${#pids[@]} >= MAX_JOBS )) && sleep 0.1 done
process_file "$f" & pids+=($!)donewait
# Method 2: xargs -P (simplest)find /var/log -name "*.log" -print0 | xargs -0 -P4 gzip
# Method 3: GNU parallelparallel -j4 gzip {} ::: /var/log/*.log
# Method 4: sem (parallel semaphore)for f in *.txt; do sem -j4 process "$f"donesem --wait
# Collect results with temp filesfor i in {1..10}; do ( result=$(compute "$i"); echo "$i $result" > "/tmp/result-$$-$i" ) &donewaitcat /tmp/result-$$-* | sort -nrm /tmp/result-$$-*Why this approach: xargs -P is the simplest parallel approach for embarrassingly parallel file operations; use sem from GNU parallel when you need a semaphore without rewriting the whole pipeline.
Answer: A cron script needs: exclusive locking, self-logging, sane environment, and clean exit codes.
xxxxxxxxxx# cron: */15 * * * * /usr/local/bin/process-queue.sh
set -euo pipefail
SCRIPT_NAME=$(basename "$0" .sh)LOG_DIR="/var/log/myapp"LOG_FILE="$LOG_DIR/${SCRIPT_NAME}.log"LOCK_FILE="/var/run/${SCRIPT_NAME}.lock"MAX_LOG_SIZE=$(( 10 * 1024 * 1024 )) # 10 MB
# Ensure log directory existsmkdir -p "$LOG_DIR"
# Rotate log if too largeif [[ -f "$LOG_FILE" ]] && (( $(stat -c%s "$LOG_FILE") > MAX_LOG_SIZE )); then mv "$LOG_FILE" "${LOG_FILE}.1"fi
# Redirect all output to logexec >> "$LOG_FILE" 2>&1
log() { echo "[$(date '+%Y-%m-%d %H:%M:%S')] [$SCRIPT_NAME] $*"; }
# Exclusive lockexec 9>"$LOCK_FILE"if ! flock -n 9; then log "Another instance running, exiting" exit 0 # not an error for cronfi
log "Starting"
# Normalize environment (cron has minimal PATH)export PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
# Cleanup on exittrap 'log "Finished (exit $?)"' EXIT
# Main logicprocess_items() { log "Processing queue..." # ... work here ...}
process_itemsWhy this approach: Exiting 0 when the lock is already held is the correct cron behavior — a non-zero exit would generate a cron email alert for every normal overlap, causing alert fatigue.
Answer: Scripts can send alerts via mail/sendmail, curl to webhook APIs, or dedicated CLI tools.
xxxxxxxxxx
# Email via sendmail/mailsend_email_alert() { local subject="$1" body="$2" echo "$body" | mail -s "$subject" [email protected]}
# Or with explicit headerssend_email() { local to="$1" subject="$2" body="$3" { echo "To: $to" echo "Subject: $subject" echo "Content-Type: text/plain" echo "" echo "$body" } | sendmail -t}
# Slack webhookslack_alert() { local message="$1" local webhook_url="${SLACK_WEBHOOK_URL:?}" curl -s -X POST "$webhook_url" \ -H 'Content-Type: application/json' \ -d "{\"text\": \"$(echo "$message" | sed 's/"/\\"/g')\"}"}
# PagerDuty Events API v2pagerduty_alert() { local summary="$1" severity="${2:-critical}" curl -s -X POST https://events.pagerduty.com/v2/enqueue \ -H 'Content-Type: application/json' \ -d "{ \"routing_key\": \"${PD_ROUTING_KEY:?}\", \"event_action\": \"trigger\", \"payload\": { \"summary\": \"$summary\", \"severity\": \"$severity\", \"source\": \"$(hostname)\" } }"}
# Integrate with trap for automatic alerting on failuretrap 'slack_alert "ALERT: $(hostname) $0 failed on line $LINENO"' ERRWhy this approach: Using curl with a Slack/PagerDuty webhook requires no additional packages and works identically across environments, making it the most portable alerting mechanism.
Answer: Implement rotation by renaming, compressing, and pruning old files — or use logrotate configuration for production.
xxxxxxxxxxset -euo pipefail
LOG_DIR="/var/log/myapp"LOG_FILE="$LOG_DIR/app.log"MAX_ROTATIONS=7MAX_SIZE=$(( 50 * 1024 * 1024 )) # 50MB
rotate_log() { local log="$1" [[ -f "$log" ]] || return 0
local size size=$(stat -c%s "$log") (( size < MAX_SIZE )) && return 0
echo "Rotating $log (${size} bytes)"
# Shift existing rotations for (( i=MAX_ROTATIONS-1; i>=1; i-- )); do local prev="${log}.${i}.gz" local next="${log}.$((i+1)).gz" [[ -f "$prev" ]] && mv "$prev" "$next" done
# Rotate current cp "$log" "${log}.1" gzip -f "${log}.1"
# Truncate original (safe: process still writes to open FD) > "$log"
# Or use mv + touch for processes that re-open on SIGHUP # mv "$log" "${log}.1" # touch "$log" # kill -HUP "$(pgrep myapp)" # gzip "${log}.1"
# Remove oldest rm -f "${log}.$((MAX_ROTATIONS+1)).gz"
echo "Rotation complete"}
rotate_log "$LOG_FILE"
# logrotate config equivalent (production)cat > /etc/logrotate.d/myapp <<'EOF'/var/log/myapp/*.log { daily rotate 7 compress delaycompress missingok notifempty postrotate systemctl kill -s HUP myapp.service endscript}EOFWhy this approach: Truncating with > "$log" (rather than rm + recreate) is safe for processes with open file descriptors — they continue writing to the same inode without error.
Answer: When inotifywait is unavailable, poll with find and a timestamp marker file, or compare directory snapshots.
xxxxxxxxxxset -euo pipefail
WATCH_DIR="/var/spool/jobs"PROCESSED_DIR="/var/spool/jobs/done"MARKER="/tmp/.last_check_$$"POLL_INTERVAL=10
mkdir -p "$PROCESSED_DIR"touch "$MARKER"
log() { echo "[$(date '+%H:%M:%S')] $*"; }
process_file() { local f="$1" log "Processing: $(basename "$f")" # ... actual work ... mv "$f" "$PROCESSED_DIR/"}
log "Watching $WATCH_DIR (polling every ${POLL_INTERVAL}s)"
while true; do # Find files newer than marker, not in done/ while IFS= read -r -d '' file; do process_file "$file" done < <(find "$WATCH_DIR" -maxdepth 1 -type f -newer "$MARKER" -print0)
# Update marker timestamp touch "$MARKER" sleep "$POLL_INTERVAL"doneWhy this approach: Using a marker file with -newer is simpler and more reliable than snapshot-diffing — touch is atomic and the -newer test leverages the kernel's mtime tracking.
Answer: A health check tests multiple dimensions: process running, port open, HTTP response, and response time.
xxxxxxxxxx
SERVICE="nginx"HOST="localhost"PORT=80URL="http://localhost/health"MAX_RESPONSE_MS=500EXIT_CODE=0
log() { echo "[$(date '+%H:%M:%S')] $*"; }fail() { log "FAIL: $*" >&2; EXIT_CODE=1; }
# 1. Process checkif pgrep -x "$SERVICE" > /dev/null; then log "OK: $SERVICE process running (PID $(pgrep -x "$SERVICE" | head -1))"else fail "$SERVICE process not found"fi
# 2. Port checkif nc -z -w3 "$HOST" "$PORT" 2>/dev/null; then log "OK: port $PORT open"else fail "port $PORT not accepting connections"fi
# 3. HTTP health endpointhttp_check() { local start end elapsed http_code start=$(date +%s%N) http_code=$(curl -s -o /dev/null -w "%{http_code}" \ --max-time 5 "$URL") end=$(date +%s%N) elapsed=$(( (end - start) / 1000000 ))
if [[ "$http_code" == "200" ]]; then log "OK: HTTP 200 in ${elapsed}ms" if (( elapsed > MAX_RESPONSE_MS )); then log "WARN: slow response (${elapsed}ms > ${MAX_RESPONSE_MS}ms)" fi else fail "HTTP $http_code from $URL" fi}http_check
# 4. Disk space checkdisk_usage=$(df / | awk 'NR==2 {print $5}' | tr -d '%')if (( disk_usage > 90 )); then fail "disk usage at ${disk_usage}%"else log "OK: disk at ${disk_usage}%"fi
exit "$EXIT_CODE"Why this approach: Accumulating failures with EXIT_CODE=1 rather than exiting on the first failure gives a complete health picture in one script run.
Answer: A deploy script captures current state before changes, applies changes atomically, verifies success, and rolls back automatically on failure.
xxxxxxxxxxset -euo pipefail
APP_DIR="/opt/myapp"DEPLOY_DIR="/opt/deploys"SERVICE="myapp"ARTIFACT="${1:?Usage: $0 <artifact.tar.gz>}"TIMESTAMP=$(date +%Y%m%d-%H%M%S)NEW_RELEASE="$DEPLOY_DIR/$TIMESTAMP"CURRENT_LINK="$APP_DIR/current"
log() { echo "[$(date '+%H:%M:%S')] $*"; }error(){ echo "[$(date '+%H:%M:%S')] ERROR: $*" >&2; }
rollback() { local prev prev=$(readlink "$CURRENT_LINK" 2>/dev/null || true) if [[ -n "$prev" && "$prev" != "$NEW_RELEASE" ]]; then log "Rolling back to $prev" ln -sfn "$prev" "$CURRENT_LINK" systemctl restart "$SERVICE" error "Deployment failed — rolled back to $(basename "$prev")" else error "No previous release to roll back to" fi exit 1}trap rollback ERR
# Unpack new releasemkdir -p "$NEW_RELEASE"tar -xzf "$ARTIFACT" -C "$NEW_RELEASE"log "Unpacked to $NEW_RELEASE"
# Run pre-deploy checks"$NEW_RELEASE/bin/check-config" || { error "Config check failed"; exit 1; }
# Atomic switchln -sfn "$NEW_RELEASE" "$CURRENT_LINK"log "Switched current -> $TIMESTAMP"
# Restart servicesystemctl restart "$SERVICE"sleep 3
# Verifyif ! curl -sf http://localhost/health > /dev/null; then error "Health check failed" exit 1 # triggers rollback trapfi
log "Deployment successful: $TIMESTAMP"
# Prune old releases (keep last 5)ls -dt "$DEPLOY_DIR"/[0-9]* | tail -n +6 | xargs rm -rfWhy this approach: ln -sfn for the current symlink is atomic — the old release stays live until the instant the symlink swaps, giving zero-downtime cutover.
os and os.path for filesystem operations?Answer: os provides POSIX-level syscall wrappers; os.path handles path string manipulation portably.
xxxxxxxxxximport osimport os.path
# Current directorycwd = os.getcwd()os.chdir('/tmp')
# Path operationsp = '/var/log/app/error.log'os.path.dirname(p) # '/var/log/app'os.path.basename(p) # 'error.log'os.path.split(p) # ('/var/log/app', 'error.log')os.path.splitext(p) # ('/var/log/app/error', '.log')os.path.join('/var', 'log', 'app.log') # '/var/log/app.log'os.path.abspath('rel/path') # absoluteos.path.expanduser('~/data') # expand ~
# Existence and type checksos.path.exists(p)os.path.isfile(p)os.path.isdir('/var/log')os.path.islink('/etc/localtime')os.path.getsize(p) # bytesos.path.getmtime(p) # float: seconds since epoch
# Directory operationsos.makedirs('/tmp/a/b/c', exist_ok=True)os.listdir('/etc')os.scandir('/etc') # faster, returns DirEntry objects
# Walk directory treefor dirpath, dirnames, filenames in os.walk('/var/log'): for fname in filenames: full = os.path.join(dirpath, fname) print(full)
# Environmenthome = os.environ.get('HOME', '/tmp')os.environ['MY_VAR'] = 'value'Why this approach: os.path.join should always be used instead of string concatenation — it handles trailing slashes and platform separators correctly.
shutil simplify high-level file operations?Answer: shutil provides copy, move, delete, and archive operations that os lacks or makes cumbersome.
xxxxxxxxxximport shutilimport os
# Copy file (content only)shutil.copy('src.txt', 'dst.txt')
# Copy with metadata (permissions, timestamps)shutil.copy2('src.txt', 'dst.txt')
# Copy entire directory treeshutil.copytree('/src/dir', '/dst/dir')# With ignore patternsshutil.copytree('/src', '/dst', ignore=shutil.ignore_patterns('*.pyc', '__pycache__', '.git'))
# Move (rename or move across filesystems)shutil.move('old_name.txt', 'new_name.txt')shutil.move('/tmp/file.txt', '/var/data/')
# Delete entire directory treeshutil.rmtree('/tmp/old_workdir')# Safe version: ignore if missingshutil.rmtree('/tmp/dir', ignore_errors=True)
# Disk usagetotal, used, free = shutil.disk_usage('/')print(f"Free: {free // (1024**3)} GB")
# Which binary is in PATHpath = shutil.which('python3') # like `which` command
# Create archivesshutil.make_archive('/tmp/backup', 'gztar', '/var/data')# Formats: 'zip', 'tar', 'gztar', 'bztar', 'xztar'
# Extract archiveshutil.unpack_archive('/tmp/backup.tar.gz', '/restore/dir')
# Copy file object to file object (streaming)with open('src', 'rb') as src, open('dst', 'wb') as dst: shutil.copyfileobj(src, dst, length=1024*1024)Why this approach: shutil.copytree with ignore_patterns is the cleanest way to copy a project directory while excluding build artifacts — far simpler than equivalent os.walk logic.
subprocess.run work and when do you use check=True and capture_output?Answer: subprocess.run is the modern high-level API for running external commands. check=True raises CalledProcessError on non-zero exit; capture_output=True captures stdout and stderr.
xxxxxxxxxximport subprocess
# Basic run (output goes to terminal)result = subprocess.run(['ls', '-la', '/tmp'])
# Capture outputresult = subprocess.run( ['git', 'log', '--oneline', '-5'], capture_output=True, # same as stdout=PIPE, stderr=PIPE text=True # decode bytes to str)print(result.stdout)print(result.stderr)print(result.returncode)
# Raise exception on failuretry: result = subprocess.run( ['systemctl', 'is-active', 'nginx'], check=True, capture_output=True, text=True ) print("nginx is active")except subprocess.CalledProcessError as e: print(f"nginx not active: {e.returncode}") print(e.stderr)
# Pass stdinresult = subprocess.run( ['grep', 'error'], input='ok\nerror found\nok again\n', capture_output=True, text=True)
# Shell pipeline (use shell=True carefully — no user input)result = subprocess.run( 'ps aux | grep python | wc -l', shell=True, capture_output=True, text=True, check=True)
# Environment variablesimport osenv = {**os.environ, 'MY_VAR': 'value'}subprocess.run(['./script.sh'], env=env, check=True)
# Timeouttry: subprocess.run(['sleep', '30'], timeout=5)except subprocess.TimeoutExpired: print("Command timed out")Why this approach: Always prefer the list form ['cmd', 'arg'] over shell=True — the list form avoids shell injection vulnerabilities and handles arguments with spaces correctly.
subprocess.Popen instead of subprocess.run?Answer: Popen is the low-level API for streaming I/O, bidirectional communication, or when you need the process handle before it finishes.
xxxxxxxxxximport subprocessimport sys
# Stream output in real time (run() buffers everything)with subprocess.Popen( ['tail', '-f', '/var/log/syslog'], stdout=subprocess.PIPE, text=True) as proc: for line in proc.stdout: print(line, end='') if 'critical' in line.lower(): proc.terminate() break
# Bidirectional: write to stdin, read from stdoutwith subprocess.Popen( ['python3', '-c', 'import sys; print(sys.stdin.read().upper())'], stdin=subprocess.PIPE, stdout=subprocess.PIPE, text=True) as proc: stdout, stderr = proc.communicate(input='hello world\n') print(stdout) # HELLO WORLD
# Build a pipeline: ls | grep .pyp1 = subprocess.Popen(['ls', '/usr/lib'], stdout=subprocess.PIPE)p2 = subprocess.Popen(['grep', r'\.py$'], stdin=p1.stdout, stdout=subprocess.PIPE, text=True)p1.stdout.close() # let p1 receive SIGPIPE if p2 exitsoutput, _ = p2.communicate()p1.wait()
# Long-running background processproc = subprocess.Popen(['./worker.sh'])# ... do other work ...rc = proc.wait() # or proc.poll() to check without blocking
# Kill if taking too longimport timeproc = subprocess.Popen(['./slow_task.sh'])time.sleep(10)if proc.poll() is None: proc.terminate() proc.wait()Why this approach: proc.stdout.close() after setting p2.stdin=p1.stdout is essential — without it, p1 won't receive SIGPIPE when p2 exits, and the pipeline hangs.
pathlib improve on os.path?Answer: pathlib.Path is an object-oriented path API that's more readable, composable, and Pythonic than os.path string functions.
xxxxxxxxxxfrom pathlib import Pathimport shutil
# Constructionp = Path('/var/log/app')home = Path.home()cwd = Path.cwd()
# Joining (/ operator)log_file = p / 'error.log' # Path('/var/log/app/error.log')config = home / '.config' / 'app.toml'
# Propertieslog_file.name # 'error.log'log_file.stem # 'error'log_file.suffix # '.log'log_file.parent # Path('/var/log/app')log_file.parts # ('/', 'var', 'log', 'app', 'error.log')
# Checkslog_file.exists()log_file.is_file()p.is_dir()log_file.stat().st_sizelog_file.stat().st_mtime
# Globfor f in p.glob('*.log'): print(f)for f in Path('/src').rglob('*.py'): print(f)
# Read/writecontent = log_file.read_text(encoding='utf-8')log_file.write_text('new content\n', encoding='utf-8')data = log_file.read_bytes()
# Create/deletep.mkdir(parents=True, exist_ok=True)log_file.unlink(missing_ok=True) # Python 3.8+shutil.rmtree(p)
# Rename / replace (atomic on same filesystem)log_file.replace(p / 'error.log.old')
# Resolve symlinksreal = log_file.resolve()Why this approach: The / operator for path joining makes path construction read like a path rather than a nested function call — base / 'subdir' / 'file.txt' is instantly clear.
argparse?Answer: argparse provides type checking, help generation, subcommands, and mutual exclusion groups.
xxxxxxxxxx#!/usr/bin/env python3import argparseimport sys
def parse_args(): parser = argparse.ArgumentParser( description='Log analyzer tool', formatter_class=argparse.RawDescriptionHelpFormatter, epilog='Examples:\n %(prog)s -f app.log -l ERROR -n 20' )
# Positional parser.add_argument('logfile', nargs='?', default='/var/log/app.log', help='Log file path (default: %(default)s)')
# Optional with type and default parser.add_argument('-n', '--lines', type=int, default=10, metavar='N', help='Number of lines (default: %(default)s)')
# Flag parser.add_argument('-v', '--verbose', action='store_true', help='Enable verbose output')
# Choices parser.add_argument('-l', '--level', choices=['DEBUG','INFO','WARN','ERROR','FATAL'], default='ERROR')
# Multiple values parser.add_argument('--exclude', nargs='+', metavar='PATTERN', help='Patterns to exclude')
# Mutually exclusive group = parser.add_mutually_exclusive_group() group.add_argument('--count', action='store_true') group.add_argument('--summary', action='store_true')
# Required option parser.add_argument('--output', required=False, type=argparse.FileType('w'), default=sys.stdout)
args = parser.parse_args()
# Additional validation if args.lines < 1: parser.error(f'--lines must be positive, got {args.lines}')
return args
def main(): args = parse_args() print(f"Analyzing {args.logfile}, level={args.level}, n={args.lines}")
if __name__ == '__main__': main()Why this approach: parser.error() prints the usage and error message then exits with code 2, matching the POSIX convention for argument errors — don't use sys.exit() directly for argument validation.
Answer: Always use with statements for automatic close; use explicit encoding; handle exceptions; use pathlib or tempfile for atomic writes.
xxxxxxxxxximport tempfileimport osfrom pathlib import Path
# Read entire filewith open('/etc/hosts', 'r', encoding='utf-8') as f: content = f.read()
# Read line by line (memory efficient for large files)with open('/var/log/app.log', 'r', encoding='utf-8', errors='replace') as f: for line in f: line = line.rstrip('\n') if 'ERROR' in line: print(line)
# Read lines into listwith open('file.txt') as f: lines = f.readlines() # with newlines lines = [l.rstrip() for l in f] # stripped
# Writewith open('output.txt', 'w', encoding='utf-8') as f: f.write('line one\n') f.writelines(['a\n', 'b\n', 'c\n'])
# Appendwith open('output.txt', 'a') as f: f.write('appended line\n')
# Binarywith open('image.png', 'rb') as f: data = f.read()with open('copy.png', 'wb') as f: f.write(data)
# Atomic write (write-then-rename)def atomic_write(path, content, encoding='utf-8'): p = Path(path) tmp = tempfile.NamedTemporaryFile( mode='w', encoding=encoding, dir=p.parent, delete=False, suffix='.tmp' ) try: tmp.write(content) tmp.flush() os.fsync(tmp.fileno()) # ensure durability tmp.close() os.replace(tmp.name, path) # atomic rename except Exception: tmp.close() os.unlink(tmp.name) raise
atomic_write('/etc/myapp/config.json', '{"key": "value"}\n')Why this approach: os.replace() (Python 3.3+) is atomic on POSIX — it wraps rename(2) — so readers never observe a partial write, even if the process crashes mid-way.
Answer: Catch specific exceptions, log the full traceback for debugging, use finally for cleanup, and distinguish expected from unexpected errors.
xxxxxxxxxximport subprocessimport loggingimport sysimport tracebackfrom pathlib import Path
logging.basicConfig(level=logging.INFO, format='%(asctime)s %(levelname)s %(message)s')log = logging.getLogger(__name__)
def read_config(path): try: return Path(path).read_text(encoding='utf-8') except FileNotFoundError: log.error("Config not found: %s", path) return None except PermissionError as e: log.error("Permission denied reading %s: %s", path, e) raise SystemExit(1) from e
def run_command(cmd): try: result = subprocess.run(cmd, check=True, capture_output=True, text=True, timeout=30) return result.stdout except subprocess.CalledProcessError as e: log.error("Command %s failed (rc=%d): %s", cmd, e.returncode, e.stderr) raise except subprocess.TimeoutExpired: log.error("Command %s timed out", cmd) raise except FileNotFoundError: log.error("Command not found: %s", cmd[0]) raise
def main(): try: cfg = read_config('/etc/myapp/config.json') output = run_command(['systemctl', 'status', 'nginx']) log.info("Done: %s", output[:80]) except KeyboardInterrupt: log.info("Interrupted by user") sys.exit(130) except SystemExit: raise # don't catch sys.exit() except Exception: log.critical("Unexpected error:\n%s", traceback.format_exc()) sys.exit(1)
if __name__ == '__main__': main()Why this approach: Catching Exception at the top level while letting SystemExit and KeyboardInterrupt propagate (they inherit from BaseException, not Exception) is the correct Python exception hierarchy usage.
logging module for production scripts?Answer: Use logging.basicConfig for simple scripts; dictConfig or fileConfig for production; add handlers for both file and stderr.
xxxxxxxxxximport loggingimport logging.handlersimport sysfrom pathlib import Path
def setup_logging(log_file=None, level=logging.INFO, verbose=False): root = logging.getLogger() root.setLevel(logging.DEBUG) # capture everything; handlers filter
fmt = logging.Formatter( fmt='%(asctime)s %(levelname)-8s %(name)s: %(message)s', datefmt='%Y-%m-%d %H:%M:%S' )
# Console handler console = logging.StreamHandler(sys.stderr) console.setLevel(logging.DEBUG if verbose else logging.WARNING) console.setFormatter(fmt) root.addHandler(console)
# File handler with rotation if log_file: Path(log_file).parent.mkdir(parents=True, exist_ok=True) fh = logging.handlers.RotatingFileHandler( log_file, maxBytes=10 * 1024 * 1024, # 10 MB backupCount=5, encoding='utf-8' ) fh.setLevel(logging.DEBUG) fh.setFormatter(fmt) root.addHandler(fh)
# Usagesetup_logging('/var/log/myapp/script.log', verbose=True)log = logging.getLogger(__name__)
log.debug("debug detail")log.info("Processing started")log.warning("Disk at 85%%")log.error("Failed to connect: %s", "timeout")log.critical("Service down")
# Structured logging with extra fieldslog.info("Request processed", extra={ 'user': 'alice', 'duration_ms': 142, 'status': 200})
# Suppress noisy librarieslogging.getLogger('urllib3').setLevel(logging.WARNING)logging.getLogger('boto3').setLevel(logging.WARNING)Why this approach: Setting the root logger to DEBUG and filtering at the handler level gives you full flexibility — you can enable verbose file logging while keeping the console clean without restarting.
Answer: Use os.environ for reading/writing; os.environ.get() for defaults; os.environ.setdefault() and python-dotenv for config loading.
xxxxxxxxxximport osimport sys
# Read (returns None if missing)db_host = os.environ.get('DB_HOST')
# Read with defaultdb_port = int(os.environ.get('DB_PORT', '5432'))
# Read required variable (fail loudly)def require_env(name): val = os.environ.get(name) if val is None: print(f"ERROR: Required environment variable {name} not set", file=sys.stderr) sys.exit(1) return val
api_key = require_env('API_KEY')
# Set (affects current process and children)os.environ['LOG_LEVEL'] = 'DEBUG'
# Deleteos.environ.pop('TEMP_VAR', None) # safe: no KeyError
# Check presenceif 'CI' in os.environ: print("Running in CI")
# All env as dictenv_copy = dict(os.environ)
# Pass modified env to subprocessimport subprocesschild_env = {**os.environ, 'OVERRIDE': 'value'}subprocess.run(['./script.sh'], env=child_env)
# Load from .env file (python-dotenv)try: from dotenv import load_dotenv load_dotenv('/etc/myapp/.env', override=False)except ImportError: pass # optional dependency
# Config class patternclass Config: DB_HOST = os.environ.get('DB_HOST', 'localhost') DB_PORT = int(os.environ.get('DB_PORT', 5432)) DEBUG = os.environ.get('DEBUG', '').lower() in ('1','true','yes') SECRET_KEY = require_env('SECRET_KEY')Why this approach: os.environ.get('KEY', default) is preferred over os.environ['KEY'] in scripts because a missing variable raises a KeyError with no context — the get + explicit error message pattern gives actionable output.
Answer: Use json.load/json.dump for files and json.loads/json.dumps for strings. Handle JSONDecodeError for invalid input.
xxxxxxxxxximport jsonimport sysfrom pathlib import Path
# Read JSON filewith open('config.json', 'r', encoding='utf-8') as f: config = json.load(f)
# Write JSON file (pretty-printed)with open('output.json', 'w', encoding='utf-8') as f: json.dump(data, f, indent=2, sort_keys=True) f.write('\n') # trailing newline for POSIX compliance
# Parse JSON stringtext = '{"host": "localhost", "port": 8080}'obj = json.loads(text)print(obj['host'])
# Serialize to strings = json.dumps(obj, indent=2)
# Handle parse errorsdef load_json_safe(path): try: with open(path, 'r', encoding='utf-8') as f: return json.load(f) except FileNotFoundError: print(f"File not found: {path}", file=sys.stderr) return None except json.JSONDecodeError as e: print(f"Invalid JSON in {path}: {e}", file=sys.stderr) return None
# Atomic write of JSONdef write_json_atomic(path, data): import tempfile, os content = json.dumps(data, indent=2) + '\n' p = Path(path) with tempfile.NamedTemporaryFile('w', dir=p.parent, delete=False, suffix='.tmp') as tmp: tmp.write(content) tmp_name = tmp.name os.replace(tmp_name, path)
# Custom types (e.g., datetime)from datetime import datetimeclass DateTimeEncoder(json.JSONEncoder): def default(self, o): if isinstance(o, datetime): return o.isoformat() return super().default(o)
json.dumps({'ts': datetime.now()}, cls=DateTimeEncoder)
# Parse dates on loaddef date_hook(d): for k, v in d.items(): if isinstance(v, str): try: d[k] = datetime.fromisoformat(v) except ValueError: pass return djson.loads(text, object_hook=date_hook)Why this approach: Adding f.write('\n') after json.dump is a small but important detail for POSIX compliance — many tools (wc -l, diff, git) behave correctly only on newline-terminated files.
Answer: Use the csv module for proper quoting and encoding; use DictReader/DictWriter for header-based access.
xxxxxxxxxximport csvimport sysfrom pathlib import Path
# Read CSV with DictReaderwith open('data.csv', newline='', encoding='utf-8') as f: reader = csv.DictReader(f) for row in reader: print(row['name'], row['age'])
# Read with positional fieldswith open('data.csv', newline='') as f: reader = csv.reader(f) header = next(reader) for row in reader: print(row[0], row[2])
# Write with DictWriterfieldnames = ['name', 'age', 'email']with open('output.csv', 'w', newline='', encoding='utf-8') as f: writer = csv.DictWriter(f, fieldnames=fieldnames) writer.writeheader() writer.writerows([ {'name': 'Alice', 'age': 30, 'email': '[email protected]'}, {'name': 'Bob', 'age': 25, 'email': '[email protected]'}, ])
# Custom delimiter (TSV)with open('data.tsv', newline='') as f: reader = csv.DictReader(f, delimiter='\t') for row in reader: print(row)
# Handle CSV with quotingwith open('complex.csv', 'w', newline='') as f: writer = csv.writer(f, quoting=csv.QUOTE_NONNUMERIC) writer.writerow(['name', 'note', 'value']) writer.writerow(['Alice', 'has, a comma', 42.0])
# Read from stdin (piped input)reader = csv.DictReader(sys.stdin)for row in reader: process(row)
# Transform: filter and rewritewith open('input.csv', newline='') as fin, \ open('output.csv', 'w', newline='') as fout: reader = csv.DictReader(fin) writer = csv.DictWriter(fout, fieldnames=reader.fieldnames) writer.writeheader() for row in reader: if int(row['age']) >= 18: writer.writerow(row)Why this approach: Always open CSV files with newline='' as recommended by the Python docs — the csv module handles its own universal newline translation, and omitting newline='' causes double-CR issues on Windows.
os.walk and pathlib.rglob for directory traversal?Answer: os.walk generates (dirpath, dirnames, filenames) tuples top-down; pathlib.rglob is cleaner for pattern matching.
xxxxxxxxxximport osfrom pathlib import Path
# os.walk — full controldef find_large_files(root, min_bytes=1024*1024): for dirpath, dirnames, filenames in os.walk(root): # Skip hidden directories dirnames[:] = [d for d in dirnames if not d.startswith('.')] for fname in filenames: fpath = os.path.join(dirpath, fname) try: size = os.path.getsize(fpath) if size > min_bytes: yield fpath, size except OSError: continue
for path, size in find_large_files('/var'): print(f"{size//1024:8d} KB {path}")
# pathlib.rglob — cleaner for patternsroot = Path('/src')
# All Python filesfor py_file in root.rglob('*.py'): print(py_file)
# All files modified in the last dayimport timecutoff = time.time() - 86400for f in root.rglob('*'): if f.is_file() and f.stat().st_mtime > cutoff: print(f)
# Count files by extensionfrom collections import Counterext_counts = Counter( f.suffix.lower() for f in root.rglob('*') if f.is_file())for ext, count in ext_counts.most_common(10): print(f"{ext or '(none)':10s} {count}")
# Total size by directoryfor dirpath, _, files in os.walk('/var/log'): total = sum( os.path.getsize(os.path.join(dirpath, f)) for f in files if os.path.isfile(os.path.join(dirpath, f)) ) if total > 10*1024*1024: print(f"{total//1024//1024} MB {dirpath}")Why this approach: Modifying dirnames in-place (dirnames[:] = [...]) is the correct way to prune os.walk — it prevents recursion into excluded directories without spawning a new walk.
Answer: Use concurrent.futures.ThreadPoolExecutor for I/O-bound subprocess tasks or a list of Popen objects for fine-grained control.
xxxxxxxxxximport subprocessfrom concurrent.futures import ThreadPoolExecutor, as_completedfrom pathlib import Path
# ThreadPoolExecutor with subprocess.run (simplest)def check_host(host): result = subprocess.run( ['ping', '-c', '1', '-W', '1', host], capture_output=True, text=True ) return host, result.returncode == 0
hosts = ['8.8.8.8', '1.1.1.1', '192.168.1.1', 'google.com']
with ThreadPoolExecutor(max_workers=10) as ex: futures = {ex.submit(check_host, h): h for h in hosts} for future in as_completed(futures): host, reachable = future.result() print(f"{host}: {'UP' if reachable else 'DOWN'}")
# Manual Popen list (lower level, full control)def compress_files(files, workers=4): procs = [] for f in files: p = subprocess.Popen(['gzip', str(f)]) procs.append((f, p)) if len(procs) >= workers: f0, p0 = procs.pop(0) rc = p0.wait() if rc != 0: print(f"Failed: {f0}")
for f, p in procs: rc = p.wait() if rc != 0: print(f"Failed: {f}")
compress_files(Path('/var/log').glob('*.log'))
# ProcessPoolExecutor for CPU-bound Python workfrom concurrent.futures import ProcessPoolExecutorimport hashlib
def file_hash(path): h = hashlib.sha256() with open(path, 'rb') as f: for chunk in iter(lambda: f.read(65536), b''): h.update(chunk) return path, h.hexdigest()
with ProcessPoolExecutor() as ex: for path, digest in ex.map(file_hash, Path('/usr/bin').iterdir()): print(f"{digest} {path}")Why this approach: ThreadPoolExecutor with subprocess.run is the cleanest parallel subprocess pattern — threads release the GIL while waiting for subprocesses, so there's no CPU contention despite using threads.
tempfile module safely?Answer: tempfile creates uniquely named, securely opened temporary files and directories, avoiding race conditions in /tmp.
xxxxxxxxxximport tempfileimport osfrom pathlib import Path
# NamedTemporaryFile — deleted on close by defaultwith tempfile.NamedTemporaryFile( mode='w', suffix='.json', prefix='myapp-', dir='/tmp', delete=True, encoding='utf-8') as tmp: tmp.write('{"key": "value"}') tmp.flush() print(f"Temp file: {tmp.name}") # process the file while it's open os.system(f'cat {tmp.name}')# File deleted here
# Keep after close (for atomic write pattern)tmp = tempfile.NamedTemporaryFile( mode='w', delete=False, dir=Path('/etc/myapp'), suffix='.tmp')try: tmp.write(generate_config()) tmp.flush() os.fsync(tmp.fileno()) tmp.close() os.replace(tmp.name, '/etc/myapp/config.json')except Exception: tmp.close() os.unlink(tmp.name) raise
# Temporary directorywith tempfile.TemporaryDirectory(prefix='build-') as tmpdir: print(f"Build dir: {tmpdir}") subprocess.run(['make', '-C', tmpdir], check=True)# Directory and all contents deleted automatically
# SpooledTemporaryFile — stays in memory until size limitwith tempfile.SpooledTemporaryFile(max_size=1024*1024, mode='w') as f: f.write(data) f.seek(0) content = f.read()
# Low-level: just get a unique pathfd, path = tempfile.mkstemp(suffix='.db', dir='/var/tmp')os.close(fd) # close the FD; we just needed the pathtry: use_file(path)finally: os.unlink(path)Why this approach: tempfile.NamedTemporaryFile(delete=False) + explicit os.replace + cleanup in except is the safest atomic write pattern — tempfile guarantees no name collision and secure file creation (mode 0600).
Answer: Use configparser for INI files, json for JSON, tomllib (3.11+) for TOML, and a validation layer with clear error messages.
xxxxxxxxxximport configparserimport jsonimport sysfrom pathlib import Path
# INI / .cfg files with configparserdef load_ini_config(path): config = configparser.ConfigParser() config.read_dict({'DEFAULT': { 'port': '8080', 'log_level': 'INFO', 'debug': 'false' }}) read = config.read(path) if not read: print(f"Config not found: {path}", file=sys.stderr) sys.exit(1) return config
cfg = load_ini_config('/etc/myapp/config.ini')host = cfg.get('database', 'host')port = cfg.getint('server', 'port')debug = cfg.getboolean('server', 'debug')
# JSON config with validationdef load_json_config(path): try: with open(path) as f: data = json.load(f) except (FileNotFoundError, json.JSONDecodeError) as e: print(f"Config error: {e}", file=sys.stderr) sys.exit(1)
# Validate required keys required = {'host', 'port', 'database'} missing = required - set(data.keys()) if missing: print(f"Missing config keys: {missing}", file=sys.stderr) sys.exit(1)
# Type validation if not isinstance(data['port'], int): print("Config error: 'port' must be an integer", file=sys.stderr) sys.exit(1) if not 1 <= data['port'] <= 65535: print("Config error: 'port' out of range", file=sys.stderr) sys.exit(1)
return data
# TOML (Python 3.11+)import tomllibwith open('config.toml', 'rb') as f: toml_cfg = tomllib.load(f)
# Pydantic for declarative validation (if available)try: from pydantic import BaseModel, validator class AppConfig(BaseModel): host: str = 'localhost' port: int = 8080 debug: bool = False cfg = AppConfig(**json.load(open('config.json')))except ImportError: passWhy this approach: Validating config at startup with clear error messages (rather than letting KeyError crash mid-execution) is a sign of production-quality tooling — operators need to know what to fix before the service starts.
Answer: Use urllib.request for no-dependency scripts; requests or httpx for production code.
xxxxxxxxxximport urllib.requestimport urllib.errorimport jsonimport os
# urllib (stdlib, no deps)def api_get(url, token=None): req = urllib.request.Request(url) if token: req.add_header('Authorization', f'Bearer {token}') req.add_header('Accept', 'application/json') try: with urllib.request.urlopen(req, timeout=10) as resp: return json.loads(resp.read().decode()) except urllib.error.HTTPError as e: print(f"HTTP {e.code}: {e.reason}", file=sys.stderr) raise except urllib.error.URLError as e: print(f"Network error: {e.reason}", file=sys.stderr) raise
# POST with JSON bodydef api_post(url, payload, token=None): data = json.dumps(payload).encode() req = urllib.request.Request(url, data=data, method='POST') req.add_header('Content-Type', 'application/json') if token: req.add_header('Authorization', f'Bearer {token}') with urllib.request.urlopen(req, timeout=10) as resp: return json.loads(resp.read().decode())
# requests library (more ergonomic)try: import requests session = requests.Session() session.headers.update({ 'Authorization': f'Bearer {os.environ["API_TOKEN"]}', 'Content-Type': 'application/json' }) session.timeout = (5, 30) # (connect, read) timeout
resp = session.get('https://api.example.com/v1/status') resp.raise_for_status() # raises HTTPError for 4xx/5xx data = resp.json()
# Retry with backoff from requests.adapters import HTTPAdapter from urllib3.util.retry import Retry retry = Retry(total=3, backoff_factor=1, status_forcelist=[429, 500, 502, 503]) session.mount('https://', HTTPAdapter(max_retries=retry))except ImportError: pass # fall back to urllibWhy this approach: Mounting a Retry adapter on a requests.Session is the correct way to add retry logic — it handles connection errors and server errors transparently without wrapping every call in a try/except loop.
hashlib and hmac for file integrity and authentication?Answer: hashlib computes digests for file integrity; hmac adds a secret key for authentication (MAC).
xxxxxxxxxximport hashlibimport hmacimport os
# File checksumdef file_sha256(path): h = hashlib.sha256() with open(path, 'rb') as f: for chunk in iter(lambda: f.read(65536), b''): h.update(chunk) return h.hexdigest()
digest = file_sha256('/etc/hosts')print(f"sha256: {digest}")
# Verify download integritydef verify_file(path, expected_sha256): actual = file_sha256(path) if actual != expected_sha256: raise ValueError( f"Checksum mismatch: expected {expected_sha256}, got {actual}" ) print(f"Verified: {path}")
# Multiple algorithmsfor algo in ['md5', 'sha1', 'sha256', 'sha512']: h = hashlib.new(algo) with open('/etc/hosts', 'rb') as f: h.update(f.read()) print(f"{algo}: {h.hexdigest()}")
# HMAC for API request signingdef sign_payload(secret_key: bytes, payload: bytes) -> str: mac = hmac.new(secret_key, payload, hashlib.sha256) return mac.hexdigest()
def verify_hmac(secret_key: bytes, payload: bytes, signature: str) -> bool: expected = sign_payload(secret_key, payload) return hmac.compare_digest(expected, signature) # timing-safe
secret = os.environ.get('SIGNING_KEY', 'default').encode()body = b'{"event": "deploy", "version": "1.2.3"}'sig = sign_payload(secret, body)
# Verify webhook signatureif verify_hmac(secret, body, sig): print("Signature valid")Why this approach: hmac.compare_digest performs a constant-time comparison — using == instead is vulnerable to timing attacks that can reveal the secret key one byte at a time.
Answer: Open the file, seek to end, and poll for new content — mirroring tail -f behavior.
xxxxxxxxxx#!/usr/bin/env python3import timeimport sysimport reimport signalfrom pathlib import Path
def tail_f(path, poll_interval=0.1): """Generator that yields new lines from a file as they appear.""" with open(path, 'r', encoding='utf-8', errors='replace') as f: f.seek(0, 2) # seek to end while True: line = f.readline() if line: yield line.rstrip('\n') else: time.sleep(poll_interval)
# Pattern-based alertingLOG_PATH = '/var/log/app/app.log'ERROR_PATTERN = re.compile( r'(?P<ts>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2})' r'\s+(?P<level>ERROR|CRITICAL)' r'\s+(?P<msg>.+)')
alert_counts = {}
def handle_line(line): m = ERROR_PATTERN.search(line) if not m: return level = m.group('level') msg = m.group('msg') alert_counts[level] = alert_counts.get(level, 0) + 1 print(f"[ALERT] {level}: {msg[:120]}")
# Graceful shutdownrunning = Truedef shutdown(sig, frame): global running running = False print(f"\nStopping. Counts: {alert_counts}") sys.exit(0)
signal.signal(signal.SIGINT, shutdown)signal.signal(signal.SIGTERM, shutdown)
print(f"Tailing {LOG_PATH} ...")for line in tail_f(LOG_PATH): handle_line(line)Why this approach: f.seek(0, 2) (seek to end before starting) ensures you only process new lines and don't replay the entire history — essential for long-running log files.
Answer: Combine argparse, logging, config loading, error handling, subprocess, and clean exit into a production-ready template.
xxxxxxxxxx#!/usr/bin/env python3"""deploy-checker.py — Verify deployment health across a list of hosts.Usage: deploy-checker.py -c config.json [-v] [-n] hosts.txt"""import argparseimport jsonimport loggingimport logging.handlersimport osimport subprocessimport sysimport tempfilefrom pathlib import Pathfrom concurrent.futures import ThreadPoolExecutor, as_completed
# ── Logging setup ──────────────────────────────────────────────────────────def setup_logging(log_file, verbose): root = logging.getLogger() root.setLevel(logging.DEBUG) fmt = logging.Formatter('%(asctime)s %(levelname)-8s %(message)s', datefmt='%Y-%m-%d %H:%M:%S') ch = logging.StreamHandler(sys.stderr) ch.setLevel(logging.DEBUG if verbose else logging.INFO) ch.setFormatter(fmt) root.addHandler(ch) if log_file: fh = logging.handlers.RotatingFileHandler( log_file, maxBytes=5*1024*1024, backupCount=3) fh.setLevel(logging.DEBUG) fh.setFormatter(fmt) root.addHandler(fh)
log = logging.getLogger(__name__)
# ── Config ─────────────────────────────────────────────────────────────────def load_config(path): try: with open(path) as f: cfg = json.load(f) except (FileNotFoundError, json.JSONDecodeError) as e: log.critical("Config error: %s", e) sys.exit(1) for key in ('check_url', 'expected_version'): if key not in cfg: log.critical("Missing config key: %s", key) sys.exit(1) return cfg
# ── Core logic ─────────────────────────────────────────────────────────────def check_host(host, cfg, dry_run=False): if dry_run: log.info("[DRY-RUN] would check %s", host) return host, True, "dry-run" try: result = subprocess.run( ['curl', '-sf', '--max-time', '5', f"http://{host}{cfg['check_url']}"], capture_output=True, text=True, timeout=10 ) if result.returncode != 0: return host, False, f"HTTP error (rc={result.returncode})" data = json.loads(result.stdout) if data.get('version') != cfg['expected_version']: return host, False, f"version mismatch: {data.get('version')}" return host, True, data.get('version') except subprocess.TimeoutExpired: return host, False, "timeout" except json.JSONDecodeError: return host, False, "invalid JSON response" except Exception as e: return host, False, str(e)
# ── CLI ────────────────────────────────────────────────────────────────────def parse_args(): p = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) p.add_argument('hosts_file', type=Path, help='File with one host per line') p.add_argument('-c', '--config', required=True, type=Path) p.add_argument('-v', '--verbose', action='store_true') p.add_argument('-n', '--dry-run', action='store_true') p.add_argument('-j', '--jobs', type=int, default=10) p.add_argument('--log-file', type=Path, default=None) return p.parse_args()
# ── Main ───────────────────────────────────────────────────────────────────def main(): args = parse_args() setup_logging(args.log_file, args.verbose) cfg = load_config(args.config)
try: hosts = [h.strip() for h in args.hosts_file.read_text().splitlines() if h.strip() and not h.startswith('#')] except FileNotFoundError: log.critical("Hosts file not found: %s", args.hosts_file) sys.exit(1)
log.info("Checking %d hosts (workers=%d)", len(hosts), args.jobs) failed = []
with ThreadPoolExecutor(max_workers=args.jobs) as ex: futures = {ex.submit(check_host, h, cfg, args.dry_run): h for h in hosts} for future in as_completed(futures): host, ok, detail = future.result() if ok: log.info("OK %-30s %s", host, detail) else: log.error("FAIL %-30s %s", host, detail) failed.append(host)
log.info("Done: %d OK, %d failed", len(hosts)-len(failed), len(failed)) if failed: log.error("Failed hosts: %s", ', '.join(failed)) sys.exit(0 if not failed else 1)
if __name__ == '__main__': try: main() except KeyboardInterrupt: log.info("Interrupted") sys.exit(130)Why this approach: This template demonstrates all the patterns Google interviewers look for: structured logging, validated config, argparse with sensible defaults, parallel execution, clean exit codes, and a __name__ == '__main__' guard that makes the module importable and testable.
Answer: Use subprocess.run() or subprocess.Popen() with stdout=subprocess.PIPE and stderr=subprocess.PIPE. For simple cases, subprocess.run() with capture_output=True is the cleanest approach. For streaming or interactive processes, use Popen with communicate().
xxxxxxxxxximport subprocess
# Simple: capture output with run()result = subprocess.run( ["ls", "-la", "/var/log"], stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, # decode bytes to str automatically timeout=30)print("STDOUT:", result.stdout)print("STDERR:", result.stderr)print("Return code:", result.returncode)
# Equivalent shorthand (Python 3.7+)result = subprocess.run( ["df", "-h"], capture_output=True, text=True)
# Popen + communicate() for more controlproc = subprocess.Popen( ["gzip", "-c"], stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)compressed, err = proc.communicate(input=b"data to compress")print(f"Compressed size: {len(compressed)} bytes")
# Piped commands (shell=True approach — acceptable for simple pipes)result = subprocess.run( "ps aux | grep python | wc -l", shell=True, capture_output=True, text=True)
# Piped commands (safe approach — no shell=True)ps = subprocess.Popen(["ps", "aux"], stdout=subprocess.PIPE)grep = subprocess.Popen(["grep", "python"], stdin=ps.stdout, stdout=subprocess.PIPE)ps.stdout.close() # allow ps to receive SIGPIPE if grep exitswc = subprocess.Popen(["wc", "-l"], stdin=grep.stdout, stdout=subprocess.PIPE)grep.stdout.close()output = wc.communicate()[0]print(output.decode().strip())
# Check for errorsif result.returncode != 0: raise subprocess.CalledProcessError(result.returncode, result.args)Why this approach: communicate() reads all output at once and closes the streams, preventing deadlocks caused by full pipe buffers.
Answer: Paramiko provides a Python SSH client. Use SSHClient.connect() to authenticate, exec_command() to run commands, and read from the returned file-like objects for stdout/stderr. For SFTP file transfers, use open_sftp().
xxxxxxxxxximport paramikoimport os
def run_remote_command(host, username, command, key_path=None, password=None): client = paramiko.SSHClient() # Auto-add unknown host keys (use RejectPolicy in production) client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
connect_kwargs = { "hostname": host, "username": username, "timeout": 10, } if key_path: connect_kwargs["key_filename"] = os.path.expanduser(key_path) if password: connect_kwargs["password"] = password
try: client.connect(**connect_kwargs) stdin, stdout, stderr = client.exec_command(command, timeout=30)
# Wait for command to complete exit_code = stdout.channel.recv_exit_status() out = stdout.read().decode("utf-8").strip() err = stderr.read().decode("utf-8").strip() return exit_code, out, err finally: client.close()
# SFTP upload and downloaddef sftp_transfer(host, username, local_path, remote_path, key_path="~/.ssh/id_rsa"): client = paramiko.SSHClient() client.set_missing_host_key_policy(paramiko.AutoAddPolicy()) client.connect(hostname=host, username=username, key_filename=os.path.expanduser(key_path)) try: sftp = client.open_sftp() # Upload sftp.put(local_path, remote_path) print(f"Uploaded {local_path} -> {host}:{remote_path}") # Download sftp.get(remote_path, local_path + ".downloaded") # List remote directory for entry in sftp.listdir_attr("/home"): print(f"{entry.filename} {entry.st_size}") sftp.close() finally: client.close()
# Example usageexit_code, out, err = run_remote_command( "server.example.com", "admin", "df -h /", key_path="~/.ssh/id_rsa")print(f"Exit: {exit_code}\n{out}")if err: print(f"Errors: {err}")Why this approach: recv_exit_status() blocks until the remote process finishes, ensuring you capture complete output rather than reading prematurely.
requests library to make GET and POST calls with headers, authentication, and timeouts?Answer: requests abstracts HTTP connections cleanly. Pass headers=, auth=, and timeout= as keyword arguments. Use a Session object to reuse connections and share headers/auth across multiple requests.
xxxxxxxxxximport requestsfrom requests.auth import HTTPBasicAuth, HTTPDigestAuth
BASE_URL = "https://api.example.com"
# Simple GET with headers and timeoutresponse = requests.get( f"{BASE_URL}/users", headers={ "Authorization": "Bearer my-token-here", "Accept": "application/json", "X-Request-ID": "abc123", }, params={"page": 1, "per_page": 50}, # appended as query string timeout=(5, 30), # (connect timeout, read timeout) in seconds)response.raise_for_status() # raises HTTPError for 4xx/5xxdata = response.json() # parse JSON response body
# POST with JSON bodypayload = {"name": "alice", "role": "admin"}response = requests.post( f"{BASE_URL}/users", json=payload, # sets Content-Type: application/json headers={"Authorization": "Bearer my-token-here"}, timeout=10,)response.raise_for_status()created = response.json()print(f"Created user ID: {created['id']}")
# POST with form-encoded bodyresponse = requests.post( "https://auth.example.com/token", data={"grant_type": "client_credentials"}, auth=HTTPBasicAuth("client_id", "client_secret"), timeout=10,)
# Session: reuses TCP connections, shares headerssession = requests.Session()session.headers.update({"Authorization": "Bearer my-token"})session.verify = "/etc/ssl/certs/ca-bundle.crt" # custom CA
for endpoint in ["/metrics", "/health", "/version"]: r = session.get(f"{BASE_URL}{endpoint}", timeout=5) r.raise_for_status() print(endpoint, r.json())
# Error handling with retryfrom requests.adapters import HTTPAdapterfrom urllib3.util.retry import Retry
retry_strategy = Retry( total=3, backoff_factor=1, status_forcelist=[429, 500, 502, 503, 504],)adapter = HTTPAdapter(max_retries=retry_strategy)session.mount("https://", adapter)Why this approach: A Session with a mounted retry adapter handles transient network failures automatically without cluttering business logic with retry loops.
json.load, json.loads, and json.dumps.Answer: Python's json module handles serialization and deserialization. load() reads from a file object, loads() parses a string, and dumps() serializes to a string (dump() writes to a file).
xxxxxxxxxximport jsonfrom pathlib import Pathfrom datetime import datetime
# Parse JSON from a filewith open("/etc/app/config.json") as f: config = json.load(f)
# Parse JSON from a string (e.g., API response body)raw = '{"host": "db.internal", "port": 5432, "ssl": true}'parsed = json.loads(raw)print(parsed["host"], parsed["port"])
# Serialize to stringdata = { "timestamp": "2026-05-18T00:00:00Z", "services": ["nginx", "postgres", "redis"], "metrics": {"cpu": 42.5, "mem_gb": 8.1},}compact = json.dumps(data) # no whitespacepretty = json.dumps(data, indent=2, sort_keys=True)print(pretty)
# Write JSON to filewith open("/tmp/report.json", "w") as f: json.dump(data, f, indent=2)
# Custom encoder for non-serializable typesclass DateTimeEncoder(json.JSONEncoder): def default(self, obj): if isinstance(obj, datetime): return obj.isoformat() return super().default(obj)
payload = {"created_at": datetime.utcnow(), "value": 99}print(json.dumps(payload, cls=DateTimeEncoder))
# Safe parsing with error handlingdef safe_json_load(path): try: with open(path) as f: return json.load(f) except (json.JSONDecodeError, FileNotFoundError) as e: print(f"Failed to parse {path}: {e}") return {}
# Merge/update nested dicts from JSONbase = json.loads('{"db": {"host": "localhost", "port": 5432}}')override = json.loads('{"db": {"port": 5433, "ssl": true}}')base["db"].update(override["db"])print(base) # {"db": {"host": "localhost", "port": 5433, "ssl": true}}Why this approach: Using raise_for_status() before json.loads() on HTTP responses and wrapping json.load() in try/except prevents silent data corruption from malformed inputs.
Answer: Use yaml.safe_load() instead of yaml.load() to avoid arbitrary code execution via YAML tags. For writing, use yaml.dump(). Install with pip install pyyaml.
xxxxxxxxxximport yamlfrom pathlib import Path
# Safe load from filewith open("/etc/app/config.yaml") as f: config = yaml.safe_load(f)
print(config.get("database", {}).get("host", "localhost"))
# Safe load from stringyaml_text = """server: host: 0.0.0.0 port: 8080 workers: 4database: host: db.internal port: 5432 name: appdb ssl: truefeatures: - feature_a - feature_b"""cfg = yaml.safe_load(yaml_text)print(cfg["server"]["port"]) # 8080print(cfg["features"]) # ['feature_a', 'feature_b']
# Load multiple documents from one file (separated by ---)yaml_multi = """---name: prodenv: production---name: stagingenv: staging"""docs = list(yaml.safe_load_all(yaml_multi))for doc in docs: print(doc["name"], doc["env"])
# Write YAMLdata = { "version": "1.2", "hosts": ["web01", "web02", "web03"], "config": {"timeout": 30, "retries": 3},}output = yaml.dump(data, default_flow_style=False, sort_keys=False)print(output)
with open("/tmp/output.yaml", "w") as f: yaml.dump(data, f, default_flow_style=False)
# Validate required keys after loadingdef load_config(path): with open(path) as f: cfg = yaml.safe_load(f) required = ["server", "database"] missing = [k for k in required if k not in cfg] if missing: raise ValueError(f"Config missing required keys: {missing}") return cfgWhy this approach: safe_load() only handles standard YAML types, preventing !!python/object tags from executing arbitrary Python code on untrusted config files.
Answer: Python's csv module handles quoting, escaping, and newlines correctly. Use DictReader and DictWriter for named-column access, which is more readable and robust than index-based access.
xxxxxxxxxximport csvimport sysfrom pathlib import Path
# Read CSV with headerswith open("/var/log/access_report.csv", newline="") as f: reader = csv.DictReader(f) for row in reader: print(row["ip"], row["status"], row["bytes"])
# Read pipe-delimited filewith open("/tmp/data.psv", newline="") as f: reader = csv.DictReader(f, delimiter="|") rows = list(reader)
# Write CSV with DictWriterfieldnames = ["hostname", "cpu_pct", "mem_pct", "disk_pct"]with open("/tmp/metrics.csv", "w", newline="") as f: writer = csv.DictWriter(f, fieldnames=fieldnames) writer.writeheader() writer.writerows([ {"hostname": "web01", "cpu_pct": 42, "mem_pct": 71, "disk_pct": 55}, {"hostname": "web02", "cpu_pct": 18, "mem_pct": 63, "disk_pct": 60}, ])
# Read CSV without headers (index-based)with open("/tmp/raw.csv", newline="") as f: reader = csv.reader(f) for row in reader: ip, ts, method, path, status = row[0], row[1], row[2], row[3], row[4]
# Stream-process large CSV to avoid loading into memorydef filter_large_csv(src, dst, status_code="500"): with open(src, newline="") as fin, open(dst, "w", newline="") as fout: reader = csv.DictReader(fin) writer = csv.DictWriter(fout, fieldnames=reader.fieldnames) writer.writeheader() for row in reader: if row.get("status") == status_code: writer.writerow(row)
filter_large_csv("/var/log/access.csv", "/tmp/errors.csv")
# Write to stdout (useful in pipelines)writer = csv.writer(sys.stdout)writer.writerow(["key", "value"])writer.writerow(["uptime_days", 42])Why this approach: DictReader/DictWriter decouple code from column order, making scripts resilient to column additions or reordering in the CSV schema.
concurrent.futures.ThreadPoolExecutor to parallelize I/O-bound tasks in Python?Answer: ThreadPoolExecutor manages a pool of threads for concurrent I/O work (network calls, disk reads). Use submit() for individual futures or map() for uniform task lists. Always handle exceptions from futures explicitly.
xxxxxxxxxximport concurrent.futuresimport requestsimport timefrom typing import List, Dict
HOSTS = ["web01", "web02", "web03", "web04", "web05"]
def check_host(host: str) -> Dict: """Check HTTP health endpoint of a host.""" url = f"http://{host}/health" try: r = requests.get(url, timeout=5) return {"host": host, "status": r.status_code, "ok": r.ok} except requests.RequestException as e: return {"host": host, "status": None, "ok": False, "error": str(e)}
# map() — simple, ordered results, re-raises first exceptionwith concurrent.futures.ThreadPoolExecutor(max_workers=10) as executor: results = list(executor.map(check_host, HOSTS, timeout=30))
for r in results: print(r["host"], "UP" if r["ok"] else "DOWN")
# submit() — more control, handle individual failureswith concurrent.futures.ThreadPoolExecutor(max_workers=10) as executor: future_to_host = {executor.submit(check_host, h): h for h in HOSTS}
for future in concurrent.futures.as_completed(future_to_host): host = future_to_host[future] try: result = future.result() print(f"{host}: {'OK' if result['ok'] else 'FAIL'}") except Exception as e: print(f"{host}: exception {e}")
# Parallelizing file downloadsdef download_file(url: str, dest: str) -> str: r = requests.get(url, timeout=30, stream=True) r.raise_for_status() with open(dest, "wb") as f: for chunk in r.iter_content(chunk_size=8192): f.write(chunk) return dest
urls = [ ("https://example.com/file1.tar.gz", "/tmp/file1.tar.gz"), ("https://example.com/file2.tar.gz", "/tmp/file2.tar.gz"),]
start = time.time()with concurrent.futures.ThreadPoolExecutor(max_workers=4) as executor: futures = [executor.submit(download_file, url, dest) for url, dest in urls] for f in concurrent.futures.as_completed(futures): print(f"Downloaded: {f.result()}")print(f"Total: {time.time() - start:.1f}s")Why this approach: as_completed() processes results as they finish rather than waiting for the slowest task, improving perceived responsiveness in monitoring scripts.
Answer: Python's GIL prevents true parallelism in threads for CPU-bound work. multiprocessing spawns separate OS processes, each with its own GIL and memory space, enabling true parallelism for CPU-intensive tasks like compression, hashing, or data processing.
xxxxxxxxxximport multiprocessingimport hashlibimport osfrom pathlib import Path
def hash_file(path: str) -> tuple: """Compute SHA-256 of a file (CPU-bound).""" h = hashlib.sha256() with open(path, "rb") as f: for chunk in iter(lambda: f.read(65536), b""): h.update(chunk) return path, h.hexdigest()
def compress_data(chunk: bytes) -> bytes: import zlib return zlib.compress(chunk, level=9)
# Using Pool.map() — distribute work across CPU coresfiles = [str(p) for p in Path("/var/log").rglob("*.log")]
with multiprocessing.Pool(processes=os.cpu_count()) as pool: results = pool.map(hash_file, files)
for path, digest in results: print(f"{digest} {path}")
# Pool.starmap() for multiple argumentsdef process_shard(shard_id: int, data: bytes) -> dict: import zlib compressed = zlib.compress(data, level=6) return {"shard": shard_id, "original": len(data), "compressed": len(compressed)}
data_shards = [(i, b"x" * 100_000) for i in range(8)]with multiprocessing.Pool() as pool: results = pool.starmap(process_shard, data_shards)
for r in results: ratio = r["compressed"] / r["original"] print(f"Shard {r['shard']}: {ratio:.2%} of original")
# Process-safe queue for producer/consumerdef producer(queue, files): for f in files: queue.put(f) queue.put(None) # sentinel
def consumer(queue, results_queue): while True: item = queue.get() if item is None: break path, digest = hash_file(item) results_queue.put((path, digest))
# When to use threading vs multiprocessing:# - threading: I/O-bound (network, disk), GIL released during syscalls# - multiprocessing: CPU-bound (hashing, parsing, compression)# - asyncio: high-concurrency I/O (thousands of connections)Why this approach: Pool.map() automatically partitions the work list and collects results in order, handling process lifecycle and IPC transparently.
tempfile.NamedTemporaryFile safely for temporary files in scripts?Answer: tempfile.NamedTemporaryFile creates a secure temporary file with a unique name in the system's temp directory. Use it as a context manager so it is automatically deleted on exit, even if an exception occurs.
xxxxxxxxxximport tempfileimport subprocessimport osimport shutil
# Basic usage — file is deleted when context manager exitswith tempfile.NamedTemporaryFile(mode="w", suffix=".conf", delete=True) as f: f.write("[section]\nkey=value\n") f.flush() # ensure data is written before passing path to subprocess result = subprocess.run(["validate-config", f.name], capture_output=True)
# Keep file after close for cross-process use (delete=False)with tempfile.NamedTemporaryFile( mode="wb", suffix=".tar.gz", delete=False, dir="/tmp") as f: tmp_path = f.name f.write(b"binary data here")
try: # Hand off to another process subprocess.run(["tar", "tzf", tmp_path], check=True)finally: os.unlink(tmp_path) # manual cleanup
# TemporaryDirectory for multiple temp fileswith tempfile.TemporaryDirectory(prefix="deploy_", dir="/tmp") as tmpdir: config_path = os.path.join(tmpdir, "app.conf") manifest_path = os.path.join(tmpdir, "manifest.json")
with open(config_path, "w") as f: f.write("port=8080\n") with open(manifest_path, "w") as f: f.write('{"version": "1.2.3"}\n')
subprocess.run(["deploy-tool", "--config", config_path, "--manifest", manifest_path], check=True)# tmpdir and all its contents are removed here
# Atomic file write using temp file + renamedef atomic_write(path: str, content: str): """Write content atomically — either whole file or nothing.""" dirpath = os.path.dirname(os.path.abspath(path)) with tempfile.NamedTemporaryFile( mode="w", dir=dirpath, delete=False, suffix=".tmp" ) as f: f.write(content) tmp_path = f.name os.replace(tmp_path, path) # atomic on POSIX filesystems
atomic_write("/etc/app/config.conf", "new config content\n")Why this approach: os.replace() on the same filesystem is atomic, so readers never see a partially written file even if the writer crashes mid-write.
glob and fnmatch to find files matching patterns in Python?Answer: glob.glob() and Path.glob() expand shell-style wildcards to file paths. fnmatch tests a single filename against a pattern without filesystem access, useful for filtering lists already in memory.
xxxxxxxxxximport globimport fnmatchimport osfrom pathlib import Path
# glob.glob() — basic wildcard expansionlog_files = glob.glob("/var/log/*.log")gz_files = glob.glob("/var/log/**/*.gz", recursive=True) # recursive with **conf_files = glob.glob("/etc/**/*.conf", recursive=True)
# glob.iglob() — iterator, avoids loading all paths into memoryfor path in glob.iglob("/var/log/**/*.log", recursive=True): print(path)
# pathlib Path.glob() — object-oriented, returns Path objectslog_dir = Path("/var/log")for log in log_dir.glob("*.log"): print(log.name, log.stat().st_size)
for log in log_dir.rglob("*.log"): # rglob = recursive glob print(log)
# fnmatch — pattern matching on filename stringsfilenames = ["app.log", "app.log.1", "access.log", "error.log", "deploy.sh"]
matched = [f for f in filenames if fnmatch.fnmatch(f, "*.log")]print(matched) # ['app.log', 'access.log', 'error.log']
matched = [f for f in filenames if fnmatch.fnmatch(f, "app.*")]print(matched) # ['app.log', 'app.log.1']
# fnmatchcase — case-sensitive versionfiltered = fnmatch.filter(filenames, "*.log") # list comprehension equivalentprint(filtered)
# Practical: find and process rotated logsdef find_rotated_logs(log_dir, base_name): """Find logs like app.log, app.log.1, app.log.2.gz""" pattern = os.path.join(log_dir, f"{base_name}*") files = sorted(glob.glob(pattern)) return files
rotated = find_rotated_logs("/var/log/nginx", "access.log")for f in rotated: print(f)
# Find recently modified configsimport timecutoff = time.time() - 86400 # last 24 hoursrecent = [ p for p in Path("/etc").rglob("*.conf") if p.stat().st_mtime > cutoff]Why this approach: Path.rglob() from pathlib returns Path objects directly, enabling method chaining like .stat(), .read_text(), and .unlink() without extra os.path calls.
re module for regex matching, searching, extracting groups, and substitution?Answer: Compile patterns with re.compile() for reuse. Use search() for anywhere-in-string matches, match() for start-of-string, findall() for all occurrences, and sub() for replacement.
xxxxxxxxxximport re
# Compile once, reuse many timesip_pattern = re.compile( r"\b(\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})\b")log_line = '192.168.1.100 - - [18/May/2026:12:34:56 +0000] "GET /api/v1/health HTTP/1.1" 200 42'
# search() — find first match anywhere in stringm = ip_pattern.search(log_line)if m: print(m.group(0)) # full match: 192.168.1.100 print(m.group(1)) # first group: 192 print(m.groups()) # ('192', '168', '1', '100')
# match() — only matches at beginningdate_pattern = re.compile(r"^(\d{4})-(\d{2})-(\d{2})")m = date_pattern.match("2026-05-18T00:00:00Z")if m: year, month, day = m.groups()
# findall() — list of all matches (or list of tuples if groups)text = "Errors at 10:01:05, 10:02:33, and 10:15:50"times = re.findall(r"\d{2}:\d{2}:\d{2}", text)print(times) # ['10:01:05', '10:02:33', '10:15:50']
# Named groups for readabilityapache_log = re.compile( r'(?P<ip>\S+) \S+ \S+ \[(?P<time>[^\]]+)\] ' r'"(?P<method>\S+) (?P<path>\S+) \S+" (?P<status>\d{3}) (?P<bytes>\S+)')m = apache_log.match(log_line)if m: print(m.group("ip"), m.group("status"), m.group("path")) print(m.groupdict())
# finditer() — iterator of match objects (memory efficient)for m in apache_log.finditer(open("/var/log/nginx/access.log").read()): if m.group("status").startswith("5"): print(f"5xx: {m.group('path')} from {m.group('ip')}")
# sub() — substitution# Redact passwords in log linessanitized = re.sub(r'password=[^&\s]+', 'password=REDACTED', log_line)# Remove ANSI color codesansi_escape = re.compile(r'\x1B(?:[@-Z\\-_]|\[[0-?]*[ -/]*[@-~])')clean = ansi_escape.sub("", "\x1b[31mERROR\x1b[0m: disk full")print(clean) # ERROR: disk full
# Flagspattern = re.compile(r"^error:", re.IGNORECASE | re.MULTILINE)errors = pattern.findall("ERROR: oops\ninfo: ok\nError: again")print(errors) # ['ERROR:', 'Error:']Why this approach: Named groups ((?P<name>...)) make regex patterns self-documenting and allow accessing captures by semantic name rather than fragile positional indices.
hashlib to compute MD5 and SHA-256 checksums of files and strings in Python?Answer: hashlib provides a uniform interface for all hash algorithms. Always read files in chunks to handle large files without exhausting memory. Use SHA-256 for security-sensitive contexts; MD5 only for non-security checksums (e.g., cache keys, deduplication).
xxxxxxxxxximport hashlibimport hmacimport os
# Hash a stringdef hash_string(s: str, algorithm: str = "sha256") -> str: h = hashlib.new(algorithm) h.update(s.encode("utf-8")) return h.hexdigest()
print(hash_string("hello world"))print(hash_string("hello world", "md5"))
# Hash a file in chunks (memory-efficient)def hash_file(path: str, algorithm: str = "sha256") -> str: h = hashlib.new(algorithm) with open(path, "rb") as f: for chunk in iter(lambda: f.read(65536), b""): h.update(chunk) return h.hexdigest()
digest = hash_file("/var/log/syslog")print(f"SHA-256: {digest}")
# Verify file integritydef verify_file(path: str, expected_sha256: str) -> bool: actual = hash_file(path) return hmac.compare_digest(actual, expected_sha256.lower())
# Compute and write a checksum file (like sha256sum tool output)def write_checksums(file_list, output_path): with open(output_path, "w") as out: for filepath in file_list: digest = hash_file(filepath) out.write(f"{digest} {filepath}\n")
# Verify against a checksum filedef verify_checksums(checksum_file): failed = [] with open(checksum_file) as f: for line in f: line = line.strip() if not line: continue digest, path = line.split(None, 1) actual = hash_file(path.strip()) if not hmac.compare_digest(actual, digest): failed.append(path) print(f"FAIL: {path}") else: print(f"OK: {path}") return failed
# HMAC for message authenticationsecret_key = b"super-secret-key"message = b"deploy version 1.2.3 to production"mac = hmac.new(secret_key, message, hashlib.sha256).hexdigest()print(f"HMAC-SHA256: {mac}")
# Check available algorithmsprint(hashlib.algorithms_available)Why this approach: hmac.compare_digest() uses a constant-time comparison that prevents timing attacks when verifying checksums in security-sensitive contexts.
Answer: An idempotent script produces the same result regardless of how many times it is run. The key principle is "check before act": test whether the desired state already exists before taking action. This is critical for safe re-runs after partial failures.
xxxxxxxxxximport osimport subprocessimport hashlibimport grpimport pwd
# Pattern 1: Create directory only if missingdef ensure_dir(path, mode=0o755, owner=None, group=None): if not os.path.isdir(path): os.makedirs(path, mode=mode) print(f"Created: {path}") else: print(f"Exists: {path}")
if owner or group: uid = pwd.getpwnam(owner).pw_uid if owner else -1 gid = grp.getgrnam(group).gr_gid if group else -1 os.chown(path, uid, gid)
# Pattern 2: Write file only if content changeddef ensure_file_content(path, content): new_hash = hashlib.sha256(content.encode()).hexdigest() if os.path.exists(path): with open(path) as f: old_hash = hashlib.sha256(f.read().encode()).hexdigest() if old_hash == new_hash: print(f"Unchanged: {path}") return False with open(path, "w") as f: f.write(content) print(f"Updated: {path}") return True # changed = caller may need to restart service
# Pattern 3: Install package only if absentdef ensure_package_installed(package): result = subprocess.run( ["dpkg", "-s", package], capture_output=True ) if result.returncode == 0: print(f"Already installed: {package}") return subprocess.run(["apt-get", "install", "-y", package], check=True) print(f"Installed: {package}")
# Pattern 4: Add line to file only if not presentdef ensure_line_in_file(filepath, line): line = line.rstrip("\n") try: with open(filepath) as f: lines = [l.rstrip("\n") for l in f] except FileNotFoundError: lines = []
if line not in lines: with open(filepath, "a") as f: f.write(line + "\n") print(f"Added to {filepath}: {line}") else: print(f"Already present in {filepath}: {line}")
# Pattern 5: Create symlink idempotentlydef ensure_symlink(src, dst): if os.path.islink(dst): if os.readlink(dst) == src: return # correct, nothing to do os.unlink(dst) # wrong target, fix it elif os.path.exists(dst): raise RuntimeError(f"{dst} exists and is not a symlink") os.symlink(src, dst) print(f"Linked: {dst} -> {src}")
# Orchestrate idempotent setupdef setup_app(): ensure_dir("/opt/app", owner="www-data", group="www-data") ensure_dir("/opt/app/logs") ensure_dir("/opt/app/tmp") changed = ensure_file_content("/opt/app/config.ini", "[app]\nport=8080\n") ensure_package_installed("nginx") ensure_line_in_file("/etc/hosts", "127.0.0.1 app.local") if changed: subprocess.run(["systemctl", "restart", "app"], check=True)Why this approach: Checking current state before acting makes scripts safe to run in CI/CD pipelines and disaster recovery scenarios where partial failures require re-running without manual cleanup.
sys.exit() with meaningful exit codes in production scripts?Answer: Scripts should communicate success, failure, and specific error conditions through exit codes. sys.exit(0) means success; non-zero indicates failure. Define named constants for codes to make scripts self-documenting and compatible with monitoring systems.
xxxxxxxxxximport sysimport loggingimport subprocess
# Define exit codes as constantsEXIT_OK = 0EXIT_GENERIC_ERROR = 1EXIT_USAGE_ERROR = 2EXIT_IO_ERROR = 3EXIT_CONFIG_ERROR = 4EXIT_DEPENDENCY_ERROR = 5EXIT_TIMEOUT = 6EXIT_PERMISSION_ERROR = 7
# Nagios/monitoring-compatible codesNAGIOS_OK = 0NAGIOS_WARNING = 1NAGIOS_CRITICAL = 2NAGIOS_UNKNOWN = 3
logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")log = logging.getLogger(__name__)
def main(): import argparse parser = argparse.ArgumentParser(description="Deploy application") parser.add_argument("--env", required=True, choices=["dev", "staging", "prod"]) parser.add_argument("--version", required=True) args = parser.parse_args()
# Config validation try: config = load_config("/etc/deploy/config.yaml") except FileNotFoundError as e: log.error("Config file not found: %s", e) sys.exit(EXIT_CONFIG_ERROR) except Exception as e: log.error("Config parse error: %s", e) sys.exit(EXIT_CONFIG_ERROR)
# Dependency check for tool in ["kubectl", "helm", "docker"]: result = subprocess.run(["which", tool], capture_output=True) if result.returncode != 0: log.error("Required tool not found: %s", tool) sys.exit(EXIT_DEPENDENCY_ERROR)
# Main work try: deploy(args.env, args.version, config) except PermissionError as e: log.error("Permission denied: %s", e) sys.exit(EXIT_PERMISSION_ERROR) except TimeoutError as e: log.error("Deployment timed out: %s", e) sys.exit(EXIT_TIMEOUT) except IOError as e: log.error("I/O error during deployment: %s", e) sys.exit(EXIT_IO_ERROR) except Exception as e: log.exception("Unexpected error: %s", e) sys.exit(EXIT_GENERIC_ERROR)
log.info("Deployment complete.") sys.exit(EXIT_OK)
# Nagios check script patterndef check_disk(): import shutil total, used, free = shutil.disk_usage("/") pct_used = (used / total) * 100 msg = f"Disk usage: {pct_used:.1f}% ({free // 2**30}GB free)"
if pct_used > 95: print(f"CRITICAL - {msg}") sys.exit(NAGIOS_CRITICAL) elif pct_used > 80: print(f"WARNING - {msg}") sys.exit(NAGIOS_WARNING) else: print(f"OK - {msg}") sys.exit(NAGIOS_OK)
def load_config(path): passdef deploy(env, version, config): pass
if __name__ == "__main__": main()Why this approach: Defined exit code constants make the script's contract explicit and allow shell scripts, Nagios, and CI systems to branch on specific failure modes rather than treating all non-zero exits identically.
configparser to read and write INI-style configuration files in Python?Answer: configparser.ConfigParser handles INI files with [sections] and key = value pairs. It supports type coercion, default values, interpolation, and fallback values — making it a robust alternative to ad-hoc config parsing.
xxxxxxxxxximport configparserimport os
# Read an INI config fileconfig = configparser.ConfigParser()config.read("/etc/app/app.conf")
# Access values with fallbackshost = config.get("database", "host", fallback="localhost")port = config.getint("database", "port", fallback=5432)ssl = config.getboolean("database", "ssl", fallback=False)timeout = config.getfloat("database", "connect_timeout", fallback=5.0)
print(f"Connecting to {host}:{port} (ssl={ssl})")
# Iterate sections and keysfor section in config.sections(): print(f"[{section}]") for key, value in config.items(section): print(f" {key} = {value}")
# Check existence before accessif config.has_section("cache") and config.has_option("cache", "ttl"): ttl = config.getint("cache", "ttl")
# Write a config file programmaticallynew_config = configparser.ConfigParser()
new_config["DEFAULT"] = { "log_level": "INFO", "timeout": "30",}new_config["server"] = { "host": "0.0.0.0", "port": "8080", "workers": "4",}new_config["database"] = { "host": "db.internal", "port": "5432", "name": "appdb", "ssl": "true",}
with open("/etc/app/generated.conf", "w") as f: new_config.write(f)
# Interpolation — reference other values with %(key)sinterp_config = configparser.ConfigParser()interp_config.read_string("""[paths]base_dir = /opt/applog_dir = %(base_dir)s/logsdata_dir = %(base_dir)s/data""")print(interp_config.get("paths", "log_dir")) # /opt/app/logs
# Environment variable override patternclass EnvConfigParser(configparser.ConfigParser): def get(self, section, option, **kwargs): env_key = f"{section.upper()}_{option.upper()}" if env_key in os.environ: return os.environ[env_key] return super().get(section, option, **kwargs)
cfg = EnvConfigParser()cfg.read("/etc/app/app.conf")# DB_HOST env var overrides [database] hostdb_host = cfg.get("database", "host", fallback="localhost")Why this approach: The fallback= parameter in get() eliminates KeyError exceptions for optional config keys while keeping the "required vs optional" semantics visible at the call site.
Answer: Credentials should come from environment variables, secret management systems (Vault, AWS Secrets Manager), or encrypted files — never hardcoded in source. Use os.environ for env vars and the relevant SDK for secret stores.
xxxxxxxxxximport osimport sys
# Pattern 1: Environment variables (simplest, works everywhere)db_password = os.environ.get("DB_PASSWORD")if not db_password: print("ERROR: DB_PASSWORD environment variable not set", file=sys.stderr) sys.exit(1)
# Pattern 2: .env file loading (dev only — never commit .env)def load_dotenv(path=".env"): if not os.path.exists(path): return with open(path) as f: for line in f: line = line.strip() if line and not line.startswith("#") and "=" in line: key, _, value = line.partition("=") os.environ.setdefault(key.strip(), value.strip())
load_dotenv()
# Pattern 3: AWS Secrets Managerdef get_secret(secret_name, region="us-east-1"): import boto3, json client = boto3.client("secretsmanager", region_name=region) response = client.get_secret_value(SecretId=secret_name) secret = response.get("SecretString") or \ response["SecretBinary"].decode("utf-8") return json.loads(secret)
# Pattern 4: HashiCorp Vaultdef get_vault_secret(path, vault_addr=None, token=None): import requests addr = vault_addr or os.environ["VAULT_ADDR"] token = token or os.environ["VAULT_TOKEN"] r = requests.get( f"{addr}/v1/{path}", headers={"X-Vault-Token": token}, timeout=5 ) r.raise_for_status() return r.json()["data"]["data"]
# Pattern 5: Require secrets via config objectclass Config: def __init__(self): self.db_host = os.environ.get("DB_HOST", "localhost") self.db_port = int(os.environ.get("DB_PORT", "5432")) self.db_password = self._require("DB_PASSWORD") self.api_key = self._require("API_KEY")
def _require(key): value = os.environ.get(key) if not value: raise EnvironmentError(f"Required environment variable not set: {key}") return value
config = Config()Why this approach: Centralizing credential loading into a Config class with _require() fails fast at startup with a clear error, rather than failing deep in business logic with a confusing NoneType error.
Answer: Configure the logging module with multiple handlers — a StreamHandler for stdout and a RotatingFileHandler for the log file. Set different levels on each handler to control verbosity per destination.
xxxxxxxxxximport loggingimport logging.handlersimport sysimport os
def setup_logging(log_file="/var/log/app/deploy.log", file_level=logging.DEBUG, console_level=logging.INFO): """Configure root logger with file and console handlers.""" os.makedirs(os.path.dirname(log_file), exist_ok=True)
root = logging.getLogger() root.setLevel(logging.DEBUG) # root captures everything; handlers filter
fmt = logging.Formatter( "%(asctime)s %(levelname)-8s %(name)s:%(lineno)d - %(message)s", datefmt="%Y-%m-%dT%H:%M:%S" )
# Console handler — INFO and above console = logging.StreamHandler(sys.stdout) console.setLevel(console_level) console.setFormatter(fmt)
# File handler — DEBUG and above, rotating 10MB x 5 files file_handler = logging.handlers.RotatingFileHandler( log_file, maxBytes=10 * 1024 * 1024, backupCount=5, encoding="utf-8" ) file_handler.setLevel(file_level) file_handler.setFormatter(fmt)
root.addHandler(console) root.addHandler(file_handler)
return root
setup_logging()log = logging.getLogger(__name__)
log.debug("Starting deployment pipeline") # file onlylog.info("Deploying version 1.2.3") # console + filelog.warning("Config file missing, using defaults")log.error("Health check failed on web03")log.critical("Rollback initiated")
# Structured logging for JSON log aggregatorsimport jsonclass JsonFormatter(logging.Formatter): def format(self, record): return json.dumps({ "ts": self.formatTime(record, "%Y-%m-%dT%H:%M:%S"), "level": record.levelname, "logger": record.name, "msg": record.getMessage(), "file": record.pathname, "line": record.lineno, })
json_handler = logging.StreamHandler(sys.stdout)json_handler.setFormatter(JsonFormatter())Why this approach: Separating log levels per handler lets operators get clean INFO summaries on the console while preserving full DEBUG traces in the file for post-incident analysis.
argparse for production-quality CLIs?Answer: argparse generates help text, type validation, and usage errors automatically. Use subcommands for scripts with multiple operations, type= for coercion, and required= groups for mutually exclusive options.
xxxxxxxxxximport argparseimport sys
def build_parser(): parser = argparse.ArgumentParser( description="Application deployment tool", formatter_class=argparse.RawDescriptionHelpFormatter, epilog="""Examples: deploy.py deploy --env prod --version 1.2.3 deploy.py rollback --env prod --steps 1 deploy.py status --env prod """ ) parser.add_argument("--verbose", "-v", action="store_true", help="Enable debug logging") parser.add_argument("--dry-run", action="store_true", help="Show actions without executing them")
subparsers = parser.add_subparsers(dest="command", required=True)
# deploy subcommand deploy_p = subparsers.add_parser("deploy", help="Deploy a new version") deploy_p.add_argument("--env", required=True, choices=["dev", "staging", "prod"], help="Target environment") deploy_p.add_argument("--version", required=True, help="Version tag to deploy (e.g. 1.2.3)") deploy_p.add_argument("--replicas", type=int, default=2, help="Number of replicas (default: 2)") deploy_p.add_argument("--timeout", type=float, default=300.0, help="Deployment timeout in seconds")
# rollback subcommand rb_p = subparsers.add_parser("rollback", help="Rollback to previous version") rb_p.add_argument("--env", required=True, choices=["dev", "staging", "prod"]) rb_p.add_argument("--steps", type=int, default=1, help="Number of versions to roll back")
# Mutually exclusive group group = rb_p.add_mutually_exclusive_group() group.add_argument("--to-version", help="Roll back to specific version") group.add_argument("--steps", dest="steps", type=int, default=1)
return parser
def main(): parser = build_parser() args = parser.parse_args()
if args.verbose: import logging logging.basicConfig(level=logging.DEBUG)
if args.command == "deploy": print(f"Deploying {args.version} to {args.env} " f"({args.replicas} replicas, timeout={args.timeout}s)") if args.dry_run: print("[DRY RUN] No changes made.") sys.exit(0)
elif args.command == "rollback": print(f"Rolling back {args.steps} step(s) in {args.env}")
if __name__ == "__main__": main()Why this approach: Subcommands keep related argument groups isolated and allow each subcommand to have its own --help, making complex scripts navigable without reading source code.
Answer: Implement __enter__ / __exit__ for class-based context managers, or use contextlib.contextmanager with a generator. Context managers guarantee cleanup even when exceptions occur.
xxxxxxxxxximport contextlibimport osimport tempfileimport subprocessimport time
# Class-based context managerclass TempWorkDir: """Creates a temporary directory and changes into it.""" def __init__(self, prefix="work_", base="/tmp"): self.prefix = prefix self.base = base self.tmpdir = None self.original_cwd = None
def __enter__(self): self.original_cwd = os.getcwd() self.tmpdir = tempfile.mkdtemp(prefix=self.prefix, dir=self.base) os.chdir(self.tmpdir) return self.tmpdir
def __exit__(self, exc_type, exc_val, exc_tb): os.chdir(self.original_cwd) import shutil shutil.rmtree(self.tmpdir, ignore_errors=True) return False # don't suppress exceptions
with TempWorkDir(prefix="deploy_") as tmpdir: subprocess.run(["git", "clone", "https://example.com/app.git", "."], check=True) # tmpdir cleaned up here
# Generator-based context manager (simpler for functions).contextmanagerdef timer(label="operation"): start = time.monotonic() try: yield finally: elapsed = time.monotonic() - start print(f"{label} took {elapsed:.3f}s")
with timer("database migration"): subprocess.run(["alembic", "upgrade", "head"], check=True)
# Suppress specific exceptions.contextmanagerdef suppress_and_log(*exc_types): import logging try: yield except exc_types as e: logging.warning("Suppressed exception: %s: %s", type(e).__name__, e)
with suppress_and_log(FileNotFoundError): os.unlink("/tmp/stale_lock") # OK if already gone
# ExitStack for dynamic number of context managersdef process_files(file_list): with contextlib.ExitStack() as stack: handles = [stack.enter_context(open(f)) for f in file_list] for handle in handles: print(handle.read(100)) # all files closed here
# Reentrant context managerlock_cm = contextlib.nullcontext() # no-op, useful for optional lockingWhy this approach: contextlib.ExitStack lets you dynamically register cleanup callbacks and open a variable number of resources under one with block, eliminating nested with pyramids.
shutil module for file system operations like copying, moving, and archiving?Answer: shutil provides high-level file and directory operations that handle edge cases (permissions, symbolic links, metadata) better than manual open()/write() approaches. Use it for deployment scripts that manipulate directory trees.
xxxxxxxxxximport shutilimport osimport tarfilefrom datetime import datetime
# Copy a single file (preserving metadata)shutil.copy2("/etc/nginx/nginx.conf", "/etc/nginx/nginx.conf.bak")
# Copy entire directory treeshutil.copytree( "/opt/app/current", "/opt/app/releases/20260518", symlinks=True, ignore=shutil.ignore_patterns("*.pyc", "__pycache__", ".git", "*.log"))
# Move file or directoryshutil.move("/opt/app/releases/20260518", "/opt/app/current")
# Remove entire directory treeshutil.rmtree("/opt/app/releases/old_release", ignore_errors=True)
# Disk usagetotal, used, free = shutil.disk_usage("/")print(f"Disk: {used//2**30}GB used / {total//2**30}GB total " f"({free//2**30}GB free)")
# Create archive (tar.gz, zip, etc.)shutil.make_archive( base_name=f"/tmp/backup_{datetime.now():%Y%m%d}", format="gztar", # tar.gz root_dir="/opt/app", base_dir="current")# Creates /tmp/backup_20260518.tar.gz
# Unpack archiveshutil.unpack_archive("/tmp/backup_20260518.tar.gz", "/opt/app/restore")
# which() — find executable in PATHkubectl_path = shutil.which("kubectl")if not kubectl_path: raise RuntimeError("kubectl not found in PATH")print(f"Using kubectl: {kubectl_path}")
# Deployment blue/green swap patterndef blue_green_swap(deploy_path, new_release_path, backup_path): """Atomically swap current with new release.""" if os.path.exists(backup_path): shutil.rmtree(backup_path)
if os.path.exists(deploy_path): shutil.copytree(deploy_path, backup_path, symlinks=True)
shutil.rmtree(deploy_path, ignore_errors=True) shutil.copytree(new_release_path, deploy_path, symlinks=True) print(f"Deployed: {new_release_path} -> {deploy_path}") print(f"Backup at: {backup_path}")
blue_green_swap( "/opt/app/current", "/opt/app/releases/v1.2.3", "/opt/app/previous")Why this approach: shutil.make_archive() and unpack_archive() handle format detection and cross-platform differences automatically, avoiding subprocess calls to tar or zip that differ between Linux and macOS.
Answer: A crontab line has the format: minute hour day-of-month month day-of-week command. Each field accepts numbers, ranges (1-5), lists (1,3,5), steps (*/15), or * (any). The command runs whenever all five fields match the current time.
xxxxxxxxxx# Format:# ┌─────────────── minute (0–59)# │ ┌───────────── hour (0–23)# │ │ ┌─────────── day of month (1–31)# │ │ │ ┌───────── month (1–12 or jan–dec)# │ │ │ │ ┌─────── day of week (0–7, 0 and 7 = Sunday, or sun–sat)# │ │ │ │ │# * * * * * /path/to/command
# Every minute* * * * * /usr/local/bin/heartbeat.sh
# Every 15 minutes*/15 * * * * /usr/local/bin/check_services.sh
# Every day at 2:30 AM30 2 * * * /usr/local/bin/db_backup.sh
# Every Monday at 6:00 AM0 6 * * 1 /usr/local/bin/weekly_report.sh
# Every weekday (Mon–Fri) at 8:00 AM0 8 * * 1-5 /usr/local/bin/business_hours_check.sh
# 1st and 15th of every month at midnight0 0 1,15 * * /usr/local/bin/billing_run.sh
# Every 6 hours0 */6 * * * /usr/local/bin/log_rotate.sh
# January 1st at midnight (New Year)0 0 1 1 * /usr/local/bin/annual_cleanup.sh
# Every 5 minutes between 9 AM and 5 PM on weekdays*/5 9-17 * * 1-5 /usr/local/bin/poll_api.sh
# Multiple times: at 6 AM, 12 PM, 6 PM every day0 6,12,18 * * * /usr/local/bin/sync.sh
# Special characters reference:# * = every valid value# , = list (1,3,5)# - = range (1-5)# / = step (*/5 = every 5 units)# Last day of month (not standard, use scripts):# 0 23 28-31 * * [ "$(date +\%d -d tomorrow)" = "01" ] && /path/to/script.shWhy this approach: Breaking the format into visual columns with a comment header prevents field transposition errors — the most common crontab mistake.
@reboot, @daily, @hourly, and other cron shortcut expressions?Answer: Cron shortcut strings replace the five-field syntax for common intervals. @reboot is particularly useful for starting services or scripts when the system boots up.
xxxxxxxxxx# Shortcut strings (supported by Vixie cron and most modern cron daemons)@reboot /usr/local/bin/start_agent.sh # once at boot@yearly /usr/local/bin/annual_report.sh # = 0 0 1 1 *@annually /usr/local/bin/annual_report.sh # same as @yearly@monthly /usr/local/bin/monthly_cleanup.sh # = 0 0 1 * *@weekly /usr/local/bin/weekly_summary.sh # = 0 0 * * 0@daily /usr/local/bin/daily_backup.sh # = 0 0 * * *@midnight /usr/local/bin/daily_backup.sh # same as @daily@hourly /usr/local/bin/check_health.sh # = 0 * * * *
# Practical @reboot usage — start a monitoring agent@reboot /usr/local/bin/monitoring-agent --config /etc/monitoring/agent.conf >> /var/log/monitoring-agent.log 2>&1
# Multiple @reboot entries are allowed@reboot /usr/local/bin/setup_tunnels.sh@reboot /usr/local/bin/warm_cache.sh
# Note: @reboot runs as the cron user, with a minimal environment.# Always use full paths and set PATH explicitly in the script:# #!/bin/bash# PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin# ...
# Verify your crontabcrontab -l # list current user's crontabcrontab -e # editcrontab -r # remove (dangerous! always verify first)sudo crontab -u www-data -l # list another user's crontab
# System cron directories (no need to crontab -e, just drop files)ls /etc/cron.hourly/ls /etc/cron.daily/ls /etc/cron.weekly/ls /etc/cron.monthly/# Files in these dirs must be executable and contain no dots in their namesWhy this approach: Shortcut strings are self-documenting and immune to typos in field positioning, reducing the chance of accidentally scheduling a daily job to run every minute.
Answer: A systemd timer unit (.timer) activates a paired service unit (.service) on a schedule defined by OnCalendar=. Timers support dependencies, logging to journald, and missed-run catch-up — advantages over cron for system services.
xxxxxxxxxx# /etc/systemd/system/db-backup.service[Unit]Description=Database Backup ServiceAfter=network.target postgresql.service
[Service]Type=oneshotUser=postgresExecStart=/usr/local/bin/db_backup.shStandardOutput=journalStandardError=journal
# /etc/systemd/system/db-backup.timer[Unit]Description=Run database backup daily at 02:30Requires=db-backup.service
[Timer]OnCalendar=*-*-* 02:30:00Persistent=true # run missed jobs if system was downRandomizedDelaySec=300 # jitter up to 5 min to avoid thundering herdAccuracySec=1m # how precisely to honor the time
[Install]WantedBy=timers.targetxxxxxxxxxx# Enable and start the timersudo systemctl daemon-reloadsudo systemctl enable --now db-backup.timer
# Check timer statussystemctl list-timers --allsystemctl status db-backup.timersystemctl status db-backup.service
# View logsjournalctl -u db-backup.service -n 50journalctl -u db-backup.service --since "2026-05-18"
# Manually trigger the service (test without waiting for timer)sudo systemctl start db-backup.service
# OnCalendar syntax examples# OnCalendar=weekly # Monday 00:00:00# OnCalendar=daily # *-*-* 00:00:00# OnCalendar=hourly # *-*-* *:00:00# OnCalendar=Mon..Fri 09:00 # weekdays at 9 AM# OnCalendar=*-*-1 00:00:00 # 1st of every month# OnCalendar=*:0/15 # every 15 minutes
# Verify calendar expressionsystemd-analyze calendar "Mon..Fri 09:00"systemd-analyze calendar "*-*-* 02:30:00"Cron vs. systemd timers comparison:
xxxxxxxxxxFeature Cron systemd timers─────────────────────── ────────────────── ──────────────────────────Logging redirect to file journald (automatic)Dependencies none full dependency graphRun if missed no yes (Persistent=true)Per-user support yes (crontab) yes (systemctl --user)Jitter/spread manual scripting RandomizedDelaySec=Fine-grained intervals 1 minute minimum 1 second minimumComplexity simple requires two unit files
Why this approach: Persistent=true ensures a missed backup (due to maintenance downtime) runs on next boot, eliminating the silent data loss gap that cron silently ignores.
at command for one-shot deferred job execution?Answer: at schedules a command to run once at a specific future time, unlike cron which repeats. It reads commands from stdin and emails output to the user. Use atq to list pending jobs and atrm to cancel them.
xxxxxxxxxx# Schedule a command at a specific timeecho "/usr/local/bin/deploy.sh --env prod" | at 02:30
# Various time formatsat 14:30 tomorrowat "3pm + 2 hours"at "next Monday"at "now + 30 minutes"at "2026-05-19 02:30"at midnight
# Run a scriptat 23:00 < /usr/local/bin/maintenance.sh
# Multi-line job via heredocat 02:00 <<'EOF'/usr/local/bin/backup.sh /opt/app/usr/local/bin/notify.sh "Backup complete"EOF
# List pending jobsatq# Output: 3 Mon May 19 02:00:00 2026 a root
# View job detailsat -c 3 # show commands for job 3
# Remove a jobatrm 3
# Practical: schedule maintenance window notificationat "now + 5 minutes" <<EOFwall "System maintenance begins in 5 minutes. Please save your work."EOF
at "now + 10 minutes" <<EOF/etc/init.d/apache2 stop/usr/local/bin/maintenance_tasks.sh/etc/init.d/apache2 startEOF
# Check if atd is runningsystemctl status atd# orservice atd status
# Run as specific user (requires root or atd configured with sudo)sudo -u deploy at 04:00 < /home/deploy/rotate_keys.shWhy this approach: at is simpler and safer than adding a one-time cron entry because it doesn't require remembering to remove the entry afterward, preventing accidental recurring execution.
flock?Answer: flock acquires an exclusive file lock before running a command. If the lock is held (previous run still active), the new invocation exits immediately. This prevents resource contention from slow-running jobs that overlap.
xxxxxxxxxx# Basic flock usage in crontab*/5 * * * * flock -n /tmp/my_script.lock /usr/local/bin/my_script.sh
# -n = non-blocking (exit immediately if lock taken)# -x = exclusive lock (default)# -e 1 = exit code when lock fails (optional)
# With explicit lock file and timeout*/5 * * * * flock --timeout=10 /var/lock/backup.lock /usr/local/bin/backup.sh
# In a shell script (self-locking)LOCK_FILE="/var/lock/$(basename "$0" .sh).lock"exec 200>"$LOCK_FILE" # open fd 200 for writing
# Try to acquire exclusive lock (non-blocking)flock -n 200 || { echo "$(date): Another instance is running, exiting." >&2 exit 1}
# Lock held — do workecho "Starting job at $(date)"/usr/local/bin/long_running_task.sh
# Lock released automatically when script exits (fd 200 closed)
# Inline cron entry with full logging*/15 * * * * flock -n /tmp/etl.lock \ bash -c '/usr/local/bin/run_etl.sh >> /var/log/etl.log 2>&1' \ || echo "$(date): ETL already running, skipped" >> /var/log/etl.log
# With PID tracking for debuggingLOCK_FILE="/var/run/myapp_cron.pid"exec {LOCK_FD}>"$LOCK_FILE"flock -n "$LOCK_FD" || { echo "Lock held by PID $(cat $LOCK_FILE), exiting" exit 0}echo $$ >&"$LOCK_FD" # write our PID to the lock filetrap "rm -f $LOCK_FILE" EXIT
echo "Running as PID $$"# ... do work ...
# Alternative: lockfile (from procmail)lockfile -1 -r 3 /tmp/my.lock && { # critical section rm -f /tmp/my.lock}Why this approach: File-based locking with flock is atomic at the kernel level and automatically released if the process crashes or is killed, unlike PID file approaches that require manual cleanup on unclean exits.
Answer: Cron captures stdout and emails it to the user (if MAILTO is set), which can flood inboxes. Redirect output to a log file in the crontab entry itself, and use 2>&1 to capture stderr alongside stdout.
xxxxxxxxxx# Redirect stdout and stderr to the same log file0 2 * * * /usr/local/bin/backup.sh >> /var/log/backup.log 2>&1
# Redirect to separate files0 2 * * * /usr/local/bin/backup.sh >> /var/log/backup.log 2>> /var/log/backup_err.log
# Add timestamp to each run0 2 * * * { echo "=== $(date) ==="; /usr/local/bin/backup.sh; } >> /var/log/backup.log 2>&1
# Use logger to send to syslog0 2 * * * /usr/local/bin/backup.sh 2>&1 | logger -t backup-cron
# Discard all output (silent cron)*/5 * * * * /usr/local/bin/heartbeat.sh > /dev/null 2>&1
# Timestamped log with rotation hint0 * * * * /usr/local/bin/check_health.sh >> /var/log/health_$(date +\%Y\%m).log 2>&1
# Using tee to log AND display (useful during testing)0 2 * * * /usr/local/bin/backup.sh 2>&1 | tee -a /var/log/backup.log | logger -t cron-backup
# Suppress email but keep local logMAILTO=""0 2 * * * /usr/local/bin/backup.sh >> /var/log/backup.log 2>&1
# Script with built-in logging (recommended for complex jobs)# backup.shLOG="/var/log/backup.log"log() { echo "$(date '+%Y-%m-%dT%H:%M:%S') $*" | tee -a "$LOG"; }log "INFO: Backup started"pg_dump mydb > /backup/db.sql && log "INFO: Dump complete" \ || { log "ERROR: Dump failed"; exit 1; }log "INFO: Backup finished"Why this approach: Appending with >> preserves history across runs, while 2>&1 before the log path ensures stderr goes to the same file, making log parsing straightforward.
Answer: Cron runs with a minimal environment — typically PATH=/usr/bin:/bin and no user profile loaded. Scripts that work interactively often fail in cron because tools are in /usr/local/bin, environment variables are unset, or shell aliases are unavailable.
xxxxxxxxxx# Problem: cron's default PATH is minimal# Default PATH in cron: /usr/bin:/bin# Interactive shell PATH might include: /usr/local/bin, /opt/homebrew/bin, etc.
# Solution 1: Use absolute paths in crontab0 2 * * * /usr/local/bin/python3 /opt/app/backup.py
# Solution 2: Set PATH at top of crontabSHELL=/bin/bashPATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/opt/homebrew/binMAILTO=[email protected]
0 2 * * * backup.sh # can now find backup.sh in PATH
# Solution 3: Source profile in the script or crontab entry0 2 * * * source /etc/profile && /usr/local/bin/backup.sh0 2 * * * bash -l -c '/usr/local/bin/backup.sh' # login shell
# Common variables cron does NOT set:# HOME — may be / or root's home# USER — not set# LOGNAME — not set# DISPLAY — not set (no X display)
# Set them explicitly in crontabHOME=/home/deployUSER=deploy0 2 * * * /usr/local/bin/backup.sh
# Virtual environment activation0 2 * * * /opt/app/venv/bin/python /opt/app/backup.py
# NVM / pyenv / rbenv — must source explicitly0 2 * * * export NVM_DIR="$HOME/.nvm"; source "$NVM_DIR/nvm.sh"; node /opt/app/script.js
# Debug: test your crontab environment* * * * * env > /tmp/cron_env.txt 2>&1 # capture environment to file
# Best practice: always use absolute paths in scripts called by cronPYTHON=/usr/bin/python3PSQL=/usr/bin/psqlBACKUP_SCRIPT=/usr/local/bin/run_backup.py$PYTHON $BACKUP_SCRIPT --host $DB_HOSTWhy this approach: Using absolute paths in both the crontab entry and within called scripts eliminates PATH-dependent failures and makes the cron job behavior identical regardless of the invoking user's shell configuration.
MAILTO work in cron, and how do you configure email notifications?Answer: Cron emails any stdout/stderr output to the user specified in MAILTO. Set MAILTO="" to disable emails entirely, or point it to an email address for alerting. Many production systems route cron mail through a local MTA to a monitoring system.
xxxxxxxxxx# Set at top of crontab (before any job lines)
# Send output to a specific addressMAILTO=[email protected]
# Send to multiple addressesMAILTO="[email protected],[email protected]"
# Disable all email (silent cron)MAILTO=""
# Per-job: override MAILTO inline (not portable, works in some crons)0 2 * * * MAILTO=[email protected] /usr/local/bin/db_backup.sh
# Only email on failure (wrap the command)0 2 * * * /usr/local/bin/backup.sh > /var/log/backup.log 2>&1 || \ echo "CRON FAILURE: backup.sh failed — see /var/log/backup.log" | \ mail -s "CRON ERROR: backup.sh" [email protected]
# Using sendmail directly for rich notifications0 2 * * * result=$(/usr/local/bin/backup.sh 2>&1); \ [ $? -ne 0 ] && echo "$result" | mail -s "[CRON FAIL] backup.sh" [email protected]
# Check that mail is workingecho "Test cron mail" | mail -s "Cron test" [email protected]
# Check mail queuemailqsudo postfix status # or sendmail, exim, depending on MTA
# Alternative: integrate with monitoring system instead of email0 2 * * * /usr/local/bin/backup.sh && \ curl -s "https://hc-ping.com/your-uuid-here" || \ curl -s "https://hc-ping.com/your-uuid-here/fail"Why this approach: Healthchecks.io-style "ping on success" alerting inverts the model — you get alerted when the job doesn't run rather than when it does, catching failures caused by cron daemon crashes or misconfigured schedules.
crontab and /etc/cron.d/?Answer: System-wide cron jobs in /etc/cron.d/ include a sixth field (the username) between the time fields and the command. This is the standard way to run scheduled jobs as non-root service accounts without modifying individual user crontabs.
xxxxxxxxxx# /etc/cron.d/app-jobs (system-wide cron, includes username field)# Format: min hour dom month dow USER commandSHELL=/bin/bashPATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/binMAILTO=[email protected]
# Run backup as the 'backup' user0 2 * * * backup /usr/local/bin/db_backup.sh >> /var/log/db_backup.log 2>&1
# Run web app maintenance as www-data30 3 * * * www-data /opt/app/manage.py cleanup_sessions >> /var/log/app_cron.log 2>&1
# Run metrics collection as prometheus user*/1 * * * * prometheus /usr/local/bin/collect_metrics.py
# Key differences from user crontab:# 1. Username field present between time and command# 2. Place file in /etc/cron.d/ (no crontab -e needed)# 3. File must be owned by root and not world-writable# 4. File must not have a dot in the namesudo chown root:root /etc/cron.d/app-jobssudo chmod 644 /etc/cron.d/app-jobs
# User crontabs: edit as the specific usersudo crontab -u www-data -esudo crontab -u deploy -l
# Verify cron picks up the file (cron.d)sudo service cron status # Debian/Ubuntusudo systemctl status crond # RHEL/CentOS
# Restrict who can use cron# /etc/cron.allow — whitelist: only listed users can crontab# /etc/cron.deny — blacklist: listed users cannot crontab# If both absent: only root can use cron (on some distros, all users can)echo "deploy" >> /etc/cron.allowecho "nobody" >> /etc/cron.denyWhy this approach: Files in /etc/cron.d/ are version-controllable, can be deployed by configuration management (Ansible, Chef, Puppet), and specify the runtime user explicitly — all of which improve auditability over ad-hoc crontab -e edits.
Answer: Cron failures are almost always environment-related (PATH, env vars) or permission-related. The debugging workflow is: capture the cron environment, replicate it in a shell, then test interactively. Check syslog/journald for cron daemon messages.
xxxxxxxxxx# Step 1: Check if cron daemon logged the job attemptgrep CRON /var/log/syslog | tail -20 # Debian/Ubuntujournalctl -u cron -n 50 # systemd systemsgrep cron /var/log/cron | tail -20 # RHEL/CentOS
# Step 2: Capture the actual cron environment* * * * * env > /tmp/cron_env.txt 2>&1# Then remove that line and inspect /tmp/cron_env.txt
# Step 3: Test script under cron-like environment# Replicate cron's environment manuallyenv -i HOME=/root LOGNAME=root PATH=/usr/bin:/bin \ SHELL=/bin/sh USER=root \ /usr/local/bin/your_script.sh
# Step 4: Run script with full output capture* * * * * /usr/local/bin/debug_script.sh >> /tmp/debug.log 2>&1; \ echo "Exit: $?" >> /tmp/debug.log
# Step 5: Common fixes# Fix 1: Use absolute pathswhich mycommand # find path# In script: /usr/local/bin/mycommand instead of mycommand
# Fix 2: Source profile or activate virtualenvsource /opt/app/venv/bin/activatepython manage.py task
# Fix 3: Fix permissionsls -la /usr/local/bin/your_script.sh # must be executablechmod +x /usr/local/bin/your_script.sh
# Fix 4: Check for tty-dependent code# Scripts using 'read -p' or interactive prompts will hang in cron
# Fix 5: Redirect stdin from /dev/null0 2 * * * /usr/local/bin/script.sh < /dev/null >> /var/log/script.log 2>&1
# Fix 6: Check if crond is enabledsystemctl is-enabled cronsystemctl is-active cronsudo systemctl start cronWhy this approach: Capturing the actual cron environment to a file is the definitive diagnostic step because it reveals the exact PATH, locale, and variable differences that cause environment-dependent failures.
tail -f and multitail to monitor live log files?Answer: tail -f follows a file as new lines are appended, making it essential for watching live application logs. multitail extends this to watch multiple files simultaneously in a split terminal window with optional color highlighting and filtering.
xxxxxxxxxx# Basic tail -f — follow log in real timetail -f /var/log/nginx/access.log
# Show last N lines then followtail -n 100 -f /var/log/nginx/error.log
# Follow multiple files (shows filename prefix)tail -f /var/log/nginx/access.log /var/log/nginx/error.log
# -F (capital) — follow by name, handles log rotation# (reopens file if it's rotated/replaced)tail -F /var/log/app/app.log
# Combine with grep to filter on the flytail -F /var/log/nginx/access.log | grep " 500 "tail -F /var/log/app.log | grep -E "(ERROR|CRITICAL)"
# Combine with awk for structured outputtail -F /var/log/nginx/access.log | awk '{print $1, $7, $9}'
# Highlight errors in colortail -F /var/log/app.log | grep --color=always -E "ERROR|WARN|$"
# multitail — split-screen monitoring of multiple logs# Install: sudo apt install multitailmultitail /var/log/nginx/access.log /var/log/nginx/error.log
# multitail with color schememultitail -ci red /var/log/nginx/error.log \ -ci green /var/log/nginx/access.log
# multitail with grep filtermultitail -I /var/log/app.log -p l ERROR # line-level grep
# Follow multiple app logs across servers (with ssh)multitail -l "ssh web01 'tail -F /var/log/app.log'" \ -l "ssh web02 'tail -F /var/log/app.log'"
# Watch a command's output (like watch but streaming)tail -f /proc/1/fd/1 # stdout of PID 1
# Using less +F as a scrollable tailless +F /var/log/syslog # Ctrl+C to scroll, F to resume followingWhy this approach: tail -F (capital F) handles log rotation transparently by following the filename rather than the file descriptor, preventing the common mistake of watching a rotated-away old file after logrotate runs.
awk to extract useful metrics?Answer: Access logs follow a well-known format. awk field separators and pattern matching extract status codes, request paths, response sizes, and client IPs efficiently in a single pass without loading the whole file into memory.
xxxxxxxxxx# Apache/nginx Combined Log Format:# 192.168.1.1 - - [18/May/2026:12:34:56 +0000] "GET /api/v1/health HTTP/1.1" 200 42 "-" "curl/7.68.0"# $1=ip $4=date $5=time $6=method+path $9=status $10=bytes $11=referer $12=user-agent
# Count requests by HTTP status codeawk '{print $9}' /var/log/nginx/access.log | sort | uniq -c | sort -rn# Output:# 15432 200# 1024 304# 532 404# 87 500
# Top 10 requested URLsawk '{print $7}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head -10
# Top 10 client IP addressesawk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head -10
# Total bytes transferredawk '{sum += $10} END {printf "Total: %.2f MB\n", sum/1024/1024}' /var/log/nginx/access.log
# Average response sizeawk '{sum += $10; count++} END {printf "Avg: %.0f bytes\n", sum/count}' /var/log/nginx/access.log
# All 5xx errors with path and IPawk '$9 ~ /^5/ {print $1, $9, $7}' /var/log/nginx/access.log
# 4xx errors onlyawk '$9 >= 400 && $9 < 500 {print}' /var/log/nginx/access.log
# Requests per hour (extract hour from timestamp)awk '{split($4,a,":"); print a[2]}' /var/log/nginx/access.log | \ sort | uniq -c | sort -k2n
# Filter by date rangeawk '$4 >= "[18/May/2026:10:00" && $4 <= "[18/May/2026:11:00"' \ /var/log/nginx/access.log | wc -l
# Unique visitors per dayawk '{print substr($4,2,11), $1}' /var/log/nginx/access.log | \ sort -u | awk '{print $1}' | uniq -c
# Slow requests (if log includes response time — $NF in some configs)awk '$NF > 1.0 {print $NF, $7}' /var/log/nginx/access.log | sort -rn | head -20
# Path-specific error rateawk '/\/api\/v1\/upload/ {total++; if($9>=500) err++} \ END {printf "Error rate: %.1f%%\n", (err/total)*100}' \ /var/log/nginx/access.logWhy this approach: Single-pass awk processing avoids loading the entire log into memory, making it practical for multi-gigabyte access logs where python or perl approaches would require careful streaming implementation.
Answer: Group log entries by the minute portion of the timestamp and count 5xx responses per minute using awk. This is useful for detecting error rate spikes in post-incident analysis or real-time monitoring pipelines.
xxxxxxxxxx# Count 5xx errors per minuteawk '$9 >= 500 && $9 < 600 { # $4 format: [18/May/2026:12:34:56 split($4, t, ":") minute = substr(t[1],2) ":" t[2] ":" t[3] counts[minute]++}END { for (m in counts) print m, counts[m]}' /var/log/nginx/access.log | sort
# Count ALL requests and 5xx errors per minute (for rate calculation)awk '{ split($4, t, ":") minute = substr(t[1],2) ":" t[2] ":" t[3] total[minute]++ if ($9 >= 500 && $9 < 600) errors[minute]++}END { print "minute\ttotal\terrors\terror_pct" for (m in total) { e = (m in errors) ? errors[m] : 0 pct = (e / total[m]) * 100 printf "%s\t%d\t%d\t%.1f%%\n", m, total[m], e, pct }}' /var/log/nginx/access.log | sort
# Tail live log and count 5xx per minute in real timetail -F /var/log/nginx/access.log | \awk '$9 >= 500 { split($4, t, ":") key = substr(t[1],2) ":" t[2] ":" t[3] counts[key]++ total[key]++ # Print running count every time minute changes if (key != prev_key && prev_key != "") { printf "%s: %d 5xx errors\n", prev_key, counts[prev_key] } prev_key = key}'
# Summarize error rate from last 60 minutesawk -v cutoff="$(date -d '60 minutes ago' '+%d/%b/%Y:%H:%M' 2>/dev/null \ || date -v-60M '+%d/%b/%Y:%H:%M')" \'$4 > "[" cutoff { total++ if ($9 >= 500) errors++}END { printf "Last 60 min: %d total, %d errors (%.1f%%)\n", total, errors, (errors/total)*100}' /var/log/nginx/access.log
# Using Python for more complex analysispython3 - <<'EOF'from collections import defaultdictimport re
pattern = re.compile( r'(?P<ip>\S+) \S+ \S+ \[(?P<time>[^\]]+)\] ' r'"[^"]*" (?P<status>\d{3})')counts = defaultdict(lambda: {"total": 0, "errors": 0})
with open("/var/log/nginx/access.log") as f: for line in f: m = pattern.match(line) if m: minute = m.group("time")[:17] # DD/Mon/YYYY:HH:MM counts[minute]["total"] += 1 if int(m.group("status")) >= 500: counts[minute]["errors"] += 1
for minute in sorted(counts): d = counts[minute] pct = d["errors"] / d["total"] * 100 if pct > 0: print(f"{minute}: {d['errors']}/{d['total']} 5xx ({pct:.1f}%)")EOFWhy this approach: Parsing the minute from the log timestamp with awk's split() avoids calling date for every line, keeping the per-line processing time O(1) and maintaining throughput on high-traffic logs.
Answer: A monitoring script reads a metric, compares it to a threshold, and sends an alert (email, Slack, PagerDuty) if exceeded. Implement hysteresis (alert-on-trigger, resolve-on-clear) using a state file to avoid alert storms.
xxxxxxxxxx# alert_on_error_rate.sh — check 5xx rate and alert if above threshold
LOG_FILE="/var/log/nginx/access.log"THRESHOLD_PCT=5 # alert if >5% of requests are 5xxWINDOW_MINUTES=5 # look back 5 minutesALERT_EMAIL="[email protected]"SLACK_WEBHOOK="https://hooks.slack.com/services/YOUR/WEBHOOK/URL"STATE_FILE="/var/run/nginx_error_alert.state"
# Calculate error rate for the last N minutescalculate_error_rate() { local cutoff cutoff=$(date -d "${WINDOW_MINUTES} minutes ago" '+%d/%b/%Y:%H:%M' 2>/dev/null \ || date -v"-${WINDOW_MINUTES}M" '+%d/%b/%Y:%H:%M')
awk -v cutoff="[$cutoff" ' $4 >= cutoff { total++ if ($9 >= 500 && $9 < 600) errors++ } END { if (total == 0) { print "0 0 0"; exit } printf "%d %d %.2f\n", total, errors, (errors/total)*100 }' "$LOG_FILE"}
send_slack_alert() { local message="$1" curl -s -X POST "$SLACK_WEBHOOK" \ -H "Content-Type: application/json" \ -d "{\"text\": \"$message\"}" > /dev/null}
send_email_alert() { local subject="$1" local body="$2" echo "$body" | mail -s "$subject" "$ALERT_EMAIL"}
# Read current state (FIRING or OK)current_state=$(cat "$STATE_FILE" 2>/dev/null || echo "OK")
# Get metricsread -r total errors rate < <(calculate_error_rate)
echo "$(date '+%Y-%m-%dT%H:%M:%S') total=$total errors=$errors rate=${rate}%"
if (( $(echo "$rate > $THRESHOLD_PCT" | bc -l) )); then if [[ "$current_state" != "FIRING" ]]; then # Transition OK -> FIRING msg="ALERT: nginx 5xx rate is ${rate}% (${errors}/${total} requests in last ${WINDOW_MINUTES}min). Threshold: ${THRESHOLD_PCT}%" send_slack_alert ":fire: $msg" send_email_alert "[ALERT] nginx high error rate: ${rate}%" "$msg" echo "FIRING" > "$STATE_FILE" echo "Alert fired: $msg" else echo "Already firing. Rate: ${rate}%" fielse if [[ "$current_state" == "FIRING" ]]; then # Transition FIRING -> OK (resolved) msg="RESOLVED: nginx 5xx rate back to normal: ${rate}% (${errors}/${total} in last ${WINDOW_MINUTES}min)" send_slack_alert ":white_check_mark: $msg" send_email_alert "[RESOLVED] nginx error rate normalized" "$msg" echo "OK" > "$STATE_FILE" echo "Alert resolved: $msg" else echo "OK. Rate: ${rate}%" fifixxxxxxxxxx# Add to crontab: check every minute* * * * * /usr/local/bin/alert_on_error_rate.sh >> /var/log/alert_check.log 2>&1Why this approach: The state file implements hysteresis — once an alert fires, subsequent checks only re-alert if it resolves and re-triggers, preventing a noisy stream of identical alerts during a sustained incident.
logrotate configuration file?Answer: logrotate automates log file rotation, compression, and cleanup. Write a config file in /etc/logrotate.d/ for each application. Key directives: rotate, compress, dateext, postrotate, create, and notifempty.
xxxxxxxxxx# /etc/logrotate.d/myapp/var/log/myapp/*.log {# Rotation frequencydaily# Keep 30 rotated copiesrotate 30# Compress rotated files with gzipcompress# Delay compression by one rotation cycle (so current+1 stays uncompressed)# Useful when the app may still have the old file opendelaycompress# Append date to rotated filename: app.log-20260518.gzdateextdateformat -%Y%m%d# Don't rotate if file is emptynotifempty# Don't error if log file is missingmissingok# Create new empty log file after rotation (mode owner group)create 0644 www-data www-data# Force rotation even if file hasn't grown# (useful combined with size threshold)# size 100M — rotate when file exceeds 100MB (overrides frequency)# Copy and truncate (for apps that can't be signaled)# copytruncate# Run commands after rotationpostrotate# Signal nginx to reopen log files[ -f /var/run/nginx.pid ] && kill -USR1 $(cat /var/run/nginx.pid) || trueendscript# Run before rotation (e.g., flush buffers)prerotate/usr/local/bin/flush_app_buffers.shendscript# Share postrotate script across all matched files (run once)sharedscripts}# Nginx access and error logs/var/log/nginx/access.log /var/log/nginx/error.log {weeklyrotate 52compressdelaycompressdateextmissingoknotifemptycreate 0640 www-data admsharedscriptspostrotateif [ -f /var/run/nginx.pid ]; thenkill -USR1 $(cat /var/run/nginx.pid)fiendscript}
xxxxxxxxxx# Test config without actually rotatinglogrotate --debug /etc/logrotate.d/myapp
# Force rotation now (bypass frequency check)logrotate --force /etc/logrotate.d/myapp
# Check logrotate status (last run times)cat /var/lib/logrotate/status
# Run all logrotate configs (as cron does)logrotate /etc/logrotate.conf
# Verify config syntaxlogrotate --debug --verbose /etc/logrotate.confWhy this approach: sharedscripts with delaycompress is the correct combination for most web servers: the postrotate signal is sent once (not once per file), and the just-rotated file stays uncompressed for one cycle in case the server still has it open.
Answer: Use df to check disk usage, compare against a threshold, and use mail or a curl webhook to notify. Track alert state in a file to implement hysteresis and prevent repeat emails.
xxxxxxxxxx# disk_alert.sh — monitor disk usage and alert on threshold breach
THRESHOLD=80 # alert when usage exceeds 80%CRITICAL_THRESHOLD=90 # critical alertEMAIL="[email protected]"HOSTNAME=$(hostname -f)STATE_DIR="/var/run/disk_alerts"mkdir -p "$STATE_DIR"
send_alert() { local partition="$1" local usage="$2" local level="$3" # WARNING or CRITICAL local subject="[$level] $HOSTNAME: Disk $partition at ${usage}%" local body body=$(cat <<EOFDisk usage alert on $HOSTNAME
Partition: $partitionUsage: ${usage}%Threshold: ${THRESHOLD}% (warning), ${CRITICAL_THRESHOLD}% (critical)Time: $(date '+%Y-%m-%d %H:%M:%S')
Full disk report:$(df -h)
Top 10 directories by size in /:$(du -sh /* 2>/dev/null | sort -rh | head -10)EOF) echo "$body" | mail -s "$subject" "$EMAIL" echo "$(date '+%Y-%m-%dT%H:%M:%S') ALERT sent: $subject"}
# Parse df output (skip header, handle % sign)df -P | grep -v "^Filesystem" | while read -r fs size used avail pct mount; do # pct is like "75%", strip the % usage="${pct%%%}"
state_file="$STATE_DIR/$(echo "$mount" | tr '/' '_').state" current_state=$(cat "$state_file" 2>/dev/null || echo "OK")
if [[ "$usage" -ge "$CRITICAL_THRESHOLD" ]]; then if [[ "$current_state" != "CRITICAL" ]]; then send_alert "$mount" "$usage" "CRITICAL" echo "CRITICAL" > "$state_file" fi elif [[ "$usage" -ge "$THRESHOLD" ]]; then if [[ "$current_state" != "WARNING" && "$current_state" != "CRITICAL" ]]; then send_alert "$mount" "$usage" "WARNING" echo "WARNING" > "$state_file" fi else # Below threshold — clear state and send resolved if needed if [[ "$current_state" != "OK" ]]; then echo "$(date '+%Y-%m-%dT%H:%M:%S') RESOLVED: $mount now at ${usage}%" echo "Disk $mount on $HOSTNAME has recovered to ${usage}% (was $current_state)" | \ mail -s "[RESOLVED] $HOSTNAME: Disk $mount recovered" "$EMAIL" echo "OK" > "$state_file" fi fidonexxxxxxxxxx# Crontab entry — check every 15 minutes*/15 * * * * /usr/local/bin/disk_alert.sh >> /var/log/disk_alert.log 2>&1
# Alternative: use Slack webhook instead of emailsend_slack() { local msg="$1" curl -s -X POST "https://hooks.slack.com/services/YOUR/WEBHOOK/URL" \ -H "Content-Type: application/json" \ -d "{\"text\": \"$msg\"}" > /dev/null}
# Test the script manuallybash /usr/local/bin/disk_alert.sh
# Check state filesls -la /var/run/disk_alerts/cat /var/run/disk_alerts/__root.stateWhy this approach: Using per-partition state files with hysteresis prevents alert flooding when disk usage hovers near the threshold, and enables a clear resolved notification when the issue is remediated.
awk, sort, and uniq to generate a full access log summary report?Answer: Chain awk, sort, and uniq -c to aggregate log data into a readable report. A wrapper script that runs multiple awk passes produces a comprehensive summary useful for daily reporting or incident analysis.
xxxxxxxxxx# log_summary.sh — generate access log summary report
LOG="${1:-/var/log/nginx/access.log}"REPORT_DATE=$(date '+%Y-%m-%d %H:%M:%S')
if [[ ! -f "$LOG" ]]; then echo "Error: Log file not found: $LOG" >&2 exit 1fi
TOTAL=$(wc -l < "$LOG")
cat <<EOF============================================================= NGINX ACCESS LOG SUMMARY File: $LOG Date: $REPORT_DATE Lines: $TOTAL=============================================================
--- HTTP Status Code Distribution ---EOF
awk '{print $9}' "$LOG" | grep -E '^[0-9]{3}$' | \ sort | uniq -c | sort -rn | \ awk '{printf " %6d %s\n", $1, $2}'
echo ""echo "--- Top 15 Requested Paths ---"awk '{print $7}' "$LOG" | \ sed 's/?.*$//' | # strip query strings sort | uniq -c | sort -rn | head -15 | \ awk '{printf " %6d %s\n", $1, $2}'
echo ""echo "--- Top 15 Client IPs ---"awk '{print $1}' "$LOG" | \ sort | uniq -c | sort -rn | head -15 | \ awk '{printf " %6d %s\n", $1, $2}'
echo ""echo "--- Requests Per Hour ---"awk '{ split($4, t, ":") hour = substr(t[1],2,11) " " t[2] ":00" count[hour]++}END { for (h in count) print h, count[h]}' "$LOG" | sort | awk '{printf " %s %d\n", $1, $NF}'
echo ""echo "--- 5xx Errors (All) ---"awk '$9 >= 500 && $9 < 600 { printf " %s %s %s %s\n", $1, $9, $7, $4}' "$LOG" | head -20
echo ""echo "--- Bandwidth by Status Code ---"awk '$10 ~ /^[0-9]+$/ { bytes[$9] += $10 count[$9]++}END { printf " %-8s %-12s %-10s\n", "Status", "Total Bytes", "Requests" for (s in bytes) printf " %-8s %-12d %-10d\n", s, bytes[s], count[s]}' "$LOG" | sort -k1
echo ""echo "--- 5xx Error Rate Summary ---"awk '{ total++ if ($9 >= 500 && $9 < 600) errors++}END { printf " Total requests: %d\n", total printf " 5xx errors: %d\n", errors printf " Error rate: %.2f%%\n", (errors/total)*100}' "$LOG"
echo ""echo "=== END OF REPORT ==="xxxxxxxxxx# Make it executable and add to cron for daily reportschmod +x /usr/local/bin/log_summary.sh
# Cron: daily report emailed at 7 AM0 7 * * * /usr/local/bin/log_summary.sh /var/log/nginx/access.log | \ mail -s "Daily Nginx Report $(date '+%Y-%m-%d')" [email protected]
# Run manually/usr/local/bin/log_summary.sh /var/log/nginx/access.log
# Run against compressed logzcat /var/log/nginx/access.log.1.gz | \ awk '{print $9}' | sort | uniq -c | sort -rnWhy this approach: Stripping query strings with sed before uniq -c aggregates /api/users?page=1 and /api/users?page=2 into a single /api/users entry, giving a clean view of endpoint popularity rather than fragmenting counts across parameter variations.
Answer: Health-check scripts must communicate severity through exit codes so orchestration tools (nagios, k8s liveness probes, systemd) can take action automatically. Exit 0 = healthy, 1 = degraded but operational, 2 = critical failure.
xxxxxxxxxx# health_check.sh — returns 0=ok, 1=warn, 2=critset -euo pipefail
WARN_DISK=80CRIT_DISK=90WARN_LOAD=4.0CRIT_LOAD=8.0
status=0 # assume OKmessages=()
# ---- Disk check ----disk_pct=$(df / | awk 'NR==2 {gsub(/%/,""); print $5}')if (( disk_pct >= CRIT_DISK )); then messages+=("CRIT: disk at ${disk_pct}%") status=2elif (( disk_pct >= WARN_DISK )); then messages+=("WARN: disk at ${disk_pct}%") [[ $status -lt 1 ]] && status=1fi
# ---- Load average (1-min) ----load=$(awk '{print $1}' /proc/loadavg 2>/dev/null || sysctl -n vm.loadavg | awk '{print $2}')if awk "BEGIN{exit !($load >= $CRIT_LOAD)}"; then messages+=("CRIT: load ${load}") status=2elif awk "BEGIN{exit !($load >= $WARN_LOAD)}"; then messages+=("WARN: load ${load}") [[ $status -lt 1 ]] && status=1fi
# ---- Service check ----if ! systemctl is-active --quiet nginx 2>/dev/null; then messages+=("CRIT: nginx is down") status=2fi
# ---- Output ----if [[ ${#messages[@]} -eq 0 ]]; then echo "OK: all checks passed"else printf '%s\n' "${messages[@]}"fi
exit $statusWhy this approach: Using a numeric accumulator for status guarantees the worst severity wins, and printing before exit lets monitoring systems capture both the human-readable message and the machine-readable code.
Answer: psutil exposes cross-platform system metrics with a clean API; combining it with a threshold dictionary makes the logic easy to extend.
xxxxxxxxxx#!/usr/bin/env python3"""monitor.py — emit alerts when thresholds are breached."""import sysimport psutil
THRESHOLDS = { "cpu_warn": 70.0, "cpu_crit": 90.0, "mem_warn": 75.0, "mem_crit": 90.0, "disk_warn": 80.0, "disk_crit": 95.0,}
def check_cpu(): pct = psutil.cpu_percent(interval=1) if pct >= THRESHOLDS["cpu_crit"]: return 2, f"CPU {pct:.1f}% >= crit {THRESHOLDS['cpu_crit']}%" if pct >= THRESHOLDS["cpu_warn"]: return 1, f"CPU {pct:.1f}% >= warn {THRESHOLDS['cpu_warn']}%" return 0, f"CPU ok ({pct:.1f}%)"
def check_memory(): mem = psutil.virtual_memory() pct = mem.percent if pct >= THRESHOLDS["mem_crit"]: return 2, f"MEM {pct:.1f}% >= crit" if pct >= THRESHOLDS["mem_warn"]: return 1, f"MEM {pct:.1f}% >= warn" return 0, f"MEM ok ({pct:.1f}%)"
def check_disk(path="/"): disk = psutil.disk_usage(path) pct = disk.percent if pct >= THRESHOLDS["disk_crit"]: return 2, f"DISK {path} {pct:.1f}% >= crit" if pct >= THRESHOLDS["disk_warn"]: return 1, f"DISK {path} {pct:.1f}% >= warn" return 0, f"DISK ok ({pct:.1f}%)"
def main(): worst = 0 for check in (check_cpu, check_memory, check_disk): level, msg = check() tag = ["OK", "WARN", "CRIT"][level] print(f"{tag}: {msg}") worst = max(worst, level) sys.exit(worst)
if __name__ == "__main__": main()Why this approach: psutil.cpu_percent(interval=1) blocks for one second to get an accurate measurement rather than a cached snapshot, making the value meaningful for alerting.
Answer: Slack's Incoming Webhooks accept a JSON POST; formatting the payload correctly and checking the response prevents silent failures.
xxxxxxxxxx# slack_alert.sh WEBHOOK_URL MESSAGE [SEVERITY]set -euo pipefail
WEBHOOK="${1:?Usage: $0 <webhook_url> <message> [ok|warn|crit]}"MESSAGE="${2:?message required}"SEVERITY="${3:-info}"
# Color: good=green, warning=yellow, danger=redcase "$SEVERITY" in ok) COLOR="good" ;; warn) COLOR="warning" ;; crit) COLOR="danger" ;; *) COLOR="#439FE0" ;;esac
HOSTNAME=$(hostname -s)TIMESTAMP=$(date -u '+%Y-%m-%d %H:%M:%S UTC')
PAYLOAD=$(cat <<EOF{ "attachments": [{ "color": "${COLOR}", "title": "Alert from ${HOSTNAME}", "text": "${MESSAGE}", "footer": "${TIMESTAMP}", "mrkdwn_in": ["text"] }]}EOF)
HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" \ -X POST \ -H 'Content-Type: application/json' \ -d "$PAYLOAD" \ "$WEBHOOK")
if [[ "$HTTP_CODE" != "200" ]]; then echo "ERROR: Slack returned HTTP ${HTTP_CODE}" >&2 exit 1fi
echo "Alert sent (HTTP ${HTTP_CODE})"Why this approach: Capturing the HTTP status code with -w "%{http_code}" and -o /dev/null lets the script fail loudly if Slack rejects the payload without printing the raw HTML error page to stdout.
Answer: tail -F follows rotations; a while read loop processes each new line in real time without polling overhead.
xxxxxxxxxx# log_watcher.sh LOGFILE PATTERN WEBHOOKset -euo pipefail
LOGFILE="${1:?logfile required}"PATTERN="${2:?pattern required}"WEBHOOK="${3:?webhook required}"
COOLDOWN=60 # seconds between repeated alerts for the same matchLAST_ALERT=0
send_slack() { local msg="$1" curl -s -X POST -H 'Content-Type: application/json' \ -d "{\"text\": \"$(hostname -s): ${msg}\"}" \ "$WEBHOOK" > /dev/null}
echo "Watching ${LOGFILE} for pattern '${PATTERN}'"
tail -F "$LOGFILE" | while IFS= read -r line; do if echo "$line" | grep -qE "$PATTERN"; then NOW=$(date +%s) ELAPSED=$(( NOW - LAST_ALERT )) if (( ELAPSED >= COOLDOWN )); then send_slack "MATCH in ${LOGFILE}: ${line}" LAST_ALERT=$NOW echo "[$(date -u)] ALERTED: $line" fi fidoneWhy this approach: The cooldown gate prevents alert storms when an error floods the log, ensuring on-call engineers get one notification per minute rather than thousands.
Answer: Manual rotation timestamps the current log, compresses it, and truncates (not removes) the live file so running processes keep their open file descriptor.
xxxxxxxxxx# rotate_log.sh LOGFILE [KEEP_DAYS]set -euo pipefail
LOGFILE="${1:?logfile required}"KEEP_DAYS="${2:-30}"
if [[ ! -f "$LOGFILE" ]]; then echo "Log file not found: $LOGFILE" >&2 exit 1fi
TIMESTAMP=$(date '+%Y%m%d_%H%M%S')ARCHIVE="${LOGFILE}.${TIMESTAMP}.gz"
# Compress a copy, then truncate the live file atomicallycp "$LOGFILE" "${LOGFILE}.tmp"> "$LOGFILE" # truncate in place; PID keeps fd opengzip -9 "${LOGFILE}.tmp"mv "${LOGFILE}.tmp.gz" "$ARCHIVE"
echo "Rotated: $ARCHIVE"
# Remove archives older than KEEP_DAYSfind "$(dirname "$LOGFILE")" -maxdepth 1 \ -name "$(basename "$LOGFILE").*.gz" \ -mtime "+${KEEP_DAYS}" \ -delete \ -print | while IFS= read -r f; do echo "Deleted old archive: $f"doneWhy this approach: Truncating with > "$LOGFILE" preserves the inode so any process writing to the file by file descriptor continues without error, unlike rm followed by touch which would cause writes to go nowhere.
Answer: JSON logs have one JSON object per line; reading line by line and calling json.loads handles multi-GB files without loading everything into memory.
xxxxxxxxxx#!/usr/bin/env python3"""json_log_parser.py — summarize error counts from JSON access logs."""import jsonimport sysfrom collections import Counterfrom pathlib import Path
def parse_log(path: str, level_field: str = "level", msg_field: str = "message"): counts: Counter = Counter() errors = []
with open(path) as fh: for lineno, raw in enumerate(fh, 1): raw = raw.strip() if not raw: continue try: record = json.loads(raw) except json.JSONDecodeError as exc: print(f"SKIP line {lineno}: {exc}", file=sys.stderr) continue
level = record.get(level_field, "UNKNOWN").upper() counts[level] += 1
if level in ("ERROR", "CRITICAL", "FATAL"): errors.append({ "line": lineno, "level": level, "message": record.get(msg_field, ""), "timestamp": record.get("timestamp", record.get("ts", "")), })
return counts, errors
def main(): logfile = sys.argv[1] if len(sys.argv) > 1 else "/var/log/app/app.log" counts, errors = parse_log(logfile)
print("=== Level Summary ===") for lvl, n in sorted(counts.items()): print(f" {lvl:<10} {n:>6}")
print(f"\n=== Recent Errors ({min(len(errors), 10)} of {len(errors)}) ===") for e in errors[-10:]: print(f" [{e['timestamp']}] {e['level']}: {e['message']}")
if __name__ == "__main__": main()Why this approach: Processing line by line keeps memory constant regardless of log size, and catching json.JSONDecodeError per line means a single malformed entry does not abort analysis of the entire file.
Answer: Collect all matching numeric values into a list, sort it, then index by percentile position; statistics module handles the math cleanly.
xxxxxxxxxx#!/usr/bin/env python3"""latency_stats.py — compute p50/p95/p99 from nginx-style access logs."""import reimport sysimport statistics
# Example log line: 127.0.0.1 - - [01/Jan/2026] "GET /api" 200 0.342LATENCY_RE = re.compile(r'"[A-Z]+ \S+ HTTP/\S+" \d{3} (\d+\.\d+)')
def percentile(data: list, pct: float) -> float: """Return the pct-th percentile (0-100) of sorted data.""" if not data: return 0.0 k = (len(data) - 1) * pct / 100 lo, hi = int(k), min(int(k) + 1, len(data) - 1) return data[lo] + (data[hi] - data[lo]) * (k - lo)
def main(): logfile = sys.argv[1] if len(sys.argv) > 1 else "-" fh = open(logfile) if logfile != "-" else sys.stdin
latencies = [] with fh: for line in fh: m = LATENCY_RE.search(line) if m: latencies.append(float(m.group(1)))
if not latencies: print("No latency data found.") sys.exit(1)
latencies.sort() print(f"Requests : {len(latencies)}") print(f"Mean : {statistics.mean(latencies)*1000:.1f} ms") print(f"p50 : {percentile(latencies, 50)*1000:.1f} ms") print(f"p95 : {percentile(latencies, 95)*1000:.1f} ms") print(f"p99 : {percentile(latencies, 99)*1000:.1f} ms") print(f"Max : {max(latencies)*1000:.1f} ms")
if __name__ == "__main__": main()Why this approach: Sorting once and indexing by fractional position is O(n log n) and handles any sample size; using statistics.mean avoids floating-point accumulation error from a hand-rolled sum.
Answer: multitail or a simple Python threading approach lets you watch several files in parallel with source labels—critical for correlating events across services.
xxxxxxxxxx#!/usr/bin/env python3"""multi_tail.py — tail multiple files concurrently with source label."""import sysimport timeimport threadingfrom pathlib import Path
def tail_file(path: str, label: str, interval: float = 0.1): """Continuously read new lines appended to path.""" with open(path) as fh: fh.seek(0, 2) # jump to end while True: line = fh.readline() if line: print(f"[{label}] {line}", end="", flush=True) else: time.sleep(interval)
def main(): if len(sys.argv) < 2: print("Usage: multi_tail.py FILE [FILE ...]") sys.exit(1)
threads = [] for filepath in sys.argv[1:]: label = Path(filepath).name t = threading.Thread(target=tail_file, args=(filepath, label), daemon=True) t.start() threads.append(t)
# Block main thread until Ctrl-C try: while True: time.sleep(1) except KeyboardInterrupt: print("\nStopped.")
if __name__ == "__main__": main()Why this approach: Daemon threads automatically die when the main thread exits, so KeyboardInterrupt cleanly terminates all watchers without needing explicit thread joins or stop events.
Answer: Auth headers are passed with -H; Bearer tokens go in the Authorization header directly, while Basic auth can use -u (curl handles base64 encoding) or an explicit header.
xxxxxxxxxx# ---- Bearer token ----TOKEN="eyJhbGciOiJSUzI1NiJ9..."
curl -sS \ -H "Authorization: Bearer ${TOKEN}" \ -H "Accept: application/json" \ "https://api.example.com/v1/users"
# ---- Basic auth (curl encodes automatically) ----curl -sS \ -u "username:password" \ "https://api.example.com/v1/status"
# ---- Basic auth as explicit header ----ENCODED=$(printf '%s:%s' "username" "password" | base64)curl -sS \ -H "Authorization: Basic ${ENCODED}" \ "https://api.example.com/v1/status"
# ---- API Key in custom header ----curl -sS \ -H "X-Api-Key: ${API_KEY}" \ "https://api.example.com/v1/data"
# ---- Store credentials in ~/.netrc to avoid shell history exposure ----# machine api.example.com login myuser password mypasscurl -sS --netrc "https://api.example.com/v1/status"Why this approach: Using -u user:pass keeps credentials out of the Authorization header construction in your script and lets curl handle encoding, but --netrc is best for production scripts where secrets must not appear in process arguments.
Answer: Set Content-Type: application/json and pass the body with -d; use a heredoc or a file for complex payloads to avoid quoting nightmares.
xxxxxxxxxxTOKEN="${API_TOKEN:?set API_TOKEN}"BASE="https://api.example.com"
# ---- Simple inline JSON ----curl -sS -X POST \ -H "Authorization: Bearer ${TOKEN}" \ -H "Content-Type: application/json" \ -d '{"name":"deploy-job","env":"prod"}' \ "${BASE}/v1/jobs"
# ---- JSON from variable (safe quoting) ----NAME="my-service"VERSION="1.2.3"PAYLOAD=$(printf '{"name":"%s","version":"%s"}' "$NAME" "$VERSION")
curl -sS -X POST \ -H "Content-Type: application/json" \ -H "Authorization: Bearer ${TOKEN}" \ -d "$PAYLOAD" \ "${BASE}/v1/deployments"
# ---- JSON from a file ----curl -sS -X POST \ -H "Content-Type: application/json" \ -H "Authorization: Bearer ${TOKEN}" \ -d @payload.json \ "${BASE}/v1/deployments"
# ---- PATCH / PUT ----curl -sS -X PATCH \ -H "Content-Type: application/json" \ -H "Authorization: Bearer ${TOKEN}" \ -d '{"status":"active"}' \ "${BASE}/v1/jobs/42"Why this approach: Passing complex JSON as a file with -d @filename eliminates shell-escaping issues and keeps the script readable when the payload has multiple fields or nested objects.
Answer: --write-out with %{http_code} appends metadata after the body; redirect body to a temp file to separate the two.
xxxxxxxxxx# api_call.sh — separate HTTP status from bodyset -euo pipefail
URL="${1:?url required}"TOKEN="${API_TOKEN:-}"
TMPFILE=$(mktemp)trap 'rm -f "$TMPFILE"' EXIT
HTTP_CODE=$(curl -sS \ -H "Authorization: Bearer ${TOKEN}" \ -H "Accept: application/json" \ -o "$TMPFILE" \ -w "%{http_code}" \ "$URL")
BODY=$(cat "$TMPFILE")
echo "HTTP Status: ${HTTP_CODE}"
case "$HTTP_CODE" in 2??) echo "SUCCESS" echo "$BODY" | python3 -m json.tool 2>/dev/null || echo "$BODY" ;; 401) echo "UNAUTHORIZED — check your token" >&2 exit 2 ;; 404) echo "NOT FOUND: ${URL}" >&2 exit 3 ;; 5??) echo "SERVER ERROR (${HTTP_CODE}): ${BODY}" >&2 exit 4 ;; *) echo "UNEXPECTED STATUS ${HTTP_CODE}: ${BODY}" >&2 exit 5 ;;esacWhy this approach: Writing the body to a temp file and status to stdout via -w "%{http_code}" cleanly separates concerns so the caller can branch on the numeric code without trying to parse a JSON body that may contain the number 200.
Answer: A loop with sleep $((base ** attempt)) (or approximated with bit-shift) retries transient failures while not hammering a struggling service.
xxxxxxxxxx# retry.sh — run a command with exponential backoffset -euo pipefail
MAX_ATTEMPTS=5BASE_DELAY=1 # secondsMAX_DELAY=60
retry() { local attempt=1 local delay=$BASE_DELAY
while true; do if "$@"; then return 0 fi
if (( attempt >= MAX_ATTEMPTS )); then echo "FATAL: command failed after ${MAX_ATTEMPTS} attempts: $*" >&2 return 1 fi
# Cap delay (( delay > MAX_DELAY )) && delay=$MAX_DELAY
# Add jitter: random 0..delay seconds JITTER=$(( RANDOM % (delay + 1) ))
echo "Attempt ${attempt}/${MAX_ATTEMPTS} failed. Retrying in $((delay + JITTER))s..." >&2 sleep $(( delay + JITTER ))
(( attempt++ )) (( delay *= 2 )) done}
# Usage examplesretry curl -sS -f "https://api.example.com/health"retry rsync -avz source/ user@host:/dest/Why this approach: Jitter (random fraction of the backoff interval) prevents the thundering-herd problem where every retry client wakes up simultaneously after the same sleep duration and overwhelms the recovering service.
Answer: tenacity decorators declaratively attach retry logic to any function, keeping business logic clean while handling jitter, wait strategies, and stop conditions.
xxxxxxxxxx#!/usr/bin/env python3"""retry_requests.py — resilient HTTP calls with tenacity."""import requestsfrom tenacity import ( retry, stop_after_attempt, wait_exponential_jitter, retry_if_exception_type, before_sleep_log, RetryError,)import logging
logging.basicConfig(level=logging.INFO)log = logging.getLogger(__name__)
RETRYABLE = ( requests.exceptions.ConnectionError, requests.exceptions.Timeout, requests.exceptions.HTTPError, # filtered below)
def is_retryable_response(exc: BaseException) -> bool: if isinstance(exc, requests.exceptions.HTTPError): code = exc.response.status_code if exc.response is not None else 0 return code in (429, 500, 502, 503, 504) return isinstance(exc, (requests.exceptions.ConnectionError, requests.exceptions.Timeout))
( retry=retry_if_exception_type(RETRYABLE), wait=wait_exponential_jitter(initial=1, max=60, jitter=2), stop=stop_after_attempt(5), before_sleep=before_sleep_log(log, logging.WARNING), reraise=True,)def get_with_retry(url: str, **kwargs) -> requests.Response: resp = requests.get(url, timeout=10, **kwargs) resp.raise_for_status() # raises HTTPError on 4xx/5xx return resp
def main(): try: resp = get_with_retry( "https://api.example.com/data", headers={"Authorization": "Bearer token123"}, ) print(resp.json()) except RetryError as e: log.error("All retries exhausted: %s", e) raise SystemExit(1)
if __name__ == "__main__": main()Why this approach: wait_exponential_jitter combines exponential growth with random jitter in one parameter, and before_sleep_log logs each retry automatically so operators can see how many retries occurred without adding manual print statements.
Answer: jq transforms JSON inline in pipelines; mastering a handful of primitives covers 90% of real-world API response processing.
xxxxxxxxxx# ---- Setup: assume API returns this ----# {"users":[{"id":1,"name":"alice","active":true},# {"id":2,"name":"bob","active":false}]}
# Extract array of namescurl -s api/users | jq '.users[].name'# "alice"# "bob"
# Get array as compact JSONcurl -s api/users | jq '[.users[].name]'
# Select only active userscurl -s api/users | jq '[.users[] | select(.active == true)]'
# Map to id:name objectcurl -s api/users | jq '[.users[] | {id, name}]'
# Pluck a single field by keycurl -s api/user/1 | jq -r '.name' # -r = raw string, no quotes
# Keys of an objectecho '{"a":1,"b":2}' | jq 'keys'# ["a","b"]
# Filter by string patterncurl -s api/services | jq '[.[] | select(.name | startswith("prod-"))]'
# Compute sum of a numeric fieldcurl -s api/jobs | jq '[.[].duration] | add'
# Construct new object from fieldscurl -s api/deployments | jq '.[] | {service:.name, env:.environment, ver:.version}'
# Null-safe field access with alternativecurl -s api/items | jq '.[] | .tags // "untagged"'
# Flatten nested arrayecho '[[1,2],[3,4]]' | jq 'flatten'
# Sort and uniquecurl -s api/tags | jq '[.[].label] | unique | sort'
# Index an array by fieldcurl -s api/users | jq 'INDEX(.users[]; .id)'# {"1":{"id":1,"name":"alice",...}, "2":{...}}
# Output as TSV for awk/cutcurl -s api/users | jq -r '.users[] | [.id, .name] | @tsv'Why this approach: Knowing select, INDEX, @tsv, and the // alternative operator transforms jq from a pretty-printer into a full ETL tool, eliminating the need for Python for many API-to-shell pipelines.
Answer: A generator that follows next links decouples pagination logic from the caller and works lazily so you can break early without fetching unnecessary pages.
xxxxxxxxxx#!/usr/bin/env python3"""paginated_api.py — exhaust all pages of a cursor/link-header API."""import requestsfrom typing import Generator, Any
SESSION = requests.Session()SESSION.headers.update({"Authorization": "Bearer TOKEN"})
# ---- Strategy 1: next URL in response body ----def paginate_body(start_url: str) -> Generator[Any, None, None]: """Yield every item from a JSON API that puts 'next' in the body.""" url: str | None = start_url while url: resp = SESSION.get(url, timeout=15) resp.raise_for_status() data = resp.json() yield from data.get("items", data.get("results", [])) url = data.get("next") # None stops the loop
# ---- Strategy 2: Link header (GitHub-style) ----def paginate_link_header(start_url: str) -> Generator[Any, None, None]: """Follow RFC 5988 Link headers (rel='next').""" url: str | None = start_url while url: resp = SESSION.get(url, timeout=15, params={"per_page": 100}) resp.raise_for_status() yield from resp.json() # Parse: Link: <https://api.github.com/...?page=2>; rel="next" link_header = resp.headers.get("Link", "") url = None for part in link_header.split(","): if 'rel="next"' in part: url = part.split(";")[0].strip().strip("<>") break
# ---- Strategy 3: offset/limit ----def paginate_offset(base_url: str, page_size: int = 100) -> Generator[Any, None, None]: offset = 0 while True: resp = SESSION.get( base_url, params={"limit": page_size, "offset": offset}, timeout=15, ) resp.raise_for_status() items = resp.json().get("results", []) if not items: break yield from items if len(items) < page_size: break offset += page_size
# Usagedef main(): for user in paginate_body("https://api.example.com/v1/users"): print(user["id"], user["name"])
if __name__ == "__main__": main()Why this approach: Generators keep memory usage proportional to one page at a time rather than the entire dataset, and the three strategies cover the most common pagination conventions without requiring changes to caller code.
Answer: http.server.BaseHTTPRequestHandler is in the standard library and handles POST bodies, request headers, and response codes with no external dependencies.
xxxxxxxxxx#!/usr/bin/env python3"""webhook_receiver.py — listen for incoming webhooks and process them."""import hashlibimport hmacimport jsonimport osimport sysfrom http.server import BaseHTTPRequestHandler, HTTPServer
SECRET = os.environ.get("WEBHOOK_SECRET", "").encode()
class WebhookHandler(BaseHTTPRequestHandler): def do_POST(self): content_length = int(self.headers.get("Content-Length", 0)) body = self.rfile.read(content_length)
# ---- Signature verification (GitHub-style HMAC-SHA256) ---- sig_header = self.headers.get("X-Hub-Signature-256", "") if SECRET: expected = "sha256=" + hmac.new(SECRET, body, hashlib.sha256).hexdigest() if not hmac.compare_digest(sig_header, expected): self._respond(401, {"error": "invalid signature"}) return
# ---- Parse JSON payload ---- try: payload = json.loads(body) except json.JSONDecodeError: self._respond(400, {"error": "invalid JSON"}) return
# ---- Dispatch by event type ---- event = self.headers.get("X-GitHub-Event", "unknown") print(f"EVENT: {event} — {json.dumps(payload)[:200]}") self.process_event(event, payload)
self._respond(200, {"status": "ok"})
def process_event(self, event: str, payload: dict): if event == "push": repo = payload.get("repository", {}).get("full_name") branch = payload.get("ref", "").removeprefix("refs/heads/") print(f"Push to {repo}:{branch}") elif event == "ping": print("Ping received — webhook configured successfully")
def _respond(self, code: int, body: dict): data = json.dumps(body).encode() self.send_response(code) self.send_header("Content-Type", "application/json") self.send_header("Content-Length", str(len(data))) self.end_headers() self.wfile.write(data)
def log_message(self, fmt, *args): print(f"[{self.address_string()}] {fmt % args}")
def main(): port = int(sys.argv[1]) if len(sys.argv) > 1 else 8080 server = HTTPServer(("0.0.0.0", port), WebhookHandler) print(f"Listening on port {port}") try: server.serve_forever() except KeyboardInterrupt: print("Shutting down.")
if __name__ == "__main__": main()Why this approach: hmac.compare_digest does constant-time comparison to prevent timing attacks on signature verification, which is essential when the webhook endpoint is publicly reachable.
Answer: openssl s_client retrieves the certificate; openssl x509 -enddate parses the expiry; Python's ssl module provides a pure-Python alternative that works in scripts without subprocess.
xxxxxxxxxx# check_ssl.sh — alert if cert expires within WARN_DAYS daysWARN_DAYS=30CRIT_DAYS=7
check_cert() { local host="${1:?host required}" local port="${2:-443}"
# Fetch certificate and get expiry EXPIRY=$(echo | openssl s_client -connect "${host}:${port}" \ -servername "$host" 2>/dev/null \ | openssl x509 -noout -enddate \ | cut -d= -f2)
if [[ -z "$EXPIRY" ]]; then echo "UNKNOWN: could not fetch cert for ${host}" return 3 fi
# Convert expiry to epoch EXPIRY_EPOCH=$(date -d "$EXPIRY" +%s 2>/dev/null || \ date -j -f "%b %e %T %Y %Z" "$EXPIRY" +%s) NOW_EPOCH=$(date +%s) DAYS_LEFT=$(( (EXPIRY_EPOCH - NOW_EPOCH) / 86400 ))
if (( DAYS_LEFT <= CRIT_DAYS )); then echo "CRIT: ${host} cert expires in ${DAYS_LEFT} days (${EXPIRY})" return 2 elif (( DAYS_LEFT <= WARN_DAYS )); then echo "WARN: ${host} cert expires in ${DAYS_LEFT} days (${EXPIRY})" return 1 else echo "OK: ${host} cert valid for ${DAYS_LEFT} more days" return 0 fi}
for host in api.example.com www.example.com; do check_cert "$host"donexxxxxxxxxx#!/usr/bin/env python3"""ssl_check.py — check multiple hosts for expiring certificates."""import sslimport socketimport datetimeimport sys
WARN_DAYS = 30CRIT_DAYS = 7
def cert_expiry(host: str, port: int = 443) -> datetime.datetime: ctx = ssl.create_default_context() with ctx.wrap_socket(socket.create_connection((host, port), timeout=5), server_hostname=host) as conn: cert = conn.getpeercert() expiry_str = cert["notAfter"] # e.g. "Jan 15 00:00:00 2026 GMT" return datetime.datetime.strptime(expiry_str, "%b %d %H:%M:%S %Y %Z")
def main(): hosts = sys.argv[1:] or ["api.example.com"] worst = 0 for host in hosts: try: expiry = cert_expiry(host) days = (expiry - datetime.datetime.utcnow()).days if days <= CRIT_DAYS: print(f"CRIT {host}: expires in {days}d ({expiry.date()})") worst = max(worst, 2) elif days <= WARN_DAYS: print(f"WARN {host}: expires in {days}d ({expiry.date()})") worst = max(worst, 1) else: print(f"OK {host}: expires in {days}d ({expiry.date()})") except Exception as exc: print(f"ERROR {host}: {exc}") worst = max(worst, 2) sys.exit(worst)
if __name__ == "__main__": main()Why this approach: The Python version uses the standard ssl module so no subprocess is needed, and connecting with a real TLS handshake also validates the certificate chain, catching misconfigured intermediates that openssl with -noverify would miss.
Answer: Use Python with concurrent.futures.ThreadPoolExecutor to check many endpoints in parallel, collecting results into a structured report.
xxxxxxxxxx#!/usr/bin/env python3"""endpoint_checker.py — bulk health test for HTTP endpoints."""import csvimport sysimport timefrom concurrent.futures import ThreadPoolExecutor, as_completedfrom dataclasses import dataclass, fieldfrom typing import List
import requests
TIMEOUT = 10WORKERS = 20EXPECTED_STATUS = 200
class Result: url: str status: int = 0 latency_ms: float = 0.0 error: str = ""
def ok(self) -> bool: return self.status == EXPECTED_STATUS
def check(url: str) -> Result: r = Result(url=url) try: start = time.monotonic() resp = requests.get(url, timeout=TIMEOUT, allow_redirects=True) r.latency_ms = (time.monotonic() - start) * 1000 r.status = resp.status_code except requests.exceptions.Timeout: r.error = "timeout" except requests.exceptions.ConnectionError as e: r.error = f"connection: {e}" except Exception as e: r.error = str(e) return r
def main(urls: List[str]): results: List[Result] = [] with ThreadPoolExecutor(max_workers=WORKERS) as pool: futures = {pool.submit(check, u): u for u in urls} for fut in as_completed(futures): results.append(fut.result())
results.sort(key=lambda r: r.url) failed = [r for r in results if not r.ok]
print(f"Checked {len(results)} endpoints — {len(failed)} failures\n") print(f"{'URL':<50} {'STATUS':>6} {'LATENCY':>10} ERROR") print("-" * 80) for r in results: tag = " " if r.ok else "!!" print(f"{tag} {r.url:<48} {r.status:>6} {r.latency_ms:>8.0f}ms {r.error}")
# Write CSV report with open("endpoint_report.csv", "w", newline="") as fh: w = csv.writer(fh) w.writerow(["url", "status", "latency_ms", "ok", "error"]) for r in results: w.writerow([r.url, r.status, f"{r.latency_ms:.1f}", r.ok, r.error])
sys.exit(1 if failed else 0)
if __name__ == "__main__": import os urls = sys.argv[1:] if not urls: urls_file = os.environ.get("URLS_FILE", "urls.txt") with open(urls_file) as fh: urls = [line.strip() for line in fh if line.strip()] main(urls)Why this approach: ThreadPoolExecutor with as_completed fills the worker pool immediately and collects results as they arrive, meaning the total runtime is roughly max_latency rather than sum_of_all_latencies.
Answer: A token-bucket or leaky-bucket algorithm in Python enforces a maximum request rate without over-sleeping; for Bash, a simple counter with sleep is often enough.
xxxxxxxxxx#!/usr/bin/env python3"""rate_limiter.py — token bucket for API calls."""import timeimport threadingfrom contextlib import contextmanager
class TokenBucket: """Thread-safe token bucket: allows up to `rate` requests per second."""
def __init__(self, rate: float, capacity: float | None = None): self.rate = rate # tokens/second self.capacity = capacity or rate # burst capacity self._tokens = float(self.capacity) self._last = time.monotonic() self._lock = threading.Lock()
def _refill(self): now = time.monotonic() elapsed = now - self._last self._tokens = min(self.capacity, self._tokens + elapsed * self.rate) self._last = now
def throttle(self): """Block until a token is available, then yield.""" while True: with self._lock: self._refill() if self._tokens >= 1: self._tokens -= 1 break time.sleep(0.01) # short spin before re-checking yield
# Usageimport requests
limiter = TokenBucket(rate=5.0) # 5 requests/second
def fetch(url: str) -> dict: with limiter.throttle(): return requests.get(url, timeout=10).json()
# Bash equivalent (simple 1 req/sec):BASH_RATE_LIMIT = """#!/usr/bin/env bashURLS=(https://api.example.com/1 https://api.example.com/2)RATE=2 # requests per secondDELAY=$(awk "BEGIN{printf \"%.3f\", 1/$RATE}")
for url in "${URLS[@]}"; do curl -sS "$url" sleep "$DELAY"done"""print(BASH_RATE_LIMIT)Why this approach: A token bucket allows short bursts (up to capacity) while enforcing a long-term average rate, which matches how most APIs define their limits and avoids wasting time with a fixed-sleep approach during quiet periods.
Answer: A simple file-based or in-memory cache with a TTL prevents hammering an API for data that rarely changes, which is important when scripts run frequently.
xxxxxxxxxx#!/usr/bin/env python3"""cached_api.py — cache API responses with TTL to reduce redundant calls."""import hashlibimport jsonimport osimport timefrom functools import wrapsfrom pathlib import Pathfrom typing import Any
import requests
CACHE_DIR = Path("/tmp/api_cache")CACHE_DIR.mkdir(exist_ok=True)
def cached(ttl_seconds: int = 300): """Decorator: cache function return value for ttl_seconds.""" def decorator(fn): (fn) def wrapper(*args, **kwargs): key = hashlib.md5( json.dumps((fn.__name__, args, sorted(kwargs.items())), sort_keys=True).encode() ).hexdigest() cache_file = CACHE_DIR / f"{key}.json"
# Return cached result if still fresh if cache_file.exists(): age = time.time() - cache_file.stat().st_mtime if age < ttl_seconds: return json.loads(cache_file.read_text())
result = fn(*args, **kwargs) cache_file.write_text(json.dumps(result)) return result return wrapper return decorator
(ttl_seconds=60)def get_services(env: str) -> list[dict]: resp = requests.get( f"https://registry.example.com/services", params={"env": env}, headers={"Authorization": f"Bearer {os.environ['API_TOKEN']}"}, timeout=10, ) resp.raise_for_status() return resp.json()["services"]
def main(): services = get_services("prod") for svc in services: print(svc["name"], svc["version"])
if __name__ == "__main__": main()Why this approach: File-based caching survives across script invocations and processes (unlike in-memory dicts), making it useful for cron jobs that share state without a running daemon.
Answer: Multipart form uploads use -F in curl and files= in requests; set content-type correctly to avoid server-side parsing errors.
xxxxxxxxxx# ---- curl: multipart upload ----curl -sS -X POST \ -H "Authorization: Bearer ${TOKEN}" \ -F "file=@/path/to/report.csv;type=text/csv" \ -F "description=monthly report" \ "https://api.example.com/v1/uploads"
# ---- curl: binary stream upload ----curl -sS -X PUT \ -H "Authorization: Bearer ${TOKEN}" \ -H "Content-Type: application/octet-stream" \ --data-binary @/path/to/artifact.tar.gz \ "https://storage.example.com/artifacts/release.tar.gz"xxxxxxxxxx#!/usr/bin/env python3"""upload.py — upload a file via multipart or binary PUT."""import osimport requests
TOKEN = os.environ["API_TOKEN"]SESSION = requests.Session()SESSION.headers["Authorization"] = f"Bearer {TOKEN}"
def multipart_upload(filepath: str, description: str = "") -> dict: with open(filepath, "rb") as fh: resp = SESSION.post( "https://api.example.com/v1/uploads", files={"file": (os.path.basename(filepath), fh, "application/octet-stream")}, data={"description": description}, timeout=60, ) resp.raise_for_status() return resp.json()
def stream_upload(filepath: str, dest_url: str) -> int: """Upload large file without reading it all into memory.""" with open(filepath, "rb") as fh: resp = SESSION.put( dest_url, data=fh, headers={"Content-Type": "application/octet-stream"}, timeout=300, ) resp.raise_for_status() return resp.status_code
if __name__ == "__main__": result = multipart_upload("/tmp/report.csv", description="Q1 report") print(result)Why this approach: Streaming the file handle directly (data=fh) avoids loading the entire file into memory, which is essential for large artifacts like VM images or database dumps.
Answer: xml.etree.ElementTree is in the standard library and handles namespaced XML; findall with XPath subsets extracts nodes efficiently.
xxxxxxxxxx#!/usr/bin/env python3"""xml_parser.py — extract data from SOAP/XML API responses."""import xml.etree.ElementTree as ETimport requests
SOAP_NS = {"soap": "http://schemas.xmlsoap.org/soap/envelope/", "m": "http://www.example.com/api"}
def parse_services_xml(xml_text: str) -> list[dict]: root = ET.fromstring(xml_text) body = root.find("soap:Body", SOAP_NS) services = [] for svc in body.findall(".//m:Service", SOAP_NS): services.append({ "name": svc.findtext("m:Name", default="", namespaces=SOAP_NS), "status": svc.findtext("m:Status", default="unknown", namespaces=SOAP_NS), "port": int(svc.findtext("m:Port", default="0", namespaces=SOAP_NS) or 0), }) return services
def fetch_and_parse(url: str) -> list[dict]: resp = requests.get(url, headers={"Accept": "application/xml"}, timeout=10) resp.raise_for_status() return parse_services_xml(resp.text)
if __name__ == "__main__": # Demonstrate with a local XML string sample = """<?xml version="1.0"?><soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:m="http://www.example.com/api"> <soap:Body> <m:Service><m:Name>auth</m:Name><m:Status>UP</m:Status><m:Port>8080</m:Port></m:Service> <m:Service><m:Name>db</m:Name><m:Status>DOWN</m:Status><m:Port>5432</m:Port></m:Service> </soap:Body></soap:Envelope>""" for s in parse_services_xml(sample): print(s)Why this approach: Registering namespace prefixes in a dictionary and passing it to findall/findtext is the correct way to handle namespaced XML, since XPath requires full namespace URIs which are too verbose to repeat inline.
Answer: HMAC signing involves canonicalizing request components, signing with the shared secret, and attaching the signature as a header—the same pattern used by AWS SigV4 and many webhook providers.
xxxxxxxxxx#!/usr/bin/env python3"""hmac_sign.py — sign API requests with HMAC-SHA256."""import hashlibimport hmacimport osimport timeimport requests
API_KEY = os.environ["API_KEY"]API_SECRET = os.environ["API_SECRET"].encode()
def sign_request(method: str, path: str, body: bytes = b"") -> dict[str, str]: """Return headers dict with HMAC-SHA256 signature.""" timestamp = str(int(time.time())) body_hash = hashlib.sha256(body).hexdigest() # Canonical string: METHOD\nPATH\nTIMESTAMP\nBODY_HASH canonical = f"{method.upper()}\n{path}\n{timestamp}\n{body_hash}" signature = hmac.new( API_SECRET, canonical.encode(), hashlib.sha256, ).hexdigest() return { "X-Api-Key": API_KEY, "X-Timestamp": timestamp, "X-Signature": signature, }
def signed_get(url: str, path: str) -> dict: headers = sign_request("GET", path) resp = requests.get(url, headers=headers, timeout=10) resp.raise_for_status() return resp.json()
def signed_post(url: str, path: str, payload: dict) -> dict: import json body = json.dumps(payload).encode() headers = { "Content-Type": "application/json", **sign_request("POST", path, body), } resp = requests.post(url, headers=headers, data=body, timeout=10) resp.raise_for_status() return resp.json()
if __name__ == "__main__": BASE = "https://api.example.com" result = signed_get(f"{BASE}/v1/account", "/v1/account") print(result)Why this approach: Including a timestamp in the canonical string prevents replay attacks (the server can reject requests older than 5 minutes), and hashing the body detects payload tampering in transit.
Answer: A production log-rotation script must handle the live file safely, compress archives efficiently, and prune old files based on a retention policy.
xxxxxxxxxx# log_rotate_full.shset -euo pipefail
LOG_DIR="${1:-/var/log/myapp}"KEEP_DAYS="${2:-7}"MAX_SIZE_MB="${3:-100}" # rotate if file exceeds this size
rotate_one() { local logfile="$1" local basename basename=$(basename "$logfile") local datestamp datestamp=$(date '+%Y-%m-%d_%H%M%S') local archive="${LOG_DIR}/${basename%.log}.${datestamp}.log.gz"
# Only rotate if file is non-empty [[ -s "$logfile" ]] || { echo "SKIP (empty): $logfile"; return 0; }
# Check size threshold local size_mb size_mb=$(du -m "$logfile" | cut -f1) if (( size_mb < MAX_SIZE_MB )); then echo "SKIP (${size_mb}MB < ${MAX_SIZE_MB}MB): $logfile" return 0 fi
# Copy-then-truncate preserves open file descriptors cp "$logfile" "${logfile}.rotating" truncate -s 0 "$logfile" # atomic truncate gzip -9 "${logfile}.rotating" mv "${logfile}.rotating.gz" "$archive"
echo "ROTATED: $archive (${size_mb}MB)"
# Signal app to reopen log if needed (e.g., nginx) # kill -USR1 "$(cat /var/run/myapp.pid)" 2>/dev/null || true}
prune_old() { local count=0 while IFS= read -r -d '' f; do rm -f "$f" echo "PRUNED: $f" (( count++ )) done < <(find "$LOG_DIR" -maxdepth 1 -name '*.log.gz' \ -mtime "+${KEEP_DAYS}" -print0) echo "Pruned ${count} old archive(s)"}
main() { echo "=== Log rotation: $(date -u) ===" find "$LOG_DIR" -maxdepth 1 -name '*.log' -type f | sort | while IFS= read -r f; do rotate_one "$f" done prune_old echo "=== Done ==="}
mainWhy this approach: truncate -s 0 is the standard way to clear a live log file because it modifies the file in place without changing its inode, so any process with an open file descriptor continues writing to the same file and does not lose log entries.
Answer: xargs -P or GNU parallel distributes SSH commands across multiple hosts simultaneously; capturing per-host output separately prevents interleaving.
xxxxxxxxxx# bulk_ssh.sh — run a command on many servers in parallelset -euo pipefail
HOSTS_FILE="${1:?hosts file required}"COMMAND="${2:?command required}"PARALLEL="${3:-10}"SSH_OPTS="-o StrictHostKeyChecking=no -o ConnectTimeout=5 -o BatchMode=yes"OUTPUT_DIR="$(mktemp -d)"
run_on_host() { local host="$1" local outfile="${OUTPUT_DIR}/${host}.out" local errfile="${OUTPUT_DIR}/${host}.err"
if ssh $SSH_OPTS "$host" "$COMMAND" >"$outfile" 2>"$errfile"; then echo "OK $host" else echo "ERR $host (exit $?)" >&2 fi}export -f run_on_hostexport OUTPUT_DIR SSH_OPTS COMMAND
# Run in parallelxargs -a "$HOSTS_FILE" -P "$PARALLEL" -I{} bash -c 'run_on_host "$@"' _ {}
# Summarize resultsecho ""echo "=== Results ==="for f in "$OUTPUT_DIR"/*.out; do host=$(basename "$f" .out) echo "--- $host ---" cat "$f"donerm -rf "$OUTPUT_DIR"xxxxxxxxxx#!/usr/bin/env python3"""bulk_ssh.py — parallel SSH using asyncio + asyncssh."""import asyncioimport sysfrom pathlib import Path
try: import asyncsshexcept ImportError: print("pip install asyncssh") sys.exit(1)
async def run_on_host(host: str, command: str, sem: asyncio.Semaphore) -> tuple[str, int, str, str]: async with sem: try: async with asyncssh.connect( host, known_hosts=None, connect_timeout=5 ) as conn: result = await conn.run(command, check=False) return host, result.exit_status, result.stdout, result.stderr except Exception as exc: return host, -1, "", str(exc)
async def main(hosts: list[str], command: str, parallelism: int = 10): sem = asyncio.Semaphore(parallelism) tasks = [run_on_host(h, command, sem) for h in hosts] results = await asyncio.gather(*tasks) for host, code, out, err in sorted(results): tag = "OK " if code == 0 else "ERR" print(f"{tag} [{code:3}] {host}: {out.strip() or err.strip()[:80]}")
if __name__ == "__main__": hosts_file, command = sys.argv[1], sys.argv[2] hosts = Path(hosts_file).read_text().split() asyncio.run(main(hosts, command))Why this approach: asyncssh with asyncio.Semaphore is more efficient than threads for I/O-bound SSH connections because it avoids thread-per-connection overhead while still limiting concurrency to avoid overwhelming the target hosts.
Answer: Idempotency means running the script twice produces the same result as running it once; check existence before create, use || to skip on already-exists, and validate the final state.
xxxxxxxxxx# provision_user.sh — idempotent user setupset -euo pipefail
USERNAME="${1:?username required}"GROUPS="${2:-sudo,docker}"SSH_KEY="${3:-}" # optional public key
# ---- Create user if not exists ----if id "$USERNAME" &>/dev/null; then echo "User ${USERNAME} already exists — skipping create"else useradd \ --create-home \ --shell /bin/bash \ --comment "Provisioned by automation" \ "$USERNAME" echo "Created user: ${USERNAME}"fi
# ---- Ensure user is in required groups ----IFS=',' read -ra GRP_LIST <<< "$GROUPS"for grp in "${GRP_LIST[@]}"; do if getent group "$grp" &>/dev/null; then if id -nG "$USERNAME" | grep -qw "$grp"; then echo "Already in group ${grp} — skip" else usermod -aG "$grp" "$USERNAME" echo "Added ${USERNAME} to ${grp}" fi else echo "WARN: group ${grp} does not exist — skipping" fidone
# ---- Set up SSH key if provided ----if [[ -n "$SSH_KEY" ]]; then SSH_DIR="/home/${USERNAME}/.ssh" AUTH_KEYS="${SSH_DIR}/authorized_keys"
install -d -m 700 -o "$USERNAME" -g "$USERNAME" "$SSH_DIR"
if grep -qF "$SSH_KEY" "$AUTH_KEYS" 2>/dev/null; then echo "SSH key already present — skip" else echo "$SSH_KEY" >> "$AUTH_KEYS" chown "$USERNAME:$USERNAME" "$AUTH_KEYS" chmod 600 "$AUTH_KEYS" echo "Added SSH key for ${USERNAME}" fifi
# ---- Verify ----echo "--- Verification ---"id "$USERNAME"groups "$USERNAME"Why this approach: Checking state before acting (id, grep -qF) ensures the script is a no-op on subsequent runs, making it safe for cron, CI, and Terraform provisioners that may execute it repeatedly.
Answer: The script must create a timestamped backup atomically, verify its integrity, and prune old backups based on count or age.
xxxxxxxxxx# backup.sh — backup a directory and keep last 7 daysset -euo pipefail
SOURCE="${1:?source directory required}"DEST="${2:?backup destination required}"KEEP="${3:-7}"LABEL="${4:-backup}"
TIMESTAMP=$(date '+%Y-%m-%d_%H%M%S')ARCHIVE="${DEST}/${LABEL}_${TIMESTAMP}.tar.gz"CHECKSUM="${ARCHIVE}.sha256"
mkdir -p "$DEST"
echo "[$(date -u)] Starting backup of ${SOURCE}"
# ---- Create archive ----tar \ --create \ --gzip \ --file "$ARCHIVE" \ --exclude='*.tmp' \ --exclude='.cache' \ "$SOURCE"
ARCHIVE_SIZE=$(du -h "$ARCHIVE" | cut -f1)
# ---- Verify archive is readable ----if ! tar --list --file "$ARCHIVE" > /dev/null 2>&1; then echo "ERROR: archive verification failed — removing corrupt file" >&2 rm -f "$ARCHIVE" exit 1fi
# ---- Generate checksum ----sha256sum "$ARCHIVE" > "$CHECKSUM"echo "Created: ${ARCHIVE} (${ARCHIVE_SIZE})"
# ---- Retention: remove oldest backups beyond KEEP count ----mapfile -t OLD_BACKUPS < <( find "$DEST" -maxdepth 1 -name "${LABEL}_*.tar.gz" \ -printf '%T@ %p\n' | sort -n | head -n -"$KEEP" | awk '{print $2}')
for old in "${OLD_BACKUPS[@]+"${OLD_BACKUPS[@]}"}"; do rm -f "$old" "${old}.sha256" echo "Pruned: ${old}"done
echo "[$(date -u)] Backup complete. Keeping last ${KEEP} backups."ls -lh "${DEST}/${LABEL}_"*.tar.gz 2>/dev/null || trueWhy this approach: Verifying the archive with tar --list before writing the checksum means the checksum file is only created for known-good archives, preventing silent corruption from disk errors during write.
Answer: A production deploy script must validate prerequisites, deploy atomically (symlink swap), and roll back automatically if smoke tests fail.
xxxxxxxxxx# deploy.sh — deploy a service with preflight checks and rollbackset -euo pipefail
APP_DIR="/opt/myapp"RELEASES_DIR="${APP_DIR}/releases"CURRENT_LINK="${APP_DIR}/current"ARTIFACT="${1:?artifact path required}"KEEP_RELEASES=5
TIMESTAMP=$(date '+%Y%m%d_%H%M%S')RELEASE_DIR="${RELEASES_DIR}/${TIMESTAMP}"
log() { echo "[$(date -u '+%Y-%m-%dT%H:%M:%SZ')] $*"; }die() { log "FATAL: $*" >&2; exit 1; }
# ---- Pre-flight checks ----preflight() { log "Running pre-flight checks..." command -v systemctl &>/dev/null || die "systemctl not found" [[ -f "$ARTIFACT" ]] || die "Artifact not found: $ARTIFACT" [[ -d "$RELEASES_DIR" ]] || mkdir -p "$RELEASES_DIR" df -P "$RELEASES_DIR" | awk 'NR==2{if($5+0>=90) exit 1}' \ || die "Disk usage >=90% on releases partition" log "Pre-flight passed"}
# ---- Deploy ----deploy() { log "Deploying to ${RELEASE_DIR}" mkdir -p "$RELEASE_DIR" tar -xzf "$ARTIFACT" -C "$RELEASE_DIR" chown -R myapp:myapp "$RELEASE_DIR"
PREVIOUS_LINK=$(readlink "$CURRENT_LINK" 2>/dev/null || true)
# Atomic symlink swap ln -snf "$RELEASE_DIR" "${CURRENT_LINK}.new" mv -Tf "${CURRENT_LINK}.new" "$CURRENT_LINK"
log "Symlink updated: ${CURRENT_LINK} -> ${RELEASE_DIR}" echo "$PREVIOUS_LINK" > "${RELEASES_DIR}/.previous"}
# ---- Smoke test ----smoke_test() { log "Restarting service..." systemctl restart myapp
log "Waiting for service to come up..." local max_wait=30 elapsed=0 while (( elapsed < max_wait )); do if curl -sf http://localhost:8080/health &>/dev/null; then log "Health check passed after ${elapsed}s" return 0 fi sleep 2 (( elapsed += 2 )) done return 1}
# ---- Rollback ----rollback() { local previous previous=$(cat "${RELEASES_DIR}/.previous" 2>/dev/null || true) if [[ -z "$previous" || ! -d "$previous" ]]; then die "No previous release to roll back to" fi log "ROLLING BACK to ${previous}" ln -snf "$previous" "${CURRENT_LINK}.rollback" mv -Tf "${CURRENT_LINK}.rollback" "$CURRENT_LINK" systemctl restart myapp log "Rollback complete"}
# ---- Prune old releases ----prune_releases() { mapfile -t OLD < <( find "$RELEASES_DIR" -maxdepth 1 -mindepth 1 -type d \ | sort | head -n -"$KEEP_RELEASES" ) for r in "${OLD[@]+"${OLD[@]}"}"; do rm -rf "$r" log "Pruned release: $r" done}
# ---- Main ----preflightdeploy
if smoke_test; then log "Deploy SUCCESSFUL" prune_releaseselse log "Smoke test FAILED — rolling back" >&2 rollback exit 1fiWhy this approach: The atomic mv -Tf symlink swap ensures the new release is either fully live or not at all—there is no window where current points to a partially-extracted directory.
Answer: Zombie processes are already dead—they have no code to kill; only reaping them (having the parent call wait) removes them. The script identifies zombies and their parents for operator action.
xxxxxxxxxx# zombie_finder.sh — find zombie processes and report their parentsset -euo pipefail
echo "=== Zombie Processes ==="printf '%-8s %-8s %-8s %s\n' "ZOMBIE_PID" "PPID" "PARENT_PID" "PARENT_CMD"printf '%s\n' "$(printf '%.0s-' {1..60})"
ZOMBIE_COUNT=0
while IFS= read -r line; do PID=$(echo "$line" | awk '{print $1}') PPID=$(echo "$line" | awk '{print $2}') CMD=$(echo "$line" | cut -d' ' -f5-)
PARENT_CMD=$(ps -p "$PPID" -o comm= 2>/dev/null || echo "GONE") printf '%-10s %-8s %-8s %s\n' "$PID" "$PPID" "$PPID" "$PARENT_CMD" (( ZOMBIE_COUNT++ ))
done < <(ps axo pid,ppid,stat,comm,args | awk '$3 ~ /^Z/ {print}')
echo ""echo "Total zombies: ${ZOMBIE_COUNT}"
if (( ZOMBIE_COUNT > 0 )); then echo "" echo "NOTE: Zombies cannot be killed directly." echo "To reap them:" echo " 1. Kill the parent process (it will reap children on exit)" echo " 2. Or send SIGCHLD to the parent: kill -CHLD <PPID>" echo " 3. If parent is init (PID 1), zombies will be reaped automatically"
# Attempt SIGCHLD to parents PPIDS=$(ps axo pid,ppid,stat | awk '$3 ~ /^Z/ {print $2}' | sort -u) for ppid in $PPIDS; do if [[ "$ppid" != "1" ]]; then echo "Sending SIGCHLD to parent PID ${ppid}" kill -CHLD "$ppid" 2>/dev/null || true fi donefiWhy this approach: Sending SIGCHLD to the parent is the correct mechanism—it prompts the parent to call wait() and collect the zombie's exit status, which is what removes the process table entry.
Answer: A watchdog polls a health endpoint or process table and restarts the service if it fails, with alerting and a back-off to prevent restart loops.
xxxxxxxxxx# watchdog.sh — restart a service if its health check failsset -euo pipefail
SERVICE="${1:?service name required}"HEALTH_URL="${2:-http://localhost:8080/health}"CHECK_INTERVAL="${3:-30}" # secondsMAX_RESTARTS="${4:-3}" # within WINDOW_SECONDSWINDOW_SECONDS=300SLACK_WEBHOOK="${SLACK_WEBHOOK:-}"
restart_count=0window_start=$(date +%s)
alert() { local msg="$1" echo "[$(date -u)] WATCHDOG: $msg" if [[ -n "$SLACK_WEBHOOK" ]]; then curl -s -X POST -H 'Content-Type: application/json' \ -d "{\"text\": \"WATCHDOG $(hostname -s): $msg\"}" \ "$SLACK_WEBHOOK" &>/dev/null || true fi}
check_health() { curl -sf --max-time 5 "$HEALTH_URL" &>/dev/null}
reset_window_if_needed() { local now now=$(date +%s) if (( now - window_start > WINDOW_SECONDS )); then restart_count=0 window_start=$now fi}
alert "Watchdog started for ${SERVICE} (checking ${HEALTH_URL} every ${CHECK_INTERVAL}s)"
while true; do sleep "$CHECK_INTERVAL" reset_window_if_needed
if ! check_health; then echo "[$(date -u)] Health check FAILED for ${SERVICE}"
(( restart_count++ )) if (( restart_count > MAX_RESTARTS )); then alert "CRITICAL: ${SERVICE} has restarted ${restart_count} times in ${WINDOW_SECONDS}s — NOT restarting again. Manual intervention needed." sleep 600 # long backoff before checking again restart_count=0 continue fi
alert "WARN: restarting ${SERVICE} (attempt ${restart_count}/${MAX_RESTARTS})" systemctl restart "$SERVICE" || true
# Wait for service to come up sleep 10 if check_health; then alert "OK: ${SERVICE} recovered after restart" else alert "ERR: ${SERVICE} still down after restart" fi fidoneWhy this approach: The sliding window counter (restart_count / window_start) distinguishes between a service that fails once versus one in a crash loop, preventing the watchdog from repeatedly restarting a persistently broken service and masking the real problem.
Answer: shellcheck is a static analysis tool for Bash; running it in CI prevents entire categories of bugs before they reach production.
xxxxxxxxxx# run_shellcheck.sh — check all scripts in a directoryset -euo pipefail
find "${1:-.}" -name '*.sh' -type f | while IFS= read -r script; do echo "Checking: $script" shellcheck --severity=warning --shell=bash "$script"donexxxxxxxxxx# ---- Common pitfalls shellcheck catches ----
# SC2086: Word splitting on unquoted variableFILES=$(ls *.log)for f in $FILES; do ... # BAD: breaks on spacesfor f in *.log; do ... # GOOD: glob is safe
# SC2046: Quote to prevent word splitting in command substitutionrm $(cat filelist.txt) # BADwhile IFS= read -r f; do rm "$f"; done < filelist.txt # GOOD
# SC2164: cd without checking successcd /tmp && do_work # BAD if /tmp doesn't existcd /tmp || exit 1 # GOOD# Or use: set -e + cd /tmp
# SC2206: Splitting array assignmentIFS=',' read PARTS <<< "$CSV" # BAD: PARTS is a stringIFS=',' read -ra PARTS <<< "$CSV" # GOOD: PARTS is an array
# SC2155: Masking return valueexport VAR=$(command_that_might_fail) # BAD: export always returns 0VAR=$(command_that_might_fail) # GOODexport VAR # GOOD
# SC2034: Variable appears unusedUNUSED_VAR="foo" # shellcheck warns
# SC2162: read without -r mangling backslashesread line # BADread -r line # GOOD
# SC2143: Unnecessary use of grep + [ ]if [ "$(grep -c pattern file)" -gt 0 ]; then # VERBOSEif grep -q pattern file; then # GOOD
# SC2236: Negating -n testif [ ! -n "$var" ]; then # confusingif [ -z "$var" ]; then # GOODWhy this approach: Integrating shellcheck into CI with --severity=warning catches 90% of Bash correctness issues automatically, turning code review from bug-hunting into logic review.
Answer: bats (Bash Automated Testing System) provides a @test DSL with assert_output, assert_failure, and setup/teardown hooks for testing Bash functions in isolation.
xxxxxxxxxx# test_utils.bats — unit tests for utils.sh
load 'test_helper/bats-support/load'load 'test_helper/bats-assert/load'
# Source the script under test (don't execute main)setup() { source "${BATS_TEST_DIRNAME}/../lib/utils.sh" export TEST_DIR TEST_DIR=$(mktemp -d)}
teardown() { rm -rf "$TEST_DIR"}
@test "trim_whitespace removes leading and trailing spaces" { result=$(trim_whitespace " hello world ") assert_output "hello world"}
@test "is_integer returns 0 for valid integers" { run is_integer 42 assert_success
run is_integer -7 assert_success}
@test "is_integer returns 1 for non-integer" { run is_integer "3.14" assert_failure
run is_integer "abc" assert_failure}
@test "create_dir creates directory idempotently" { local dir="${TEST_DIR}/subdir/nested" run create_dir "$dir" assert_success assert [ -d "$dir" ]
# Running again should not fail (idempotent) run create_dir "$dir" assert_success}
@test "rotate_log creates archive and truncates original" { echo "test log content" > "${TEST_DIR}/app.log" run rotate_log "${TEST_DIR}/app.log" 7
assert_success # Archive was created assert [ "$(ls "${TEST_DIR}"/*.gz 2>/dev/null | wc -l)" -gt 0 ] # Original is truncated assert [ ! -s "${TEST_DIR}/app.log" ]}
@test "parse_config reads KEY=VALUE pairs" { printf 'HOST=localhost\nPORT=8080\n' > "${TEST_DIR}/config.env" result=$(parse_config "${TEST_DIR}/config.env" HOST) assert_output "localhost"}Why this approach: bats runs each @test in a subshell so setup state does not leak between tests, and the assert_* helpers produce readable diff output on failure rather than cryptic exit code mismatches.
Answer: pytest with monkeypatch and tmp_path lets you test scripts that interact with the filesystem, environment variables, and external commands without touching real resources.
xxxxxxxxxx# test_backup.py — pytest tests for backup.pyimport jsonimport osimport timefrom pathlib import Pathfrom unittest.mock import patch, MagicMock
import pytest
# Import the module under testimport backup # assumes backup.py is in PYTHONPATH
class TestRotateLog: def test_rotates_file_above_threshold(self, tmp_path): log = tmp_path / "app.log" log.write_text("x" * 1024) # 1 KB file backup.rotate_log(str(log), min_size_bytes=512, keep=3) archives = list(tmp_path.glob("*.gz")) assert len(archives) == 1 assert log.stat().st_size == 0 # truncated
def test_skips_empty_file(self, tmp_path): log = tmp_path / "empty.log" log.write_text("") backup.rotate_log(str(log), min_size_bytes=1, keep=3) assert not list(tmp_path.glob("*.gz"))
def test_prunes_old_archives(self, tmp_path): log = tmp_path / "app.log" log.write_text("data" * 100) # Create 5 existing fake archives for i in range(5): (tmp_path / f"app.{i}.log.gz").write_text("fake") time.sleep(0.01) backup.rotate_log(str(log), min_size_bytes=1, keep=3) archives = list(tmp_path.glob("*.gz")) assert len(archives) == 3 # only 3 kept
class TestHealthCheck: def test_ok_when_all_thresholds_clear(self, monkeypatch): monkeypatch.setattr("psutil.cpu_percent", lambda interval=0: 40.0) monkeypatch.setattr( "psutil.virtual_memory", lambda: MagicMock(percent=50.0) ) monkeypatch.setattr( "psutil.disk_usage", lambda path: MagicMock(percent=60.0) ) code, msgs = backup.run_health_checks() assert code == 0 assert all("ok" in m.lower() for m in msgs)
def test_crit_when_cpu_above_threshold(self, monkeypatch): monkeypatch.setattr("psutil.cpu_percent", lambda interval=0: 95.0) monkeypatch.setattr( "psutil.virtual_memory", lambda: MagicMock(percent=30.0) ) monkeypatch.setattr( "psutil.disk_usage", lambda path: MagicMock(percent=30.0) ) code, msgs = backup.run_health_checks() assert code == 2 assert any("CRIT" in m for m in msgs)
class TestPaginatedAPI: def test_exhausts_all_pages(self, requests_mock): requests_mock.get( "https://api.test/items", [ {"json": {"items": [1, 2, 3], "next": "https://api.test/items?page=2"}}, {"json": {"items": [4, 5], "next": None}}, ] ) items = list(backup.paginate_body("https://api.test/items")) assert items == [1, 2, 3, 4, 5]Why this approach: monkeypatch.setattr replaces real system calls (psutil, requests) with controlled stubs, making tests deterministic and fast without needing a real network or filesystem state.
Answer: bash -n syntax-checks without running; set -x traces execution; trap DEBUG gives per-command hooks for custom logging.
xxxxxxxxxx# ---- Method 1: Syntax check only (no execution) ----# bash -n myscript.sh
# ---- Method 2: Trace mode (prints each command before running) ----set -x # enable trace# ... your code ...set +x # disable trace
# ---- Method 3: Trace to a file (avoids polluting stdout) ----exec {BASH_XTRACEFD}>/tmp/trace_$$.logPS4='+ ${BASH_SOURCE}:${LINENO}:${FUNCNAME[0]:+${FUNCNAME[0]}(): }'set -x
# ---- Method 4: trap DEBUG — log every command with context ----debug_trap() { echo "DEBUG [line ${LINENO}] cmd: ${BASH_COMMAND}" >> /tmp/debug_$$.log}trap debug_trap DEBUG
# ---- Method 5: ERR trap — print stack trace on error ----err_trap() { local code=$? echo "ERROR: command '${BASH_COMMAND}' failed with exit ${code}" >&2 echo "Stack trace:" >&2 local i=0 while caller $i; do (( i++ )) done >&2}trap err_trap ERR
# ---- Method 6: Verbose function entry/exit ----trace_func() { echo "ENTER: ${FUNCNAME[1]} args=($*)" >&2}
my_function() { trace_func "$@" local result result=$(do_something "$1") echo "EXIT: ${FUNCNAME[0]} result=${result}" >&2 echo "$result"}
# ---- Common debugging one-liners ----# Check if variable is set and non-emptyecho "VAR=[${MY_VAR:-UNSET}]"
# Print all set variables( set -o posix; set ) | grep -E '^[A-Z_]+'
# Show all functions defineddeclare -F
# Check which command would be runtype my_commandwhich my_commandWhy this approach: Setting PS4 with ${BASH_SOURCE}:${LINENO} makes set -x output include file and line number, turning the trace from an unstructured dump into a navigable log.
Answer: Idempotency requires testing preconditions before acting and using atomic operations; the patterns are consistent across file creation, directory setup, service configuration, and database migrations.
xxxxxxxxxx# idempotent_patterns.sh — collection of safe-to-run-twice patternsset -euo pipefail
# ---- File creation (only if not exists) ----CONFIG="/etc/myapp/config.ini"if [[ ! -f "$CONFIG" ]]; then install -D -m 640 /dev/null "$CONFIG" cat > "$CONFIG" <<'EOF'[server]port = 8080EOF echo "Created $CONFIG"fi
# ---- Directory with exact permissions ----ensure_dir() { local dir="$1" mode="${2:-755}" owner="${3:-root}" install -d -m "$mode" -o "$owner" "$dir" # install -d is idempotent: no-op if dir exists with correct attrs}ensure_dir /opt/myapp/data 750 myapp
# ---- Symlink update ----ensure_symlink() { local target="$1" link="$2" if [[ "$(readlink "$link" 2>/dev/null)" == "$target" ]]; then return 0 # already correct fi ln -snf "$target" "${link}.new" mv -Tf "${link}.new" "$link"}ensure_symlink /opt/myapp/releases/2026-05-17 /opt/myapp/current
# ---- Cron job (no duplicate entries) ----CRON_LINE="0 2 * * * /opt/myapp/scripts/backup.sh"CRON_FILE="/etc/cron.d/myapp"if ! grep -qF "$CRON_LINE" "$CRON_FILE" 2>/dev/null; then echo "$CRON_LINE" >> "$CRON_FILE" echo "Added cron entry"fi
# ---- Line in config file ----ensure_line_in_file() { local line="$1" file="$2" grep -qxF "$line" "$file" 2>/dev/null || echo "$line" >> "$file"}ensure_line_in_file "net.ipv4.ip_forward = 1" /etc/sysctl.conf
# ---- Package installation (apt) ----ensure_package() { local pkg="$1" dpkg -l "$pkg" 2>/dev/null | grep -q '^ii' && return 0 apt-get install -y "$pkg"}
# ---- Service enable + start ----ensure_service() { local svc="$1" systemctl is-enabled "$svc" &>/dev/null || systemctl enable "$svc" systemctl is-active "$svc" &>/dev/null || systemctl start "$svc"}
# ---- Marker file pattern (run-once initializer) ----INIT_MARKER="/var/lib/myapp/.initialized"if [[ ! -f "$INIT_MARKER" ]]; then run_first_time_setup touch "$INIT_MARKER"fiWhy this approach: The mv -Tf symlink pattern is atomic—the old link is replaced in one syscall so concurrent readers always see either the old or new target, never a dangling intermediate state.
Answer: These three features interact to cause subtle bugs that work in testing but fail silently in production with real filenames, empty variables, or unexpected characters.
xxxxxxxxxx# pitfalls.sh — demonstrations of common Bash traps
# ---- Pitfall 1: Unquoted variable with spaces ----FILENAME="my file.txt"cp $FILENAME /tmp/ # BAD: treated as "cp my file.txt /tmp/" — 3 argscp "$FILENAME" /tmp/ # GOOD
# ---- Pitfall 2: Unquoted command substitution ----FILES=$(find . -name "*.log")for f in $FILES; do # BAD: word-splits on newlines AND spaces process "$f"done
find . -name "*.log" -print0 | while IFS= read -r -d '' f; do # GOOD process "$f"done
# ---- Pitfall 3: Glob expansion in the wrong place ----PATTERN="*.txt"ls $PATTERN # BAD: expands if CWD has .txt files, error if notls "$PATTERN" # GOOD (passes literal *.txt to ls)ls ./*.txt # GOOD when you DO want expansion
# ---- Pitfall 4: Empty variable unintended glob ----DIR=""rm -rf $DIR/* # BAD: expands to rm -rf /* !!!rm -rf "${DIR:?DIR must not be empty}"/* # GOOD: aborts if empty
# ---- Pitfall 5: Integer comparison with empty string ----COUNT=""if [[ $COUNT -gt 0 ]]; then # BAD: -gt fails with non-integer error echo "has items"fiif [[ "${COUNT:-0}" -gt 0 ]]; then # GOOD: default to 0
# ---- Pitfall 6: Pipefail and exit codes in pipelines ----set -o pipefail # make grep | head propagate grep's exit codefalse | true # without pipefail: exit 0; with pipefail: exit 1
# ---- Pitfall 7: Subshell variable scope ----COUNT=0echo "a b c" | while read -r word; do (( COUNT++ )) # BAD: runs in subshell; COUNT stays 0 outsidedoneecho "$COUNT" # prints 0!
while read -r word; do # GOOD: redirect into while (same shell) (( COUNT++ ))done <<< "a b c"echo "$COUNT" # prints 3
# ---- Pitfall 8: errexit swallowed by if/&&/|| ----set -e# These DON'T trigger errexit even with set -e:if false_command; then :; fi # false_command can fail without abortingfalse_command || true # false_command failure is OKresult=$(false_command) || true # but: assignment to result is also 0
# ---- Pitfall 9: Arithmetic with set -e ----set -eCOUNT=0(( COUNT++ )) # BAD: (( 0++ )) evaluates to 0 (false) → exits script!(( COUNT++ )) || true # GOOD: suppress exit(( COUNT += 1 )) # GOOD: evaluates to 1 (true) — only safe when initial != -1Why this approach: Adding set -euo pipefail to every script and always quoting variable expansions catches the majority of these pitfalls at the cost of minor extra typing, while the specific patterns above address the cases where quoting alone is insufficient.
Answer: A debug harness combines all three techniques to produce a structured trace with timestamps, function names, and line numbers that can be enabled/disabled by an environment variable.
xxxxxxxxxx# debug_harness.sh — reusable debug harness# Usage: DEBUG=1 bash myscript.shset -euo pipefail
# ---- Enable debug mode via environment variable ----if [[ "${DEBUG:-0}" == "1" ]]; then # Redirect xtrace to separate file descriptor exec {BASH_XTRACEFD}>"/tmp/debug_${$}.log" export BASH_XTRACEFD
# Rich PS4: timestamp + source + line + function PS4='$(date "+%T.%3N") + ${BASH_SOURCE[0]##*/}:${LINENO}:${FUNCNAME[0]:+${FUNCNAME[0]}(): }' export PS4
set -x echo "Debug log: /tmp/debug_${$}.log" >&2fi
# ---- ERR trap with full stack trace ----_err_handler() { local exit_code=$? local line=$1 echo "--- STACK TRACE ---" >&2 echo "Error at line ${line}, exit code ${exit_code}" >&2 echo "Command: ${BASH_COMMAND}" >&2 local i=0 echo "Call stack:" >&2 while caller $i 2>/dev/null; do (( i++ )) done | awk '{printf " #%d %s() at %s:%s\n", NR-1, $3, $2, $1}' >&2 echo "-------------------" >&2}trap '_err_handler $LINENO' ERR
# ---- Syntax check function (call before sourcing untrusted scripts) ----syntax_check() { local script="$1" if ! bash -n "$script" 2>&1; then echo "SYNTAX ERROR in $script" >&2 return 1 fi}
# ---- DEBUG trap for variable watching ----WATCH_VAR=""_debug_watch() { [[ -n "$WATCH_VAR" ]] || return 0 local val="${!WATCH_VAR:-UNSET}" # Only print when value changes if [[ "${_PREV_VAL:-}" != "$val" ]]; then echo "WATCH ${WATCH_VAR}=${val} (line ${LINENO})" >&2 _PREV_VAL="$val" fi}trap '_debug_watch' DEBUG
# ---- Example usage ----process_file() { local file="$1" [[ -f "$file" ]] || { echo "File not found: $file" >&2; return 1; } wc -l < "$file"}
main() { echo "Processing..." process_file "${1:-/etc/hostname}"}
main "$@"Why this approach: Writing xtrace to a separate file descriptor with BASH_XTRACEFD keeps the debug trace out of stdout so it does not corrupt piped output, while the ERR trap with caller provides a human-readable stack trace without requiring a debugger.
Answer: Language choice is an engineering decision based on complexity, performance, portability, maintenance burden, and the skill level of people who will need to modify the code later.
xxxxxxxxxxDecision matrix:USE BASH when:- Orchestrating existing CLI tools (curl, git, tar, systemctl)- < ~100 lines of logic- Script is run by ops team members who know Bash- No complex data structures needed- Gluing together pipelines of Unix tools- One-off automation that won't be heavily maintainedUSE PYTHON when:- Complex logic, conditionals, or data manipulation- Parsing/generating structured data (JSON, YAML, CSV, XML)- HTTP API calls with retries, pagination, auth- Unit tests required (pytest is far better than bats)- Rich error handling needed- Cross-platform portability required- Working with databases, S3, cloud APIs- Script will be maintained by multiple engineers- 100+ lines or multiple functions neededUSE GO when:- Performance-critical (high throughput, low latency)- Single binary distribution (no runtime dependency)- Concurrent workloads (replace xargs -P with goroutines)- Tool will be distributed to end users/customers- Strong typing needed to prevent runtime bugs- Replacing a Bash script that has grown beyond 500 lines- Container sidecar tools, kubectl plugins, CLIsEXAMPLES:Bash is fine:deploy.sh — run git pull, docker build, kubectl applyrotate_logs.sh — find, gzip, rmPython is better:Parsing 10GB JSON logsAPI integration with paginationData pipeline with transformationsGo is better:kubectl plugin distributed to 100 engineersHigh-throughput log processor (>100k lines/sec)Service binary that handles signals, goroutines, TLS
xxxxxxxxxx#!/usr/bin/env python3"""Benchmark: when Python actually beats Bash.
Bash (pipe-based): grep | awk | sort | uniq -c - Spawns 4 processes - Good for simple aggregations on files
Python equivalent: - Single process - Faster for complex joins or multi-pass analysis - Required when logic doesn't fit Unix primitives"""import sysfrom collections import Counter
def count_errors(logfile: str) -> dict: """Count error types — cleaner and faster than grep|awk|sort|uniq.""" counts: Counter = Counter() with open(logfile) as fh: for line in fh: if "ERROR" in line: # Extract error code (would need complex awk for this) parts = line.split() if len(parts) > 2: counts[parts[2]] += 1 return dict(counts.most_common(10))
if __name__ == "__main__": result = count_errors(sys.argv[1] if len(sys.argv) > 1 else "/var/log/app.log") for error, count in result.items(): print(f"{count:>6} {error}")Why this approach: Choosing the right tool is itself an interview signal — demonstrating that you reach for Python when Bash will create maintenance debt, and Go when distribution or performance matters, shows systems engineering judgment beyond syntax knowledge.
xxxxxxxxxx# Exit codes: 0=OK, 1=WARN, 2=CRITexit_code=$(( ${exit_code:-0} > $new_code ? ${exit_code} : $new_code ))
# Disk usage %df / | awk 'NR==2 {gsub(/%/,""); print $5}'
# Memory free %free | awk '/^Mem/ {printf "%.0f\n", $4/$2*100}'
# CPU 1-min loadawk '{print $1}' /proc/loadavg
# Tail log and alert on patterntail -F /var/log/app.log | grep --line-buffered "ERROR" | while read -r l; do send_slack "$l"; donexxxxxxxxxx# GET with Bearer token, show HTTP codecurl -sS -o /dev/null -w "%{http_code}" -H "Authorization: Bearer $TOKEN" URL
# POST JSONcurl -sS -X POST -H "Content-Type: application/json" -d '{"k":"v"}' URL
# Separate body from statusHTTP=$(curl -sS -o /tmp/body -w "%{http_code}" URL); cat /tmp/body
# Check SSL expiry daysecho | openssl s_client -connect host:443 2>/dev/null \ | openssl x509 -noout -enddate | cut -d= -f2
# Upload filecurl -sS -X POST -F "file=@path/to/file" -H "Authorization: Bearer $TOKEN" URLxxxxxxxxxxjq '.[].name' # extract field from arrayjq '[.[] | select(.active)]' # filter arrayjq '[.[] | {id, name}]' # project fieldsjq -r '.items[] | [.id,.name] | @tsv' # TSV outputjq 'INDEX(.[]; .id)' # array to lookup mapjq '[.[].score] | add / length' # averagejq '.[] | select(.status | startswith("ERR"))' # prefix matchjq 'keys' # object keysjq '.foo // "default"' # null coalescingxxxxxxxxxx# Bash retry with backoffretry() { local n=1 max=5 delay=1 while ! "$@"; do (( n >= max )) && { echo "FAILED after $max attempts"; return 1; } sleep $((delay * 2 ** n)); (( n++ )) done}retry curl -sf https://api.example.com/healthxxxxxxxxxx# Python tenacityfrom tenacity import retry, wait_exponential_jitter, stop_after_attempt(wait=wait_exponential_jitter(1, 60), stop=stop_after_attempt(5))def call(): requests.get(url).raise_for_status()
# Token bucket (5 req/sec)import timeclass Bucket: def __init__(self, rate): self.rate=rate; self._t=time.monotonic(); self._tokens=rate def wait(self): while True: now=time.monotonic(); self._tokens=min(self.rate, self._tokens+(now-self._t)*self.rate); self._t=now if self._tokens>=1: self._tokens-=1; return time.sleep(0.01)xxxxxxxxxx# xargs parallel (10 workers)cat hosts.txt | xargs -P 10 -I{} ssh {} 'uptime'
# GNU parallelparallel -j10 ssh {} uptime :::: hosts.txt
# Wait all background jobs, capture failurespids=(); for h in "${HOSTS[@]}"; do ssh "$h" cmd & pids+=($!); donefor pid in "${pids[@]}"; do wait "$pid" || echo "FAILED: $pid"; donexxxxxxxxxx# Create only if absent[[ -f /etc/cfg ]] || cp template /etc/cfg
# Add line oncegrep -qxF "line" file || echo "line" >> file
# Atomic symlink swapln -snf target link.new && mv -Tf link.new link
# Directory with permissions (idempotent)install -d -m 750 -o app /opt/app/data
# Truncate live log (preserves inode/fd)truncate -s 0 /var/log/app.logxxxxxxxxxxbash -n script.sh # syntax check, no runset -x; ...; set +x # trace blockset -euo pipefail # catch errors, unset vars, pipeline failuresPS4='${BASH_SOURCE}:${LINENO}: ' # rich trace prefixtrap 'echo "ERR line $LINENO: $BASH_COMMAND"' ERRshellcheck --severity=warning script.sh| Task | Bash | Python | Go |
|---|---|---|---|
| Wrap CLI tools | ✓✓ | ✓ | - |
| Parse JSON/CSV | - | ✓✓ | ✓ |
| HTTP API calls | ✓ | ✓✓ | ✓ |
| Parallel work | ✓ (xargs) | ✓ | ✓✓ |
| Single binary | - | - | ✓✓ |
| Unit tests | bats (ok) | ✓✓ | ✓✓ |
| > 200 lines | caution | ✓✓ | ✓✓ |
| Cross-platform | - | ✓✓ | ✓✓ |