Shell & Scripting

What is a shell?

A shell is a user-space program that reads input, expands syntax, and executes commands—by calling fork/exec on binaries or running shell builtins.

Shell	Typical use	Notes
`bash`	Linux servers, CI, Docker `ENTRYPOINT`	Default on most distros; bash 4+ for associative arrays. Write scripts for bash if you target Linux.
`zsh`	macOS default since Catalina, interactive dev	Better tab completion, globbing; scripts often still target bash for portability.
`sh` / `dash`	`/bin/sh` on Debian/Ubuntu (dash)	POSIX subset; faster, stricter. Use when you need minimal POSIX-only scripts.

echo $SHELL
echo $0                       # shell or script name in a script
bash --version
ps -p $$ -o comm=            # current shell process name

Pro Tip: Interactive config lives in ~/.bashrc (bash) or ~/.zshrc (zsh). Non-interactive CI jobs often run with a minimal environment—don't assume aliases exist.

Variables and environment

Shell variables are strings (arrays exist in bash). Environment variables are exported to child processes— how PATH, HOME, and app secrets reach your commands.

Form	Meaning
`name=value`	Shell variable (no spaces around `=`)
`export name=value`	Variable visible to child processes
`${name}`	Expansion; required when adjacent to alphanumerics
`${name:-default}`	Use default if unset or empty
`$(command)`	Command substitution — stdout becomes a string
`$?`	Exit code of last command

APP_ENV=production
export APP_ENV

echo "Deploying to ${APP_ENV:-staging}"
RELEASE_DIR="/opt/app/releases/$(date +%Y%m%d_%H%M%S)"
mkdir -p "$RELEASE_DIR"

# Read-only pass through env
env | grep -E '^AWS_' | sort

Quoting and special characters

Quoting controls word splitting and expansion. Getting this wrong is the #1 source of "it worked in my terminal" bugs.

Single quotes '...'

Literal string—no expansion of $, backticks, or backslash (except ' itself cannot contain single quote easily).

Double quotes "..."

Expands $var and $(cmd); preserves spaces as one word. Prefer for paths and messages.

Backslash \

Escapes the next character; line continuation at end of line.

# BAD: word splitting on spaces
for f in $(ls *.log); do echo "$f"; done

# GOOD: glob or find -print0
for f in *.log; do echo "$f"; done

name="O'Brien"
grep "$name" users.txt       # double quotes OK with apostrophe inside

# Special chars often need quoting: * ? [ ] & | ; $ ` !

Warning: Never interpolate unchecked user input into eval or unquoted variables—command injection. Use quoted variables and validate paths.

Pipes and redirection

Unix composes small tools: stdout of one program becomes stdin of the next. File descriptors 0 (stdin), 1 (stdout), 2 (stderr) can be redirected.

  cmd1  ──stdout──►  |  ──stdin──►  cmd2  ──stdout──►  terminal
                         pipe (kernel buffer)

Operator	Effect
`\|`	Pipe stdout of left command to stdin of right
`> file`	Redirect stdout (overwrite file)
`>> file`	Redirect stdout (append)
`2> file`	Redirect stderr
`&> file`	Redirect stdout and stderr (bash)
`2>&1`	Send stderr to same place as stdout
`< file`	Read stdin from file
`<<'EOF'`	Here-document (literal until EOF)

# Pipeline — filter then count
grep -h ERROR /var/log/app/*.log | sort | uniq -c | sort -rn | head

# Separate stdout and stderr
./deploy.sh > deploy.log 2> deploy.err

# Both to one file (common in cron)
./backup.sh >> /var/log/backup.log 2>&1

# Discard stdout, keep errors
./noisy.sh > /dev/null

# Here-doc — feed config to a command
cat <<'EOF' | sudo tee /etc/myapp/config.toml
listen = ":8080"
debug = false
EOF

Pro Tip: In bash scripts use set -o pipefail so a failure in the middle of a pipeline fails the whole pipeline—not just the last command.

Your first shell script

A script is a text file with a shebang line telling the kernel which interpreter to use, plus commands you'd type manually—automated and repeatable.

#!/usr/bin/env bash
# deploy.sh — pull release artifact and restart app (example)
set -euo pipefail

readonly APP_ROOT="/opt/myapp"
readonly RELEASE_URL="${1:?Usage: $0 <release-url>}"

log() { printf '[%s] %s\n' "$(date -Is)" "$*"; }

tmpdir=$(mktemp -d)
trap 'rm -rf "$tmpdir"' EXIT

log "Downloading release"
curl -fsSL "$RELEASE_URL" -o "$tmpdir/release.tar.gz"

log "Extracting"
tar -xzf "$tmpdir/release.tar.gz" -C "$tmpdir"

log "Activating"
rsync -a --delete "$tmpdir/dist/" "$APP_ROOT/current/"
systemctl restart myapp

log "Health check"
for i in {1..30}; do
  if curl -fsS http://127.0.0.1:8080/health; then
    log "OK"
    exit 0
  fi
  sleep 2
done

log "Health check failed"
exit 1

chmod +x deploy.sh
./deploy.sh https://releases.example.com/app-1.2.3.tar.gz

Shebang options

#!/bin/bash — explicit path (may differ across distros)
#!/usr/bin/env bash — finds bash on PATH (portable)
#!/bin/sh — POSIX mode when you avoid bash-only features

Pro Tip: set -euo pipefail — exit on error, treat unset vars as errors, fail pipelines on any stage failure. Standard for maintainable bash.

Loops, conditionals, functions

Control flow turns one-off commands into tools. Prefer explicit tests and quoted variables.

Conditionals

if [[ -f /etc/myapp/config.yml ]]; then
  echo "Config exists"
elif [[ -d /opt/myapp ]]; then
  echo "App dir only"
else
  echo "Missing setup" >&2
  exit 1
fi

# Test commands: [[ ]] (bash), [ ] (POSIX)
# Files: -f file  -d dir  -r readable  -x executable
# Strings: -z empty  -n non-empty  str1 = str2
# Numbers: -eq -ne -lt -gt

Loops

# For each argument
for arg in "$@"; do
  echo "Processing: $arg"
done

# C-style counter
for ((i = 0; i < 5; i++)); do
  echo "attempt $i"
done

# While read lines (safe for whitespace in paths)
while IFS= read -r line; do
  echo "$line"
done < /etc/hosts

# While command succeeds
while curl -fsS http://localhost:8080/health; do
  sleep 5
done

Functions

retry() {
  local max="${1:-5}"
  local delay="${2:-2}"
  shift 2
  local n=1
  until "$@"; do
    if ((n >= max)); then
      return 1
    fi
    sleep "$delay"
    ((n++))
  done
}

retry 3 2 curl -fsS https://api.example.com/ready

Case statement

case "${1:-}" in
  start)
    systemctl start myapp
    ;;
  stop)
    systemctl stop myapp
    ;;
  status)
    systemctl status myapp
    ;;
  *)
    echo "Usage: $0 {start|stop|status}" >&2
    exit 2
    ;;
esac

Production habits

Scripts that run in cron, CI, or deploy pipelines need stricter discipline than interactive typing.

Strict mode: set -euo pipefail at the top after shebang.
Cleanup: trap on EXIT to remove temp dirs and release locks.
Logging: timestamped messages to stderr; redirect stdout/stderr in cron.
Absolute paths: cron has minimal PATH—use full paths or set PATH explicitly.
Lint: run shellcheck deploy.sh before merge.

#!/usr/bin/env bash
set -euo pipefail
IFS=$'\n\t'

LOCK_FILE="/var/lock/myapp-backup.lock"
exec 9>"$LOCK_FILE"
flock -n 9 || { echo "Another backup running" >&2; exit 1; }

# shellcheck disable=SC2317 — called via trap
cleanup() { rm -f "$LOCK_FILE"; }
trap cleanup EXIT

/usr/local/bin/backup.sh

Related: schedule with crontab (see Commands Reference), process behavior in Process Management, and text tools in grep, sed, awk.

Pro Tip: For anything beyond ~100 lines, consider Python or Go—but bash excels at gluing CLI tools, deploy hooks, and CI steps. dev