Docker Images & Layers

What an image contains

• Ordered read-only layers — each layer is a tar archive of files added, modified, or deleted (whiteout markers for deletes)
• Image manifest — JSON listing layer digests + config digest; may be a manifest list for multi-arch
• Image config — env vars, entrypoint, cmd, exposed ports, labels, build history metadata
• Content-addressed storage — layers identified by SHA256 digest; same bytes = same digest everywhere

┌─────────────────────────────────────────┐  ← image config (JSON)
│  CMD, ENV, ENTRYPOINT, ExposedPorts     │
├─────────────────────────────────────────┤
│  Layer 4  (RUN npm run build)     diff  │  ← read-only
├─────────────────────────────────────────┤
│  Layer 3  (COPY src/ ./src/)      diff  │
├─────────────────────────────────────────┤
│  Layer 2  (RUN npm ci)            diff  │
├─────────────────────────────────────────┤
│  Layer 1  (COPY package*.json)    diff  │
├─────────────────────────────────────────┤
│  Layer 0  (FROM node:20-alpine)   base  │  ← parent image layers
└─────────────────────────────────────────┘
         merged view → container rootfs (+ writable upperdir at runtime)

Manifest and config

When you docker pull nginx:1.25, the client fetches the manifest for that tag, then pulls each layer blob by digest. The tag is a mutable pointer; the digest (nginx@sha256:abc…) is immutable. Production deploys should pin digests.

Dangling images

After rebuilding, old layer chains lose their tag—they become dangling images (<none> in docker image ls). They still consume disk until pruned. CI runners accumulate these rapidly without docker image prune or registry lifecycle policies.

# Inspect image layers and sizes
docker history --no-trunc myapp:1.0

# Show manifest digest (immutable reference)
docker inspect --format '{{index .RepoDigests 0}}' nginx:1.25-alpine

# Find dangling images
docker images -f dangling=true

How the cache works

For each Dockerfile instruction, BuildKit/Docker computes a cache key from:

• The instruction text itself
• The parent layer digest
• A hash of files referenced by COPY/ADD (build context)

Cache hit → skip execution, reuse existing layer. Cache miss → execute instruction, create new layer, invalidate all subsequent instructions.

The golden pattern

Order from least-changing to most-changing:

1. OS packages and system dependencies
2. Language dependency manifests (package-lock.json, pom.xml, requirements.txt)
3. Install dependencies (npm ci, mvn dependency:go-offline)
4. Application source code last

# syntax=docker/dockerfile:1.6
FROM node:20-alpine
WORKDIR /app
# 1. Copy only lockfiles — cache survives source changes
COPY package.json package-lock.json ./
RUN npm ci --omit=dev
# 2. Source changes only invalidate this layer
COPY . .
RUN npm run build

Cache invalidation triggers

FROM — base image and stages

Every Dockerfile begins with FROM. Special bases: scratch (empty—used for static Go binaries), distroless (minimal runtime, no shell). Always pin to digest in production: FROM debian:bookworm-slim@sha256:….

RUN — shell form vs exec form

Shell form: RUN npm install → runs as /bin/sh -c "npm install". Exec form: RUN ["npm", "install"] — no shell, no variable expansion. Chain commands with && in one RUN to avoid extra layers and ensure fail-fast.

RUN apt-get update \
 && apt-get install -y --no-install-recommends curl ca-certificates \
 && rm -rf /var/lib/apt/lists/*

COPY vs ADD

COPY is explicit—files from build context only. ADD additionally auto-extracts local tar archives and can fetch URLs (without cache benefits of COPY). Docker official best practice: use COPY unless you specifically need tar extraction.

ARG vs ENV — build-time vs runtime

ARG exists only during docker build—not in running containers. ENV persists into runtime and appears in docker inspect. Neither is safe for secrets: ARG shows in history, ENV shows in inspect and child images.

ENTRYPOINT vs CMD — PID 1 semantics

USER — non-root by default in production

Set USER before the final CMD/ENTRYPOINT. Prefer numeric UID (USER 1001) over username to avoid base-image-specific user tables. Use COPY --chown=1001:1001 during build—don't chown at runtime.

HEALTHCHECK

Defines how Docker (and Compose/K8s via translation) determines if the container is healthy. Parameters: --interval, --timeout, --retries, --start-period. Unhealthy status appears in docker ps STATUS column.

Named stages and COPY --from

Each FROM begins a new stage. Name stages with AS builder. COPY --from=builder pulls files from a previous stage—not from the build context. Only the final stage becomes the tagged image (unless you --target a specific stage).

Java — Maven build → JRE runtime

# syntax=docker/dockerfile:1.6
FROM maven:3.9-eclipse-temurin-21 AS builder
WORKDIR /build
COPY pom.xml .
RUN mvn -B dependency:go-offline
COPY src ./src
RUN mvn -B package -DskipTests

FROM eclipse-temurin:21-jre-alpine
WORKDIR /app
COPY --from=builder /build/target/*.jar app.jar
USER 1001
ENTRYPOINT ["java", "-jar", "app.jar"]

Node.js — build → alpine runtime

FROM node:20 AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

FROM node:20-alpine
WORKDIR /app
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
USER node
CMD ["node", "dist/server.js"]

Go — static binary → scratch

FROM golang:1.22-alpine AS builder
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build -ldflags="-s -w" -o /app .

FROM scratch
COPY --from=builder /app /app
USER 65532:65532
ENTRYPOINT ["/app"]

Test stage — fail build if tests fail

FROM builder AS test
RUN mvn -B test

FROM eclipse-temurin:21-jre-alpine AS runtime
COPY --from=builder /build/target/*.jar /app.jar

CI runs docker build --target test to execute tests in the build graph without shipping the test stage.

Alpine and musl gotchas

Alpine uses musl libc instead of glibc. Many prebuilt native binaries (Oracle JDK, some Python wheels, Node native modules) assume glibc. Java on Alpine needs a musl-aware build or an extra glibc compatibility layer— often negating size wins. For JVM production, prefer distroless/java21 or eclipse-temurin:21-jre-alpine with tested native deps.

Distroless — Google's minimal production base

Distroless images contain only your app and runtime dependencies—no shell, no package manager, no curl. Attack surface shrinks dramatically. Debug with :debug tags (include busybox shell) during development only.

Measure before optimizing

# List images by size
docker image ls --format 'table {{.Repository}}\t{{.Tag}}\t{{.Size}}'

# Per-layer breakdown
docker history --human --no-trunc myapp:latest

# Deep analysis (install dive: https://github.com/wagoodman/dive)
dive myapp:latest

Techniques ranked by impact

.dockerignore essentials

.git
.gitignore
node_modules
npm-debug.log
target/
*.md
.env
.env.*
coverage/
.idea/
.vscode/
**/*_test.go
Dockerfile*

Enable BuildKit

# Per-build
DOCKER_BUILDKIT=1 docker build -t myapp .

# Permanent (daemon.json)
# { "features": { "buildkit": true } }

# Dockerfile syntax directive (unlocks latest features)
# syntax=docker/dockerfile:1.6

Cache mounts — persist across builds

Unlike regular layers, cache mounts are not committed to the image. Maven, npm, and Go module caches survive between builds without bloating the final image.

# syntax=docker/dockerfile:1.6
FROM maven:3.9-eclipse-temurin-21 AS builder
WORKDIR /build
COPY pom.xml .
RUN --mount=type=cache,target=/root/.m2 \
    mvn -B dependency:go-offline
COPY src ./src
RUN --mount=type=cache,target=/root/.m2 \
    mvn -B package -DskipTests

Secret mounts — never in layers

# Build with secret (not stored in image)
docker build --secret id=npmrc,src=$HOME/.npmrc -t myapp .

# Dockerfile
# RUN --mount=type=secret,id=npmrc,target=/root/.npmrc npm ci

SSH mounts — private git clones

RUN --mount=type=ssh forwards your SSH agent into the build—clone private repos without embedding keys in the image. Build with docker build --ssh default.

Registry cache — fast CI

docker buildx build \
  --cache-from type=registry,ref=myregistry/myapp:buildcache \
  --cache-to type=registry,ref=myregistry/myapp:buildcache,mode=max \
  --push -t myregistry/myapp:latest .

BuildKit features reference

flowchart LR
  CTX[Build context]
  BK[BuildKit solver]
  S1[Stage: builder]
  S2[Stage: runtime]
  CACHE[(Cache mounts\n.m2 / npm)]
  SEC[Secret mounts]
  REG[(Registry cache)]
  IMG[Final image]
  CTX --> BK
  BK --> S1
  BK --> S2
  S1 --> CACHE
  S1 --> SEC
  BK --> REG
  S2 --> IMG