Docker Core · companion to Unix Core & DevOps

Learn Docker from kernel primitives to production containers

Internals-first guides for developers, DevOps engineers, and architects—what Linux actually does when you docker run, why layers cost disk and cache, and what breaks in production.

Browse all sections ↓ Why containers? Start with internals

Why containers? The "works on my machine" problem

You shipped code that passed CI, passed QA, and crashed in production because prod ran a different libc, a different JDK patch level, or a missing libssl. VMs solved isolation but each one carries a full guest OS—heavy, slow to boot, expensive to patch.

Containers package your app and its dependencies into an immutable image, then run it as an isolated process on a shared Linux kernel. Same bytes everywhere: laptop, CI runner, staging, prod. Think of an image as a class and a container as an instance—one blueprint, many running copies, each with its own filesystem view and resource budget.

VM vs container: what's different at the kernel level

A VM runs a hypervisor + guest OS + your app. A container runs your app directly on the host kernel, isolated by namespaces and cgroups—not by virtualizing hardware.

Dimension Virtual machine Linux container
Isolation mechanism Hardware virtualization + guest kernel Namespaces + cgroups on shared kernel
Startup time Minutes (boot guest OS) Milliseconds (fork + mount)
Memory overhead GB per VM (full OS) MB (process + writable layer)
Density per host Tens of VMs Hundreds of containers
Kernel sharing Each VM has its own kernel All containers share host kernel (Linux only)
Security boundary Stronger (hardware + guest kernel) Weaker—kernel bugs can affect all containers
⚖️ Trade-off

Containers win on speed, density, and portability. VMs win on hard multi-tenant isolation and running non-Linux kernels. Most teams use both: VMs for tenant boundaries, containers for app packaging inside them.

🎯 Interview Tip

Containers are not mini-VMs. They are isolated processes. If asked "how is a container isolated?", name the Linux primitives: PID, NET, MNT, UTS, IPC, USER namespaces + cgroup resource limits.

Docker's place in the ecosystem

Docker popularized containers, then donated the runtime to the community. Today the stack is layered: CLI and build tools on top, OCI-compliant runtime at the bottom, Linux kernel underneath everything.

🔬 Under the Hood

When you docker run nginx, the CLI talks to dockerd over a Unix socket. dockerd asks containerd to create a bundle (rootfs + config.json per the OCI Runtime Spec). containerd invokes runc create then runc start. runc calls clone() with CLONE_NEW* flags and writes cgroup limits to /sys/fs/cgroup/.

Docker ecosystem map

Docker is more than docker run—understand which tool owns build, orchestration, distribution, and security scanning in your pipeline.

Developer workflow

  • Docker CLI — build, run, exec, logs, inspect
  • Docker Desktop — local engine + GUI + Kubernetes toggle (Mac/Windows/Linux)
  • Docker Compose V2 — multi-container local stacks via docker compose

Build & cache

  • BuildKit — parallel stages, cache mounts, secret mounts (default Docker 23+)
  • buildx — multi-platform builds, remote builders, --platform
  • hadolint — Dockerfile linting in CI

Distribution

  • Docker Hub — public registry, official images, rate limits on free tier
  • Harbor / ECR / GHCR — private registries with scanning and lifecycle policies
  • Cosign + Syft — image signing and SBOM generation (Sigstore ecosystem)

Security & supply chain

  • Docker Scout — CVE analysis integrated into CLI and Hub (2022+)
  • Trivy / Grype — open-source image scanners for CI
  • Distroless / UBI — minimal, hardened base image choices

Runtime stack

  • dockerd — Docker Engine daemon (API, networking, volumes)
  • containerd — image and container lifecycle (CNCF graduated 2019)
  • runc — low-level OCI runtime spawning the container process

Production orchestration

  • Docker Compose — single-server / VPS production deployments
  • Kubernetes — multi-node orchestration; containerd/CRI-O on nodes
  • ECS / Fargate — AWS-managed containers without managing nodes
📦 Real World

GitHub Actions runs CI jobs in ephemeral containers built from your Dockerfile or service images. Netflix uses containers (Titus on AWS) for batch and streaming workloads. Google invented distroless images because they run billions of containers and need minimal attack surface.

💡 Pro Tip

Enable BuildKit permanently: add "features": { "buildkit": true } to ~/.docker/daemon.json or export DOCKER_BUILDKIT=1. You get cache mounts and secret mounts that never touch image layers.

Docker timeline Docker 24+ BuildKit Compose V2

From dotCloud side project to industry standard—and the deliberate unbundling of the runtime from the Docker brand.

  1. 2013

    dotCloud open-sources Docker

    Solomon Hykes demos lightweight containers at PyCon. LXC + namespaces + union filesystems packaged for developers.

  2. 2014

    Docker Hub & Docker Compose

    Central image registry launches. Fig (acquired) becomes Docker Compose—define multi-container apps in YAML.

  3. 2015

    OCI formed

    Docker donates runc to the Open Container Initiative. Runtime, image, and distribution specs prevent vendor lock-in.

  4. 2016

    Docker Swarm mode

    Native clustering built into Docker Engine. Largely superseded by Kubernetes but still used for simple multi-node setups.

  5. 2017

    Moby project & containerd → CNCF

    Docker splits into Moby (open-source assembly) and Docker CE/EE products. containerd donated to CNCF—becomes K8s default runtime.

  6. 2019

    BuildKit default

    BuildKit becomes opt-in default builder. Parallel multi-stage builds, inline cache, --mount=type=secret.

  7. 2020

    Docker Desktop for Apple M1

    ARM64 support mainstream. Multi-arch docker buildx workflows become essential for Apple Silicon devs shipping to amd64 prod.

  8. 2022

    Docker Scout

    Supply chain security: CVE dashboards, base image recommendations, policy gates in CI and Hub.

  9. 2024

    containerd image store default

    Docker Desktop and Engine 25+ use containerd for image storage by default—faster pull, better multi-platform manifest handling.

Quick architecture: docker run end to end

Every container starts the same way—whether you typed the command or Kubernetes did it via the CRI.

docker run

CLI parses flags, sends Create + Start to dockerd API

dockerd

Pull image, create network endpoint, allocate volume mounts

containerd → runc

Snapshot OverlayFS layers, write OCI bundle, clone() + exec

Container process

PID 1 in isolated namespaces, cgroup limits applied

flowchart TB
  subgraph user["Your machine"]
    CLI["Docker CLI\ndocker run / compose"]
  end
  subgraph engine["Docker Engine"]
    D["dockerd\nAPI · networks · volumes"]
    C["containerd\nimages · snapshots"]
    S["containerd-shim"]
    R["runc\nOCI runtime"]
  end
  subgraph kernel["Linux kernel"]
    NS["namespaces"]
    CG["cgroups v2"]
    OV["OverlayFS"]
  end
  P["Container process\nPID 1"]
  CLI --> D
  D --> C
  C --> S
  S --> R
  R --> NS
  R --> CG
  R --> OV
  R --> P
terminal — docker run 
$ docker run -d --name web -p 8080:80 --memory 256m nginx:1.25-alpine
Unable to find image 'nginx:1.25-alpine' locally
1.25-alpine: Pulling from library/nginx
Digest: sha256:a21a...c4f2
Status: Downloaded newer image for nginx:1.25-alpine
a3f8c2e91b04d7f6e8a1c9b2d3e4f5a6b7c8d9e0f1a2b3c4d5e6f7a8b9c0d1e2

$ docker inspect --format '{{.State.Pid}}' web
48291

$ ls -la /proc/48291/ns/
lrwxrwxrwx 1 root root 0 ipc -> ipc:[4026532987]
lrwxrwxrwx 1 root root 0 mnt -> mnt:[4026532985]
lrwxrwxrwx 1 root root 0 net -> net:[4026532988]
lrwxrwxrwx 1 root root 0 pid -> pid:[4026532986]
lrwxrwxrwx 1 root root 0 uts -> uts:[4026532984]
⚠️ Pitfall

Running containers without --memory or --cpus limits lets one container consume the entire host—OOM kills other containers or the host itself. Always set resource limits in production, even on single-server Compose deployments.

🔒 Security

Never run production workloads with --privileged. It grants nearly all capabilities and disables most seccomp filters—one container escape compromises the host.

Explore the guide — all sections

Thirteen deep-dive chapters plus cheat sheets. Recommended path: Internals → Images → Containers → Networking, then Compose, security, and CI/CD as your role requires.

Learning path: Internals · Images · Containers · Networking · Dockerfile Patterns

Developer

developer

Dockerfiles, Docker Compose for local dev, debugging with exec/logs, volume mounts for hot reload, and pushing images to registries.

DevOps / Platform Engineer

devops

CI/CD pipelines, multi-stage builds, BuildKit cache, image scanning, registry management, runtime security, and production Compose deployments.

Architect / Tech Lead

architect

Container strategy, base image governance, Docker vs Kubernetes vs Podman decisions, security posture, and VM-to-container migration patterns.