Docker Compose
Compose turns a multi-container application into a declarative project—one YAML file describes services, networks, volumes, and secrets; docker compose up materializes the whole stack. Master the Compose Spec and you can reproduce any environment from laptop to single-server production without rewriting orchestration logic.
Compose architecture
Compose is not a separate runtime—it is a client that drives the Docker Engine API. It reads YAML, resolves variables, merges files, and creates labeled resources grouped under a project name. Every container, network, and volume gets com.docker.compose.* labels so docker compose down can tear down exactly what up created.
compose.yaml vs docker-compose.yml
Both filenames work. Docker Compose searches, in order: compose.yaml, compose.yml, docker-compose.yaml, docker-compose.yml. The modern convention is compose.yaml—shorter, tool-agnostic, and aligned with the Compose Specification. Legacy repos still use docker-compose.yml; migrate when you touch the file anyway.
The Compose Specification
The Compose Spec is an open, vendor-neutral schema for multi-container apps. Docker Compose V2, Podman Compose, and other implementations target the same fields: services, networks, volumes, configs, secrets. Fields like deploy are implementation-specific—honored by Swarm and some production Compose setups, ignored by default local Engine.
docker compose V2 vs docker-compose V1 (deprecated)
| Aspect | docker compose (V2) | docker-compose (V1) |
|---|---|---|
| Distribution | Docker CLI plugin (docker compose) | Standalone Python binary |
| Engine API | Native BuildKit integration, compose watch | Legacy build path; no watch |
| Spec support | Compose Spec 3.x+ (no top-level version: required) | Frozen at Compose file format v3; unmaintained |
| Status | Current default (Docker Desktop, Engine 20.10+) | Deprecated June 2023; remove from CI images |
# Verify V2 plugin (note: no hyphen)
docker compose version
# V1 still on PATH? Uninstall or alias away
which docker-compose # should be empty or a warning wrapper
Project name
Compose groups resources under a project name, defaulting to the directory basename (folder my-api → project my-api). Override with COMPOSE_PROJECT_NAME, -p myproject, or top-level name: in compose.yaml. Project name prefixes container names (my-api-api-1) and default network (my-api_default).
Resource types
| Resource | Purpose | Engine object |
|---|---|---|
| service | A container definition (image or build, env, ports, deps) | Container(s) — one per replica |
| network | Private DNS + L3 connectivity between services | Bridge/overlay network |
| volume | Persistent or shared storage | Named volume or bind mount |
| config | Non-secret config file mounted into containers | Swarm config / file bind (local) |
| secret | Sensitive data (passwords, keys) — not in env vars | Swarm secret / file in /run/secrets |
Compose V2 is written in Go and ships as a CLI plugin. It builds an internal model from merged YAML, then issues the same API calls you would script with docker container create, network create, and volume create—with consistent labels for project tracking.
Running two clones of the same repo in different directories creates two projects with separate networks and volumes. Port collisions happen when both publish 8080:8080. Use -p or per-developer .env overrides for host ports.
When asked "Docker vs Compose," say: Docker runs one container; Compose orchestrates a labeled project of many containers with shared networking and lifecycle commands. Compose does not replace Kubernetes—it solves single-host, declarative multi-container workflows.
compose.yaml deep dive
Every production-grade compose file balances build vs pull, startup ordering, and network isolation. The fields below are the ones you will edit weekly—not the full spec appendix.
Minimal project skeleton
name: order-platform
services:
api:
image: ghcr.io/acme/api:${API_TAG:-latest}
ports:
- "${API_PORT:-8080}:8080"
environment:
DATABASE_URL: postgres://app:${DB_PASSWORD}@db:5432/orders
depends_on:
db:
condition: service_healthy
networks:
- frontend
- backend
db:
image: postgres:16-alpine
volumes:
- pgdata:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U app -d orders"]
interval: 5s
timeout: 3s
retries: 5
networks:
- backend
networks:
frontend:
backend:
internal: true
volumes:
pgdata:
image vs build
Use image: when CI already built and pushed the artifact—you want reproducible digests in prod. Use build: for local iteration where source changes should trigger rebuilds. You can specify both: Compose tags the build result with the image name.
build: context, args, target
services:
api:
build:
context: ./services/api # directory sent as build context
dockerfile: Dockerfile.prod # relative to context
args:
NODE_ENV: development
APP_VERSION: ${GIT_SHA:-dev}
target: runtime # multi-stage: stop at this stage
image: acme/api:${GIT_SHA:-dev}
ports, volumes, environment
| Field | Syntax | Notes |
|---|---|---|
| ports | "HOST:CONTAINER" or "127.0.0.1:8080:8080" | Long syntax supports published, target, protocol |
| volumes | named_vol:/path or ./host:/container | Named volumes survive down; bind mounts reflect host files live |
| environment | Map or list (KEY=value) | Prefer env_file: for long lists; never commit secrets |
depends_on with conditions
Plain depends_on: [db] only waits for the container to start, not for Postgres to accept connections. Use condition with healthchecks for real readiness:
services:
api:
depends_on:
db:
condition: service_healthy
redis:
condition: service_started # no healthcheck defined
db:
healthcheck:
test: ["CMD-SHELL", "pg_isready -U $$POSTGRES_USER"]
interval: 5s
retries: 10
start_period: 30s
healthcheck
Healthchecks run inside the container. Use CMD-SHELL for shell pipelines, CMD for exec form. start_period suppresses failure counts during slow boot (migrations, JVM warmup).
networks
Services join networks listed under networks:. DNS name = service name. internal: true blocks outbound internet—good for databases and message brokers that should only talk to app tiers.
deploy block
The deploy key configures Swarm mode and is ignored by default on plain Docker Engine. Some teams use docker compose --compatibility to map replicas and limits to Engine. For single-host prod without Swarm, set mem_limit, cpus, and restart at service level instead.
services:
worker:
image: acme/worker:2.1.0
deploy:
replicas: 2
resources:
limits:
cpus: "0.50"
memory: 512M
reservations:
memory: 256M
restart_policy:
condition: on-failure
max_attempts: 3
profiles
Profiles gate optional services. Only services without a profile, or whose profile you activate, start on up.
# Start core stack only
docker compose up -d
# Add observability and mail catcher
docker compose --profile observability --profile devtools up -d
services:
prometheus:
image: prom/prometheus:v2.52.0
profiles: [observability]
mailhog:
image: mailhog/mailhog
profiles: [devtools]
extends (legacy pattern)
extends let one service inherit another's definition from a different file. The Compose Spec deprecated extends in favor of include (Compose 2.20+) or YAML anchors. Prefer multiple -f merges or shared fragments via include:.
Run docker compose config before every PR that touches YAML—it resolves variables, merges overrides, and prints the effective file. Catch typos and circular dependencies before CI does.
Drop the top-level version: "3.8" key in new files. Compose V2 infers the schema from the Spec; the version field is obsolete and confuses reviewers.
build: in compose.yaml is convenient locally but couples deploy artifacts to developer laptops. Architects often use image: with immutable tags in prod compose files and keep build: only in override files.
Multi-environment compose
One repository, three runtimes: laptop, staging VM, production server. The pattern is a base compose.yaml plus environment-specific overlays—never three unrelated copies that drift apart.
File layout
deploy/
├── compose.yaml # shared services, networks, volumes
├── compose.override.yaml # auto-loaded locally (gitignored or committed)
├── compose.prod.yaml # prod: image tags, limits, no bind mounts
├── .env.example # documented defaults (committed)
└── .env # local secrets (gitignored)
Base compose.yaml
name: order-platform
services:
api:
image: ghcr.io/acme/api:${API_TAG:-latest}
environment:
LOG_LEVEL: ${LOG_LEVEL:-info}
DATABASE_URL: postgres://app:${DB_PASSWORD}@db:5432/orders
depends_on:
db:
condition: service_healthy
networks: [app]
db:
image: postgres:16-alpine
environment:
POSTGRES_USER: app
POSTGRES_PASSWORD: ${DB_PASSWORD}
POSTGRES_DB: orders
volumes:
- pgdata:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U app -d orders"]
interval: 5s
retries: 5
networks: [app]
networks:
app:
volumes:
pgdata:
compose.override.yaml (local dev)
Compose automatically merges compose.override.yaml when present—no -f flag needed. Override files typically add bind mounts, published ports, and debug env vars.
services:
api:
build:
context: ../services/api
target: dev
ports:
- "127.0.0.1:8080:8080"
volumes:
- ../services/api/src:/app/src:cached
environment:
LOG_LEVEL: debug
DEBUG: "true"
compose.prod.yaml
services:
api:
image: ghcr.io/acme/api:${API_TAG} # required in prod — no default
restart: unless-stopped
ports:
- "443:8080"
deploy:
resources:
limits:
memory: 1G
cpus: "1.0"
logging:
driver: json-file
options:
max-size: "10m"
max-file: "5"
db:
restart: unless-stopped
# no host port publish — DB reachable only on app network
Merging with -f
Later files override earlier keys. Production deploy:
docker compose \
-f compose.yaml \
-f compose.prod.yaml \
--env-file .env.prod \
up -d
Variable substitution: ${VAR:-default}
| Syntax | Behavior |
|---|---|
| ${VAR} | Required; empty string if unset (may break startup) |
| ${VAR:-default} | Use default when unset or empty |
| ${VAR:?error} | Fail fast with message if unset—ideal for prod secrets |
.env file
Compose loads .env from the project directory for interpolation only—it does not automatically inject vars into containers unless referenced in environment: or env_file:. Precedence: shell env > --env-file > .env.
# .env.example — commit this
API_TAG=latest
API_PORT=8080
LOG_LEVEL=info
DB_PASSWORD=change-me-local
# .env — gitignore; copy from example
DB_PASSWORD=dev-secret-42
Never commit .env with production credentials. Use ${DB_PASSWORD:?set DB_PASSWORD} in prod overlays so a missing secret aborts deploy instead of starting with empty passwords.
Teams on GitOps often generate the final compose from templates (Helm-style) at deploy time. The source of truth remains the base + overlay pattern; CI injects image digests and secrets from a vault.
Compose for local development
Local Compose should optimize for fast feedback: edit code on the host, see changes in the container, attach a debugger, seed realistic data. Production concerns (immutable images, tight resource caps) stay in overlays.
Bind mounts for hot reload
Mount source directories over the container path so interpreters with watch mode (Node, Go air, Spring DevTools) reload without rebuilds. Use :cached on macOS Docker Desktop to reduce filesystem sync overhead.
services:
api:
build:
context: ./api
target: dev
command: npm run dev
volumes:
- ./api/src:/app/src:cached
- ./api/package.json:/app/package.json:ro
environment:
NODE_ENV: development
CHOKIDAR_USEPOLLING: "true" # file watch on Docker Desktop VM
Debugger entrypoint override
Override command or entrypoint in an override file to start the JVM or Node inspector listening on a published port.
services:
api:
ports:
- "8080:8080"
- "9229:9229" # Node inspector
command: node --inspect=0.0.0.0:9229 src/index.js
# Java example
billing:
ports:
- "5005:5005"
environment:
JAVA_TOOL_OPTIONS: "-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:5005"
Database seeding
Postgres and MySQL official images run /docker-entrypoint-initdb.d/* scripts on first volume init. Mount SQL or shell seeders for consistent local data.
services:
db:
image: postgres:16-alpine
environment:
POSTGRES_USER: app
POSTGRES_PASSWORD: dev
POSTGRES_DB: orders
volumes:
- pgdata:/var/lib/postgresql/data
- ./db/init:/docker-entrypoint-initdb.d:ro # 01-schema.sql, 02-seed.sql
volumes:
pgdata:
Full local stack example
name: shop-local
services:
api:
build:
context: ./api
target: dev
ports:
- "127.0.0.1:3000:3000"
volumes:
- ./api:/app:cached
environment:
REDIS_URL: redis://cache:6379
DATABASE_URL: postgres://app:dev@db:5432/shop
depends_on:
db:
condition: service_healthy
cache:
condition: service_started
networks: [frontend, backend]
db:
image: postgres:16-alpine
environment:
POSTGRES_USER: app
POSTGRES_PASSWORD: dev
POSTGRES_DB: shop
volumes:
- pgdata:/var/lib/postgresql/data
- ./db/init:/docker-entrypoint-initdb.d:ro
healthcheck:
test: ["CMD-SHELL", "pg_isready -U app -d shop"]
interval: 3s
retries: 10
networks: [backend]
cache:
image: redis:7-alpine
networks: [backend]
worker:
build:
context: ./worker
command: npm run jobs:dev
volumes:
- ./worker:/app:cached
environment:
REDIS_URL: redis://cache:6379
depends_on: [cache]
networks: [backend]
profiles: [jobs]
networks:
frontend:
backend:
volumes:
pgdata:
docker compose watch (sync / rebuild)
Compose Watch (V2.22+) watches host paths and syncs files into running containers or rebuilds on change—no manual volume gymnastics for some workflows.
services:
api:
build: ./api
develop:
watch:
- path: ./api/src
action: sync
target: /app/src
- path: ./api/package.json
action: rebuild
docker compose watch
# or
docker compose up --watch
Add an anonymous volume over node_modules (/app/node_modules) when bind-mounting a Node app—otherwise host node_modules (wrong arch) shadows the container install.
Bind-mounting over a directory that also contains build artifacts can hide files the image expects. Use multi-stage Dockerfiles with a dev target that installs deps inside the image, then mount only src/.
Compose networking
Every Compose project gets a default bridge network with embedded DNS. Service name db resolves to the DB container IP—no hard-coded addresses. Tiered networks add defense in depth; external networks connect to existing infrastructure.
Default project network
If you omit networks:, all services join <project>_default. Containers reach each other by service name on any internal port—only ports: publishes to the host.
# From api container — DNS resolves "db"
docker compose exec api getent hosts db
# Inspect the project network
docker network inspect shop-local_backend
Custom tier networks
Split frontend (edge + API) from backend (API + data stores). Mark backend internal: true to block outbound internet while preserving east-west traffic.
services:
gateway:
image: nginx:alpine
ports:
- "80:80"
networks: [frontend]
api:
image: acme/api:1.4.0
networks: [frontend, backend]
db:
image: postgres:16-alpine
networks: [backend]
networks:
frontend:
driver: bridge
backend:
driver: bridge
internal: true
External networks
Join a network created outside the project—shared reverse proxy, monitoring agent, or legacy VM bridge. Compose will not delete external networks on down.
# Create once on the host
docker network create proxy
# compose.yaml references it
services:
api:
image: acme/api:1.4.0
networks:
- default
- proxy
networks:
proxy:
external: true
name: proxy
Compose uses the embedded DNS server at 127.0.0.11 inside each container. It returns A records for service names scoped to networks that container shares—if api is only on frontend, a container on backend-only cannot resolve it.
Host networking (network_mode: host) removes NAT overhead but breaks cross-platform portability and DNS-based service discovery. Reserve for performance-sensitive agents on Linux prod servers—not for default dev stacks.
Compose commands reference
V2 commands live under docker compose (space, not hyphen). Global flags: -f (files), -p (project), --profile, --env-file.
| Command | Purpose | Common flags / example |
|---|---|---|
| up | Create and start services | docker compose up -d --build --remove-orphans |
| down | Stop and remove containers, networks; optional volumes | docker compose down -v --rmi local |
| logs | Stream aggregated service logs | docker compose logs -f --tail=100 api worker |
| exec | Run command in running container | docker compose exec api sh |
| ps | List project containers and state | docker compose ps -a |
| build | Build or rebuild images | docker compose build --no-cache api |
| pull | Pull service images | docker compose pull |
| scale | Scale service replicas (non-Swarm) | docker compose up -d --scale worker=3 |
| run | One-off command in new container | docker compose run --rm api npm test |
| config | Validate and print merged compose | docker compose config --quiet |
| top | Show running processes per service | docker compose top api |
Day-one workflow
docker compose config # validate YAML + env
docker compose pull # refresh base images
docker compose up -d --build # build local services, start detached
docker compose ps # health status
docker compose logs -f api # tail one service
docker compose exec db psql -U app -d orders
docker compose run --rm api npm test
docker compose down # stop; add -v to wipe named volumes
docker compose run does not publish ports by default and creates a one-off container—use it for migrations and tests. exec requires the service to already be running.
Know the difference: down -v deletes named volumes declared in the compose file—data loss in dev if you forget. stop halts containers but keeps them for inspection.
Production compose patterns
Compose on a single VM or bare-metal server is a valid production model for small teams—especially before Kubernetes ROI makes sense. Treat the compose file like infrastructure code: pinned digests, secrets, health gates, and log rotation.
Single-server production compose.yaml
name: order-platform-prod
services:
api:
image: ghcr.io/acme/api@sha256:abc123... # immutable digest
restart: unless-stopped
ports:
- "443:8080"
environment:
DATABASE_URL: postgres://app@db:5432/orders
LOG_LEVEL: info
secrets:
- db_password
depends_on:
db:
condition: service_healthy
deploy:
resources:
limits:
cpus: "2.0"
memory: 2G
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
interval: 30s
timeout: 5s
retries: 3
start_period: 60s
logging:
driver: json-file
options:
max-size: "25m"
max-file: "10"
networks: [edge, data]
db:
image: postgres:16-alpine
restart: unless-stopped
environment:
POSTGRES_USER: app
POSTGRES_PASSWORD_FILE: /run/secrets/db_password
POSTGRES_DB: orders
secrets:
- db_password
volumes:
- pgdata:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U app -d orders"]
interval: 10s
retries: 5
deploy:
resources:
limits:
memory: 4G
logging:
driver: json-file
options:
max-size: "10m"
max-file: "5"
networks: [data]
secrets:
db_password:
file: ./secrets/db_password.txt # chmod 600; not in git
networks:
edge:
data:
internal: true
volumes:
pgdata:
driver: local
Pattern checklist
| Pattern | Why | Compose mechanism |
|---|---|---|
| Resource limits | Prevent one container OOM-killing the host | deploy.resources.limits or mem_limit / cpus |
| Healthchecks + depends_on | Avoid traffic to half-ready APIs | condition: service_healthy |
| Secrets | Keep credentials out of env and image layers | secrets: → /run/secrets/ |
| restart: unless-stopped | Survive daemon restarts and reboots | Service-level restart policy |
| Log rotation | json-file logs fill disk without limits | logging.options.max-size / max-file |
| No bind mounts | Immutable deploy surface | Named volumes only; images from CI |
Deploy command
# On production host
export API_TAG=sha256:abc123...
docker compose -f compose.yaml -f compose.prod.yaml pull
docker compose -f compose.yaml -f compose.prod.yaml up -d --remove-orphans
docker compose ps
Mount the Docker socket (/var/run/docker.sock) only in trusted sidecars. Any container with socket access has effective root on the host. Prefer read-only root filesystem and dropped capabilities where your images support it.
Compose vs Kubernetes: Compose excels at 1–3 node deployments with simple rollouts. When you need zero-downtime rolling updates across many nodes, pod autoscaling, or CRDs—migrate the same images to K8s. The compose file remains valuable as documentation of service relationships.
Many teams run Traefik or Caddy as a separate compose project on an external proxy network. App stacks join that network; TLS termination stays in the edge stack while app compose files stay portable.