/user/kayd @ devops :~$ cat docker-compose-multi-container-guide.md

Docker Compose, Bake & ECR: Build and Ship Apps Docker Compose, Bake & ECR: Build and Ship Apps

QR Code linking to: Docker Compose, Bake & ECR: Build and Ship Apps
Karandeep Singh
Karandeep Singh
• 7 minutes

Summary

A hands-on guide that takes a web + Postgres + Redis stack from a single Docker Compose file in local dev, to building images with Docker Bake, to pushing them to Amazon ECR for ECS, Fargate, or EKS.

Running one container with docker run is easy. Running a web app plus its database plus a cache — each needing the right flags, network, and startup order — turns into a wall of fragile shell commands. Docker Compose fixes that by describing your whole stack in a single YAML file you can start with one command.

This guide is hands-on and follows the full path an image takes: define the stack in compose.yaml for local dev, build the images with Docker Bake, and push them to Amazon ECR so they’re ready to run on ECS, Fargate, or EKS.

Docker Compose, Bake, and ECR — build and ship a multi-container app

The Problem Compose Solves

Say your app needs three containers. By hand, that looks like this:

docker network create appnet
docker run -d --name db --network appnet -e POSTGRES_PASSWORD=secret postgres:16
docker run -d --name cache --network appnet redis:7
docker run -d --name web --network appnet -p 8080:8080 -e DATABASE_URL=... myapp

Every restart means retyping (or scripting) all of it. There’s no single source of truth, teammates set it up differently, and tearing it down cleanly is its own chore. Compose replaces all of that with one declarative file.

Your First compose.yaml

Create a file named compose.yaml in your project root. This defines the same three-container stack declaratively:

services:
  web:
    build: .                       # build from the local Dockerfile
    ports:
      - "8080:8080"
    environment:
      - DATABASE_URL=postgres://app:secret@db:5432/app
      - REDIS_URL=redis://cache:6379
    depends_on:
      - db
      - cache
  db:
    image: postgres:16
    environment:
      - POSTGRES_USER=app
      - POSTGRES_PASSWORD=secret
      - POSTGRES_DB=app
    volumes:
      - pgdata:/var/lib/postgresql/data
  cache:
    image: redis:7

volumes:
  pgdata:

Start the entire stack with one command:

docker compose up -d        # -d runs in the background (detached)
docker compose ps           # see what's running
docker compose logs -f web  # follow one service's logs

That’s the whole value proposition: three coordinated containers, one file, one command.

How Services Talk to Each Other

Notice the DATABASE_URL points at db:5432, not an IP. Compose creates a shared network for the project and registers each service name as a DNS hostname. So web reaches Postgres at db and Redis at cache — automatically.

    graph LR
  subgraph Compose network
    W[web :8080] --> D[(db - postgres :5432)]
    W --> C[(cache - redis :6379)]
  end
  U[Your browser] -->|localhost:8080| W
  

This is why you should never hard-code container IPs. Service names are stable; IPs are not.

Persisting Data with Volumes

Containers are ephemeral — delete one and its filesystem is gone. That’s fine for the web service but catastrophic for the database. The pgdata named volume mounted at Postgres’s data directory keeps your data across restarts and rebuilds.

docker compose down          # stops containers, KEEPS the volume (data safe)
docker compose up -d         # data is still there

Environment Variables and the .env File

Hard-coding secret in the file is fine for a demo, but real projects pull config from a .env file that Compose reads automatically:

# .env (do NOT commit this)
POSTGRES_PASSWORD=supersecret
  db:
    image: postgres:16
    environment:
      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD}

Add .env to your .gitignore. Commit a .env.example with blank values so teammates know what to set.

Startup Order Is Not Readiness

depends_on controls start order, but here’s the trap that bites everyone: it does not wait for a dependency to be ready. Postgres can be “started” yet still be initializing and refusing connections, so your web app crashes on boot. Fix it with a healthcheck:

  db:
    image: postgres:16
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U app"]
      interval: 5s
      retries: 5
  web:
    build: .
    depends_on:
      db:
        condition: service_healthy   # now web waits until db actually accepts connections

The Compose Commands You’ll Use Daily

docker compose up -d --build     # rebuild images and (re)start everything
docker compose ps                # list services + status + ports
docker compose logs -f           # tail logs from all services
docker compose exec db psql -U app   # open a shell/CLI inside a running service
docker compose restart web       # restart a single service
docker compose stop              # stop without removing
docker compose down              # stop and remove containers + network
docker compose down -v           # ...and delete volumes (data)

Building Images with Docker Bake

Compose’s build: works, but docker compose build builds services one at a time and tangles build config into your runtime file. Docker Bake (docker buildx bake) is the build-focused tool: it builds many images in parallel, shares a build cache, targets multiple CPU architectures, and reads a dedicated docker-bake.hcl (or your existing Compose file). It’s the same BuildKit engine Compose uses, driven by a build-first config you can reuse in CI.

Bake can read your Compose file directly:

docker buildx bake --file compose.yaml    # build every service that has a build: section

For anything beyond the basics, a dedicated docker-bake.hcl is clearer:

# docker-bake.hcl
variable "TAG" {
  default = "latest"
}

group "default" {
  targets = ["web"]
}

target "web" {
  context    = "."
  dockerfile = "Dockerfile"
  tags       = ["myapp:${TAG}"]
  platforms  = ["linux/amd64", "linux/arm64"]   # multi-arch in a single build
}
docker buildx bake             # build the default group
docker buildx bake --print     # show the resolved config without building
TAG=v2 docker buildx bake      # override a variable at run time

The payoff: one declarative file builds every image, in parallel, with caching and multi-architecture support — and the same file drives both your local builds and your CI pipeline.

Pushing Images to Amazon ECR

Local images are fine for dev, but to run on AWS — ECS, Fargate, or EKS — your images need to live in a registry. Amazon ECR (Elastic Container Registry) is the private registry that integrates with IAM and the rest of AWS.

Create a repository and authenticate Docker to it:

# one-time: create the repository
aws ecr create-repository --repository-name myapp --region us-east-1

# authenticate Docker to your private registry (the token is valid ~12 hours)
aws ecr get-login-password --region us-east-1 \
  | docker login --username AWS --password-stdin \
    <account-id>.dkr.ecr.us-east-1.amazonaws.com

Point your Bake tags at the ECR registry, then build and push in a single step with --push:

# docker-bake.hcl
variable "REGISTRY" {
  default = "<account-id>.dkr.ecr.us-east-1.amazonaws.com"
}
variable "TAG" {
  default = "latest"
}

target "web" {
  context   = "."
  tags      = ["${REGISTRY}/myapp:${TAG}"]
  platforms = ["linux/amd64"]
}
docker buildx bake --push      # build AND push to ECR in one command
    graph LR
  A[compose.yaml<br/>local dev] --> B[docker buildx bake<br/>build images]
  B --> C[(Amazon ECR<br/>private registry)]
  C --> D[ECS / Fargate / EKS<br/>run in the cloud]
  

The registry is the hand-off point between “builds on my laptop” and “runs in the cloud” — once the image is in ECR, ECS, Fargate, and EKS can all pull it.

Common Pitfalls

  • Scaling a service with a fixed host port. docker compose up --scale web=3 fails if web maps "8080:8080", because three containers can’t share one host port. Use a range or put a load balancer in front.
  • Committing secrets. The .env file and inline passwords end up in Git history. Use .env (ignored) and .env.example (committed).
  • Assuming depends_on waits for readiness. It doesn’t — add healthchecks.
  • Losing data to down -v. Only use -v when you intend to wipe volumes.
  • Stale ECR login. The get-login-password token expires (~12 hours). In CI, re-authenticate at the start of every pipeline run rather than caching credentials.

When to Graduate Beyond Compose

Compose is ideal for local development and small single-host deployments. Once your images are in ECR and you need self-healing across multiple machines, rolling updates, and autoscaling, that’s the job of an orchestrator — see Kubernetes fundamentals: pods, deployments, and services and EKS setup with eksctl. If you want to understand what a container actually is under the hood, containers from scratch in Go builds the primitives by hand.

Question

What's your build-and-ship pipeline today — docker compose build, Docker Bake, or something in CI?

0

References and Further Reading

Similar Articles

More from devops

Knowledge Quiz

Test your general knowledge with this quick quiz!

A set of multiple-choice questions to test your knowledge.

Take as much time as you need.

Your score will be shown at the end.