Stop Docker Builds from Exploding: Fix Layer Bloat with BuildKit Caching

You push a one-line code change to your CI/CD pipeline, but the build takes 15 minutes because Docker is re-downloading base images and re-executing heavy compilation steps. Your container image size balloons to 4GB because every RUN command creates a new layer, and your registry storage costs are skyrocketing. This happens when you ignore layer ordering or disable BuildKit, causing cache misses on every build.

The 5-minute setup

First, ensure BuildKit is enabled in your Docker daemon or build command. Then, restructure your Dockerfile to maximize cache hits by moving static dependencies (like package.json or requirements.txt) before source code. Finally, use multi-stage builds to exclude build tools from the final image.

# syntax=docker/dockerfile:1

# Stage 1: Build
FROM node:20-alpine AS builder
WORKDIR /app

# Copy package files first to leverage cache
COPY package*.json ./
RUN npm ci

# Copy source code last
COPY . .
RUN npm run build

# Stage 2: Production
FROM node:20-alpine
WORKDIR /app

# Only copy necessary artifacts from builder
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
COPY package*.json ./

EXPOSE 3000
CMD ["node", "dist/index.js"]

To enforce this in CI, add this to your .github/workflows/build.yml or equivalent:

- name: Build and push Docker image
  run: |
    docker buildx build \
      --platform linux/amd64 \
      --tag myapp:latest \
      --cache-from type=registry,ref=myapp:latest \
      --cache-to type=registry,ref=myapp:buildcache,mode=max \
      --push .

Why it works

Docker caches layers based on the checksum of the command and its context. If you copy package.json before your source code, changes to your source files won't invalidate the npm ci layer, saving minutes of install time. Multi-stage builds ensure the final image only contains the runtime dependencies and compiled code, stripping out compilers, package managers, and source code that bloat the image size. BuildKit’s remote caching (--cache-to) allows CI runners to share cached layers across builds, drastically reducing cold-start build times.

Pro Tip

Check your image size and layer breakdown with this one-liner to identify which layer is causing bloat:

docker history --no-trunc --format "table {{.Size}}\t{{.CreatedBy}}" myapp:latest | sort -h

Look for large npm install or pip install layers in the final stage. If you see them, you likely forgot to use a multi-stage build or you’re copying node_modules from the build stage instead of using COPY --from. Also, enable --progress=plain in your docker build command to see exactly which layers are cached vs. rebuilt, helping you debug cache misses in real-time.