Debugging Container Failures Without Breaking Security Boundaries

You are trying to debug a container that crashes immediately upon startup or behaves erratically, but you cannot simply exec into it because the entrypoint fails before the shell is ready, or the container exits too quickly to attach. A common but dangerous workaround is running the container with --privileged to investigate, which grants full host access and disables security constraints. While effective for quick debugging, this opens the door to accidental privilege escalation and security misconfigurations if the flag is accidentally left in production manifests or CI pipelines.

The 5-minute setup

Instead of granting full privileges, use a temporary debug container that shares the PID namespace and mounts the target container's filesystem, allowing you to inspect the state of the failing process without running the entrypoint. First, identify the problematic container ID:

docker ps -a --filter "status=exited" --filter "status=restarting"

Suppose the container ID is abc123. Instead of restarting it with --privileged, create a throwaway debug container that attaches to the same namespaces and volumes as the original, but uses a stable shell entrypoint:

# Stop the broken container if it's still trying to restart
docker stop abc123 2>/dev/null

# Launch a debug container that shares the PID namespace and mounts the root filesystem
# of the original container. Note: --pid=container:abc123 requires the target to be running,
# so for exited containers, we inspect and rebuild the context manually for full fidelity.

# For a quick inspection of a crashed container's filesystem without running its code:
docker run --rm -it \
  --volumes-from abc123 \
  --network=container:abc123 \
  --pid=host \
  alpine:3.19 sh

If the container is already exited, --pid=container:abc123 will fail. In that case, inspect the image and mount the specific volume or bind mount manually:

# Inspect the original container to find mount points
docker inspect abc123 --format '{{json .HostConfig.Binds}}'

# Run a debug container with the same image but override the entrypoint to a shell
# This allows you to run the binary manually with debug flags
docker run --rm -it \
  --volumes-from abc123 \
  --network=host \
  nginx:1.25-alpine sh

Inside this shell, you can manually execute the original entrypoint command with strace or gdb if available, or simply inspect config files in /etc/nginx/ directly. This approach keeps the container running in a controlled state without granting --privileged capabilities.

Why it works

By using --volumes-from and --network=container: (or host), you replicate the exact filesystem and network context of the failing container without executing its potentially crashing entrypoint. This allows you to inspect logs, configuration files, and binaries in a stable environment. The key is that you are not granting extra capabilities via --privileged; you are simply reusing the existing resource context. This maintains security integrity while providing full visibility into what the application expects to find.

Pro Tip

Create a shell alias for rapid debugging that automatically extracts the image and mounts from any stopped container:

debug-container() {
  local cid=$1
  local image=$(docker inspect $cid --format '{{.Config.Image}}')
  docker run --rm -it \
    --volumes-from $cid \
    --network=host \
    $image sh
}

Then simply run debug-container abc123 to drop into a shell with the exact same context as the broken container. This saves time compared to manually parsing docker inspect output for mount points and environment variables.