Docker Troubleshooting — 问题排查与调试
Systematic approach to diagnosing and resolving common Docker issues.
When to Use
ALWAYS use this skill when the user mentions:
- "docker 排查", "docker debug", "容器问题"
- "容器启动失败", "exit code", "crash"
- "OOM", "磁盘满", "disk full"
- "网络不通", "network issues"
- "docker daemon", "daemon not running"
Diagnostic Framework
1. Check container state → docker ps -a, docker inspect
2. Check logs → docker logs --tail 100
3. Check resources → docker stats, docker system df
4. Check exit code → docker inspect --format='{{.State.ExitCode}}'
5. Interactive debug → docker run --rm -it image sh
Container Startup Failure
Exit Code Reference
| Code | Meaning | Action |
|------|---------|--------|
| 0 | Success | — |
| 1 | Application error | Check docker logs |
| 125 | Docker daemon error | Check daemon status |
| 126 | Command cannot execute | Check permissions |
| 127 | Command not found | Check CMD/ENTRYPOINT path |
| 137 | SIGKILL (OOM or docker kill) | Check memory limits |
| 139 | SIGSEGV (segfault) | Application bug |
| 143 | SIGTERM (graceful stop) | Health check timeout? |
# Quick diagnosis
docker ps -a | grep Exited
docker inspect --format='{{.State.ExitCode}} {{.State.Error}}' container
docker logs container
Disk Space Exhaustion
# Check disk usage
docker system df
# What's using space?
docker system df -v # Verbose breakdown
# Clean up
docker system prune -a # Remove all unused: images, containers, volumes, networks
docker image prune -a # Unused images only
docker volume prune # Unused volumes only
# Log rotation (prevent future issues)
docker run --log-opt max-size=10m --log-opt max-file=3 myapp
OOM (Out of Memory)
# Check if container was OOM-killed
docker inspect --format='{{.State.OOMKilled}}' container
# Check memory usage
docker stats --no-stream
# Solution: increase memory limit
docker update --memory=1g --memory-swap=1g container
Network Connectivity
# Check network configuration
docker network inspect bridge
docker exec container ip addr show
docker exec container ping other-container
# DNS check
docker exec container nslookup other-container
docker exec container cat /etc/resolv.conf
# Host networking check
iptables -t nat -L DOCKER -n # Port forwarding rules
Permission Issues
# Bind mount permission denied
# Solution 1: Match UID
docker run -u $(id -u):$(id -g) -v $PWD:/app myapp
# Solution 2: Fix permissions in entrypoint
# In entrypoint script: chown -R appuser:appgroup /data
# Docker socket permission
sudo usermod -aG docker $USER # Add user to docker group
newgrp docker # Refresh group (or re-login)
Workflow — 排查流程
Step 1: 查看状态: docker ps -a → 确认退出码和 OOMKilled
Step 2: 查看日志: docker logs --tail 100 <container> 获取错误信息
Step 3: 深入检查: docker inspect → 检查网络/挂载/资源限制
Step 4: 交互调试: docker run -it --rm --entrypoint sh <image> 进入排查
Step 5: 资源排查: docker stats + docker system df 检查 CPU/内存/磁盘
Gotchas — Common Pitfalls
docker system prune is destructive: Removes ALL unused objects. Don't run blindly in production. → Recovery: Use docker system df first to preview; add --filter "until=24h" for safe cleanup; never run prune -a without confirmation.- Logs fill disk silently: JSON-file driver has NO rotation by default. A noisy app can fill the disk in hours. → Recovery:
docker run --log-opt max-size=10m --log-opt max-file=3 app; existing: truncate -s 0 $(docker inspect -f '{{.LogPath}}' myapp).
- OOMKilled=TRUE but no error in logs: SIGKILL can't be logged. → Recovery: Check
docker inspect -f '{{.State.OOMKilled}}' myapp; increase memory: docker update --memory 512m myapp.
- Exited (0) but service not working: Application started but crashed after health check passed. → Recovery:
docker logs --tail 100 myapp; check if app listens on 0.0.0.0 (not 127.0.0.1); verify with docker exec myapp netstat -tlnp.
Boundary — 能力边界（适用与不适用场景）
| 分类 | 场景 | 说明 |
|------|------|------|
| ✅ 能做 | 容器启动失败排查 | 退出码分析 + docker logs + inspect |
| ✅ 能做 | 磁盘/日志/资源问题 | docker system df + prune + log rotation |
| ✅ 能做 | OOM/CPU 排查 | docker stats + --memory 限制 + OOMKilled |
| ⚠️ 需条件 | 网络包级分析 | 需 tcpdump/Wireshark + 网络知识 |
| ⚠️ 需条件 | 内核级问题 | 需 Linux 内核调试技能 |
| ❌ 超范围 | 应用代码 bug 修复 | 开发者修复代码 |
| ❌ 超范围 | 生产环境部署 | 使用 docker-production |
| ❌ 超范围 | 安全漏洞修复 | 使用 docker-security + docker-scout |
When NOT to Use This Skill
| ❌ Skip | ✅ Use Instead |
|---------|---------------|
| Setting up production | docker-production |
| Security issues | docker-security |
| Performance tuning | docker-run (resource limits) |
| Docker basics | docker-basics |
Security & Stability
docker exec into production containers can expose sensitive data. Prefer log-based debugging.docker system prune -a removes ALL unused images and volumes — use with caution in production.- Inspecting containers with
--privileged debug tools can bypass security controls.
- Never expose Docker socket (
/var/run/docker.sock) to containers for debugging — it grants root access.
- Rotate debug logs — they may contain sensitive information. Use log drivers with TTL.
📚 官方文档参考
| 文档 | 地址 |
|------|------|
| Docker 引擎故障排查 | https://docs.docker.com/engine/troubleshoot/ |
| docker logs 命令 | https://docs.docker.com/reference/cli/docker/container/logs/ |
| docker inspect | https://docs.docker.com/reference/cli/docker/inspect/ |
| Docker CLI 参考 | https://docs.docker.com/reference/cli/docker/ |
| 守护进程配置 | https://docs.docker.com/reference/cli/dockerd/ |
| 常见问题 FAQ | https://docs.docker.com/faq/ |
🧭 Docker Skills Journey
📍 You are here: docker-troubleshooting — 问题排查
← Previous: docker-production