🐍
Code Snippet — August 2, 2026Bash: Parallel Command Runner with Concurrency, Progress & Retry
Replace serial
for loops and opaque
xargs -P with a pure-Bash runner that gives you
live ✓/✗ progress,
configurable concurrency (
-j 8),
auto-retry with exponential backoff (
-r 3),
per-task timeout (
-t 60),
fail-fast for CI, and
dry-run for safety. Zero dependencies — Bash 4+ only.
# Run 3 commands in parallel (max 2 at a time, with retry)
par-run -j 2 -r 3 "echo task1" "echo task2" "echo task3"
# From stdin (pipe find/grep results)
find . -name "*.py" | par-run --from-stdin -j 8 "python -m pyflakes {}"
# With timeout and fail-fast for CI
par-run -j 4 -t 60 --fail-fast --results ./artifacts/ \\
"pytest tests/unit/" \\
"pytest tests/integration/"
# Dry-run first (safety!)
par-run --dry-run -j 4 \\
"rsync -avz /data/ backup:/data/" \\
"pg_dump mydb | gzip > backup.sql.gz"
Key options: -j N (max jobs),
-r N (retries),
-t SEC (timeout),
-f (fail-fast),
-d (dry-run),
--from-stdin,
--results DIR. Uses
wait -n (Bash 4.3+) for true slot-based concurrency — no semaphores, no temp files. Full version with self-tests in today's workspace.
💡 When xargs -P is too opaque and GNU parallel isn't available, this gives you everything in pure Bash. Drop it in ~/.bash_aliases or source it in your Makefile.