Docker Compose Builder · Python / FastAPI + Celery

Docker Compose FastAPI + PostgreSQL 16 + Celery Worker Healthcheck Guide

Generate a Compose Specification docker-compose.yml and matching .env.example for Python / FastAPI + Celery with native binary healthchecks, service_healthy boot ordering, and firewall-safe port bindings.

Written & verified by Chathura Prabashana · Updated November 2026

1. Adjust stack settings or load a test scenario2. Inspect live diagnostics3. Copy generated config
Quick-Test Scenarios:

1. Runtime & Data Services

2. Healthchecks, Workers & Security

# docker-compose.yml — Modern Compose Specification (no obsolete 'version:' attribute)
# Generated by OpsCheck DevTools (Python FastAPI)
name: fastapi_celery_stack

services:
  api:
    build:
      context: .
      dockerfile: Dockerfile
    restart: unless-stopped
    env_file:
      - .env
    ports:
      - "127.0.0.1:8000:8000"
    environment:
      DB_HOST: postgres
      DB_PORT: "5432"
      REDIS_HOST: redis
      REDIS_PORT: "6379"
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy
    networks:
      - app_network

  worker:
    build:
      context: .
      dockerfile: Dockerfile
    command: celery -A app.worker.celery_app worker --loglevel=info --concurrency=4
    restart: unless-stopped
    env_file:
      - .env
    environment:
      DB_HOST: postgres
      DB_PORT: "5432"
      REDIS_HOST: redis
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy
    networks:
      - app_network

  postgres:
    image: postgres:16-alpine
    restart: unless-stopped
    environment:
      POSTGRES_DB: ${DB_DATABASE:-app_db}
      POSTGRES_USER: ${DB_USERNAME:-app}
      POSTGRES_PASSWORD: ${DB_PASSWORD:-change_me_strong}
    # Kept strictly internal on app_network (prevents Docker iptables UFW bypass)
    expose:
      - "5432"
    volumes:
      - db_data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U $${POSTGRES_USER:-app} -d $${POSTGRES_DB:-app}"]
      interval: 10s
      timeout: 5s
      retries: 5
      start_period: 20s
    networks:
      - app_network

  redis:
    image: redis:7-alpine
    command: redis-server --appendonly yes
    restart: unless-stopped
    expose:
      - "6379"
    volumes:
      - redis_data:/data
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 10s
      timeout: 5s
      retries: 5
      start_period: 10s
    networks:
      - app_network

networks:
  app_network:
    driver: bridge

volumes:
  db_data:
  redis_data:
Service Boot Orderbridge: app_network
postgres(healthy)+redis(healthy)apiworker

Healthcheck & Security Diagnostics

Compose Spec v2+
  • passStartup Race Condition Prevented (condition: service_healthy)

    api and worker will wait until postgres and redis pass their native healthchecks before starting.

  • infoInternal Bridge DNS Hostnames (Never Use localhost Inside Containers)

    Connect over the shared app_network using Docker service names: DB_HOST=postgres (port 5432), REDIS_HOST=redis (port 6379). Inside the "api" container, 127.0.0.1/localhost points only to "api" itself and always refuses DB/broker connections.

  • passFirewall-Safe Port Exposure Enabled (127.0.0.1 / Internal expose:)

    Database and broker ports stay strictly isolated on app_network via expose:, and app ports bind to 127.0.0.1 so Docker never bypasses host UFW/nftables rules.

  • passNative Probe Binaries Used (pg_isready & redis-cli ping)

    Generated healthchecks invoke built-in service binaries rather than curl or wget, avoiding false "unhealthy" states in minimal Alpine and slim official images.

  • passPersistent Named Volumes Declared (db_data, redis_data)

    Top-level named volumes preserve database tables and broker queues across container rebuilds and "docker compose down" (data is only removed if you pass -v).

Runs locally in your browser. Compose YAML and environment templates are generated client-side.

Typical Container Boot & Connection Error (Python / FastAPI + Celery)

When depends_on lacks condition: service_healthy or internal service hostnames are misconfigured in .env, container logs typically surface this error during startup:

docker compose logs (Startup Failure)

sqlalchemy.exc.OperationalError: (psycopg.OperationalError) connection failed: Connection refused — Is the server running on host "postgres" (172.19.0.2) and accepting TCP/IP connections on port 5432?

Container Boot Ordering, Bridge DNS & Persistence for Python / FastAPI + Celery

Combining Python FastAPI (Uvicorn) with PostgreSQL 16, Redis 7, and a Celery background worker is the standard production blueprint for async AI pipelines, document processing, and webhook-driven Python SaaS backends. Because both the FastAPI web process and Celery workers share the same Python codebase, SQLAlchemy models, and Alembic migrations, building one Docker image and overriding only the command: in the worker service keeps builds fast and dependencies identical.

Without explicit Compose healthchecks, FastAPI's lifespan startup hooks (such as creating SQLAlchemy async connection pools or running Alembic migrations) and Celery's Redis broker handshake routinely outrace PostgreSQL 16's initial database creation. The result is an immediate OperationalError or Redis ConnectionError on cold boots, followed by noisy CrashLoop restarts in container logs.

Using PostgreSQL's built-in pg_isready -U app -d app_db probe and Redis's redis-cli ping probe inside Docker Compose healthchecks guarantees deterministic boot ordering. Neither postgres:16-alpine nor redis:7-alpine ships with curl, so calling the native client binaries directly eliminates false unhealthy flags while verifying genuine protocol readiness.

Inside both the FastAPI and Celery containers, always assemble SQLAlchemy's DATABASE_URL and Celery's CELERY_BROKER_URL using Docker Compose service names (postgres:5432 and redis:6379) rather than localhost. Isolating PostgreSQL and Redis on app_network via expose: also protects your data tier from accidental public port binding on cloud VPS deployments.

Before pushing to production, validate your merged YAML with docker compose config so escaped variables (such as $${POSTGRES_USER}) and .env credentials interpolate accurately, and enable Redis appendonly yes on a named redis_data volume so queued Celery tasks survive unexpected container restarts.

In production FastAPI and Celery deployments, keep Uvicorn worker counts and Celery concurrency aligned with available container CPU cores and PostgreSQL max_connections so bursts of async tasks never exhaust your database pool. Combining docker compose logs -f api worker with pg_isready and redis-cli ping healthchecks ensures rapid visibility into cold-boot timing, Alembic schema upgrades, and queue backlog health.

Step-by-Step Python / FastAPI + Celery Docker Compose Setup Guide

Follow each step below to pair the generated Compose file with accurate container DNS variables, safe database migrations, and zero-downtime restarts.

  1. 1Configure Async SQLAlchemy & Celery URLs Using Compose Service Names

    .env

    Point SQLAlchemy (asyncpg or psycopg3) to host "postgres" and Celery's broker and result backend to host "redis" over the shared Docker bridge network.

    .envdotenv
    DB_DATABASE=app_db
    DB_USERNAME=app
    DB_PASSWORD=change_me_strong
    
    DATABASE_URL=postgresql+asyncpg://app:change_me_strong@postgres:5432/app_db
    CELERY_BROKER_URL=redis://redis:6379/0
    CELERY_RESULT_BACKEND=redis://redis:6379/1
  2. 2Configure Celery Startup Broker Retries Alongside service_healthy

    app/worker.py

    Set broker_connection_retry_on_startup = True in Celery 5.3+ so workers also reconnect gracefully if Redis restarts independently during maintenance.

    app/worker.pypython
    import os
    from celery import Celery
    
    celery_app = Celery(
        "fastapi_worker",
        broker=os.environ["CELERY_BROKER_URL"],
        backend=os.environ["CELERY_RESULT_BACKEND"],
    )
    celery_app.conf.broker_connection_retry_on_startup = True
    celery_app.conf.task_acks_late = True
  3. 3Run Alembic Database Migrations Cleanly Before Serving Traffic

    terminal

    Bring up PostgreSQL and Redis first (waiting for healthy probes), run Alembic once as an ephemeral command, and verify all containers report healthy/running.

    terminalbash
    docker compose up -d --build
    docker compose exec api alembic upgrade head
    docker compose ps
  4. 4Monitor FastAPI and Celery Worker Logs Together

    terminal

    Tail both containers simultaneously to verify FastAPI enqueues tasks and Celery acknowledges them over Redis without DNS or serialization failures.

    terminalbash
    docker compose logs -f api worker

Common Python / FastAPI + Celery Docker Compose Pitfalls

Avoid these three container orchestration mistakes before deploying your Compose stack onto staging or production servers.

Pitfall 1

Using pg_isready Without -d and -U Flags Passing Prematurely

Calling bare pg_isready defaults to checking Unix socket / root user and can return exit code 0 during PostgreSQL's temporary initdb phase before POSTGRES_DB is ready for TCP connections. Always specify -U $${POSTGRES_USER} -d $${POSTGRES_DB}.

Pitfall 2

Running Alembic Migrations Inside Every FastAPI Uvicorn Worker

If your Dockerfile CMD runs alembic upgrade head and you scale to multiple API containers or workers, concurrent migrations collide on alembic_version table locks. Run migrations as a single one-off docker compose exec or dedicated migration job.

Pitfall 3

Celery Dropping Queued Tasks on Redis Restart Without AOF Persistence

Default Redis containers only snapshot memory periodically. Passing command: redis-server --appendonly yes mounted on a named redis_data volume ensures queued Celery jobs persist cleanly across container restarts.

Related Tools

Pair Your Docker Compose Stack With .env Linting, Nginx TLS & CORS Checks

Validate your .env file for unescaped $$ interpolation or leaked secrets, configure Nginx DNS resolution, and verify cross-origin headers:

Frequently Asked Questions

Why does FastAPI say "could not translate host name postgres" in Docker Compose?
That psycopg error means your FastAPI container and Postgres container are not attached to the same user-defined Docker bridge network, or the service name in docker-compose.yml (for example "db") does not match the hostname in DATABASE_URL ("postgres").
Should Celery run as root inside the Python Docker container?
No. Celery prints a security warning when running as root. Create a non-root user (such as appuser) in your Dockerfile and drop privileges with USER appuser so both FastAPI and Celery execute safely.
How do I inspect PostgreSQL from my laptop if ports are kept internal with expose:?
Use an SSH tunnel (ssh -L 5432:127.0.0.1:5432 user@vps) paired with 127.0.0.1:5432:5432 in Compose, or run psql interactively via docker compose exec postgres psql -U app -d app_db without ever opening port 5432 publicly.
Can I swap Redis for RabbitMQ as the Celery broker in this Compose generator?
Yes. Select RabbitMQ in the Cache / Message Broker dropdown above and the engine automatically generates rabbitmq:3.13-management-alpine with rabbitmq-diagnostics -q ping healthchecks and AMQP_URL variables.
Browse all 10 Compose stacks →