FastAPI background tasks: when to use Celery

FastAPI background tasks let an API return a response before doing a small follow-up job. They suit work that can stay in the same process, such as a nonessential notification. A separate worker becomes useful when a job needs its own CPU capacity, a retry policy, or a way to recover from interruption.

The choice depends on what happens if the work fails. A quick HTTP response can hide an unfinished job just as easily as it can improve the experience for a user.

Validation status, November 2, 2026: The example below passed 10 local application checks on Windows with Python 3.12.14, FastAPI 0.162.0, and Starlette 1.10.0. They cover input validation, file output, response ordering at the application boundary, and simulated task failure. The server-launch command, Celery, Redis, Linux deployment, Hivenet networking, process termination, and load behavior have not been tested for this article.

Choose by task requirements

• Use BackgroundTasks for a small, best-effort action. The caller does not need its result immediately, and you can accept losing unfinished work if the application stops. Put a time limit on any external call.
• Consider a separate worker for long-running tasks. A report export or sustained data-processing job can compete with API requests for CPU and memory. Separation lets you allocate resources and manage its lifecycle independently.
• Design recovery for work that must not disappear. Store an accepted job durably, decide how it reaches the queue, and define retries, duplicate handling, and status reporting. Installing Celery alone does not settle those decisions.

FastAPI's BackgroundTasks documentation recommends the built-in mechanism for small work and points to Celery for heavier computation that can run independently. Task importance matters as much as duration: a short billing action still needs explicit recovery.

What happens after the response is sent

A path operation function can receive a BackgroundTasks instance and attach a function to it with add_task(). The first argument is the function itself; the remaining positional or keyword arguments are passed to that function later. Calling the function while adding it would execute the work immediately.

Starlette, which supplies this behavior to FastAPI, runs the attached work after sending the response. It still runs inside the application process. An async def task does not become a separate worker process simply because it runs in the background. See Starlette's background-task reference.

An HTTP 202 response means this endpoint accepted the request. It does not certify that the task finished, that its output exists, or that a durable queue accepted it. The example deliberately returns accepted, rather than claiming that a receipt has been written.

A small FastAPI background-task example

This local exercise writes a short JSON receipt after accepting a label. It sends no email, contacts no external service, and stores no real customer data. Use an empty project directory and save this as app.py.

import json
from pathlib import Path
from uuid import uuid4

from fastapi import BackgroundTasks, FastAPI
from pydantic import BaseModel, Field


class ReceiptRequest(BaseModel):
    label: str = Field(min_length=1, max_length=80)


def write_receipt(directory: Path, receipt_id: str, label: str) -> None:
    path = directory / f"{receipt_id}.json"
    with path.open("x", encoding="utf-8") as output:
        json.dump({"receipt_id": receipt_id, "label": label}, output)
        output.write("\n")


def create_app(directory: Path = Path("receipts")) -> FastAPI:
    directory.mkdir(parents=True, exist_ok=True)
    api = FastAPI()

    @api.post("/receipts", status_code=202)
    async def receipt(payload: ReceiptRequest, background_tasks: BackgroundTasks):
        receipt_id = uuid4().hex
        background_tasks.add_task(write_receipt, directory, receipt_id, payload.label)
        return {"receipt_id": receipt_id, "status": "accepted"}

    return api

The endpoint generates the filename, so the submitted label cannot choose a path. Each request gets a separate ID. The label is limited to 80 characters, and invalid input receives HTTP 422 before a task is added. These are parsed-input limits, not a limit on the raw request body.

write_receipt uses ordinary synchronous file I/O, so it is a def function. The file appears in the project's receipts directory after the task runs. That local directory is demonstration output, not a persistent job registry, backup, or recovery system.

Run it locally

In a Python 3.12 virtual environment, install the versions used for the local checks and start one worker on the loopback address:

python -m pip install "fastapi==0.162.0" "starlette==1.10.0" "pydantic==2.14.1" "uvicorn==0.54.0"
python -m uvicorn app:create_app --factory --host 127.0.0.1 --port 8000 --workers 1

This is an unexecuted launch procedure, not a verified server session. The application checks used FastAPI's test client and an application-level response test. The package accompanying the editorial review records the installed dependencies; recheck compatibility when changing versions.

With the server running, open http://127.0.0.1:8000/docs. In Swagger UI, expand POST /receipts, choose “Try it out,” and submit:

{"label": "local-demo"}

Expect HTTP 202 with a generated receipt_id and status: accepted. Check for the matching JSON file in receipts; do not infer completion from the response alone. An empty label should return HTTP 422. The example has no authentication or public-service controls, so keep it bound to your local machine.

A passing test-client request can wait for the background function before returning to the test. It is therefore unsuitable as evidence of real client latency. The separate ordering check confirms the application's response body is sent before its attached task starts; it does not measure network timing.

Async functions and the shared thread pool

Match the function to the work it performs. An asynchronous task should use asynchronous I/O and yield while waiting. A blocking call or a long CPU loop inside async def can occupy the same event loop used to handle other requests.

Synchronous background functions use Starlette's thread pool. That pool also serves other synchronous work, including FastAPI dependencies. Its documented default limit is 40 tokens, shared across those uses. More concurrent background calls can exhaust that capacity; increasing the limit also has resource costs. See the thread-pool documentation.

Moving a function to a thread does not create extra CPU capacity. For CPU-heavy work, assess a separate process and measure the actual job. Avoid copying a worker-count formula without checking memory, task duration, and the API's other responsibilities.

Error handling: a 202 can outlive a failed task

The receipt example can fail after the response, for example if its directory becomes unwritable. The local failure check replaces the writer with a function that raises an error. The response remains HTTP 202, the writer is called once, and no receipt is created. That check demonstrates the absence of a successful result; it does not simulate a killed server or failed disk.

Starlette explains that an exception handler cannot replace a response already sent when a background task raises an exception. Record failures where the operator can find them. If users need progress or a final result, build explicit status storage and an authorized status endpoint. An in-memory dictionary will not provide a shared, durable record across server processes.

Multiple tasks attached to one response run in order. If one raises an exception, later tasks on that response do not run. Do not treat a list of background tasks as an independent parallel batch. The local checks cover this stop-on-failure behavior too.

Unfinished in-process work has no built-in durable queue to resume after a crash. A graceful shutdown may give tasks time to finish, depending on the server and its timeout, but it cannot protect against every interruption. Treat process-recovery behavior as something to test, not as an implied guarantee.

What changes with FastAPI and Celery

Celery separates accepting work from executing it. In a typical arrangement, the FastAPI application acts as a producer, a message broker carries task messages, and a worker consumes them. An optional result backend stores task state and return values. Celery's introduction to task queues describes these roles.

The route still needs to handle failure while submitting a message. Decide whether the application records the job first, how submission is retried, and how it reconciles records with the broker. Keep that boundary visible when returning an accepted response to the client.

Celery and Redis are different components. Redis can be a broker and, separately, a result backend; the latter needs its own configuration. Persistence, access controls, connection failures, and redelivery behavior need attention. Redis visibility timeouts can lead to a task being delivered again. Read the Celery Redis guide before selecting settings.

A Celery task needs a deliberate retry and acknowledgment policy. Make repeated execution safe where possible: for example, use a stable job identifier so retrying an export does not create multiple customer-visible results. Celery's task documentation explains idempotency and acknowledgment choices. Neither a broker nor a retry setting provides an automatic exactly-once guarantee.

This is an architectural comparison. No Celery worker, Redis broker, or recovery scenario was executed for this article. Their deployment belongs in a separate walkthrough with its own validation record.

For a private worker example with task submission and result polling, follow the Celery with Redis deployment guide. It separates local code checks from untested container, recovery, and Hivenet procedures.

‍

Where RQ fits

If you want a smaller Python job-queue setup, RQ is another option. It uses Redis or Valkey and separate workers, with configurable retry support. Compare the task features and operating effort you need. Choosing a smaller tool still leaves you responsible for persistence and failure handling.

Connect the worker decision to CPU sizing

Start with a representative job and record what successful completion means. Measure its CPU use, peak memory, and completion time, then assess how simultaneous jobs affect the API. An API response-time measurement alone misses time spent waiting in a queue.

Putting an API and worker on one VM is a possible starting design, but both still compete for the same resources and share the VM's failure boundary. Separating them onto different VMs can change those tradeoffs; it adds deployment and operating work.

The vCPU vs. physical core guide explains why advertised vCPU count alone cannot establish capacity. Use the CPU VM buying guide to relate the workload to CPU, memory, storage, and access needs.

For Docker, private access, and the checks around an HTTPS route, continue with the FastAPI deployment guide. Its untested cloud steps remain labeled. This receipt example needs no GPU, but it supplies no Hivenet capacity or performance result.

To discuss a CPU deployment with Hivenet, contact the team with the job type, expected volume, location, and recovery requirements. Confirm current configurations and availability before committing to a deployment.

‍