One slow endpoint stalls every concurrent request — diagnose it
A single uvicorn worker serves this service. Under load the p99 latency of every route rises together — including /health, which touches nothing — and throughput collapses to roughly one request per /report call. The process is almost idle on CPU while this happens.
Both endpoints are shown below.
Constraints for your answer:
- name the fault in each endpoint, not just the symptom
- explain why
/healthslows down even though it does no work - say what you would change, and why that change removes the stall
import requests
from fastapi import BackgroundTasks, FastAPI
app = FastAPI()
@app.get("/health")
async def health() -> dict:
return {"status": "ok"}
@app.post("/report")
async def report(tasks: BackgroundTasks) -> dict:
rows = requests.get("https://upstream.example/rows", timeout=30).json()
tasks.add_task(render_pdf, rows) # pure-CPU, ~4 s
return {"rows": len(rows)}Diagnose the cause.
requests is a synchronous client, so the call blocks the worker's single event loop for the whole upstream round trip and /health cannot be served meanwhile. The CPU-bound render_pdf then fights for the interpreter in the same process after the response. Fix: an async client for the call, and a broker-backed queue for the render.
- ✗Assuming FastAPI offloads a synchronous client automatically inside
async def - ✗Reading rising latency on unrelated routes as a networking problem rather than a blocked loop
- ✗Believing background tasks run in a separate process, so CPU-bound work is safe there
- →Why does adding worker processes hide the symptom without fixing the cause?
- →What would you measure to prove the event loop, not the upstream, is the bottleneck?
Task
One worker, p99 rising on every route at once, and an idle CPU. Find both faults in /report and explain why /health suffers.
Analysis
@app.post("/report")
async def report(tasks: BackgroundTasks) -> dict:
rows = requests.get(...).json() # ❌ synchronous client inside async def
tasks.add_task(render_pdf, rows) # ❌ CPU-bound work in the same process
return {"rows": len(rows)}async def means the coroutine is awaited directly on the event loop. requests.get never yields back to the loop — it holds the very thread the loop runs on for the whole 200-400 ms upstream round trip. While that happens the loop can serve no other request, so /health queues up despite doing nothing itself. The CPU is idle throughout: the worker is not computing, it is waiting on a socket.
The second fault shows up later. A background task runs in the same process after the response is sent, and render_pdf is CPU-bound — for its ~4 s it holds the interpreter and stalls request handling again.
Solution
import httpx
from fastapi import FastAPI
app = FastAPI()
client = httpx.AsyncClient(timeout=30)
@app.post("/report")
async def report() -> dict:
rows = (await client.get("https://upstream.example/rows")).json() # ✅ loop stays free
enqueue_render(rows) # ✅ broker-backed queue, outside the request process
return {"rows": len(rows)}Key points
- There are two ways to free the loop: an async client (
httpx.AsyncClient), or making the endpoint a plaindefso Starlette moves it to the threadpool. The first scales further — the threadpool is bounded. - ⚠️ Adding workers only smears the symptom: each one still stalls completely for the duration of its own blocking call.
- CPU-bound work in the background is not "free after the response" — it is the same process and the same interpreter.
- The signature of a blocked loop specifically is latency rising on unrelated routes while the CPU sits idle.
