Simon Willison's write-up on stateless MCP made the horizontal-scaling case in prose. We wanted the numbers. So we built a minimal Streamable-HTTP MCP server, put two identical copies behind a naive round-robin load balancer, and fired the same workload at both a stateful and a stateless build. Stateless served every call. Stateful lost half of them. And the tradeoff the write-ups skip is real but small.
What we did
One tiny MCP server on the official Python SDK (mcp 1.27.0), one
add tool, Streamable HTTP transport, a single environment
switch: STATELESS=1 or 0. Each backend process
stamps its own instance id into every tool result, so we can see which
machine served each request. Then a 40-line harness plays the part of a
load balancer with no sticky sessions: 20 tool calls, round-robined across
two identical backends. A stateful deployment expects the same session to
come back to the same process. A load balancer that does not guarantee
that is exactly the case that breaks.
Why it was worth doing
The results are clean. Stateless served 20/20 calls,
split evenly 10/10 across both backends, with no session id anywhere in
the request. Stateful served 10/20: the ten requests that
round-robined onto the second backend all failed with
Session not found, because the session lived in the first
process only. That is the sticky-session tax, on the meter.
The bigger surprise was ergonomic. Against the stateless server, a bare
tools/call with no initialize handshake and no
Mcp-Session-Id returns 200 and the right answer.
The whole three-step lifecycle collapses into one self-contained POST. For
a read-only tool that is the entire request.
What's still off
Stateless is not free. A single-POST stateless call cost about 5.2 ms p50 against 2.5 ms for a stateful call that reuses one warm session. Roughly twice the per-call latency, because each stateless request rebuilds a fresh server and transport instead of amortising one initialize. When you can guarantee sticky routing and you are firing many calls down one connection, stateful is cheaper per call. And stateless drops anything that needs a persistent stream back to the client: sampling, elicitation, progress notifications, subscriptions. Those still want the session.
For our own fleet the call is straightforward. Our hosted labs-api MCP tools are all read-only request/response, so the scheduled MCP 2.0 cutover should target stateless Streamable HTTP: it turns the server into a plain horizontally-scalable web endpoint with no sticky routing to babysit, and we lose nothing we use. The local self-hosted path stays on stdio, which is single-process and stateful anyway. The 2x latency buys you the ability to stop caring which machine answers. For most deployments that is the right trade.