Add gateway load_test_mode config
CHANGELOG · original
Added an opt-in load_test_mode block to the Claude apps gateway config: requests are built and signed but not sent upstream, and clients get a canned reply, so a deployment can be load tested Open official changelog ↗ Documentation
Documentation excerpt
load_test_mode
The load_test_mode block lets you load test a gateway without calling a model provider. While it's on, the gateway builds and signs each provider request as usual, discards it instead of sending it, and streams a canned reply back through its normal response path. The reply is filler text that begins with a sentence saying it is canned.
Requires Claude Code v2.1.282 or later on the gateway server. An earlier gateway refuses to start when it finds the key. Upgrade every replica before you add the block, and remove the block before you roll back.
The example below turns the mode on with the defaults, a reply of roughly 750 tokens of text streamed over about 10 seconds:
load_test_mode:
enabled: true
reply_tokens: 750 # roughly how many tokens of text each canned reply carries
reply_seconds: 9.5 # how long a streamed reply takes
| Field | Required | Description |
|---|---|---|
enabled |
Yes | true turns the mode on. false keeps your numbers in the file with the mode off. The gateway refuses to start if the block is present without it. |
reply_tokens |
No | Default 750. Roughly how many tokens of text each canned reply carries, a whole number from 1 to 100000. |
reply_seconds |
No | Default 9.5. How long a streamed reply takes, from 0 to 600. 0 sends the whole reply at once. A reply to a non-streaming request always comes back at once. |
A load test in this mode covers the gateway, your Postgres, and everything in front of the gateway. It doesn't cover the provider's limits, speed, and network path, or the wait at each replica's limit on concurrent upstream requests. A replica still logs the client requests are open warning when more requests are open than that limit, the point where production requests start to wait.
No model request is sent to the provider, so a replica's CPU per request is an estimate and reads lower than production, which also encrypts its traffic to the provider. Confirm a replica count with a small pilot against the real provider. Before v2.1.283, the estimate reads much lower.
While the mode is on, a request can carry an x-load-test-user header holding a whole number of up to seven digits. The gateway counts each number as a separate developer, with the email and groups of the developer whose token came with the request.
Give the load-test deployment its own empty database, because the gateway refuses to start with the mode on against a database in which any developer has already spent anything.
[!WARNING]
Never turn this on for a gateway that developers use. Every request gets the canned reply and no model is called. The gateway logs a
load_test_mode is onwarning at boot and marks eachinferenceaudit event withload_test: truewhile the mode is on.
Documentation snapshot · 2026-11-20