429 Retry-After: Stopping a Rate-Limit Retry Storm

A rate limiter can enforce the right quota and still create the wrong system behavior. In this anonymized Mythos V3 incident, a valid quota rejection omitted the one response field clients needed most: 429 Retry-After.

During one five-minute window, the application emitted 1,050 HTTP 429 responses, the edge emitted 220, and there were zero 5xx responses. Most application rejections came from a genuinely exhausted shared quota. The 429 body was only 353 bytes, but it did not tell callers when retrying could succeed.

429 Retry-After changes client behavior

Without a Retry-After header, SDKs and user scripts must guess. Some retry immediately, some use a small generic backoff, and parallel workers can synchronize into a retry storm. Every rejected attempt then creates more logging, connection work, and user-visible noise without increasing successful throughput.

A useful 429 response needs the correct status, a stable non-sensitive reason category, and an accurate Retry-After value for the quota window that rejected the request.

HTTP/1.1 429 Too Many Requests
Retry-After: 42
Content-Type: application/json

{"error":{"type":"rate_limit_error","message":"Quota window exhausted"}}

The value can be seconds or an HTTP date. We chose whole seconds until the relevant window resets, rounded up so a client does not retry a fraction too early.

Separate gates need separate reset calculations

“Rate limit” was not one rule. Requests could be rejected by key-level windows, shared-account windows, or active-concurrency reservations. A generic retry value would be misleading.

We labeled each gate internally and made it return its own reset time. The public response remained deliberately small: no account identifiers, internal nodes, capacity details, or routing topology. Structured logs recorded the gate category and a safe request trace so operators could aggregate causes without exposing credentials or payloads.

Keep quota inspection available

An exhausted user needs to inspect usage more than anyone else. If usage and status endpoints pass through the same quota-consuming gate as generation requests, the user cannot discover why requests are failing or when the window resets.

We changed those endpoints to be quota-read-only: they still require authentication and normal abuse controls, but reading the current limit does not consume the depleted generation allowance. See the public rate-limit documentation and error guide for the client-facing contract.

Reservations must remain idempotent

Concurrency accounting introduced a second risk. A retried or duplicated trace must not reserve capacity twice, and cleanup must not release someone else's slot. We aligned reservation identity with a stable request trace and made acquire/release operations idempotent.

That protects two invariants: one logical request owns at most one active reservation, and releasing the same request more than once has no additional effect. Those invariants matter during timeouts and disconnects, where happy-path-only counters tend to drift.

Result: 1,050 application 429s became 7

After releasing correct reset calculations, Retry-After, quota-read-only inspection endpoints, and idempotent reservations, application 429s in the comparable five-minute view fell from 1,050 to 7. Server errors remained at zero.

That does not mean quota disappeared. It means clients stopped hammering a window that could not accept work. A rejection became a scheduling signal rather than an invitation to guess.

Checklist for a production 429

• Return Retry-After from the gate that made the decision.
• Round reset seconds up and test boundary behavior.
• Let authenticated users read quota state while exhausted.
• Make concurrency reservations traceable and idempotent.
• Log safe categories, never keys or customer payloads.
• Test parallel retries and disconnect cleanup, not only one sequential request.
• Monitor application 429, edge 429, and 5xx separately.

The key lesson from Mythos V3 was simple: correct enforcement is only half of rate-limit design. The other half is giving callers enough information to cooperate.

This article is based on a real production improvement. Customer and account identities, infrastructure addresses, provider names, credentials, exact topology, and internal implementation names have been removed.