Context limits and output budgets

A model's context window describes the tokens it can process in a request, including input and generated output according to the provider's rules. Different routes of the same model can have different limits. Read context_window, max_output_tokens, and capabilities from the model catalog, rather than assuming a model name implies a limit.

Your prompt is forwarded without shortening

AnonRouter does not automatically delete messages, summarize your conversation, or truncate your prompt to make it fit. For ordinary plaintext chat, token estimates help order eligible routes and choose an omitted output budget; an estimate alone does not prove that the provider can or cannot accept a request. The provider applies its tokenizer and context rules.

This applies to compatibility mode, the private ticket flow, browser chat, and automatic routing, including plaintext TEE routes. E2EE and non-chat endpoints have their own request contracts. Request-size, capability, balance, and rate limits continue to apply independently of context fit.

Set an output budget

Use max_tokens or max_completion_tokens to set an explicit output maximum. AnonRouter preserves the chosen cap when forwarding it, with the parameter translation required by the provider. An unsupported cap is an error; the gateway does not silently replace it with a smaller explicit cap.

A maximum is not a promise that the model will produce that many tokens. The provider can stop earlier, and its context or reasoning rules can leave less room for visible output. Some providers count reasoning tokens against the completion budget; see reasoning controls.

{
  "model": "<exact-model-id-from-the-catalog>",
  "max_completion_tokens": 2048,
  "messages": [
    { "role": "user", "content": "Explain this design." }
  ]
}

In the private ticket flow, bind the same model and output limit when issuing the ticket. Follow the quickstart for the ticket request and completion pair.

If you omit the output limit

Ordinary chat uses a finite route-specific default: normally up to 8,192 tokens, or 16,384 for configured reasoning behavior. Known output maxima and estimated remaining context can lower an automatic budget. Free requests use their own smaller default of at most 1,024 tokens.

Set an explicit cap when your application needs a predictable maximum. The temporary balance hold can be larger than the eventual charge; leaving the cap out does not mean unlimited output or an unbounded request.

Routing and fallback

Context handling does not override model pinning, provider order, price ceilings, workspace settings, or your privacy boundary. A request that names only one provider cannot escape that pin to find a larger window elsewhere. The gateway's routing estimate is not an exact token count or a guarantee of fit.

A recognized provider context or output-cap refusal ends that request. It does not automatically retry the prompt with another provider. Other eligible pre-output failures can use the normal provider-fallback rules. After output has begun, AnonRouter does not switch providers.

Errors and retries

Recognized context errors are treated as request-limit failures, rather than evidence that the provider is unhealthy. An unrecognized upstream failure can still appear as a provider error. Disable automatic SDK retries and handle errors deliberately; see compatibility mode.

Why did the response finish with length?

finish_reason: "length" can mean the output cap or a known context boundary was reached. When a provider reports stop but its own token counts establish that a known boundary was reached, AnonRouter can correct the reason to length. Missing or estimated usage does not establish that correction.

Neither a stop reason nor reported token counts independently prove that an upstream processed every part of a prompt. AnonRouter's guarantee here is that it does not shorten the prompt itself.

Long contexts can have different prices

Some routes charge higher rates when provider-reported prompt tokens cross a threshold. Those rates can apply to the whole request, not just tokens beyond the threshold. The temporary hold can use a higher tier than the final charge because fit estimates and billing exposure are different questions. Read long-context pricing and settlement before sending large requests, and check the selected route's current prices.