Skip to content

Ingress dialects

Prism accepts requests in five API dialects. Anthropic Messages, Gemini and Ollama chat requests are translated to a Chat Completions request, routed like any chat request, and the answer is translated back. Routing, budgets, key policy and cost accounting are therefore the same in every dialect. The model name is read the same way everywhere; see Choosing a model.

POST /v1/chat/completions (also /chat/completions). The body is forwarded as sent, apart from the parameter adjustments a provider needs. Streaming (stream: true) uses SSE.

Prism reads and removes two extra fields before forwarding:

  • prism: a routing intent object, see Choosing a model.
  • session_id: the same as the X-Prism-Session header. If both are present and differ, the request fails with 400 session_identity_conflict.

POST /v1/responses (also /responses). A concrete model goes unchanged to a provider that offers the Responses API for it.

A selector such as auto is chosen from all models Prism serves, as on /v1/chat/completions, and the request is served through Chat Completions; the Responses object’s model names the model that answered. A selector request that cannot be translated (see below) is chosen only among models with the Responses API; without one the answer is 503 providers_unavailable with the message Responses API auto-routing not configured.

A concrete model that has only a Chat Completions route is also served through Chat Completions:

  • carried: instructions (as a system message); input as a string or as items of type message (developer becomes system), function_call, function_call_output, custom_tool_call, custom_tool_call_output; content parts input_text, output_text, text, input_image (with image_url), refusal; tools of type function, namespace (flattened to ns__name) and custom; tool_choice, parallel_tool_calls: false, max_output_tokens, temperature, top_p, stream, user, safety_identifier, reasoning.effort, text.format, and Prism’s own prism and session_id;
  • dropped: store, include, prompt_cache_key, service_tier, truncation, metadata, background, top_logprobs, prompt, and web_search tools.

The answer then carries X-Prism-Responses-Translated: chat_completions and X-Prism-Responses-Dropped with the dropped fields, for example store. Requests that use previous_response_id, conversation, other hosted tools or unknown item types cannot be translated; they need a provider with the Responses API.

GET /v1/responses/{id}, /cancel and /input_items are not served (404 route_not_found).

POST /v1/messages. Required: model (non-empty) and messages (each with role). Otherwise the answer is 400 “body is not an Anthropic Messages request”.

Field Handling
system (string or blocks) System message; block cache_control kept
Content blocks text, image (base64 or URL), tool_use, tool_result (with is_error, cache_control), signed thinking, redacted_thinking Carried
max_tokens, temperature, top_p, stream Carried. With stream: true Prism asks the provider for token usage, so the final message_delta reports tokens
top_k Sent only to Anthropic providers
stop_sequences Carried as stop
thinking: {type: "enabled", budget_tokens: N} Anthropic providers get the object unchanged. Other providers get a reasoning effort: N below 4096 is low, below 16384 medium, otherwise high
thinking: {type: "adaptive"} or {type: "disabled"} Accepted. Any other type is a 400
metadata.user_id Carried as user. Must be a string of at most 512 characters, otherwise 400
tools, tool_choice (auto, any, none, tool), disable_parallel_tool_use Carried
Other top-level fields (service_tier, container, mcp_servers, …) Dropped

A request that cannot be translated without loss is not rejected. This covers server tools such as web_search, Anthropic-defined tools such as bash or text_editor, unknown content blocks, and non-empty citations. Prism sends it unchanged, and only to Anthropic providers. Claude Code’s WebSearch works this way.

Anthropic’s message threads (beta message-threads-2026-08-12) work through Prism.

  • {"type": "create"}: when a connection that keeps threads can serve the model, the request goes only to such connections, unchanged, and the thread is kept on Anthropic’s side. Prism adds the beta value if it is missing. Otherwise the request is served like any other, without thread. Threads are kept by Anthropic’s own API (api.anthropic.com), Claude Platform on AWS (aws-external-anthropic.{region}.api.aws), Claude on Google Cloud managed by Anthropic (claude.googleapis.com/.../invoke) and Claude in Microsoft Foundry ({resource}.services.ai.azure.com/anthropic). Amazon Bedrock and Vertex AI do not keep them.
  • {"type": "continue", "previous_message_id": ...}: sent to the same connection and model that served the previous response, with no other attempt. Prism answers 404 with error.details.error_code thread_not_found when it holds no thread for that response (expired after 5 minutes, served elsewhere, or Prism restarted), when the client asks for another model, and when that one attempt fails for any reason. The Anthropic SDKs then send the full conversation again as a new create request. A continue stays in the create’s routing session.
  • A continue with system or tools, without previous_message_id, or with another type is a 400.
  • A create that Anthropic refuses (for example thread_unsupported_request) gets Anthropic’s error with its error.details, so the SDK thread helper can carry on without thread.

Errors use the Anthropic shape {"type":"error","error":{"type","message"}}; error.details from an Anthropic upstream is kept. The type follows the status: 401 authentication_error, 403 permission_error, 404 not_found_error, 413 request_too_large, 429 rate_limit_error, 529 overloaded_error, 5xx api_error, other 4xx invalid_request_error.

POST /v1/messages/count_tokens needs a concrete model and a provider that speaks the Anthropic protocol with an API key (not a subscription sign-in). Prism forwards the request and returns the provider’s count; it never estimates. A selector or role alias gets 404 “token counting needs an explicit model”, as does a model without such a provider. On the managed address, and for a request with a Prism key on the main address, the endpoint always answers 404. Claude Code estimates tokens locally when it gets 404.

POST /v1beta/models/{model}:generateContent and POST /v1beta/models/{model}:streamGenerateContent. The model comes from the path. ?alt=sse streams SSE; without it the stream is a JSON array sent in pieces, as Google’s API does. countTokens, embedContent, batchEmbedContents and model listing answer 404 ... is not implemented.

Field Handling
systemInstruction.parts[].text System message
contents[] with role user, model or empty; parts text, inlineData (image), functionCall, functionResponse Carried. Empty or missing contents, an entry without parts, another role, or a user turn without text or image is a 400
generationConfig.temperature, topP, maxOutputTokens, stopSequences Carried
generationConfig.candidateCount, responseMimeType, safetySettings, other keys Dropped
tools[].functionDeclarations Function tools
toolConfig.functionCallingConfig.mode NONE, ANY, AUTO (default)

Errors use Google’s shape {"error":{"code","message","status"}} with status names such as INVALID_ARGUMENT, UNAUTHENTICATED, PERMISSION_DENIED, NOT_FOUND, RESOURCE_EXHAUSTED, UNAVAILABLE, DEADLINE_EXCEEDED, INTERNAL.

Main address only; the managed address does not serve it.

Route Behavior
GET, HEAD / Ollama is running, the exact text clients check; the Server header names Prism and its version
/api/version {"version":"0.12.0","prism_version":"…"}: the Ollama API version Prism follows, and Prism’s own
/api/ps {"models":[]}
/api/tags Models from /v1/models. Provider models appear under their own id without the provider:: prefix; sorted, no duplicates
/api/show Model details by model or name; unknown model: 404 {"error":"model not found"}
/api/pull Succeeds for a model Prism serves, downloads nothing; unknown model: 404
/api/chat, /api/generate Translated to Chat Completions
/api/embed, /api/embeddings Translated to /v1/embeddings
/api/create, /api/delete, /api/copy, /api/push, /api/blobs/ 501

Chat and generate:

  • model is required; a :latest tag is ignored.
  • stream defaults to true; streams are NDJSON.
  • format: "json" turns on JSON mode; a schema object asks for that schema.
  • think: true asks for medium reasoning effort; "low", "medium" and "high" are passed on.
  • options carried: temperature, top_p, seed, stop, presence_penalty, frequency_penalty, num_predict (as max_tokens), top_k. Other options (num_ctx, keep_alive, …) are ignored.
  • /api/chat: roles system, user, assistant, tool; images become image parts. A tool message without an earlier call, or another role, is a 400.
  • /api/generate: prompt and system become messages. raw: true and a non-empty suffix are a 400. An empty prompt returns a load acknowledgement (done_reason: "load") without calling a model.

Errors use Ollama’s shape {"error":"<message>"}.

POST /v1/embeddings and POST /v1/rerank forward the body as sent; the prism routing intent is not read here.

  • A concrete model, bare or provider/model, goes to a provider that lists it. A provider/model whose provider has no model list (some embedding vendors) is sent to that provider with the prefix removed.
  • auto, an empty model, or provider/auto picks among models whose task type is embedding (or rerank): local providers first, then direct, then aggregators, then by name; each next candidate is a failover attempt. Prices do not decide the order.
  • With no eligible candidate the request goes to the provisioned local embeddings server when one is set up, otherwise to the first [[providers]] entry, with the model name unchanged.
  • Rerank: top_n defaults to 64; above 128 the answer is 400 invalid_request_size.