Ingress dialects
Prism accepts requests in five API dialects. Anthropic Messages, Gemini and Ollama chat requests are translated to a Chat Completions request, routed like any chat request, and the answer is translated back. Routing, budgets, key policy and cost accounting are therefore the same in every dialect. The model name is read the same way everywhere; see Choosing a model.
OpenAI Chat Completions
Section titled “OpenAI Chat Completions”POST /v1/chat/completions (also /chat/completions). The body is forwarded
as sent, apart from the parameter adjustments
a provider needs. Streaming (stream: true) uses SSE.
Prism reads and removes two extra fields before forwarding:
prism: a routing intent object, see Choosing a model.session_id: the same as theX-Prism-Sessionheader. If both are present and differ, the request fails with400 session_identity_conflict.
OpenAI Responses
Section titled “OpenAI Responses”POST /v1/responses (also /responses). A concrete model goes unchanged to
a provider that offers the Responses API for it.
A selector such as auto is chosen from all models Prism serves, as on
/v1/chat/completions, and the request is served through Chat Completions;
the Responses object’s model names the model that answered. A selector
request that cannot be translated (see below) is chosen only among models
with the Responses API; without one the answer is 503 providers_unavailable
with the message Responses API auto-routing not configured.
A concrete model that has only a Chat Completions route is also served through Chat Completions:
- carried:
instructions(as a system message);inputas a string or as items of typemessage(developerbecomessystem),function_call,function_call_output,custom_tool_call,custom_tool_call_output; content partsinput_text,output_text,text,input_image(withimage_url),refusal; tools of typefunction,namespace(flattened tons__name) andcustom;tool_choice,parallel_tool_calls: false,max_output_tokens,temperature,top_p,stream,user,safety_identifier,reasoning.effort,text.format, and Prism’s ownprismandsession_id; - dropped:
store,include,prompt_cache_key,service_tier,truncation,metadata,background,top_logprobs,prompt, andweb_searchtools.
The answer then carries X-Prism-Responses-Translated: chat_completions and
X-Prism-Responses-Dropped with the dropped fields, for example store.
Requests that use previous_response_id, conversation, other hosted tools or
unknown item types cannot be translated; they need a provider with the
Responses API.
GET /v1/responses/{id}, /cancel and /input_items are not served
(404 route_not_found).
Anthropic Messages
Section titled “Anthropic Messages”POST /v1/messages. Required: model (non-empty) and messages (each with
role). Otherwise the answer is 400 “body is not an Anthropic Messages
request”.
| Field | Handling |
|---|---|
system (string or blocks) |
System message; block cache_control kept |
Content blocks text, image (base64 or URL), tool_use, tool_result (with is_error, cache_control), signed thinking, redacted_thinking |
Carried |
max_tokens, temperature, top_p, stream |
Carried. With stream: true Prism asks the provider for token usage, so the final message_delta reports tokens |
top_k |
Sent only to Anthropic providers |
stop_sequences |
Carried as stop |
thinking: {type: "enabled", budget_tokens: N} |
Anthropic providers get the object unchanged. Other providers get a reasoning effort: N below 4096 is low, below 16384 medium, otherwise high |
thinking: {type: "adaptive"} or {type: "disabled"} |
Accepted. Any other type is a 400 |
metadata.user_id |
Carried as user. Must be a string of at most 512 characters, otherwise 400 |
tools, tool_choice (auto, any, none, tool), disable_parallel_tool_use |
Carried |
Other top-level fields (service_tier, container, mcp_servers, …) |
Dropped |
A request that cannot be translated without loss is not rejected. This covers
server tools such as web_search, Anthropic-defined tools such as bash or
text_editor, unknown content blocks, and non-empty citations. Prism sends
it unchanged, and only to Anthropic providers. Claude Code’s WebSearch works
this way.
Message threads
Section titled “Message threads”Anthropic’s message threads (beta message-threads-2026-08-12) work through
Prism.
{"type": "create"}: when a connection that keeps threads can serve the model, the request goes only to such connections, unchanged, and the thread is kept on Anthropic’s side. Prism adds the beta value if it is missing. Otherwise the request is served like any other, withoutthread. Threads are kept by Anthropic’s own API (api.anthropic.com), Claude Platform on AWS (aws-external-anthropic.{region}.api.aws), Claude on Google Cloud managed by Anthropic (claude.googleapis.com/.../invoke) and Claude in Microsoft Foundry ({resource}.services.ai.azure.com/anthropic). Amazon Bedrock and Vertex AI do not keep them.{"type": "continue", "previous_message_id": ...}: sent to the same connection and model that served the previous response, with no other attempt. Prism answers404witherror.details.error_codethread_not_foundwhen it holds no thread for that response (expired after 5 minutes, served elsewhere, or Prism restarted), when the client asks for another model, and when that one attempt fails for any reason. The Anthropic SDKs then send the full conversation again as a new create request. A continue stays in the create’s routing session.- A continue with
systemortools, withoutprevious_message_id, or with anothertypeis a400. - A create that Anthropic refuses (for example
thread_unsupported_request) gets Anthropic’s error with itserror.details, so the SDK thread helper can carry on withoutthread.
Errors use the Anthropic shape {"type":"error","error":{"type","message"}};
error.details from an Anthropic upstream is kept.
The type follows the status: 401 authentication_error, 403
permission_error, 404 not_found_error, 413 request_too_large, 429
rate_limit_error, 529 overloaded_error, 5xx api_error, other 4xx
invalid_request_error.
Token counting
Section titled “Token counting”POST /v1/messages/count_tokens needs a concrete model and a provider that
speaks the Anthropic protocol with an API key (not a subscription sign-in).
Prism forwards the request and returns the provider’s count; it never
estimates. A selector or role alias gets 404 “token counting needs an
explicit model”, as does a model without such a provider. On the managed
address, and for a request with a Prism key on the main address, the endpoint
always answers 404. Claude Code estimates tokens locally when it gets 404.
Gemini
Section titled “Gemini”POST /v1beta/models/{model}:generateContent and
POST /v1beta/models/{model}:streamGenerateContent. The model comes from the
path. ?alt=sse streams SSE; without it the stream is a JSON array sent in
pieces, as Google’s API does. countTokens, embedContent,
batchEmbedContents and model listing answer 404 ... is not implemented.
| Field | Handling |
|---|---|
systemInstruction.parts[].text |
System message |
contents[] with role user, model or empty; parts text, inlineData (image), functionCall, functionResponse |
Carried. Empty or missing contents, an entry without parts, another role, or a user turn without text or image is a 400 |
generationConfig.temperature, topP, maxOutputTokens, stopSequences |
Carried |
generationConfig.candidateCount, responseMimeType, safetySettings, other keys |
Dropped |
tools[].functionDeclarations |
Function tools |
toolConfig.functionCallingConfig.mode |
NONE, ANY, AUTO (default) |
Errors use Google’s shape {"error":{"code","message","status"}} with status
names such as INVALID_ARGUMENT, UNAUTHENTICATED, PERMISSION_DENIED,
NOT_FOUND, RESOURCE_EXHAUSTED, UNAVAILABLE, DEADLINE_EXCEEDED,
INTERNAL.
Ollama
Section titled “Ollama”Main address only; the managed address does not serve it.
| Route | Behavior |
|---|---|
GET, HEAD / |
Ollama is running, the exact text clients check; the Server header names Prism and its version |
/api/version |
{"version":"0.12.0","prism_version":"…"}: the Ollama API version Prism follows, and Prism’s own |
/api/ps |
{"models":[]} |
/api/tags |
Models from /v1/models. Provider models appear under their own id without the provider:: prefix; sorted, no duplicates |
/api/show |
Model details by model or name; unknown model: 404 {"error":"model not found"} |
/api/pull |
Succeeds for a model Prism serves, downloads nothing; unknown model: 404 |
/api/chat, /api/generate |
Translated to Chat Completions |
/api/embed, /api/embeddings |
Translated to /v1/embeddings |
/api/create, /api/delete, /api/copy, /api/push, /api/blobs/ |
501 |
Chat and generate:
modelis required; a:latesttag is ignored.streamdefaults totrue; streams are NDJSON.format: "json"turns on JSON mode; a schema object asks for that schema.think: trueasks for medium reasoning effort;"low","medium"and"high"are passed on.optionscarried:temperature,top_p,seed,stop,presence_penalty,frequency_penalty,num_predict(asmax_tokens),top_k. Other options (num_ctx,keep_alive, …) are ignored./api/chat: rolessystem,user,assistant,tool;imagesbecome image parts. A tool message without an earlier call, or another role, is a400./api/generate:promptandsystembecome messages.raw: trueand a non-emptysuffixare a400. An emptypromptreturns a load acknowledgement (done_reason: "load") without calling a model.
Errors use Ollama’s shape {"error":"<message>"}.
Embeddings and rerank
Section titled “Embeddings and rerank”POST /v1/embeddings and POST /v1/rerank forward the body as sent; the
prism routing intent is not read here.
- A concrete model, bare or
provider/model, goes to a provider that lists it. Aprovider/modelwhose provider has no model list (some embedding vendors) is sent to that provider with the prefix removed. auto, an empty model, orprovider/autopicks among models whose task type is embedding (or rerank): local providers first, then direct, then aggregators, then by name; each next candidate is a failover attempt. Prices do not decide the order.- With no eligible candidate the request goes to the provisioned local
embeddings server when one is set up, otherwise to the first
[[providers]]entry, with the model name unchanged. - Rerank:
top_ndefaults to 64; above 128 the answer is400 invalid_request_size.