---
title: "Prism — Self-Hosted Adaptive AI Gateway"
description: "Prism puts local models, paid API providers and subscription accounts behind one OpenAI-compatible API, with policy routing, bounded failover and budgets. Prism Free costs nothing."
url: https://prism.tm/
---

PRISM — ADAPTIVE AI GATEWAY

# One AI gateway for providers, accounts and models.

Prism is a self-hosted adaptive AI gateway. Connect supported AI providers once — paid APIs, subscription accounts and local models — and give every app and agent one OpenAI-compatible API with policy-based LLM routing, bounded failover, budgets and per-request cost accounting.

[Get started](https://prism.tm/pricing/) [Read the docs](https://prism.tm/docs/)

One integration

Applications and agents use one API instead of separate provider integrations.

Adaptive routing

Prism selects among eligible models, providers and accounts using policy and live signals, then performs bounded failover.

Operational control

Operators manage credentials, quotas, budgets, health, cost and diagnostics centrally.

Self-hosted

You run the gateway and control its configuration and operational data.

SCALE

## 170+ providers. One endpoint.

Connect provider presets, an embedded catalog of OpenAI-compatible providers, subscription accounts, local runtimes and self-hosted model servers — and route across all of them from one address.

11

provider presets, one key each

165+

catalog providers, ready to connect

∞

any OpenAI-compatible endpoint, added by URL and key

3,250+

priced models in the catalog

-   API key
-   OAuth subscriptions
-   local runtimes
-   self-hosted servers
-   any endpoint

PROVIDERS

![](https://prism.tm/brands/openai.svg)![OpenAI](https://prism.tm/brands/openai-text.svg) ![](https://prism.tm/brands/anthropic.svg)![Anthropic](https://prism.tm/brands/anthropic-text.svg) ![](https://prism.tm/brands/gemini.svg)![Gemini](https://prism.tm/brands/gemini-text.svg) ![](https://prism.tm/brands/openrouter.svg)![OpenRouter](https://prism.tm/brands/openrouter-text.svg) ![](https://prism.tm/brands/groq.svg)![Groq](https://prism.tm/brands/groq-text.svg) ![](https://prism.tm/brands/deepseek.svg)![DeepSeek](https://prism.tm/brands/deepseek-text.svg)

Preset and catalog marks shown; the catalog has 165+ OpenAI-compatible providers.

HOW IT WORKS

## How a request flows through Prism.

Your agents and apps call one API. Prism picks the model, provider and account for each request. The request can go to a paid API, to one of your subscription accounts or to a model on your own hardware.

YOUR AGENTS AND APPS

**Coding agents**Agents and IDE assistants that take a custom OpenAI, Anthropic or Ollama endpoint

**Apps and scripts**Your own code on chat completions, Responses, embeddings and rerank

**People**You and up to two others, free on Prism Free

your policy + live signals

capability, health, quota, funds, cost

WHERE REQUESTS GO

**Paid APIs**OpenAI, Anthropic, Google Gemini, OpenRouter and any OpenAI-compatible endpoint

**Subscription accounts**Chat and coding plans you already pay for, several accounts each

**Local models**Ollama, llama.cpp, vLLM, LM Studio and TEI on your own hardware

ONE API · ADAPTIVE ROUTING

## One address for most API formats .

Point your applications and agents at your Prism address. They keep sending requests in the API format they already use. Your policy determines which routes are eligible, and Prism chooses among them using live signals. Agents keep their own work loop, goals, tools and workspace; Prism is the gateway they call.

REQUEST

```
curl https://[your-prism-host]/v1/chat/completions \
  -H "Authorization: Bearer $PRISM_KEY" \
  -d '{
    "model": "auto",
    "messages": [{"role": "user", "content": "Hello"}]
  }'
```

● model: auto, resolved per request

Endpoints

OPENAI

/v1/chat/completions /v1/responses /v1/embeddings /v1/models

ANTHROPIC

/v1/messages

GEMINI

/v1beta/models/{model}:generateContent

OLLAMA

/api/chat /api/generate /api/embed /api/tags

RERANK

/v1/rerank

-   Bounded failover on a rate limit, billing error, timeout, outage, or a content-filter refusal that arrives before any output, Prism sends the request to the next eligible route
-   Out of funds a provider with no funds is skipped and rechecked automatically

ROUTE

request model: auto

\[provider A\] · \[account 1\] quota exhausted

● \[provider A\] · \[account 2\] served

Illustration of an account switch

MODEL SELECTION

## Let Prism choose the model .

Instead of a model name, put one of these five keywords in the request's `model` field. Prism calls them selectors. Each one sets the order in which Prism tries the models that can serve the request.

-   `auto` Tries the strongest cloud model first Applies when the `model` field is `auto` or empty. Strength is the model's Epoch Capabilities Index. Between two equally strong models, the cheaper one is tried first. Local models are tried last.
-   `fast` Tries the model with the lowest latency first Prism uses the latency it has measured on its own requests to each model. Between equally fast models, the cheaper one is tried first.
-   `smart` Tries the highest quality tier first The model catalog has four quality tiers: `frontier`, `strong`, `good` and `baseline`. Within a tier, the model with the highest Epoch Capabilities Index is tried first. Between two equally strong models, the cheaper one is tried first.
-   `cheap` Tries local models first, then the cheapest cloud model For work where cost matters more than answer quality.
-   `local` Uses local models only The request is never sent to a cloud provider. A default strategy, a budget rule or failover cannot change that.

-   If the first model fails with a retryable error, such as a rate limit or a timeout, Prism tries the next model in the order.
-   A selector applies to one request. In the console you can set a default strategy for the whole installation. It applies to requests that send `auto`.

[Read more in the docs](https://prism.tm/docs/api/choosing-a-model/)

COST AND ACCESS

## See what every request costs.

Prism records the cost and tokens of every request and totals them by app, client program, agent, person and model. It shows metered API spend separately from what your subscription traffic would have cost at API prices.

-   Budgets for everyone, one person or one app: warn, switch to cheaper models, use only local models or reject
-   Keys per person and agent issue, rotate and revoke in the console; each person gets a key in one click
-   Lower-cost routes used when a request has the cheap selector or a budget rule switches to cheaper models
-   Health and diagnostics provider health checks, request log, status endpoint and Prometheus metrics

SPEND ESTIMATE EXAMPLE

API SPEND

$412.80

SUBSCRIPTION VALUE

$1,236.40

PLAN PRICES

$220.00

Last 30 days. Subscription value is what subscription traffic would cost at API prices; plan prices are what the subscriptions cost per month.

API spend subscription value per day illustration

UNDER LOAD

## How much overhead Prism adds.

Every request from your agents to a model provider passes through Prism, which decides where to send it. We measured three things: how much time Prism adds to a response, how many requests and open streams an ordinary 4 vCPU machine handles, and how much memory Prism uses.

1.0 ms

Prism's own latency per request. Providers take seconds to answer, so this delay is not noticeable.

2,000

concurrent streaming responses on 4 vCPU. Every one completed; median time to first byte 0.1–0.3 s.

~250 MB

of Prism's own memory after 80,000 requests from 32 clients, not counting database pages the operating system keeps cached.

2,000/s

requests per second on 4 vCPU. Prism recorded the cost, tokens and a request log entry for each one, with per-person spend accounting, and no record was lost.

Measured on 16 October 2026 on 4 logical cores of an Intel Core i5-13500 with NVMe storage, with one provider and one model; the host was shared with other work. The provider answers instantly, so the numbers show only Prism's own overhead. The figures apply to Prism v0.44.0 and later. Other hardware and disks will give different numbers.

[How we measured](https://prism.tm/blog/how-we-measured-prism-under-load/)

USE CASES

## What people run through Prism.

One endpoint for every coding agent

Point any coding agent, script or IDE assistant at Prism if it accepts a custom OpenAI-compatible or Anthropic endpoint. You configure your providers once, in Prism, not in every tool. With your API keys, Codex can use Claude, Gemini and other models, and Claude Code keeps its web search.

one API · selectors · keys per agent

Your Ollama, llama.cpp, vLLM or LM Studio models

Enter Prism's address in the Ollama URL field of Open WebUI, Zed, Continue, Home Assistant or a similar app, and the app can use every model Prism routes to, cloud and local. The console's quick start shows the address to use. If the app has no field for a key, you can put your Prism key in the address (with Prism Teams, a key is required).

Ollama API · main address · any model

Use your subscription accounts

Connect subscription accounts through OAuth sign-in and route through them alongside API keys. Prism picks the account per request. When an account returns an error, is rate-limited or has used up its quota, Prism sends the request to the next eligible account. Sign-in to Claude, Gemini, Cursor and Grok CLI subscriptions is closed by default.

OAuth sign-in · per-request choice · failover

Local models for routine work, cloud for hard requests

Send routine requests with `local`, and they stay on your own machines. Send hard requests with `auto`, and Prism picks the strongest cloud model.

local runtimes · local selector · private requests

The model layer of your own product

Your application calls one API. Prism connects to the providers, moves a failed request to another provider or account and records the cost of each request.

one API · routing · operational controls

MANAGED INFERENCE | TEAMS

## Prism manages your model servers and GPU machines.

Prism starts the model server on your own GPU machine or a cloud GPU machine when a request needs that model, and stops it after the idle time you set.

Add a machine in a few steps

The Add machine wizard on the console's Compute providers page adds the server Prism runs on (often with no command at all), another Linux machine (with one command) or a new cloud instance, which connects itself when you paste the text the wizard shows into its user data. Existing machines in a Nebius account or another cloud account are added on the same page. Prism finds llama-server, vLLM, Ollama and SGLang servers automatically, including ones that run as system services. Tick the ones Prism should manage. Prism does nothing on the machine until you confirm, and you do not need to edit a config file or restart Prism.

Servers start on request and stop when idle

A request for a model waits while its server starts and is answered by that model once it is ready. An existing cloud machine is powered on for the request and powered off after the idle time you set. Support for Nebius is built in; Amazon EC2, Google Compute Engine, Microsoft Azure, Oracle Cloud, Alibaba Cloud ECS, Scaleway, Crusoe, Hyperstack, Paperspace and Oblivus are added with plugin packages. A machine on your own network is powered on with Wake-on-LAN and powered off through its own agent. Prism never powers a machine off while other processes are using its GPU. Prism never creates, resizes or deletes machines.

See what your GPUs are doing

The console shows GPU load, memory, temperature, power, cost and idle time for every machine, with history charts. Usage history shows GPU machine cost next to API spend; an opt-in monthly machine budget notifies you at 80% and 100% and does not power machines off. Machines that need attention are listed first, and the page works on a phone too. Prism alerts you when GPU memory fills up, a GPU gets too hot, a powered-on machine is idle or a machine stops responding.

Your agents can start and stop model servers

Give an agent a scoped key and it can start and stop model servers and power machines on and off through Prism, without SSH. That key cannot call models and gives no access to provider keys, users or the subscription.

Managed inference is part of Prism Pro and the editions above it. Connecting to model servers that are already running is included in Prism Free.

SELF-HOSTED

## Run Prism on your own machine .

Prism runs on Linux and macOS. You control its configuration and operational data. Requests that Prism forwards to a provider are processed under that provider's own terms.

-   A single Go binary
    
    Prism needs no Python, Docker or database server. One command installs it as a service.
    
-   Local by default
    
    Prism listens on localhost and runs as a systemd service on Linux or a launchd service on macOS.
    
-   Prism Free costs nothing
    
    Free for any use, with no limit on your agents. Prism takes no percentage of what you pay providers.
    

[Compare the editions](https://prism.tm/pricing/)  [See how Prism compares](https://prism.tm/compare/)

Full feature list What the gateway does

API

-   OpenAI-compatible endpoints: chat completions, Responses, embeddings, rerank and models
-   Native Anthropic Messages and Gemini endpoints: Codex and Claude Code work through Prism with your API keys
-   Ollama API on Prism's main address
-   OpenAI-compatible image, audio, file and batch endpoints
-   Streaming and non-streaming responses
-   Name a specific provider and model as `provider::model`, or use a selector: `auto`, `fast`, `smart`, `cheap`, `local`
-   Chat Completions requests can be served through an upstream Responses endpoint

Routing

-   Quality first by default: `auto` gives each request a quality floor, set higher for agent steps that use tools, and sends it to the strongest model at or above it, with price deciding only between equals; `cheap` tries an eligible local model first, then the cheapest; `smart` ranks by quality, `fast` by latency
-   When a selector such as `auto` picks a model that is available through several routes, Prism tries them in the order you choose
-   In the console you can exclude models from automatic selection: for example, one pattern removes all free variants, which are rate-limited and may train on your data
-   Failover across providers on rate limits, billing errors, timeouts and outages, and on content-filter refusals and answers that end abnormally before any output, with every attempt listed in the response headers
-   Prism records which models accept tools, image input and JSON output, from their answers and from the provider's model list, and does not choose a model for a request that needs something the model has rejected. A model whose support is unknown is still tried, after the models with confirmed support. If neither the provider nor an earlier answer has shown this, Prism sends one small test request per capability to API-key and local models; API providers bill these requests. The tests can be switched off
-   Providers that run out of funds are skipped and rechecked automatically
-   The `local` selector uses local models only, and `cheap` tries them first. A request marked `private` is never sent to a cloud provider

Providers and accounts

-   Add a provider with just an API key; models are discovered automatically
-   Presets for OpenAI, Anthropic, Google Gemini, Meta Model API, OpenRouter, Groq, DeepSeek, Together AI, Fireworks AI, DeepInfra and Token Broker
-   Plugin packages from the catalog at [prism.tm/plugins](https://prism.tm/plugins/), installed, updated and removed on the console's Plugins page; Prism checks each package's signature before it enables it
-   Amazon Bedrock (Claude included), Google Vertex AI (Gemini and Claude models), Azure OpenAI / Foundry, Meta Muse Code, Cloudflare Workers AI, Z.ai, MiniMax, Novita AI, Parasail, SambaNova Cloud and Vercel AI Gateway through plugin packages. Bedrock connects with a Bedrock API key or an IAM access key, Vertex AI with a service-account key, Azure with your resource's API key, Meta Muse Code with your Meta account
-   Connect subscription accounts through OAuth sign-in; tokens are stored encrypted and refreshed automatically. Sign-in to subscriptions of providers whose terms forbid third-party programs is closed by default and opens only if the Prism team allows it for your installation
-   Local runtimes: Ollama, llama-server, LM Studio, vLLM, TEI
-   Connect a self-hosted model server by its address alone: Prism detects which API the server uses and connects without a key when none is needed
-   Your model servers and GPU machines started when a request needs them (Pro and higher)
-   An egress proxy per provider, HTTP or SOCKS5

Control and cost

-   Opt-in budgets for everyone, one person or one app, with actions: warn, switch to cheaper models, use only local models, reject requests
-   Spending limits for one API-key provider account or one model: an account that reaches its limit takes no requests until its period resets, and `auto` uses another model in place of one that has reached its limit
-   Spend alerts through a plugin package: a Telegram or Slack message when the day's spend crosses the percentages of a daily budget that you choose in the package and, with the first request of a new day, a summary of the previous day
-   Cost and tokens recorded for every request, including the reasoning tokens the provider reports; when the provider returns the amount it charged, Prism records that amount
-   Request log with the requested model, the reason for the choice and the trace ID of each request; usage history
-   Prism keys for your agents and for each person: issue, rotate and revoke them, and change a key's permissions without replacing it. Each person gets their own key in one click under My Prism key in the console, and requests made with it are recorded under that person's name
-   Status endpoint and Prometheus metrics
-   Role health without paid test requests: for each role (a name for a list of models tried in order), the console shows whether its first model, a backup model or no model is serving it. Prism notifies you when a role has no working model
-   Prism notifies you in the console and by webhook when it stops using a provider key after an authorization failure or stops using a model, and when the key or model is back in use; email notifications need Teams
-   Web console in English, German, Spanish, French, Italian, Hebrew, Russian, Ukrainian and Simplified Chinese. You change providers, routing, budgets and notifications in the console, without editing a config file. When Prism runs as a service, a saved setting that needs a restart is applied with the Restart now button

Deployment

-   A single Go binary: no Python, no Docker, no database server
-   Linux with systemd and macOS with launchd
-   Listens on localhost by default
-   One-command installer

Optional, off by default

-   Anonymous usage statistics, opt-in
-   OpenTelemetry export through a plugin package: one span per model attempt (model, provider, tokens, cost, latency, outcome) to Langfuse or any OTLP/HTTP collector, without prompt or response text
-   A line at the end of each chat answer that names the model that served it, for clients that hide response headers
-   Compression of repeated and long tool output before it is sent to the model, so agents use fewer tokens
-   Fleet telemetry, opt-in

**Free:** up to 3 people **Pro, Teams, Enterprise:** from €4 per user / month, before tax (coming soon)

[See pricing](https://prism.tm/pricing/)

## One gateway for your providers, accounts and models.

Self-host it, connect your providers and point your agents at one API. Prism Free costs nothing, for any use.

[Get started](https://prism.tm/pricing/) [Read the docs](https://prism.tm/docs/)
