Start on request and stop when idle
Both settings are off by default and are set per model service.
Turn it on
Section titled “Turn it on”In the console: Compute providers → the service → Edit → Start on
request, and optionally Stop after idle. Or in config.toml:
[[inference.services]]id = "qwen-coder"provider = "gpu-coder"model = "qwen-coder" # exact ID returned by /v1/modelstransport = "agent"agent_url = "https://192.0.2.10:18093"agent_token_env = "PRISM_GPU_CODER_AGENT_TOKEN"agent_service = "qwen-coder"dedicated = trueautostart = truestart_timeout = "10m" # longer than the model's real load time# autostart_wait = "10m" # default: start_timeout# autostart_max_waiters = 32 # default 32# idle_stop_after = "30m" # default: off| Setting | Console label | Default | Meaning |
|---|---|---|---|
autostart |
Start on request | off | A request that names this model starts the stopped server |
start_timeout |
Start timeout | 3 minutes | How long a start may take, until the model answers; applies to the Start button too |
autostart_wait |
Request wait | start_timeout |
How long a request waits for the start |
autostart_max_waiters |
Waiting requests limit | 32 | Requests that may wait at the same time |
idle_stop_after |
Stop after idle | off | Stop the server after this long without requests |
What a request sees
Section titled “What a request sees”- A request that names the model (for example
gpu-coder::qwen-coder, orqwen-coder) waits while the model loads and is then answered by that model, never by another one. When it waits longer thanautostart_wait, or more thanautostart_max_waitersrequests are waiting, it gets HTTP 503 withRetry-After. - A request where Prism chooses the model (
auto, a role alias, a routing profile) does not wait. It goes to another eligible model, and the server starts in the background only when Prism would have chosen it first. - Console views,
/v1/models, catalog refresh and health probes never start a server.
While the model loads, every request that names it waits. Plan loads for times when that is acceptable, or leave autostart off.
Stop reasons
Section titled “Stop reasons”- Stop in the console blocks autostart until you press Start, also across Prism restarts. The service shows “Stopped by the owner: autostart waits for the Start button”.
- An idle stop, or a stop outside Prism (
systemctl --user stop, a crash, a reboot), does not block autostart.
The service row shows whether autostart is on, how many requests wait, and why the service stopped.
Stopping does not power off the machine
Section titled “Stopping does not power off the machine”Stopping a model server does not power off its machine or end cloud billing. A cloud machine is powered off only by its own idle setting; see Compute providers and cloud machines.
Errors
Section titled “Errors”| State or code | Meaning | What to do |
|---|---|---|
start_failed |
The unit did not start | Read the unit’s log on the machine (journalctl --user -u llama-coder.service) |
readiness_timeout |
The model did not answer within start_timeout |
Raise start_timeout above the real load time |
catalog_refresh_failed |
The server answered, but Prism could not read its model list after the start | Check that /v1/models on the server returns exactly the model set in model |
readiness_lost |
A ready server stopped answering | Check the server process |
drain_timeout |
Requests were still running 5 minutes after Stop; the server keeps running with no new requests | Finish or cancel the clients, then press Stop again |
HTTP 503 license_required |
The subscription is not active, so the stopped model was not started | Renew the subscription, or start the server yourself |