Skip to content

Service agent

prism-service-agent runs on a model machine as the account that owns the model servers. It lets Prism check, start and stop those servers without SSH access. The console installs it for you (Add a machine); this page describes what it does and how to install it by hand.

  • Run /usr/bin/systemctl --user for units on its allowlist only: status, start and stop. No shell, no unit names or commands from the caller, no root, no sudo, no enabling or disabling units, no model downloads.
  • Report each service as stopped, starting, running (active, model not answering yet), ready (the readiness URL answers 2xx), stopping or failed.
  • Find model servers of its account (discover): units that run llama-server, vllm, ollama or sglang, or that listen on a port whose /v1/models answers. Nothing is started or changed by discovery.
  • With nvidia_smi = "/usr/bin/nvidia-smi" in its configuration: list the processes on each GPU and report machine telemetry (GPU utilization, memory, temperature, power; CPU, memory, disk, network). It reads process status and cgroups, never command lines or environments. Prism uses the GPU process list before it powers a machine off: a process outside the agent’s own units keeps the machine on.
  • Report its network cards (name, MAC address, IPv4 and IPv6 addresses with their prefix, and the card’s wake-up setting where Linux shows it without root) and whether it may power the machine off.
  • Send a Wake-on-LAN packet for another machine, only to the broadcast address of a subnet its own machine is in, at most 10 times a minute (Wake-on-LAN).
  • Shut its own machine down through logind when Prism asks, after Prism stopped the machine’s model servers. This needs the one-time grant below; the agent never asks for a password.
  • Requests without the token in Authorization: Bearer.
  • Requests from a browser (any request with an Origin header).
  • Unknown fields, unknown services and other actions.
  • Starting, stopping or reading the status of a unit not on its allowlist (HTTP 404).

A start waits up to start_timeout_seconds (1 to 600, default 120) for the unit to become active and the model to answer. When a model loads for longer, Prism keeps checking readiness until the service’s own start timeout in Prism, so a model that needs ten minutes still starts.

Prism keeps every confirmed agent at its own version. Every 10 minutes, and right after a machine is confirmed, Prism asks the agent for its version. When the agent is older, Prism sends it the agent archive of its own release with the signed checksum list. The agent:

  1. refuses a downgrade, its own version, and development builds;
  2. checks the signature with the release keys built into it, and the archive against the signed checksum;
  3. runs the new executable to check its version and its configuration;
  4. replaces itself, keeps the previous executable as prism-service-agent.prev, and exits with status 75 so its unit starts the new one.

If the new executable cannot start, the agent puts the previous one back. The console shows per machine the agent’s version and whether it is the same as Prism’s, being updated, updated, or failed to update, with the reason. A failed update is retried after an hour, at most three times. No root is needed.

Agents from v0.28.1 and older cannot update themselves. The console says the agent is too old and offers Show the command: a re-pairing command for that machine, valid 10 minutes and usable once. Run it on the machine as the agent’s account. Prism keeps the machine’s ID, cloud link and model servers.

Run these steps as the account that owns the model servers.

  1. Install the binary from the prism-service-agent archive of your Prism’s release into ~/.local/bin.

  2. Create the token file: 32 to 512 printable ASCII characters, mode 0600, a regular file:

    Terminal window
    mkdir -p ~/.config/prism-service-agent && cd ~/.config/prism-service-agent
    python3 -c 'import secrets;print(secrets.token_hex(32))' > token && chmod 600 token
  3. Write ~/.config/prism-service-agent/config.toml. The shell fills in the absolute path of your home directory:

    Terminal window
    cat > ~/.config/prism-service-agent/config.toml <<EOF
    bind = "127.0.0.1:18093"
    token_file = "$HOME/.config/prism-service-agent/token"
    [[services]]
    id = "qwen-coder"
    unit = "llama-coder.service"
    ready_url = "http://127.0.0.1:8080/v1/models"
    start_timeout_seconds = 300
    EOF

    ready_url must use a literal loopback address. A non-loopback bind needs tls_cert_file and tls_key_file. Unknown keys stop the agent at start. Check the file with prism-service-agent check-config --config ~/.config/prism-service-agent/config.toml.

  4. Create ~/.config/systemd/user/prism-service-agent.service:

    [Service]
    ExecStart=%h/.local/bin/prism-service-agent --config %h/.config/prism-service-agent/config.toml
    Restart=on-failure
    NoNewPrivileges=true
    UMask=0077
    [Install]
    WantedBy=default.target

    and run systemctl --user enable --now prism-service-agent.service. Enable lingering (loginctl enable-linger) so it starts at boot.

  5. Register the service in Prism’s config.toml with transport = "agent", agent_url, agent_token_env (the environment variable that holds the same token) and agent_service (the id above). See the inference configuration reference. Prism refuses to start when that variable is empty.

agent_url must point at the same host as the provider’s base_url. Plain HTTP is accepted only for a loopback address.

Command What it does
prism-service-agent --config FILE Run the agent
prism-service-agent pair --prism URL --code CODE [--port 18093] [--dir DIR] Pair with Prism (what the console command runs)
prism-service-agent check-config [--config FILE] Load the configuration and token without listening
prism-service-agent system-units [--ungranted] List model servers that run as system services
sudo prism-service-agent grant-system-units --user ACCOUNT Allow the account to start and stop those system services and to power the machine off (System services, Wake-on-LAN)
prism-service-agent verify-self Check that this executable matches the signed release
prism-service-agent version Print the version