Skip to content

Compute providers and cloud machines

Prism can power an existing cloud virtual machine on before it starts a model server there, and power it off after idle time. It never creates, rents, resizes or deletes machines. The only calls to the provider read the instance state, start it and stop it. Nebius Compute is built in; other clouds are added by plugin packages (see Other clouds), starting with Amazon EC2 and Google Compute Engine.

In the Nebius console, create a service account for Prism and grant it only what reading, starting and stopping your instances needs. Do not grant editor or admin roles. Add an authorized key to the service account (Service account → Authorized keys). Keep the private key: it is a long-lived credential.

An Object Storage access key (ID NAKI…) cannot control Compute instances; Prism names it and refuses it.

Install the model server and the service agent as user units, and enable lingering for the agent’s account (loginctl enable-linger) so the agent answers after every power-on without a login. For idle power-off, the agent needs nvidia_smi = "/usr/bin/nvidia-smi" in its configuration (pairing sets it when that file exists).

Pair the machine as described in Add a machine.

Compute providers → Cloud accounts → Add cloud account. Enter:

  • Service account ID (serviceaccount-…, under the account’s name in the Nebius console),
  • Public key ID (publickey-…, on the service account’s Authorized keys tab),
  • Private key (paste the PEM, or Or choose the key file).

A credentials JSON from nebius iam auth-public-key generate is accepted too and split into these fields. Instead of the key you can enter a 1Password reference (op://Prism/nebius/credentials) when 1Password is configured as the secrets backend. Prism keeps the key in its encrypted vault; it never appears in the console, the API, logs or state files.

Press List instances. Prism lists the instances in every region of the account (read only). Pick the instance and press Add as machine, then set:

Setting Default Meaning
Power-on timeout 10 minutes How long a power-on may take until the agent answers
Power off after idle off Power the machine off after this long without requests to its model servers

A paired machine with the same host name or address is linked to the cloud machine automatically; otherwise choose it in Linked cloud machine.

  • A request that names a model on the machine waits while Prism powers the machine on, waits for the agent, starts the model server and checks that the model answers. Several waiting requests share one power-on.
  • A request where Prism chooses the model goes to another model at once. The machine is powered on in the background only when its model would have been Prism’s first choice.
  • Power off after idle: after that long without requests, Prism asks the agent for GPU processes outside the agent’s own units. If there are any, the machine stays on and its card says “Not powered off: other processes are using the GPU” with the processes (name, PID, memory). Otherwise Prism stops the model servers, checks again, and powers the machine off.
  • Power off on the machine card powers it off (model servers first) and stops Prism from powering it on by itself, also across restarts: requests that name its models get HTTP 503 host_owner_stopped at once. Power on, or Start on one of its services, allows it again. A machine powered off in the Nebius console or with shutdown blocks nothing.
Code Meaning
host_power_on_failed The cloud did not start the instance
host_capacity_unavailable The cloud has no free capacity (GPU) for the instance now; requests that name its models fail at once instead of waiting for the power-on timeout, and the next request tries again
host_unreachable The instance runs, but the agent did not answer within the power-on timeout
host_credentials_unavailable The key is missing or Nebius rejected it (“The machine access key is not set or was rejected.”)
host_owner_stopped The owner powered the machine off in the console

After these, the model-server stages apply: start_failed, readiness_timeout (see Start on request and stop when idle).

A cloud other than Nebius is a signed plugin package with the compute capability. After the owner installs and enables it on the Plugins page (see Installed packages), Add cloud account offers the cloud next to Nebius, with the fields its package declares, their help text and the minimal rights the key needs.

  • Prism keeps the field values in its encrypted vault. The package’s worker receives them only while it lists, powers on or powers off that account’s machines, and signs its requests to the cloud itself. Prism sends them only to the addresses the package declared, never follows redirects, and keeps them out of the console, the API, logs and state files.
  • A package update cannot change the cloud’s addresses, so a stored key never starts going somewhere new.
  • After a power-on Prism reads the machine’s addresses again. When Prism reaches the machine’s agent at the machine’s public address and the cloud gave the machine a new one, Prism moves the agent and its model servers to the new address before it waits for them. An agent reached over a private network, such as a tailnet, is not moved.
  • Some clouds bill a stopped machine like a running one. When the package says so for a machine, its settings warn that Power off after idle does not lower the bill.

The Amazon EC2 package lists the instances of the regions you name and starts and stops one instance; it never creates, changes or terminates instances. Create an IAM user for Prism with only this policy and an access key for it:

{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Action":["ec2:DescribeInstances","ec2:StartInstances","ec2:StopInstances"],"Resource":"*"}]}

In the account form enter Access key ID, Secret access key and Regions (region codes separated by commas, for example us-east-1, eu-west-1; empty means us-east-1; at most eight). Only instances with an EBS root volume can be stopped; an instance-store instance is listed as billed while stopped. When AWS answers InsufficientInstanceCapacity, the machine shows host_capacity_unavailable. A stopped instance keeps its EBS volumes (billed) and loses a public address that is not an Elastic IP.

The Google Compute Engine package lists the instances of one project and starts and stops one instance; it never creates, changes or deletes instances. Create a custom role for Prism with only these permissions:

compute.instances.list, compute.instances.get, compute.instances.start,
compute.instances.stop, compute.zoneOperations.get

Create a service account, grant it that role on the project, and create a JSON key for it (Keys → Add key → Create new key → JSON). In the account form paste the whole key file into Service account key (JSON); Project ID (empty means the project in the key file) and Zones (for example us-central1-a, europe-west4-b; empty means every zone; at most 16) are optional. An API key or an OAuth client file is refused with a message that names the mistake.

For each operation the package signs a short-lived token request with the key, and only the token, never the key, goes to the Compute Engine API. A start waits for Google’s operation: when the zone has no free GPUs or the project’s GPU quota is used up (ZONE_RESOURCE_POOL_EXHAUSTED, QUOTA_EXCEEDED), the machine shows host_capacity_unavailable at once. A stopped instance keeps its disks and static addresses (billed) and loses an ephemeral external address. GPU instances cannot be suspended, only stopped.

The Microsoft Azure package lists the virtual machines of the subscriptions you name and starts and deallocates one machine; it never creates, changes or deletes machines. Register an application for Prism (Microsoft Entra ID → App registrations), create a client secret for it, and give it only this custom role on the subscription (or on the resource group of your GPU machines), with your subscription ID in place of SUBSCRIPTION_ID:

{"properties":{"roleName":"Prism power","permissions":[{"actions":["Microsoft.Compute/virtualMachines/read","Microsoft.Compute/virtualMachines/instanceView/read","Microsoft.Compute/virtualMachines/start/action","Microsoft.Compute/virtualMachines/deallocate/action","Microsoft.Network/networkInterfaces/read","Microsoft.Network/publicIPAddresses/read"]}],"assignableScopes":["/subscriptions/SUBSCRIPTION_ID"]}}

In the account form enter Directory (tenant) ID, Application (client) ID, Client secret (its Value, not its ID) and Subscription IDs (separated by commas; at most eight). The client secret goes only to login.microsoftonline.com for a token; the requests to Azure Resource Manager carry that token.

Power-off always deallocates the machine: a machine that is only powered off stays allocated and Azure bills it as running. A deallocated machine keeps its disks (billed) and loses a dynamic public IP address; Prism reads the new one after the next start. Azure accepts a start before it finds hardware; when the allocation then fails (AllocationFailed, ZonalAllocationFailed, OverconstrainedAllocationRequest, SkuNotAvailable or a quota error), the machine shows host_capacity_unavailable and waiting requests fail at once. A machine in a resource group whose name contains parentheses is not listed.

The Crusoe Cloud package lists the virtual machines of one project and starts and stops one machine; it never creates, changes or deletes machines. Crusoe Cloud API keys act with the rights of the user who creates them, and the narrowest role that can start and stop machines is project-editor. Give Prism its own user (Admin → User Access → Team → Invite User) with no organization role and only project-editor on the project of your GPU machines, and as that user create a key under Security → Cloud API keys.

In the account form enter Access key, Secret key (shown once) and Project ID. The package signs each request with the secret key; the key itself is never sent.

A stopped on-demand machine is not billed for compute. A machine under a reservation or an unexpired commitment is billed whether it runs or not; its settings warn that Power off after idle does not lower the bill. Public addresses are dynamic by default and change at every stop and start; Prism reads the new one after the next start. Crusoe accepts a start and runs it in the background; when that start fails for lack of capacity, the machine shows host_capacity_unavailable and waiting requests fail at once.

The Scaleway Instances package lists the servers of the zones you name and powers one on and off; it never creates, changes or deletes servers. Create an IAM application for Prism (IAM → Applications), a policy attached to it with one rule on the project of your GPU machines and only these permission sets, then generate an API key for the application:

InstancesServerList, InstancesServerRead, InstancesServerStart, InstancesServerStop

In the account form enter Secret key (the API key’s secret key, not the access key that starts with SCW), optionally Project ID (list only that project’s servers) and Zones (zone codes separated by commas, for example fr-par-2; empty means every Instance zone). Requests go only to api.scaleway.com.

Power-off always uses power off, never stop in place or standby: those keep the hardware allocated and Scaleway bills it as running. A powered-off server keeps its volumes and flexible IPs (billed) and loses a dynamic public IP address; Prism reads the new one after the next start. The scratch disk of an H100 or L40S server is erased at power-off. When Scaleway refuses a start (out_of_stock, quotas_exceeded), the machine shows host_capacity_unavailable and waiting requests fail at once.

The Alibaba Cloud ECS package lists the instances of the regions you name and starts and stops one instance; it never creates, changes or releases instances. Create a RAM user for Prism with permanent AccessKey access and give it only this policy:

{"Version":"1","Statement":[{"Effect":"Allow","Action":["ecs:DescribeInstances","ecs:StartInstance","ecs:StopInstance"],"Resource":"*"}]}

In the account form enter AccessKey ID, AccessKey secret and Regions (region IDs such as cn-hangzhou, ap-southeast-1, separated by commas; at most eight). The package signs each request with the AccessKey secret (signature V3); the secret itself is never sent.

Power-off stops the instance in economical mode: a pay-as-you-go instance in a VPC without local disks stops billing for vCPUs, memory and GPUs, while its cloud disks and EIP are still billed and a public IP that is not an EIP is released; Prism reads the new one after the next start. Alibaba Cloud keeps billing a subscription instance, an instance in the classic network and an instance with local disks (the d and i families) when it is stopped; their settings warn that Power off after idle does not lower the bill. Because an instance in economical mode gives its hardware back, the zone may have none left when it starts again (OperationDenied.NoStock); the machine then shows host_capacity_unavailable and waiting requests fail at once.

The Paperspace package lists the machines of one Paperspace team and starts and stops one machine; it never creates, changes or deletes machines. In the Paperspace console open the profile icon → Team settings → API keys, enter a name (for example prism), click Add and copy the key, which is shown only once. Paste it into API key in the account form. A Paperspace key acts for the whole team and cannot be limited to listing, starting and stopping machines, so give Prism a key of its own that you can delete separately. The key goes only to api.paperspace.com, in the Authorization header.

A stopped machine is not billed for compute; its storage and a static public IP address stay billed. A dynamic public IP address is released when the machine is off; Prism reads the new one after the next start. When Paperspace accepts a start and then cannot finish it for lack of hardware, the machine shows host_capacity_unavailable and waiting requests fail at once.

The Oblivus package lists the virtual machines of an Oblivus Cloud account and starts and stops one machine; it never deploys, changes or deletes machines. In the Oblivus console open API → Credentials and generate an API key for Prism with only these permissions:

virtual-server:list, virtual-server:details, virtual-server:start,
virtual-server:stop

In the account form enter the API key; Prism does not use the API token that Oblivus shows next to it. The key goes only to api.oblivus.com.

A stopped machine is billed for its storage and IP address only, and keeps its IP address. The listing shows each machine’s running price per hour; a machine whose stopped price equals its running price is marked as billed while stopped. Oblivus sends the whole machine list in one answer, so an account with more than about 30 machines does not fit Prism’s 64 KiB answer limit and its listing fails with a message that says so. Oblivus does not document the answer to a start without free hardware; a refusal that mentions stock or capacity shows host_capacity_unavailable.

The Hyperstack package lists the virtual machines of a Hyperstack account, hibernates one machine and restores it; it never creates, resizes or deletes machines. Create the API key in the Hyperstack console under API Keys. A key acts with the role of the user who created it, so create it as an organization member whose role allows only viewing, hibernating and restoring virtual machines, plus billing:view and billing:credit-view, which Hyperstack checks before a restore. In the account form enter the API key; it goes only to infrahub-api.nexgencloud.com.

Power-off is always hibernation: a machine stopped in place (SHUTOFF) keeps its hardware and Hyperstack bills it as running, while a hibernated machine is billed for its root disk only. Prism never uses the plain stop. A flavor that cannot hibernate (labelled no-hibernation: spot flavors, large-root-disk flavors and some GPU families such as B200-SXM) is marked as billed while stopped, and Prism leaves such a machine on. A hibernated machine loses its public IP address; Prism reads the new one after the restore. Its hardware is not reserved either: a restore needs the flavor in stock, and a refusal that mentions stock or capacity shows host_capacity_unavailable.

The OCI package lists the Compute instances of the regions and compartments you name and starts and stops one instance; it never launches, changes or terminates instances. It needs Prism 0.37.1 or later. Create a user for Prism in a group, for example prism-power, and give the group only this policy (replace COMPARTMENT with the compartment’s name, or write in tenancy):

Allow group prism-power to read instances in compartment COMPARTMENT
Allow group prism-power to use instances in compartment COMPARTMENT where request.permission = 'INSTANCE_POWER_ACTIONS'
Allow group prism-power to read vnic-attachments in compartment COMPARTMENT
Allow group prism-power to read vnics in compartment COMPARTMENT

On the user’s page add an API key (API keys → Add API key → Download private key → Add) and copy the Configuration file preview into the account form: Tenancy OCID, User OCID, Fingerprint, Private key (the whole downloaded PEM, without a passphrase), Regions (at most 8) and optionally Compartment OCIDs (empty means the root compartment; regions times compartments at most 8). Prism checks that the fingerprint belongs to the key; the key itself never leaves Prism, only signatures go to iaas.<region>.oraclecloud.com. Twelve of Oracle’s 44 commercial regions are not reachable because a package declares at most 32 addresses: ap-batam-1, ap-kulai-2, af-casablanca-1, sa-bogota-1, sa-valparaiso-1, eu-turin-1, eu-madrid-3, ap-singapore-2, ap-chuncheon-1, uk-cardiff-1, mx-monterrey-1 and me-jeddah-1.

Power-off is a soft stop: the operating system shuts down, and OCI powers the instance off after 15 minutes at most. A stopped instance whose shape holds GPUs (all except the VM.GPU.A10 series), local NVMe (Dense I/O, HPC) or a whole Optimized bare metal host is still billed in full; the console marks it and warns that powering it off does not lower the bill. Other shapes stop billing for compute and keep billing their volumes; a stopped instance keeps its ephemeral public IP. When OCI answers a start with “Out of host capacity” or a service limit, the machine shows host_capacity_unavailable and waiting requests fail at once. The listing reads addresses for running instances within Prism’s 32 calls per listing; the rest show their addresses once their state is read.

[secrets]
backend = "op-cli" # only for credentials_ref
[[inference.hosts]]
id = "gpu-1"
provider = "nebius"
instance_id = "computeinstance-e00example"
credentials_ref = "op://Prism/nebius/credentials" # or:
# credentials_env = "PRISM_NEBIUS_CREDENTIALS" # the credentials JSON itself
power_on_timeout = "10m"
# power_off_after = "30m"
[[inference.services]]
# ... as in the service agent page, plus:
host_id = "gpu-1"

Exactly one of credentials_ref and credentials_env is required. Plugin clouds are added in the console only. See the inference configuration reference.