Ctrlk
For the complete documentation index, see llms.txt. This page is also available as Markdown.

Inference

Cedana inference control plane

List activations joined with profiles

get
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Responses
200

Activations joined with profiles

application/json
activation_started_atstring · nullableOptional
compatibility_keystringRequired
desired_statestringRequired
endpointstring · nullableOptional
estimated_cold_start_msinteger · int64Required
exact_modelstringRequired
hosting_modestringRequired
logical_modelstringRequired
observed_generationinteger · int64Required
phasestringRequired
profile_idstringRequired
profile_revisionstringRequired
readybooleanRequired
ready_atstring · nullableOptional
updated_atstring · date-timeRequired
get/v1/inference/activations
GET /v1/inference/activations HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

[
  {
    "activation_started_at": null,
    "compatibility_key": "text",
    "desired_state": "text",
    "endpoint": null,
    "estimated_cold_start_ms": 1,
    "exact_model": "text",
    "hosting_mode": "text",
    "logical_model": "text",
    "observed_generation": 1,
    "phase": "text",
    "profile_id": "text",
    "profile_revision": "text",
    "ready": true,
    "ready_at": null,
    "updated_at": "2026-01-01T00:00:00.000Z"
  }
]

Report activation state from the controller

post
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Body
desired_statestringRequired
endpointstring · nullableOptional
observed_generationinteger · int64Optional
phasestringRequired
profile_idstringRequired
readybooleanRequired
Responses
204

Activation recorded

No content
post/v1/inference/activations/report
POST /v1/inference/activations/report HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 112

{
  "desired_state": "text",
  "endpoint": null,
  "observed_generation": 1,
  "phase": "text",
  "profile_id": "text",
  "ready": true
}
No content

Startup, serving, and storage aggregate

get
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Query parameters
profile_idstring · nullableOptional
experiment_idstring · nullableOptional
hoursinteger · int64Optional
Responses
200

Startup/storage/serving aggregate

application/json
evidencestringRequired
experiment_idstring · nullableOptional
profile_idstring · nullableOptional
sample_countintegerRequired
window_hoursinteger · int64Required
get/v1/inference/analytics
GET /v1/inference/analytics HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

{
  "evidence": "text",
  "experiment_id": null,
  "profile_id": null,
  "sample_count": 1,
  "serving": {
    "average_ttft_ms": null,
    "completion_tokens": 1,
    "deadline_misses": 1,
    "fallbacks": 1,
    "prompt_tokens": 1,
    "requests": 1,
    "tokens_per_second": null
  },
  "startup": {
    "cold": null,
    "restore": null,
    "saved_p50_ms": null,
    "speedup_p50": null,
    "warm": null
  },
  "storage": {
    "dedup_ratio": null,
    "incremental_ratio": null,
    "logical_bytes": 1,
    "physical_bytes": 1,
    "reused_bytes": 1,
    "transferred_bytes": 1,
    "written_bytes": 1
  },
  "window_hours": 1
}

List checkpoint artefacts

get
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Query parameters
profile_idstring · nullableOptional
statusstring · nullableOptional
modestring · nullableOptional
limitinteger · int64Optional
Responses
200

Checkpoint artefacts

application/json
artifact_idstringRequired
checkpoint_modestringRequired
checksumstring · nullableOptional
compatibility_keystringRequired
created_atstring · date-timeRequired
exact_modelstringRequired
hosting_modestringRequired
last_restored_atstring · nullableOptional

When a worker last became ready from this artefact.

logical_bytesinteger · nullableOptional
logical_modelstringRequired
metadataanyRequired
observed_restore_attemptsinteger · nullableOptional

Worker pods started from this artefact, ready or not. The gap against observed_restores is the interesting part.

observed_restoresinteger · nullableOptional

Worker pods started from this artefact that went on to serve. Counted from the pods that ran, so it answers how many times the checkpoint was actually used.

parent_artifact_idstring · nullableOptional
physical_bytesinteger · nullableOptional
profile_idstringRequired
restore_countinteger · int64Required

Unmaintained; do not display. Only restore_count_increment on the artifact PATCH writes it, and no caller sends that. Use observed_restores.

reused_bytesinteger · nullableOptional
sourcestringRequired
statusstringRequired
storage_tierstringRequired
updated_atstring · date-timeRequired
uristringRequired
written_bytesinteger · nullableOptional
get/v1/inference/artifacts
GET /v1/inference/artifacts HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

[
  {
    "artifact_id": "text",
    "checkpoint_mode": "text",
    "checksum": null,
    "compatibility_key": "text",
    "created_at": "2026-01-01T00:00:00.000Z",
    "exact_model": "text",
    "hosting_mode": "text",
    "last_restored_at": null,
    "logical_bytes": null,
    "logical_model": "text",
    "metadata": null,
    "observed_restore_attempts": null,
    "observed_restores": null,
    "parent_artifact_id": null,
    "physical_bytes": null,
    "profile_id": "text",
    "residency": null,
    "restore_count": 1,
    "reused_bytes": null,
    "source": "text",
    "status": "text",
    "storage_tier": "text",
    "updated_at": "2026-01-01T00:00:00.000Z",
    "uri": "text",
    "written_bytes": null
  }
]

Register a checkpoint artefact

post
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Body
artifact_idstringRequired
checkpoint_modestringRequired
checksumstring · nullableOptional
compatibility_keystringRequired
logical_bytesinteger · nullableOptional
metadataanyOptional
parent_artifact_idstring · nullableOptional
physical_bytesinteger · nullableOptional
profile_idstringRequired
reused_bytesinteger · nullableOptional
sourcestring · nullableOptional
storage_tierstring · nullableOptional
uristringRequired
written_bytesinteger · nullableOptional
Responses
200

Existing artefact (idempotent)

application/json
artifact_idstringRequired
checkpoint_modestringRequired
checksumstring · nullableOptional
compatibility_keystringRequired
created_atstring · date-timeRequired
exact_modelstringRequired
hosting_modestringRequired
last_restored_atstring · nullableOptional

When a worker last became ready from this artefact.

logical_bytesinteger · nullableOptional
logical_modelstringRequired
metadataanyRequired
observed_restore_attemptsinteger · nullableOptional

Worker pods started from this artefact, ready or not. The gap against observed_restores is the interesting part.

observed_restoresinteger · nullableOptional

Worker pods started from this artefact that went on to serve. Counted from the pods that ran, so it answers how many times the checkpoint was actually used.

parent_artifact_idstring · nullableOptional
physical_bytesinteger · nullableOptional
profile_idstringRequired
restore_countinteger · int64Required

Unmaintained; do not display. Only restore_count_increment on the artifact PATCH writes it, and no caller sends that. Use observed_restores.

reused_bytesinteger · nullableOptional
sourcestringRequired
statusstringRequired
storage_tierstringRequired
updated_atstring · date-timeRequired
uristringRequired
written_bytesinteger · nullableOptional
post/v1/inference/artifacts
POST /v1/inference/artifacts HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 283

{
  "artifact_id": "text",
  "checkpoint_mode": "text",
  "checksum": null,
  "compatibility_key": "text",
  "logical_bytes": null,
  "metadata": null,
  "parent_artifact_id": null,
  "physical_bytes": null,
  "profile_id": "text",
  "reused_bytes": null,
  "source": null,
  "storage_tier": null,
  "uri": "text",
  "written_bytes": null
}
{
  "artifact_id": "text",
  "checkpoint_mode": "text",
  "checksum": null,
  "compatibility_key": "text",
  "created_at": "2026-01-01T00:00:00.000Z",
  "exact_model": "text",
  "hosting_mode": "text",
  "last_restored_at": null,
  "logical_bytes": null,
  "logical_model": "text",
  "metadata": null,
  "observed_restore_attempts": null,
  "observed_restores": null,
  "parent_artifact_id": null,
  "physical_bytes": null,
  "profile_id": "text",
  "residency": null,
  "restore_count": 1,
  "reused_bytes": null,
  "source": "text",
  "status": "text",
  "storage_tier": "text",
  "updated_at": "2026-01-01T00:00:00.000Z",
  "uri": "text",
  "written_bytes": null
}

Get a checkpoint artefact

get
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Path parameters
artifact_idstringRequired
Responses
200

Artefact detail

application/json
artifact_idstringRequired
checkpoint_modestringRequired
checksumstring · nullableOptional
compatibility_keystringRequired
created_atstring · date-timeRequired
exact_modelstringRequired
hosting_modestringRequired
last_restored_atstring · nullableOptional

When a worker last became ready from this artefact.

logical_bytesinteger · nullableOptional
logical_modelstringRequired
metadataanyRequired
observed_restore_attemptsinteger · nullableOptional

Worker pods started from this artefact, ready or not. The gap against observed_restores is the interesting part.

observed_restoresinteger · nullableOptional

Worker pods started from this artefact that went on to serve. Counted from the pods that ran, so it answers how many times the checkpoint was actually used.

parent_artifact_idstring · nullableOptional
physical_bytesinteger · nullableOptional
profile_idstringRequired
restore_countinteger · int64Required

Unmaintained; do not display. Only restore_count_increment on the artifact PATCH writes it, and no caller sends that. Use observed_restores.

reused_bytesinteger · nullableOptional
sourcestringRequired
statusstringRequired
storage_tierstringRequired
updated_atstring · date-timeRequired
uristringRequired
written_bytesinteger · nullableOptional
get/v1/inference/artifacts/{artifact_id}
GET /v1/inference/artifacts/{artifact_id} HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

{
  "artifact_id": "text",
  "checkpoint_mode": "text",
  "checksum": null,
  "compatibility_key": "text",
  "created_at": "2026-01-01T00:00:00.000Z",
  "exact_model": "text",
  "hosting_mode": "text",
  "last_restored_at": null,
  "logical_bytes": null,
  "logical_model": "text",
  "metadata": null,
  "observed_restore_attempts": null,
  "observed_restores": null,
  "parent_artifact_id": null,
  "physical_bytes": null,
  "profile_id": "text",
  "residency": null,
  "restore_count": 1,
  "reused_bytes": null,
  "source": "text",
  "status": "text",
  "storage_tier": "text",
  "updated_at": "2026-01-01T00:00:00.000Z",
  "uri": "text",
  "written_bytes": null
}

Soft-delete an artefact (golden requires force)

delete
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Path parameters
artifact_idstringRequired
Query parameters
forceboolean · nullableOptional
Responses
200

Artefact soft-deleted

application/json
anyOptional
delete/v1/inference/artifacts/{artifact_id}
DELETE /v1/inference/artifacts/{artifact_id} HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

No content

Update artefact status / restore count

patch
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Path parameters
artifact_idstringRequired
Body
restore_count_incrementboolean · nullableOptional

Bumps the stored restore_count. Kept for compatibility; do not build on it. No caller sends it, and observed_restores replaces it.

statusstring · nullableOptional
Responses
200

Artefact updated

application/json
artifact_idstringRequired
checkpoint_modestringRequired
checksumstring · nullableOptional
compatibility_keystringRequired
created_atstring · date-timeRequired
exact_modelstringRequired
hosting_modestringRequired
last_restored_atstring · nullableOptional

When a worker last became ready from this artefact.

logical_bytesinteger · nullableOptional
logical_modelstringRequired
metadataanyRequired
observed_restore_attemptsinteger · nullableOptional

Worker pods started from this artefact, ready or not. The gap against observed_restores is the interesting part.

observed_restoresinteger · nullableOptional

Worker pods started from this artefact that went on to serve. Counted from the pods that ran, so it answers how many times the checkpoint was actually used.

parent_artifact_idstring · nullableOptional
physical_bytesinteger · nullableOptional
profile_idstringRequired
restore_countinteger · int64Required

Unmaintained; do not display. Only restore_count_increment on the artifact PATCH writes it, and no caller sends that. Use observed_restores.

reused_bytesinteger · nullableOptional
sourcestringRequired
statusstringRequired
storage_tierstringRequired
updated_atstring · date-timeRequired
uristringRequired
written_bytesinteger · nullableOptional
patch/v1/inference/artifacts/{artifact_id}
PATCH /v1/inference/artifacts/{artifact_id} HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 46

{
  "restore_count_increment": null,
  "status": null
}
{
  "artifact_id": "text",
  "checkpoint_mode": "text",
  "checksum": null,
  "compatibility_key": "text",
  "created_at": "2026-01-01T00:00:00.000Z",
  "exact_model": "text",
  "hosting_mode": "text",
  "last_restored_at": null,
  "logical_bytes": null,
  "logical_model": "text",
  "metadata": null,
  "observed_restore_attempts": null,
  "observed_restores": null,
  "parent_artifact_id": null,
  "physical_bytes": null,
  "profile_id": "text",
  "residency": null,
  "restore_count": 1,
  "reused_bytes": null,
  "source": "text",
  "status": "text",
  "storage_tier": "text",
  "updated_at": "2026-01-01T00:00:00.000Z",
  "uri": "text",
  "written_bytes": null
}

Get the artefact parent chain

get
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Path parameters
artifact_idstringRequired
Responses
200

Parent chain oldest-first

application/json
artifact_idstringRequired
checkpoint_modestringRequired
checksumstring · nullableOptional
compatibility_keystringRequired
created_atstring · date-timeRequired
exact_modelstringRequired
hosting_modestringRequired
last_restored_atstring · nullableOptional

When a worker last became ready from this artefact.

logical_bytesinteger · nullableOptional
logical_modelstringRequired
metadataanyRequired
observed_restore_attemptsinteger · nullableOptional

Worker pods started from this artefact, ready or not. The gap against observed_restores is the interesting part.

observed_restoresinteger · nullableOptional

Worker pods started from this artefact that went on to serve. Counted from the pods that ran, so it answers how many times the checkpoint was actually used.

parent_artifact_idstring · nullableOptional
physical_bytesinteger · nullableOptional
profile_idstringRequired
restore_countinteger · int64Required

Unmaintained; do not display. Only restore_count_increment on the artifact PATCH writes it, and no caller sends that. Use observed_restores.

reused_bytesinteger · nullableOptional
sourcestringRequired
statusstringRequired
storage_tierstringRequired
updated_atstring · date-timeRequired
uristringRequired
written_bytesinteger · nullableOptional
get/v1/inference/artifacts/{artifact_id}/ancestry
GET /v1/inference/artifacts/{artifact_id}/ancestry HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

[
  {
    "artifact_id": "text",
    "checkpoint_mode": "text",
    "checksum": null,
    "compatibility_key": "text",
    "created_at": "2026-01-01T00:00:00.000Z",
    "exact_model": "text",
    "hosting_mode": "text",
    "last_restored_at": null,
    "logical_bytes": null,
    "logical_model": "text",
    "metadata": null,
    "observed_restore_attempts": null,
    "observed_restores": null,
    "parent_artifact_id": null,
    "physical_bytes": null,
    "profile_id": "text",
    "residency": null,
    "restore_count": 1,
    "reused_bytes": null,
    "source": "text",
    "status": "text",
    "storage_tier": "text",
    "updated_at": "2026-01-01T00:00:00.000Z",
    "uri": "text",
    "written_bytes": null
  }
]

Promote an artefact to a hotter tier

post
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Path parameters
artifact_idstringRequired
Body
deadline_msinteger · nullableOptional
tierstringRequired
Responses
200

Artefact promoted

application/json
artifact_idstringRequired
checkpoint_modestringRequired
checksumstring · nullableOptional
compatibility_keystringRequired
created_atstring · date-timeRequired
exact_modelstringRequired
hosting_modestringRequired
last_restored_atstring · nullableOptional

When a worker last became ready from this artefact.

logical_bytesinteger · nullableOptional
logical_modelstringRequired
metadataanyRequired
observed_restore_attemptsinteger · nullableOptional

Worker pods started from this artefact, ready or not. The gap against observed_restores is the interesting part.

observed_restoresinteger · nullableOptional

Worker pods started from this artefact that went on to serve. Counted from the pods that ran, so it answers how many times the checkpoint was actually used.

parent_artifact_idstring · nullableOptional
physical_bytesinteger · nullableOptional
profile_idstringRequired
restore_countinteger · int64Required

Unmaintained; do not display. Only restore_count_increment on the artifact PATCH writes it, and no caller sends that. Use observed_restores.

reused_bytesinteger · nullableOptional
sourcestringRequired
statusstringRequired
storage_tierstringRequired
updated_atstring · date-timeRequired
uristringRequired
written_bytesinteger · nullableOptional
post/v1/inference/artifacts/{artifact_id}/promote
POST /v1/inference/artifacts/{artifact_id}/promote HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 34

{
  "deadline_ms": null,
  "tier": "text"
}
{
  "artifact_id": "text",
  "checkpoint_mode": "text",
  "checksum": null,
  "compatibility_key": "text",
  "created_at": "2026-01-01T00:00:00.000Z",
  "exact_model": "text",
  "hosting_mode": "text",
  "last_restored_at": null,
  "logical_bytes": null,
  "logical_model": "text",
  "metadata": null,
  "observed_restore_attempts": null,
  "observed_restores": null,
  "parent_artifact_id": null,
  "physical_bytes": null,
  "profile_id": "text",
  "residency": null,
  "restore_count": 1,
  "reused_bytes": null,
  "source": "text",
  "status": "text",
  "storage_tier": "text",
  "updated_at": "2026-01-01T00:00:00.000Z",
  "uri": "text",
  "written_bytes": null
}

List the latest capacity intent per profile. One per profile, newest first by update, whatever its status, so a caller can show what was last asked for.

get
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Responses
200

Latest capacity intent per profile

application/json
created_atstring · date-timeRequired

When this intent row was first created. Not when the request it currently describes was made: the row is reused for a repeated ask. Use requested_at for that.

deadlinestring · nullableOptional
desired_replicasinteger · int32Required
desired_statestringRequired
generationinteger · int64Required
idstring · uuidRequired
profile_idstringRequired
requested_atstring · date-timeOptional

When a caller last asked for this exact state and replica count, so it orders requests as they were made rather than as the row happened to change.

statusstringRequired
updated_atstring · date-timeRequired

When this row last changed for any reason, including the expiry sweep. Not an ordering key for what a caller asked for.

get/v1/inference/capacity-intents
GET /v1/inference/capacity-intents HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

[
  {
    "created_at": "2026-01-01T00:00:00.000Z",
    "deadline": null,
    "desired_replicas": 1,
    "desired_state": "text",
    "generation": 1,
    "id": "123e4567-e89b-12d3-a456-426614174000",
    "profile_id": "text",
    "requested_at": "2026-01-01T00:00:00.000Z",
    "status": "text",
    "updated_at": "2026-01-01T00:00:00.000Z"
  }
]

OpenAI-compatible chat with routing. Switchyard owns the external surface and does its own selection, so this is a simpler duplicate kept for existing callers.

post
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Body
anyOptional
Responses
200

Chat completion (streaming or not)

application/json
anyOptional
post/v1/inference/chat/completions
POST /v1/inference/chat/completions HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*

No content

Per-experiment correlation view

get
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Query parameters
experiment_idstring · uuidRequired
Responses
200

Experiment correlation

application/json
experiment_idstring · uuidRequired
namestringRequired
scenariostringRequired
statusstringRequired
get/v1/inference/correlation
GET /v1/inference/correlation?experiment_id=123e4567-e89b-12d3-a456-426614174000 HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

{
  "activations": [
    {
      "count": 1,
      "name": "text"
    }
  ],
  "artifacts": [
    {
      "artifact_id": "text",
      "checkpoint_mode": "text",
      "status": "text"
    }
  ],
  "experiment_id": "123e4567-e89b-12d3-a456-426614174000",
  "intents": [
    {
      "count": 1,
      "name": "text"
    }
  ],
  "name": "text",
  "observations": [
    {
      "count": 1,
      "name": "text"
    }
  ],
  "policy_decisions": [
    {
      "count": 1,
      "name": "text"
    }
  ],
  "scenario": "text",
  "status": "text",
  "usage": {
    "cold_start_avoided_ms": 1,
    "fallbacks": 1,
    "requests": 1,
    "successful_requests": 1
  }
}

Per-profile cost breakdown

get
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Query parameters
hoursinteger · int64Optional
profile_idstring · nullableOptional
experiment_idstring · nullableOptional
Responses
200

Per-profile cost breakdown

application/json
activation_countinteger · int64Required
active_duration_snumber · doubleRequired
compute_usdnumber · doubleRequired
fallback_usdnumber · doubleRequired
idle_time_snumber · doubleRequired
node_secondsnumber · doubleRequired
profile_idstringRequired
release_latency_msnumber · nullableOptional
storage_usdnumber · doubleRequired
tokens_servedinteger · int64Required
transfer_usdnumber · doubleRequired
get/v1/inference/costs/breakdown
GET /v1/inference/costs/breakdown HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

[
  {
    "activation_count": 1,
    "active_duration_s": 1,
    "compute_usd": 1,
    "fallback_usd": 1,
    "idle_time_s": 1,
    "node_seconds": 1,
    "profile_id": "text",
    "release_latency_ms": null,
    "storage_usd": 1,
    "tokens_served": 1,
    "transfer_usd": 1
  }
]

List cost rates

get
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Responses
200

Cost rates

application/json
node_hourly_usdnumber · doubleRequired
profile_idstringRequired
storage_gb_hour_usdnumber · doubleRequired
transfer_gb_usdnumber · doubleRequired
get/v1/inference/costs/config
GET /v1/inference/costs/config HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

[
  {
    "node_hourly_usd": 1,
    "profile_id": "text",
    "storage_gb_hour_usd": 1,
    "transfer_gb_usd": 1
  }
]

Set cost rates for a profile

post
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Body
node_hourly_usdnumber · doubleOptional
profile_idstringRequired
storage_gb_hour_usdnumber · doubleOptional
transfer_gb_usdnumber · doubleOptional
Responses
200

Cost rates saved

application/json
node_hourly_usdnumber · doubleRequired
profile_idstringRequired
storage_gb_hour_usdnumber · doubleRequired
transfer_gb_usdnumber · doubleRequired
post/v1/inference/costs/config
POST /v1/inference/costs/config HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 85

{
  "node_hourly_usd": 1,
  "profile_id": "text",
  "storage_gb_hour_usd": 1,
  "transfer_gb_usd": 1
}
{
  "node_hourly_usd": 1,
  "profile_id": "text",
  "storage_gb_hour_usd": 1,
  "transfer_gb_usd": 1
}

Baseline vs treatment savings report

get
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Query parameters
hoursinteger · int64Optional
profile_idstring · nullableOptional
experiment_idstring · nullableOptional
Responses
200

Baseline vs treatment report

application/json
hoursinteger · int64Required
node_seconds_savednumber · doubleRequired
savings_pctnumber · doubleRequired
savings_usdnumber · doubleRequired
treatment_escalation_ratenumber · doubleRequired
treatment_idle_time_snumber · doubleRequired
treatment_release_latency_msnumber · nullableOptional
variancestring · nullableOptional
get/v1/inference/costs/report
GET /v1/inference/costs/report HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

{
  "baseline": {
    "cost_usd": 1,
    "heavy_activation_count": 1,
    "heavy_active_duration_s": 1,
    "node_seconds": 1
  },
  "hours": 1,
  "node_seconds_saved": 1,
  "savings_pct": 1,
  "savings_usd": 1,
  "treatment": {
    "cost_usd": 1,
    "heavy_activation_count": 1,
    "heavy_active_duration_s": 1,
    "node_seconds": 1
  },
  "treatment_escalation_rate": 1,
  "treatment_idle_time_s": 1,
  "treatment_quality": {
    "avg_ttft_ms": null,
    "refusal_rate": 1
  },
  "treatment_release_latency_ms": null,
  "variance": null
}

List experiments

get
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Responses
200

Experiments with observation counts

application/json
completed_atstring · nullableOptional
idstring · uuidRequired
namestringRequired
observationsinteger · int64Required
parametersanyRequired
profile_idstring · nullableOptional
scenariostringRequired
started_atstring · date-timeRequired
statusstringRequired
get/v1/inference/experiments
GET /v1/inference/experiments HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

[
  {
    "completed_at": null,
    "id": "123e4567-e89b-12d3-a456-426614174000",
    "name": "text",
    "observations": 1,
    "parameters": null,
    "profile_id": null,
    "scenario": "text",
    "started_at": "2026-01-01T00:00:00.000Z",
    "status": "text"
  }
]

Create a load-test experiment

post
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Body
namestringRequired
parametersanyOptional
profile_idstring · nullableOptional
scenariostringRequired
Responses
201

Experiment created

application/json
completed_atstring · nullableOptional
idstring · uuidRequired
namestringRequired
parametersanyRequired
profile_idstring · nullableOptional
scenariostringRequired
started_atstring · date-timeRequired
statusstringRequired
post/v1/inference/experiments
POST /v1/inference/experiments HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 69

{
  "name": "text",
  "parameters": null,
  "profile_id": null,
  "scenario": "text"
}
{
  "completed_at": null,
  "id": "123e4567-e89b-12d3-a456-426614174000",
  "name": "text",
  "parameters": null,
  "profile_id": null,
  "scenario": "text",
  "started_at": "2026-01-01T00:00:00.000Z",
  "status": "text"
}

Get an experiment's desired lifecycle state. Controllers use this after an outbox wake-up rather than trusting a replayed event payload.

get
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Path parameters
idstring · uuidRequired

Experiment ID

Responses
200

Experiment

application/json
completed_atstring · nullableOptional
idstring · uuidRequired
namestringRequired
parametersanyRequired
profile_idstring · nullableOptional
scenariostringRequired
started_atstring · date-timeRequired
statusstringRequired
get/v1/inference/experiments/{id}
GET /v1/inference/experiments/{id} HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

{
  "completed_at": null,
  "id": "123e4567-e89b-12d3-a456-426614174000",
  "name": "text",
  "parameters": null,
  "profile_id": null,
  "scenario": "text",
  "started_at": "2026-01-01T00:00:00.000Z",
  "status": "text"
}

Complete/cancel a running experiment

post
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Path parameters
idstring · uuidRequired
Body
statusstringRequired
Responses
200

Experiment updated

application/json
completed_atstring · nullableOptional
idstring · uuidRequired
namestringRequired
parametersanyRequired
profile_idstring · nullableOptional
scenariostringRequired
started_atstring · date-timeRequired
statusstringRequired
post/v1/inference/experiments/{id}/complete
POST /v1/inference/experiments/{id}/complete HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 17

{
  "status": "text"
}
{
  "completed_at": null,
  "id": "123e4567-e89b-12d3-a456-426614174000",
  "name": "text",
  "parameters": null,
  "profile_id": null,
  "scenario": "text",
  "started_at": "2026-01-01T00:00:00.000Z",
  "status": "text"
}

List fleets

get
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Responses
200

Fleets

application/json
availabilitystring · nullableOptional
budgetsanyRequired
created_atstring · date-timeRequired
fleet_idstringRequired
logical_modelstringRequired
namestringRequired
profile_idsanyRequired
residencystring · nullableOptional
get/v1/inference/fleets
GET /v1/inference/fleets HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

[
  {
    "availability": null,
    "budgets": null,
    "created_at": "2026-01-01T00:00:00.000Z",
    "fleet_id": "text",
    "logical_model": "text",
    "name": "text",
    "profile_ids": null,
    "residency": null
  }
]

Create or update a fleet

post
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Body
availabilitystring · nullableOptional
budgetsanyOptional
fleet_idstring · nullableOptional
logical_modelstringRequired
namestringRequired
profile_idsstring[]Required
residencystring · nullableOptional
Responses
200

Fleet saved

application/json
availabilitystring · nullableOptional
budgetsanyRequired
created_atstring · date-timeRequired
fleet_idstringRequired
logical_modelstringRequired
namestringRequired
profile_idsanyRequired
residencystring · nullableOptional
post/v1/inference/fleets
POST /v1/inference/fleets HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 129

{
  "availability": null,
  "budgets": null,
  "fleet_id": null,
  "logical_model": "text",
  "name": "text",
  "profile_ids": [
    "text"
  ],
  "residency": null
}
{
  "availability": null,
  "budgets": null,
  "created_at": "2026-01-01T00:00:00.000Z",
  "fleet_id": "text",
  "logical_model": "text",
  "name": "text",
  "profile_ids": null,
  "residency": null
}

Get a fleet

get
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Path parameters
fleet_idstringRequired
Responses
200

Fleet detail

application/json
availabilitystring · nullableOptional
budgetsanyRequired
created_atstring · date-timeRequired
fleet_idstringRequired
logical_modelstringRequired
namestringRequired
profile_idsanyRequired
residencystring · nullableOptional
get/v1/inference/fleets/{fleet_id}
GET /v1/inference/fleets/{fleet_id} HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

{
  "availability": null,
  "budgets": null,
  "created_at": "2026-01-01T00:00:00.000Z",
  "fleet_id": "text",
  "logical_model": "text",
  "name": "text",
  "profile_ids": null,
  "residency": null
}

Delete a fleet

delete
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Path parameters
fleet_idstringRequired
Responses
204

Fleet deleted

No content
delete/v1/inference/fleets/{fleet_id}
DELETE /v1/inference/fleets/{fleet_id} HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

No content

List issued inference keys

get
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Responses
200

Issued keys

application/json
budget_usdnumber · nullableOptional
created_atstring · date-timeRequired
key_idstringRequired
modelsanyRequired
namestringRequired
rate_limit_rpsinteger · nullableOptional
revoked_atstring · nullableOptional
tenant_idstring · nullableOptional
get/v1/inference/keys
GET /v1/inference/keys HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

[
  {
    "budget_usd": null,
    "created_at": "2026-01-01T00:00:00.000Z",
    "key_id": "text",
    "models": null,
    "name": "text",
    "rate_limit_rps": null,
    "revoked_at": null,
    "tenant_id": null
  }
]

Issue a scoped inference API key

post
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Body
budget_usdnumber · nullableOptional
modelsstring[]Optional
namestringRequired
rate_limit_rpsinteger · nullableOptional
tenant_idstring · nullableOptional
Responses
201

Key issued (raw shown once)

application/json
budget_usdnumber · nullableOptional
created_atstring · date-timeRequired
key_idstringRequired
modelsanyRequired
namestringRequired
rate_limit_rpsinteger · nullableOptional
revoked_atstring · nullableOptional
tenant_idstring · nullableOptional
raw_keystringRequired

The raw key, shown exactly once at creation.

post/v1/inference/keys
POST /v1/inference/keys HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 90

{
  "budget_usd": null,
  "models": [
    "text"
  ],
  "name": "text",
  "rate_limit_rps": null,
  "tenant_id": null
}
{
  "budget_usd": null,
  "created_at": "2026-01-01T00:00:00.000Z",
  "key_id": "text",
  "models": null,
  "name": "text",
  "rate_limit_rps": null,
  "revoked_at": null,
  "tenant_id": null,
  "raw_key": "text"
}

Revoke an inference API key

delete
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Path parameters
key_idstringRequired
Responses
204

Key revoked

No content
delete/v1/inference/keys/{key_id}
DELETE /v1/inference/keys/{key_id} HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

No content

The cluster's recent lifecycle events across all profiles, time-ascending, so a UI can replay how the routing topology changed over time.

get
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Query parameters
limitinteger · int64Optional
sincestring · nullableOptional

Only events at or after this instant, so a poller can page forward.

Responses
200

Lifecycle events across all profiles

application/json

One durably-persisted lifecycle event, across every profile in the cluster.

event_idstring · uuidRequired
generationinteger · nullableOptional
kindstringRequired

Lifecycle kind, e.g. model.ready, route.published, route.withdrawn.

occurred_atstring · date-timeRequired
payloadanyOptional
profile_idstring · nullableOptional

The profile the event belongs to, when known. Null for cluster-wide events and for rows written before the column existed.

sequenceinteger · nullableOptional
get/v1/inference/lifecycle-events
GET /v1/inference/lifecycle-events HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

[
  {
    "event_id": "123e4567-e89b-12d3-a456-426614174000",
    "generation": null,
    "kind": "text",
    "occurred_at": "2026-01-01T00:00:00.000Z",
    "payload": null,
    "profile_id": null,
    "sequence": null
  }
]

Ingest one trusted lifecycle envelope over HTTP. Same event_id dedupe and transactional ingestion as the RabbitMQ path.

post
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Body

Shared envelope for trusted lifecycle events.

action_idstring · nullableOptional

Checkpoint/restore action UUID when the event belongs to one.

activation_idstring · nullableOptional

Model activation row UUID when applicable.

artifact_idstring · nullableOptional

Checkpoint artifact UUID for checkpoint events.

boot_idstring · nullableOptional
cluster_idstring · uuidRequired

Nebius cluster (clusters.id UUID).

event_idstring · uuidRequired

Idempotency key: generated once per logical event and reused across transport retries (RabbitMQ redelivery, HTTP retry).

generationinteger · int64Optional

Activation observed generation.

kindstringRequired

One of SUPPORTED_LIFECYCLE_KINDS.

node_uidstring · nullableOptional

Node the event came from (checkpoint tmpfs artifacts are boot-tied).

occurred_atstring · date-timeRequired

When the event happened at the source (RFC3339 UTC).

payloadanyOptional

Free-form JSON; required fields vary per kind.

profile_idstringRequired

Deployment profile id (e.g. qwen3-30b-a3b-l40s-fp8).

schema_versioninteger · int32Required

MUST be 1; anything else is dead-lettered on ingestion.

sequenceinteger · int64Optional

Per-(cluster_id, activation_id) monotonic counter; 0 when not applicable.

workload_uidstring · nullableOptional

k8s pod/DGD UID when relevant.

Responses
204

Lifecycle event accepted

No content
post/v1/inference/lifecycle/events
POST /v1/inference/lifecycle/events HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 348

{
  "action_id": null,
  "activation_id": null,
  "artifact_id": null,
  "boot_id": null,
  "cluster_id": "123e4567-e89b-12d3-a456-426614174000",
  "event_id": "123e4567-e89b-12d3-a456-426614174000",
  "generation": 1,
  "kind": "text",
  "node_uid": null,
  "occurred_at": "2026-01-01T00:00:00.000Z",
  "payload": null,
  "profile_id": "text",
  "schema_version": 1,
  "sequence": 1,
  "workload_uid": null
}
No content

Routing metrics summary

get
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Responses
200

Routing metrics summary

application/json
average_backend_latency_msnumber · doubleRequired
average_routing_latency_msnumber · doubleRequired
estimated_cold_start_avoided_msinteger · int64Required
fallbacksinteger · int64Required
requestsinteger · int64Required
successful_requestsinteger · int64Required
get/v1/inference/metrics
GET /v1/inference/metrics HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

{
  "average_backend_latency_ms": 1,
  "average_routing_latency_ms": 1,
  "estimated_cold_start_avoided_ms": 1,
  "fallbacks": 1,
  "requests": 1,
  "successful_requests": 1
}

OpenAI-compatible model aliases

get
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Responses
200

OpenAI models list

application/json
anyOptional
get/v1/inference/models
GET /v1/inference/models HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

No content

List recent observations

get
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Query parameters
profile_idstring · nullableOptional
experiment_idstring · nullableOptional
event_typestring · nullableOptional
limitinteger · int64Optional
Responses
200

Recent observations

application/json
completion_tokensinteger · nullableOptional
deadline_missedboolean · nullableOptional
duration_msinteger · nullableOptional
event_typestringRequired
evidencestringRequired
experiment_idstring · nullableOptional
fallback_usedboolean · nullableOptional
idstring · uuidRequired
logical_bytesinteger · nullableOptional
observation_keystringRequired
physical_bytesinteger · nullableOptional
profile_idstring · nullableOptional
prompt_tokensinteger · nullableOptional
sourcestringRequired
started_atstring · date-timeRequired
successboolean · nullableOptional
ttft_msinteger · nullableOptional
get/v1/inference/observations
GET /v1/inference/observations HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

[
  {
    "completion_tokens": null,
    "deadline_missed": null,
    "duration_ms": null,
    "event_type": "text",
    "evidence": "text",
    "experiment_id": null,
    "fallback_used": null,
    "id": "123e4567-e89b-12d3-a456-426614174000",
    "logical_bytes": null,
    "observation_key": "text",
    "physical_bytes": null,
    "profile_id": null,
    "prompt_tokens": null,
    "source": "text",
    "started_at": "2026-01-01T00:00:00.000Z",
    "success": null,
    "ttft_ms": null
  }
]

Record an observation

post
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Body
action_idstring · nullableOptional
checkpoint_idstring · nullableOptional
checkpoint_modestring · nullableOptional
cluster_idstring · nullableOptional
completed_atstring · nullableOptional
completion_tokensinteger · nullableOptional
deadline_missedboolean · nullableOptional
deployment_idstring · nullableOptional
dimensionsanyOptional
duration_msinteger · nullableOptional
event_typestringRequired
evidencestringRequired
experiment_idstring · nullableOptional
fallback_usedboolean · nullableOptional
logical_bytesinteger · nullableOptional
metricsanyOptional
node_idstring · nullableOptional
observation_keystringRequired
physical_bytesinteger · nullableOptional
profile_idstring · nullableOptional
prompt_tokensinteger · nullableOptional
request_idstring · nullableOptional
reused_bytesinteger · nullableOptional
sourcestringRequired
started_atstring · date-timeRequired
storage_tierstring · nullableOptional
successboolean · nullableOptional
transferred_bytesinteger · nullableOptional
ttft_msinteger · nullableOptional
written_bytesinteger · nullableOptional
Responses
201

Observation recorded

application/json
duration_msinteger · nullableOptional
event_typestringRequired
evidencestringRequired
experiment_idstring · nullableOptional
idstring · uuidRequired
observation_keystringRequired
profile_idstring · nullableOptional
sourcestringRequired
started_atstring · date-timeRequired
post/v1/inference/observations
POST /v1/inference/observations HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 613

{
  "action_id": null,
  "checkpoint_id": null,
  "checkpoint_mode": null,
  "cluster_id": null,
  "completed_at": null,
  "completion_tokens": null,
  "deadline_missed": null,
  "deployment_id": null,
  "dimensions": null,
  "duration_ms": null,
  "event_type": "text",
  "evidence": "text",
  "experiment_id": null,
  "fallback_used": null,
  "logical_bytes": null,
  "metrics": null,
  "node_id": null,
  "observation_key": "text",
  "physical_bytes": null,
  "profile_id": null,
  "prompt_tokens": null,
  "request_id": null,
  "reused_bytes": null,
  "source": "text",
  "started_at": "2026-01-01T00:00:00.000Z",
  "storage_tier": null,
  "success": null,
  "transferred_bytes": null,
  "ttft_ms": null,
  "written_bytes": null
}
{
  "duration_ms": null,
  "event_type": "text",
  "evidence": "text",
  "experiment_id": null,
  "id": "123e4567-e89b-12d3-a456-426614174000",
  "observation_key": "text",
  "profile_id": null,
  "source": "text",
  "started_at": "2026-01-01T00:00:00.000Z"
}

Read durable outbox events

get
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Query parameters
consumer_idstringRequired
afterinteger · nullableOptional
limitinteger · int64Optional
Responses
200

Outbox events after cursor

application/json
aggregate_idstringRequired
aggregate_typestringRequired
created_atstring · date-timeRequired
event_typestringRequired
idinteger · int64Required
payloadanyRequired
get/v1/inference/outbox
GET /v1/inference/outbox?consumer_id=text HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

[
  {
    "aggregate_id": "text",
    "aggregate_type": "text",
    "created_at": "2026-01-01T00:00:00.000Z",
    "event_type": "text",
    "id": 1,
    "payload": null
  }
]

Advance an outbox consumer cursor

post
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Body
consumer_idstringRequired
event_idinteger · int64Required
Responses
204

Cursor advanced

No content
post/v1/inference/outbox/ack
POST /v1/inference/outbox/ack HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 35

{
  "consumer_id": "text",
  "event_id": 1
}
No content

List deployment profiles

get
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Responses
200

List of deployment profiles

application/json
backend_urlstringRequired
compatibility_keystringRequired
dynamo_templateanyOptional
estimated_cold_start_msinteger · int64Required
exact_modelstringRequired
hosting_modestringRequired
logical_modelstringRequired
max_queue_depthinteger · int32Required
max_replicasinteger · int32Required
min_replicasinteger · int32Required
profile_idstringRequired
profile_revisionstringRequired
scale_down_idle_secondsinteger · int64Required
specanyRequired
target_concurrencyinteger · int32Required
get/v1/inference/profiles
GET /v1/inference/profiles HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

[
  {
    "backend_url": "text",
    "compatibility_key": "text",
    "dynamo_template": null,
    "estimated_cold_start_ms": 1,
    "exact_model": "text",
    "hosting_mode": "text",
    "logical_model": "text",
    "max_queue_depth": 1,
    "max_replicas": 1,
    "min_replicas": 1,
    "profile_id": "text",
    "profile_revision": "text",
    "scale_down_idle_seconds": 1,
    "spec": null,
    "target_concurrency": 1
  }
]

Create an immutable deployment profile

post
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Body
backend_urlstringRequired
compatibilityanyOptional
dynamo_templateanyOptional
estimated_cold_start_msinteger · int64Optional
exact_modelstringRequired
hosting_modestringOptional
logical_modelstringRequired
max_queue_depthinteger · int32Optional
max_replicasinteger · int32Optional
min_replicasinteger · int32Optional
profile_idstringRequired
scale_down_idle_secondsinteger · int64Optional
specanyRequired
target_concurrencyinteger · int32Optional
Responses
201

Profile created

application/json
backend_urlstringRequired
compatibility_keystringRequired
dynamo_templateanyOptional
estimated_cold_start_msinteger · int64Required
exact_modelstringRequired
hosting_modestringRequired
logical_modelstringRequired
max_queue_depthinteger · int32Required
max_replicasinteger · int32Required
min_replicasinteger · int32Required
profile_idstringRequired
profile_revisionstringRequired
scale_down_idle_secondsinteger · int64Required
specanyRequired
target_concurrencyinteger · int32Required
post/v1/inference/profiles
POST /v1/inference/profiles HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 297

{
  "backend_url": "text",
  "compatibility": null,
  "dynamo_template": null,
  "estimated_cold_start_ms": 1,
  "exact_model": "text",
  "hosting_mode": "text",
  "logical_model": "text",
  "max_queue_depth": 1,
  "max_replicas": 1,
  "min_replicas": 1,
  "profile_id": "text",
  "scale_down_idle_seconds": 1,
  "spec": null,
  "target_concurrency": 1
}
{
  "backend_url": "text",
  "compatibility_key": "text",
  "dynamo_template": null,
  "estimated_cold_start_ms": 1,
  "exact_model": "text",
  "hosting_mode": "text",
  "logical_model": "text",
  "max_queue_depth": 1,
  "max_replicas": 1,
  "min_replicas": 1,
  "profile_id": "text",
  "profile_revision": "text",
  "scale_down_idle_seconds": 1,
  "spec": null,
  "target_concurrency": 1
}

The latest checkpoint request for a profile, outstanding or finished

get
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Path parameters
profile_idstringRequired
Responses
200

Latest checkpoint request

application/json

An on-demand checkpoint request and how far it has got.

artifact_idstring · nullableOptional

The artifact the checkpoint produced, once it succeeded.

completed_atstring · nullableOptional
errorstring · nullableOptional

Why it failed, in the controller's words.

kindstringRequired

checkpoint or restart. One row per profile, so asking twice returns the open one.

profile_idstringRequired
reasonstring · nullableOptional
replace_existingbooleanRequired
request_idstring · uuidRequired

Identifies this attempt. A caller polling with the id it was given cannot be shown a later request's outcome as though it were its own.

requested_atstring · date-timeRequired
started_atstring · nullableOptional
statusstringRequired

pending → running → succeeded | failed

get/v1/inference/profiles/{profile_id}/checkpoint
GET /v1/inference/profiles/{profile_id}/checkpoint HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

{
  "artifact_id": null,
  "completed_at": null,
  "error": null,
  "kind": "text",
  "profile_id": "text",
  "reason": null,
  "replace_existing": true,
  "request_id": "123e4567-e89b-12d3-a456-426614174000",
  "requested_at": "2026-01-01T00:00:00.000Z",
  "started_at": null,
  "status": "text"
}

Ask for a checkpoint of this profile's worker now. Covers the two cases the automatic path refuses: a restored worker, and replacing an existing checkpoint.

post
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Path parameters
profile_idstringRequired
Body

What a caller asks for when it wants a checkpoint taken now.

reasonstring · nullableOptional

What this checkpoint is for, carried onto the artifact so a tier holding several has something to tell them apart by.

replace_existingbooleanOptional

Whether the previous checkpoint may be deleted to make room. On by default, because the tier holds barely more than one dump.

Responses
202

Checkpoint requested

application/json

An on-demand checkpoint request and how far it has got.

artifact_idstring · nullableOptional

The artifact the checkpoint produced, once it succeeded.

completed_atstring · nullableOptional
errorstring · nullableOptional

Why it failed, in the controller's words.

kindstringRequired

checkpoint or restart. One row per profile, so asking twice returns the open one.

profile_idstringRequired
reasonstring · nullableOptional
replace_existingbooleanRequired
request_idstring · uuidRequired

Identifies this attempt. A caller polling with the id it was given cannot be shown a later request's outcome as though it were its own.

requested_atstring · date-timeRequired
started_atstring · nullableOptional
statusstringRequired

pending → running → succeeded | failed

post/v1/inference/profiles/{profile_id}/checkpoint
POST /v1/inference/profiles/{profile_id}/checkpoint HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 39

{
  "reason": null,
  "replace_existing": true
}
{
  "artifact_id": null,
  "completed_at": null,
  "error": null,
  "kind": "text",
  "profile_id": "text",
  "reason": null,
  "replace_existing": true,
  "request_id": "123e4567-e89b-12d3-a456-426614174000",
  "requested_at": "2026-01-01T00:00:00.000Z",
  "started_at": null,
  "status": "text"
}

Report progress on a checkpoint request (controller only)

patch
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Path parameters
profile_idstringRequired
Body

What the controller reports back about a request it picked up.

artifact_idstring · nullableOptional
errorstring · nullableOptional
request_idstring · uuidRequired

Correlates this report to the request it describes, so a late report from a superseded attempt cannot overwrite the current outcome.

statusstringRequired

running when the dump starts, then succeeded or failed.

Responses
200

Updated

application/json

An on-demand checkpoint request and how far it has got.

artifact_idstring · nullableOptional

The artifact the checkpoint produced, once it succeeded.

completed_atstring · nullableOptional
errorstring · nullableOptional

Why it failed, in the controller's words.

kindstringRequired

checkpoint or restart. One row per profile, so asking twice returns the open one.

profile_idstringRequired
reasonstring · nullableOptional
replace_existingbooleanRequired
request_idstring · uuidRequired

Identifies this attempt. A caller polling with the id it was given cannot be shown a later request's outcome as though it were its own.

requested_atstring · date-timeRequired
started_atstring · nullableOptional
statusstringRequired

pending → running → succeeded | failed

patch/v1/inference/profiles/{profile_id}/checkpoint
PATCH /v1/inference/profiles/{profile_id}/checkpoint HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 101

{
  "artifact_id": null,
  "error": null,
  "request_id": "123e4567-e89b-12d3-a456-426614174000",
  "status": "text"
}
{
  "artifact_id": null,
  "completed_at": null,
  "error": null,
  "kind": "text",
  "profile_id": "text",
  "reason": null,
  "replace_existing": true,
  "request_id": "123e4567-e89b-12d3-a456-426614174000",
  "requested_at": "2026-01-01T00:00:00.000Z",
  "started_at": null,
  "status": "text"
}

Hosting plan for a profile

get
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Path parameters
profile_idstringRequired
Responses
200

Hosting plan

application/json
checkpoint_ownerstringRequired
compatibility_keystringRequired
cpu_fixturebooleanRequired
dynamo_templateanyOptional
hosting_modestringRequired
profile_idstringRequired
profile_revisionstringRequired
get/v1/inference/profiles/{profile_id}/hosting
GET /v1/inference/profiles/{profile_id}/hosting HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

{
  "checkpoint_owner": "text",
  "checkpoint_request": null,
  "compatibility_key": "text",
  "cpu_fixture": true,
  "dynamo_template": null,
  "hosting_mode": "text",
  "profile_id": "text",
  "profile_revision": "text"
}

Live prefix-cache counters, read from the worker. The 30s scrape series is too coarse to attribute to one chat turn.

get
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Path parameters
profile_idstringRequired
Responses
200

Live cache counters

application/json

What a profile's engine is currently reusing.

block_sizeinteger · nullableOptional

Tokens per KV block. Reuse happens in whole blocks, so a prompt shorter than two of them reports no hits however often it repeats.

gpu_blocksinteger · nullableOptional
pod_namestring · nullableOptional

Which worker answered, so a caller can tell a restart apart from a counter that merely moved.

prefix_cachingboolean · nullableOptional
profile_idstringRequired
queried_tokensinteger · int64Required

Prompt tokens the engine has looked up, cumulatively.

reused_tokensinteger · int64Required

Of those, how many skipped prefill because they were already resident.

get/v1/inference/profiles/{profile_id}/kv-cache
GET /v1/inference/profiles/{profile_id}/kv-cache HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

{
  "block_size": null,
  "gpu_blocks": null,
  "pod_name": null,
  "prefix_caching": null,
  "profile_id": "text",
  "queried_tokens": 1,
  "reused_tokens": 1
}

Deployment progress and cold-start timing for a profile

get
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Path parameters
profile_idstringRequired
Responses
200

Deployment progress

application/json

How far a profile has got, and how long each stage took.

activation_started_atstring · nullableOptional
cold_start_msinteger · nullableOptional

Measured activation-start-to-ready, absent until the profile is ready. A time-to-ready, and only a cold start when start_kind is not restore.

phasestring · nullableOptional
profile_idstringRequired
readybooleanRequired
ready_atstring · nullableOptional
restore_checkpoint_pathstring · nullableOptional

The checkpoint the worker was told to restore from, when it was.

start_kindstring · nullableOptional

How the worker behind this activation started: restore, cold, or null when no live worker pod is known.

get/v1/inference/profiles/{profile_id}/progress
GET /v1/inference/profiles/{profile_id}/progress HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

{
  "activation_started_at": null,
  "cold_start_ms": null,
  "phase": null,
  "profile_id": "text",
  "ready": true,
  "ready_at": null,
  "restore_checkpoint_path": null,
  "start_kind": null,
  "steps": [
    {
      "elapsed_ms": null,
      "kind": "text",
      "occurred_at": "2026-01-01T00:00:00.000Z",
      "payload": null
    }
  ]
}

Replace this profile's worker so it restores from its last checkpoint, coming back with the KV cache it had in about a minute rather than eleven.

post
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Path parameters
profile_idstringRequired
Responses
202

Restart requested

application/json

An on-demand checkpoint request and how far it has got.

artifact_idstring · nullableOptional

The artifact the checkpoint produced, once it succeeded.

completed_atstring · nullableOptional
errorstring · nullableOptional

Why it failed, in the controller's words.

kindstringRequired

checkpoint or restart. One row per profile, so asking twice returns the open one.

profile_idstringRequired
reasonstring · nullableOptional
replace_existingbooleanRequired
request_idstring · uuidRequired

Identifies this attempt. A caller polling with the id it was given cannot be shown a later request's outcome as though it were its own.

requested_atstring · date-timeRequired
started_atstring · nullableOptional
statusstringRequired

pending → running → succeeded | failed

post/v1/inference/profiles/{profile_id}/restart
POST /v1/inference/profiles/{profile_id}/restart HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

{
  "artifact_id": null,
  "completed_at": null,
  "error": null,
  "kind": "text",
  "profile_id": "text",
  "reason": null,
  "replace_existing": true,
  "request_id": "123e4567-e89b-12d3-a456-426614174000",
  "requested_at": "2026-01-01T00:00:00.000Z",
  "started_at": null,
  "status": "text"
}

Set a profile Active or Suspended (capacity intent)

post
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Path parameters
profile_idstringRequired
Body
deadline_msinteger · nullableOptional
desired_replicasinteger · nullableOptional
desired_statestringRequired
Responses
202

Capacity intent accepted

application/json
created_atstring · date-timeRequired

When this intent row was first created. Not when the request it currently describes was made: the row is reused for a repeated ask. Use requested_at for that.

deadlinestring · nullableOptional
desired_replicasinteger · int32Required
desired_statestringRequired
generationinteger · int64Required
idstring · uuidRequired
profile_idstringRequired
requested_atstring · date-timeOptional

When a caller last asked for this exact state and replica count, so it orders requests as they were made rather than as the row happened to change.

statusstringRequired
updated_atstring · date-timeRequired

When this row last changed for any reason, including the expiry sweep. Not an ordering key for what a caller asked for.

post/v1/inference/profiles/{profile_id}/state
POST /v1/inference/profiles/{profile_id}/state HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 67

{
  "deadline_ms": null,
  "desired_replicas": null,
  "desired_state": "text"
}
{
  "created_at": "2026-01-01T00:00:00.000Z",
  "deadline": null,
  "desired_replicas": 1,
  "desired_state": "text",
  "generation": 1,
  "id": "123e4567-e89b-12d3-a456-426614174000",
  "profile_id": "text",
  "requested_at": "2026-01-01T00:00:00.000Z",
  "status": "text",
  "updated_at": "2026-01-01T00:00:00.000Z"
}

Record where a profile's worker publishes its metrics (controller only). The propagator can reach the port but cannot find the pod IP itself.

put
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Path parameters
profile_idstringRequired
Body

Where a profile's worker publishes engine metrics (controller only).

metrics_urlstringRequired
pod_namestring · nullableOptional
Responses
204

Recorded

No content
put/v1/inference/profiles/{profile_id}/worker-endpoint
PUT /v1/inference/profiles/{profile_id}/worker-endpoint HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 38

{
  "metrics_url": "text",
  "pod_name": null
}
No content

List retention policies

get
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Responses
200

Retention policies

application/json
enabledbooleanRequired
max_age_daysinteger · nullableOptional
max_countinteger · nullableOptional
min_tierstring · nullableOptional
namestringRequired
policy_idstringRequired
profile_idstring · nullableOptional
get/v1/inference/retention-policies
GET /v1/inference/retention-policies HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

[
  {
    "enabled": true,
    "max_age_days": null,
    "max_count": null,
    "min_tier": null,
    "name": "text",
    "policy_id": "text",
    "profile_id": null
  }
]

Create or update a retention policy

post
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Body
enabledbooleanOptional
max_age_daysinteger · nullableOptional
max_countinteger · nullableOptional
min_tierstring · nullableOptional
namestringRequired
policy_idstring · nullableOptional
profile_idstring · nullableOptional
Responses
200

Policy saved

application/json
enabledbooleanRequired
max_age_daysinteger · nullableOptional
max_countinteger · nullableOptional
min_tierstring · nullableOptional
namestringRequired
policy_idstringRequired
profile_idstring · nullableOptional
post/v1/inference/retention-policies
POST /v1/inference/retention-policies HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 118

{
  "enabled": true,
  "max_age_days": null,
  "max_count": null,
  "min_tier": null,
  "name": "text",
  "policy_id": null,
  "profile_id": null
}
{
  "enabled": true,
  "max_age_days": null,
  "max_count": null,
  "min_tier": null,
  "name": "text",
  "policy_id": "text",
  "profile_id": null
}

Delete a retention policy

delete
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Path parameters
policy_idstringRequired
Responses
204

Policy deleted

No content
delete/v1/inference/retention-policies/{policy_id}
DELETE /v1/inference/retention-policies/{policy_id} HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

No content

List route policies

get
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Responses
200

List of route policies

application/json
classifier_profile_idstring · nullableOptional
classifier_promptstring · nullableOptional
classifier_response_schemastring · nullableOptional
generationinteger · int64Required
logical_modelstringRequired
modestringRequired
queue_timeout_msinteger · int64Required
get/v1/inference/routes
GET /v1/inference/routes HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

[
  {
    "classifier_profile_id": null,
    "classifier_prompt": null,
    "classifier_response_schema": null,
    "generation": 1,
    "logical_model": "text",
    "mode": "text",
    "queue_timeout_ms": 1,
    "targets": [
      {
        "priority": 1,
        "profile_id": "text",
        "tier": "text"
      }
    ]
  }
]

Create or update a route policy

post
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Body
classifier_profile_idstring · nullableOptional
classifier_promptstring · nullableOptional
classifier_response_schemastring · nullableOptional
logical_modelstringRequired
modestringOptional
queue_timeout_msinteger · int64Optional
Responses
200

Route policy saved

application/json
classifier_profile_idstring · nullableOptional
classifier_promptstring · nullableOptional
classifier_response_schemastring · nullableOptional
generationinteger · int64Required
logical_modelstringRequired
modestringRequired
queue_timeout_msinteger · int64Required
post/v1/inference/routes
POST /v1/inference/routes HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 208

{
  "classifier_profile_id": null,
  "classifier_prompt": null,
  "classifier_response_schema": null,
  "logical_model": "text",
  "mode": "text",
  "queue_timeout_ms": 1,
  "targets": [
    {
      "priority": 1,
      "profile_id": "text",
      "tier": "text"
    }
  ]
}
{
  "classifier_profile_id": null,
  "classifier_prompt": null,
  "classifier_response_schema": null,
  "generation": 1,
  "logical_model": "text",
  "mode": "text",
  "queue_timeout_ms": 1,
  "targets": [
    {
      "priority": 1,
      "profile_id": "text",
      "tier": "text"
    }
  ]
}

Project routing outcomes for a traffic mix. A step function on p50, and a profile with nothing measured yet falls back to estimates rather than measurements.

post
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Body
cold_sharenumber · doubleRequired
deadline_msinteger · int64Required
logical_modelstringRequired
requests_per_hournumber · doubleRequired
restore_sharenumber · doubleRequired
Responses
200

Projected outcomes

application/json
cold_sharenumber · doubleRequired
evidencestringRequired
logical_modelstringRequired
projected_cold_p50_msnumber · nullableOptional
projected_deadline_miss_ratenumber · doubleRequired
projected_fallback_ratenumber · doubleRequired
projected_p50_msnumber · doubleRequired
projected_restore_p50_msnumber · nullableOptional
requests_per_hournumber · doubleRequired
restore_sharenumber · doubleRequired
post/v1/inference/routes/simulate
POST /v1/inference/routes/simulate HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 95

{
  "cold_share": 1,
  "deadline_ms": 1,
  "logical_model": "text",
  "requests_per_hour": 1,
  "restore_share": 1
}
{
  "cold_share": 1,
  "evidence": "text",
  "logical_model": "text",
  "projected_cold_p50_ms": null,
  "projected_deadline_miss_rate": 1,
  "projected_fallback_rate": 1,
  "projected_p50_ms": 1,
  "projected_restore_p50_ms": null,
  "requests_per_hour": 1,
  "restore_share": 1
}

Live route-state snapshot per logical model

get
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Responses
200

Route-state snapshot

application/json
endpointstring · nullableOptional
fallback_profile_idstring · nullableOptional
fallback_readybooleanRequired
logical_modelstringRequired
pending_activationbooleanRequired

A Pending intent asks for this profile to be Active on at least one replica, so something is trying to bring it up.

pending_intentbooleanRequired

Any capacity intent for the primary is still Pending, either direction. Rarely the field you want; prefer pending_activation or pending_release.

pending_releasebooleanRequired

A Pending intent asks for this profile to stop holding an accelerator (Suspended, or Active at zero replicas).

primary_phasestring · nullableOptional
primary_profile_idstringRequired
primary_readybooleanRequired
queue_timeout_msinteger · int64Required
restore_checkpoint_pathstring · nullableOptional

The checkpoint the live worker was told to restore from, when it was.

start_kindstring · nullableOptional

How the worker now on this profile started, read from the pod spec: restore when it was given a checkpoint, cold otherwise.

statestringRequired
get/v1/inference/routes/status
GET /v1/inference/routes/status HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

[
  {
    "endpoint": null,
    "fallback_profile_id": null,
    "fallback_ready": true,
    "logical_model": "text",
    "pending_activation": true,
    "pending_intent": true,
    "pending_release": true,
    "primary_phase": null,
    "primary_profile_id": "text",
    "primary_ready": true,
    "queue_timeout_ms": 1,
    "restore_checkpoint_path": null,
    "start_kind": null,
    "state": "text"
  }
]

Cold start against restore, per profile. Measured from the workers that ran, not estimated.

get
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Responses
200

Cold versus restore worker startup, per profile

application/json

Cold starts against restores, for one profile.

logical_modelstringRequired
profile_idstringRequired
saved_msinteger · nullableOptional

Milliseconds a restore saves at the median.

speedupnumber · nullableOptional

Cold median divided by restore median. Above 1 means restoring is faster.

get/v1/inference/startup-comparison
GET /v1/inference/startup-comparison HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

[
  {
    "cold": null,
    "logical_model": "text",
    "profile_id": "text",
    "restored": null,
    "saved_ms": null,
    "speedup": null,
    "starts": [
      {
        "checkpoint_path": null,
        "pod_name": "text",
        "ready_at": null,
        "restored": true,
        "start_time_ms": 1
      }
    ]
  }
]

Per-tier storage summary

get
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Responses
200

Per-tier storage totals

application/json
total_artifactsinteger · int64Required
total_bytesinteger · int64Required
get/v1/inference/storage/summary
GET /v1/inference/storage/summary HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

{
  "tiers": [
    {
      "artifacts": 1,
      "bytes": 1,
      "residency_hours": 1,
      "tier": "text",
      "transfers": 1
    }
  ],
  "total_artifacts": 1,
  "total_bytes": 1
}

Bucketed usage time series

get
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Query parameters
hoursinteger · int64Optional
bucketstringOptional
profile_idstring · nullableOptional
experiment_idstring · nullableOptional
Responses
200

Bucketed usage

application/json
avg_backend_msnumber · nullableOptional
avg_routing_msnumber · nullableOptional
avg_ttft_msnumber · nullableOptional
bucketstring · date-timeRequired
cold_start_avoided_msinteger · int64Required
completion_tokensinteger · int64Required
deadline_missesinteger · int64Required
fallbacksinteger · int64Required
prompt_tokensinteger · int64Required
requestsinteger · int64Required
successesinteger · int64Required
get/v1/inference/usage/series
GET /v1/inference/usage/series HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

[
  {
    "avg_backend_ms": null,
    "avg_routing_ms": null,
    "avg_ttft_ms": null,
    "bucket": "2026-01-01T00:00:00.000Z",
    "cold_start_avoided_ms": 1,
    "completion_tokens": 1,
    "deadline_misses": 1,
    "fallbacks": 1,
    "prompt_tokens": 1,
    "requests": 1,
    "successes": 1
  }
]
PreviousOtel
NextDynamo

Last updated

Was this helpful?