Ctrlk
For the complete documentation index, see llms.txt. This page is also available as Markdown.

Router

Cedana API for the inference router

Authenticate with the inbound Cedana inference key and issue an opaque authorization that resolve consumes exactly once.

post
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Body
estimated_prompt_tokensinteger · int64Optional
logical_modelstringRequired
request_idstringRequired
Responses
200

Authorization issued

application/json
authorization_idstring · uuidRequired
expires_atstring · date-timeRequired
route_generationinteger · int64Required
post/v1/inference/router/authorize
POST /v1/inference/router/authorize HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 72

{
  "estimated_prompt_tokens": 1,
  "logical_model": "text",
  "request_id": "text"
}
{
  "authorization_id": "123e4567-e89b-12d3-a456-426614174000",
  "expires_at": "2026-01-01T00:00:00.000Z",
  "route_generation": 1
}

Full routing snapshot for Switchyard, service-token-only. Generation is the sum of the route generations; `If-None-Match: "<generation>"` gets a 304.

get
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Responses
200

Routing snapshot

application/json
generationinteger · int64Required
schema_versioninteger · int32Required
get/v1/inference/router/config
GET /v1/inference/router/config HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

{
  "generation": 1,
  "routes": [
    {
      "classifier_profile_id": null,
      "classifier_prompt": null,
      "classifier_response_schema": null,
      "generation": 1,
      "logical_model": "text",
      "mode": "text",
      "queue_timeout_ms": 1,
      "targets": [
        {
          "current_replicas": 1,
          "draining": true,
          "max_queue_depth": 1,
          "max_replicas": 1,
          "min_replicas": 1,
          "node_hourly_usd": 1,
          "priority": 1,
          "profile_id": "text",
          "ready": true,
          "service_url": "text",
          "target_concurrency": 1,
          "tier": "text",
          "upstream_model": "text"
        }
      ]
    }
  ],
  "schema_version": 1
}

Accept batched metric samples from Switchyard into ClickHouse, service-token-only.

post
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Body
Responses
200

Samples ingested

No content
post/v1/inference/router/metrics
POST /v1/inference/router/metrics HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 95

{
  "samples": [
    {
      "metric_name": "text",
      "profile_id": "text",
      "route": "text",
      "timestamp": 1,
      "value": 1
    }
  ]
}
No content

Assign exactly one target for an authorized request, service-token only. A row lock consumes the authorization once.

post
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Body
authorization_idstring · uuidRequired
logical_modelstringRequired
preferred_profile_idstring · nullableOptional
queued_requestsinteger · int64Optional
request_idstringRequired
Responses
200

Target assigned

application/json
activation_pathstringRequired
decision_sourcestringRequired
fallback_usedbooleanRequired
profile_idstringRequired
route_generationinteger · int64Required
target_namestringRequired
upstream_modelstringRequired
post/v1/inference/router/resolve
POST /v1/inference/router/resolve HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 208

{
  "active_streams_by_profile": {
    "ANY_ADDITIONAL_PROPERTY": 1
  },
  "authorization_id": "123e4567-e89b-12d3-a456-426614174000",
  "logical_model": "text",
  "preferred_profile_id": null,
  "queued_requests": 1,
  "request_id": "text"
}
{
  "activation_path": "text",
  "decision_source": "text",
  "fallback_used": true,
  "profile_id": "text",
  "route_generation": 1,
  "target_name": "text",
  "upstream_model": "text"
}

Accept one completed-request record from Switchyard, service-token-only. Idempotent by request_id: replays of the same completed record are no-ops.

post
Authorizations
AuthorizationstringRequired
Bearer authentication header of the form Bearer <token>.
Body
activation_pathstring · nullableOptional
classifier_latency_msinteger · nullableOptional
classifier_profile_idstring · nullableOptional
completion_tokensinteger · nullableOptional
decision_sourcestring · nullableOptional
end_to_end_latency_msinteger · nullableOptional
fallback_usedboolean · nullableOptional
logical_modelstringRequired
mean_inter_token_msnumber · nullableOptional
p95_inter_token_msnumber · nullableOptional
profile_idstring · nullableOptional
prompt_tokensinteger · nullableOptional
queue_msinteger · nullableOptional
request_idstringRequired
route_generationinteger · nullableOptional
status_codeinteger · nullableOptional
stream_duration_msinteger · nullableOptional
ttft_msinteger · nullableOptional
upstream_attemptsinteger · nullableOptional
Responses
200

Record ingested

No content
post/v1/inference/router/telemetry
POST /v1/inference/router/telemetry HTTP/1.1
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 440

{
  "activation_path": null,
  "classifier_latency_ms": null,
  "classifier_profile_id": null,
  "completion_tokens": null,
  "decision_source": null,
  "end_to_end_latency_ms": null,
  "fallback_used": null,
  "logical_model": "text",
  "mean_inter_token_ms": null,
  "p95_inter_token_ms": null,
  "profile_id": null,
  "prompt_tokens": null,
  "queue_ms": null,
  "request_id": "text",
  "route_generation": null,
  "status_code": null,
  "stream_duration_ms": null,
  "ttft_ms": null,
  "upstream_attempts": null
}
No content
PreviousDynamo
NextPricing

Last updated

Was this helpful?