Skip to content
Reference /API reference

Reference

Page 6 of 6

API reference

Every route Charon serves: the OpenAI-compatible /v1 API your apps call, and the /admin API the app and CLI use.

11 sections

Charon listens on http://127.0.0.1:7777. Apps call /v1 exactly as they would call OpenAI. The app and the charon command use /admin to manage Charon. Version0.1.0.

Base URL
http://127.0.0.1:7777
Admin token
~/.config/charon/admin.token

Authentication

Apps on this machine can call /v1 with no key. Otherwise, send a client key (Authorization: Bearer chk_…, made with charon keys create). A key is required for everyone when allow_anonymous_local is off or allow_remote is on.

/admin always needs the admin token, sent as Authorization: Bearer <token>. It is in ~/.config/charon/admin.token.

Errors look like OpenAI's: {"error": {"type", "message", "code"}}. When no backend can take a request, reasons says why for each one.

Status

GET/

Identifies the daemon (the app and CLI use it to find charond); no token needed

Responses

  • 200Name and versionobject
GET/admin/status

Version, uptime, backends, loaded models, spend and hardware summary

Responses

  • 200Daemon statusStatus
  • 401Missing or wrong admin tokenError
GET/admin/hardware

Latest hardware snapshot (sampled every 500 ms)

Responses

GET/admin/events

Server-sent events

event: names and data: payloads:

  • request.routed — RoutedEvent
  • request.done — RequestRecord
  • hardware — Hardware (about once per second)
  • backend.health — BackendHealthEvent (every probe, and once per backend on connect)
  • backend.removed — { "id": string }
  • model.progress — ModelProgressEvent
  • config.reloaded — { "what": "all" | "policy" | "settings" }

A : ping comment is sent every 15 s.

Responses

  • 200text/event-streamstring
  • 401Missing or wrong admin tokenError
GET/metrics

Prometheus text exposition (admin token required when allow_remote)

Responses

  • 200Metricsstring

Backends

GET/admin/backends

List backends with health and models

Responses

  • 200Backends in config orderobject
  • 401Missing or wrong admin tokenError
POST/admin/backends

Add a backend (persisted to charon.toml)

Secrets are never accepted here. Store the key in the OS keychain (service charon) and send key_ref: "keychain:<account>", or name an environment variable with key_env.

Request body

BackendConfig

Responses

  • 201AddedBackend
  • 400Invalid inputError
  • 401Missing or wrong admin tokenError
  • 409Already exists or in the wrong stateError
GET/admin/backends/{id}

One backend

Parameters

Parameters
NameInTypeDescription
idRequiredpathstring

Responses

DELETE/admin/backends/{id}

Remove a backend (persisted to charon.toml)

Parameters

Parameters
NameInTypeDescription
idRequiredpathstring

Responses

  • 204Removed
  • 401Missing or wrong admin tokenError
  • 404Not foundError
POST/admin/backends/{id}/test

Send a probe request and return its latency

Parameters

Parameters
NameInTypeDescription
idRequiredpathstring

Responses

  • 200Probe result (ok=false carries the error)ProbeResult
  • 401Missing or wrong admin tokenError
  • 404Not foundError
POST/admin/backends/{id}/start

Start an EC2 GPU instance and register it once vLLM is healthy

Returns immediately; progress arrives as backend.health events (state pending → starting → running).

Parameters

Parameters
NameInTypeDescription
idRequiredpathstring

Responses

POST/admin/backends/{id}/stop

Stop an EC2 GPU instance

Parameters

Parameters
NameInTypeDescription
idRequiredpathstring

Responses

Models

GET/admin/models

Local GGUF models with estimated VRAM and load state

Responses

  • 200Models from models.yamlobject
  • 401Missing or wrong admin tokenError
POST/admin/models

Register a GGUF file in the models directory, or download one

Either catalog_id (curated download), url + sha256 (Hugging Face download, verified), or path (a file already inside the models directory). Downloads run in the background and report model.progress events.

Request body

AddModel

Responses

  • 202Accepted (download started or model registered)LocalModel
  • 400Invalid inputError
  • 401Missing or wrong admin tokenError
  • 409Already exists or in the wrong stateError
DELETE/admin/models/{id}

Unload and unregister a model (the file is kept)

Parameters

Parameters
NameInTypeDescription
idRequiredpathstring

Responses

  • 204Removed
  • 401Missing or wrong admin tokenError
  • 404Not foundError
POST/admin/models/{id}/load

Start a worker for the model

Parameters

Parameters
NameInTypeDescription
idRequiredpathstring

Responses

POST/admin/models/{id}/unload

Stop the model's worker

Parameters

Parameters
NameInTypeDescription
idRequiredpathstring

Responses

Policy

GET/admin/policy

The active policy as YAML and as parsed JSON

Responses

PUT/admin/policy

Validate and replace policy.yaml

Request body

objectyaml

Responses

Keys

GET/admin/keys

Client API keys (secrets are never returned)

Responses

  • 200Keysobject
  • 401Missing or wrong admin tokenError
POST/admin/keys

Create a client key; the secret is returned once

Request body

objectname, tags

Responses

  • 201Createdobject
  • 400Invalid inputError
  • 401Missing or wrong admin tokenError
  • 409Already exists or in the wrong stateError
DELETE/admin/keys/{id}

Revoke a key

Parameters

Parameters
NameInTypeDescription
idRequiredpathstring

Responses

  • 204Revoked
  • 401Missing or wrong admin tokenError
  • 404Not foundError

Requests

GET/admin/requests

Request log, newest first

Parameters

Parameters
NameInTypeDescription
limitqueryinteger
beforequerystringRequest id cursor

Responses

  • 200Page of requestsobject
  • 401Missing or wrong admin tokenError
GET/admin/requests/{id}

One request with its full decision trace (score table and attempts)

Parameters

Parameters
NameInTypeDescription
idRequiredpathstring

Responses

GET/admin/spend

Estimated cloud spend by backend

Parameters

Parameters
NameInTypeDescription
periodquery"day" | "month"

Responses

Settings

POST/admin/reload

Re-read charon.toml and policy.yaml (the running config is kept on errors)

Responses

  • 200Reloadedobject
  • 401Missing or wrong admin tokenError
  • 422Invalid config; nothing changed. One entry per problem, with its position.ValidationError
GET/admin/settings

Daemon settings

Responses

PATCH/admin/settings

Change settings; persisted to charon.toml

Request body

SettingsPatch

Responses

  • 200Updated settingsSettings & object
  • 400Invalid inputError
  • 401Missing or wrong admin tokenError

AWS

GET/admin/aws/instances

EC2 instances for the instance picker (DescribeInstances)

Parameters

Parameters
NameInTypeDescription
regionquerystring
profilequerystring
taggedquerybooleanOnly instances tagged charon=true

Responses

  • 200Instancesobject
  • 401Missing or wrong admin tokenError
  • 502Upstream failureError

OpenAI-compatible

Drop-in OpenAI API. Authenticated with a Charon client key or anonymously on loopback.

POST/v1/chat/completions

OpenAI Chat Completions (stream and non-stream), routed by policy

Parameters

Parameters
NameInTypeDescription
X-Charon-Policyheader"local-only"`local-only` forbids cloud backends for this request

Request body

object

Responses

  • 200OpenAI chat completion, or an SSE stream of chunks
  • 400Invalid inputError
  • 401A client key was sent but is unknown or revoked, or anonymous access is disabledError
  • 402`budget_exceeded` (402), `policy_denied` (403) or `no_backend` (503), with per-candidate reasonsError
  • 403`budget_exceeded` (402), `policy_denied` (403) or `no_backend` (503), with per-candidate reasonsError
  • 429The client key's concurrency limit (`[server] per_client_concurrency`), the daemon-wide limit (`[server] max_concurrent_requests`, with Retry-After: 1), too many failed authentication attempts from a remote peer, or the provider's rate limit when no other candidate could serve the requestError
  • 502Upstream failureError
  • 503`budget_exceeded` (402), `policy_denied` (403) or `no_backend` (503), with per-candidate reasonsError
POST/v1/embeddings

OpenAI Embeddings, routed like chat

Request body

object

Responses

  • 200OpenAI embeddings response
  • 400Invalid inputError
  • 401A client key was sent but is unknown or revoked, or anonymous access is disabledError
  • 402`budget_exceeded` (402), `policy_denied` (403) or `no_backend` (503), with per-candidate reasonsError
  • 403`budget_exceeded` (402), `policy_denied` (403) or `no_backend` (503), with per-candidate reasonsError
  • 429The client key's concurrency limit (`[server] per_client_concurrency`), the daemon-wide limit (`[server] max_concurrent_requests`, with Retry-After: 1), too many failed authentication attempts from a remote peer, or the provider's rate limit when no other candidate could serve the requestError
  • 502Upstream failureError
  • 503`budget_exceeded` (402), `policy_denied` (403) or `no_backend` (503), with per-candidate reasonsError
GET/v1/models

Aliases and backend models (`alias` or `backend/model`)

Responses

  • 200OpenAI model list
  • 401A client key was sent but is unknown or revoked, or anonymous access is disabledError
POST/v1/responses

OpenAI Responses API, translated to chat completions

Request body

object

Responses

  • 200OpenAI response object, or an SSE stream of response events
  • 400Invalid inputError
  • 401A client key was sent but is unknown or revoked, or anonymous access is disabledError
  • 402`budget_exceeded` (402), `policy_denied` (403) or `no_backend` (503), with per-candidate reasonsError
  • 403`budget_exceeded` (402), `policy_denied` (403) or `no_backend` (503), with per-candidate reasonsError
  • 429The client key's concurrency limit (`[server] per_client_concurrency`), the daemon-wide limit (`[server] max_concurrent_requests`, with Retry-After: 1), too many failed authentication attempts from a remote peer, or the provider's rate limit when no other candidate could serve the requestError
  • 502Upstream failureError
  • 503`budget_exceeded` (402), `policy_denied` (403) or `no_backend` (503), with per-candidate reasonsError

Schemas

Every object the API sends or accepts. Links in the reference above jump here.

Error1 field
Error fields
FieldTypeDescription
errorRequiredobject
ValidationError1 field
ValidationError fields
FieldTypeDescription
errorRequiredobject
Locality

"local" | "cloud"

Price2 fields
Price fields
FieldTypeDescription
input_per_mtoknumber
output_per_mtoknumber
BackendConfig28 fields

Which keys apply depends on `kind`.

BackendConfig fields
FieldTypeDescription
idRequiredstring
kindRequired"local" | "openai_compat" | "bedrock" | "ec2_vllm" | "azure_openai" | "vertex" | "mock"
base_urlstringopenai_compat, ec2_vllm: …/v1. azure_openai: the resource endpoint, https://<resource>.openai.azure.com
localbooleanopenai_compat, mock: runs on this machine
key_envstring
key_refstring
modelsstring[]Static model list (skips discovery). Required for azure_openai (deployment names) and vertex.
pricesmap<string, Price>
context_lengthinteger
timeout_sinteger
regionstringbedrock, ec2_vllm: AWS region (default us-east-1). vertex: location (default global)
projectstringvertex: Google Cloud project id
auth"auto" | "api_key" | "sigv4"
aws_profilestring
endpointstringbedrock, ec2_vllm, vertex: API host override (VPC or Private Service Connect)
instance_idstring
portinteger
scheme"http" | "https"ec2_vllm without a published TLS pin. Plain HTTP is only used to loopback.
tunnel"ssm" | "none"ec2_vllm (default ssm). With none, Charon connects to the instance's address over HTTPS.
local_portinteger
hourly_usdnumber
idle_stop_minutesnumber
stop_on_exitboolean
slotsinteger
latency_msinteger
tokens_per_secnumber
fail_ratenumber
replystring
BackendModel5 fields
BackendModel fields
FieldTypeDescription
idRequiredstring
context_lengthinteger
embeddingboolean
loadedboolean
detailsobject
Backend15 fields
Backend fields
FieldTypeDescription
idRequiredstring
kindRequiredstring
localityRequiredLocality
healthyRequiredboolean
stateRequiredstringec2_vllm: the instance lifecycle (unknown | stopped | pending | starting | running | stopping | error). local: a summary such as "1 of 3 models loaded". Other kinds leave it empty and rely on `healthy`.
p50_msnumber
error_ratenumber
consecutive_failuresinteger
last_errorstring
last_probestring | null
credentialobject | null
aws_credentialobject | nullbedrock and ec2_vllm — where the AWS credentials for SigV4 come from
modelsRequiredBackendModel[]
configRequiredBackendConfig
instanceEc2Instance
ProbeResult4 fields
ProbeResult fields
FieldTypeDescription
okRequiredboolean
latency_msnumber
modelsstring[]
errorstring
Ec2Instance15 fields
Ec2Instance fields
FieldTypeDescription
instance_idRequiredstring
namestring
stateRequiredstring
instance_typestring
public_dnsstring
public_ipstring
launch_timestring
taggedbooleanHas the charon=true tag
tls_pinstringThe charon:tls-pin tag ("sha256//…" of vLLM's self-signed public key), published by instances from the template; Charon then uses HTTPS and trusts exactly that key.
hourly_usdnumber
running_secondsnumberTime running since Charon first saw it running
ec2_statestringRaw EC2 state (the lifecycle `state` adds starting and error)
idle_stop_minutesnumber
tunnel"none" | "ssm"
errorstring
Gpu10 fields
Gpu fields
FieldTypeDescription
indexRequiredinteger
nameRequiredstring
vendorRequired"nvidia" | "amd" | "apple" | "other"
unified_memoryboolean
vram_total_bytesnumber
vram_used_bytesnumber
vram_free_bytesnumber
utilization_pctnumber | null
temperature_cnumber | null
power_wnumber | null
Hardware11 fields
Hardware fields
FieldTypeDescription
tsRequiredstring
gpusRequiredGpu[]
cpu_pctRequirednumber
cpu_coresinteger
ram_total_bytesRequirednumber
ram_free_bytesRequirednumber
on_batteryRequiredboolean
battery_pctnumber | null
ac_onlineboolean | null
temperature_cnumber | null
thermal_state"nominal" | "fair" | "serious" | "critical" | null
LocalModel20 fields
LocalModel fields
FieldTypeDescription
idRequiredstring
pathRequiredstring
file_size_bytesnumber
context_lengthinteger
parallel_slotsinteger
quantizationstring
embeddingboolean
stateRequired"on_disk" | "downloading" | "loading" | "loaded" | "restarting" | "failed" | "missing"
estimated_vram_bytesnumber
measured_vram_bytesnumber | null
n_gpu_layersinteger | null
fitsboolean
fit_notestring
slots_busyinteger
tokens_per_secnumber | null
errorstring
progressnumber | nullDownload progress 0-1
bytesnumberBytes downloaded so far
total_bytesnumber
bytes_per_secnumber
CatalogModel11 fields
CatalogModel fields
FieldTypeDescription
idRequiredstring
nameRequiredstring
descriptionstring
size_bytesRequirednumber
urlRequiredstring
sha256string
context_lengthinteger
min_vram_bytesnumber
embeddingboolean
installedboolean
recommendedboolean
AddModel8 fields
AddModel fields
FieldTypeDescription
idstring
catalog_idstring
urlstringhttps://huggingface.co/…/resolve/…/file.gguf
sha256string
pathstringFile name inside the models directory
context_lengthinteger
parallel_slotsinteger
embeddingboolean
PolicyDocument4 fields
PolicyDocument fields
FieldTypeDescription
yamlRequiredstring
pathstring
warningsstring[]
policyRequiredobject
Weights4 fields
Weights fields
FieldTypeDescription
latencynumber
costnumber
preferencenumber
loadnumber
ClientKey6 fields
ClientKey fields
FieldTypeDescription
idRequiredstring
nameRequiredstring
prefixRequiredstring
tagsRequiredstring[]
created_atRequiredstring
last_used_atstring | null
RequestRecord18 fields
RequestRecord fields
FieldTypeDescription
idRequiredstring
tsRequiredstring
endpoint"chat" | "embeddings" | "responses"
modelRequiredstring
backendstring
backend_modelstring
localitystring
reasonstring
prompt_tokensinteger
completion_tokensinteger
ttft_msnumber
total_msnumber
est_cost_usdnumber
statusRequired"ok" | "error" | "cancelled"
http_statusinteger
errorstring
streamboolean
clientstring
Candidate15 fields
Candidate fields
FieldTypeDescription
backendRequiredstring
modelRequiredstring
localitystring
positioninteger
survivedRequiredboolean
filterstringunknown_backend | paused | local_only | unhealthy | rate_limited | model | context | vram | saturated | budget | battery
reasonstring
expected_latency_msnumber
latency_normnumber
cost_usdnumber
cost_normnumber
preferencenumber
loadnumber
scorenumber
rankinteger | null
RequestDetail

RequestRecord & object

Spend5 fields
Spend fields
FieldTypeDescription
periodRequired"day" | "month"
total_usdRequirednumber
budget_usdnumber | null
estimatedRequiredboolean
by_backendRequiredobject[]
Settings9 fields
Settings fields
FieldTypeDescription
hoststring
portinteger
allow_anonymous_localboolean
log_level"trace" | "debug" | "info" | "warn" | "error" | "critical" | "off"
log_bodiesboolean
pause_cloudboolean
force_local_onlyboolean
idle_unload_minutesinteger
vram_reserve_mbinteger
SettingsPatch8 fields
SettingsPatch fields
FieldTypeDescription
portinteger
allow_anonymous_localboolean
log_level"trace" | "debug" | "info" | "warn" | "error" | "critical" | "off"
log_bodiesboolean
pause_cloudboolean
force_local_onlyboolean
idle_unload_minutesinteger
vram_reserve_mbinteger
Status16 fields
Status fields
FieldTypeDescription
versionRequiredstring
git_shastring
pidinteger
started_atstring
uptime_sRequiredinteger
listenRequiredstring
config_dirstring
data_dirstring
drainingboolean
routingRequiredobject
backendsRequiredBackend[]
modelsLocalModel[]
hardwareHardware
requestsobject
spendRequiredobject
suggestionsobject[]
RoutedEvent10 fields
RoutedEvent fields
FieldTypeDescription
idstring
tsstring
modelstring
aliasstring
backendstring
backend_modelstring
localitystring
reasonstring
streamboolean
attemptinteger
BackendHealthEvent8 fields
BackendHealthEvent fields
FieldTypeDescription
idstring
kindstring
localitystring
healthyboolean
statestring
p50_msnumber
error_ratenumber
last_errorstring
ModelProgressEvent7 fields
ModelProgressEvent fields
FieldTypeDescription
idstring
statestring
progressnumber | null
bytesnumber
total_bytesnumber
bytes_per_secnumber
errorstring
↑↓to move↵to open