Reference
Page 6 of 6
API reference
Every route Charon serves: the OpenAI-compatible /v1 API your apps call, and the /admin API the app and CLI use.
Charon listens on http://127.0.0.1:7777. Apps call /v1 exactly as they would call OpenAI. The app and the charon command use /admin to manage Charon. Version0.1.0.
- Base URL
http://127.0.0.1:7777- Admin token
~/.config/charon/admin.token
Authentication
Apps on this machine can call /v1 with no key. Otherwise, send a client key (Authorization: Bearer chk_…, made with charon keys create). A key is required for everyone when allow_anonymous_local is off or allow_remote is on.
/admin always needs the admin token, sent as Authorization: Bearer <token>. It is in ~/.config/charon/admin.token.
Errors look like OpenAI's: {"error": {"type", "message", "code"}}. When no backend can take a request, reasons says why for each one.
Status
Identifies the daemon (the app and CLI use it to find charond); no token needed
Responses
- 200Name and versionobject
Server-sent events
event: names and data: payloads:
request.routed— RoutedEventrequest.done— RequestRecordhardware— Hardware (about once per second)backend.health— BackendHealthEvent (every probe, and once per backend on connect)backend.removed—{ "id": string }model.progress— ModelProgressEventconfig.reloaded—{ "what": "all" | "policy" | "settings" }
A : ping comment is sent every 15 s.
Responses
- 200text/event-streamstring
- 401Missing or wrong admin tokenError
Backends
List backends with health and models
Responses
- 200Backends in config orderobject
- 401Missing or wrong admin tokenError
Add a backend (persisted to charon.toml)
Secrets are never accepted here. Store the key in the OS keychain (service charon) and send
key_ref: "keychain:<account>", or name an environment variable with key_env.
Request body
Send a probe request and return its latency
Parameters
| Name | In | Type | Description |
|---|---|---|---|
| idRequired | path | string |
Responses
- 200Probe result (ok=false carries the error)ProbeResult
- 401Missing or wrong admin tokenError
- 404Not foundError
Start an EC2 GPU instance and register it once vLLM is healthy
Returns immediately; progress arrives as backend.health events (state pending → starting → running).
Parameters
| Name | In | Type | Description |
|---|---|---|---|
| idRequired | path | string |
Models
Local GGUF models with estimated VRAM and load state
Responses
- 200Models from models.yamlobject
- 401Missing or wrong admin tokenError
Register a GGUF file in the models directory, or download one
Either catalog_id (curated download), url + sha256 (Hugging Face download, verified),
or path (a file already inside the models directory). Downloads run in the background and
report model.progress events.
Request body
Responses
- 202Accepted (download started or model registered)LocalModel
- 400Invalid inputError
- 401Missing or wrong admin tokenError
- 409Already exists or in the wrong stateError
Start a worker for the model
Parameters
| Name | In | Type | Description |
|---|---|---|---|
| idRequired | path | string |
Responses
- 200LoadedLocalModel
- 401Missing or wrong admin tokenError
- 404Not foundError
- 409Already exists or in the wrong stateError
Stop the model's worker
Parameters
| Name | In | Type | Description |
|---|---|---|---|
| idRequired | path | string |
Responses
- 200UnloadedLocalModel
- 401Missing or wrong admin tokenError
- 404Not foundError
Policy
The active policy as YAML and as parsed JSON
Responses
- 200PolicyPolicyDocument
- 401Missing or wrong admin tokenError
Validate and replace policy.yaml
Request body
objectyaml
Responses
- 200AppliedPolicyDocument
- 401Missing or wrong admin tokenError
- 422Validation errors, one per lineValidationError
Keys
Client API keys (secrets are never returned)
Responses
- 200Keysobject
- 401Missing or wrong admin tokenError
Requests
Request log, newest first
Parameters
| Name | In | Type | Description |
|---|---|---|---|
| limit | query | integer | |
| before | query | string | Request id cursor |
Responses
- 200Page of requestsobject
- 401Missing or wrong admin tokenError
One request with its full decision trace (score table and attempts)
Parameters
| Name | In | Type | Description |
|---|---|---|---|
| idRequired | path | string |
Responses
- 200RequestRequestDetail
- 401Missing or wrong admin tokenError
- 404Not foundError
Settings
Re-read charon.toml and policy.yaml (the running config is kept on errors)
Responses
- 200Reloadedobject
- 401Missing or wrong admin tokenError
- 422Invalid config; nothing changed. One entry per problem, with its position.ValidationError
Change settings; persisted to charon.toml
Request body
AWS
OpenAI-compatible
Drop-in OpenAI API. Authenticated with a Charon client key or anonymously on loopback.
OpenAI Chat Completions (stream and non-stream), routed by policy
Parameters
| Name | In | Type | Description |
|---|---|---|---|
| X-Charon-Policy | header | "local-only" | `local-only` forbids cloud backends for this request |
Request body
object
Responses
- 200OpenAI chat completion, or an SSE stream of chunks
- 400Invalid inputError
- 401A client key was sent but is unknown or revoked, or anonymous access is disabledError
- 402`budget_exceeded` (402), `policy_denied` (403) or `no_backend` (503), with per-candidate reasonsError
- 403`budget_exceeded` (402), `policy_denied` (403) or `no_backend` (503), with per-candidate reasonsError
- 429The client key's concurrency limit (`[server] per_client_concurrency`), the daemon-wide limit (`[server] max_concurrent_requests`, with Retry-After: 1), too many failed authentication attempts from a remote peer, or the provider's rate limit when no other candidate could serve the requestError
- 502Upstream failureError
- 503`budget_exceeded` (402), `policy_denied` (403) or `no_backend` (503), with per-candidate reasonsError
OpenAI Embeddings, routed like chat
Request body
object
Responses
- 200OpenAI embeddings response
- 400Invalid inputError
- 401A client key was sent but is unknown or revoked, or anonymous access is disabledError
- 402`budget_exceeded` (402), `policy_denied` (403) or `no_backend` (503), with per-candidate reasonsError
- 403`budget_exceeded` (402), `policy_denied` (403) or `no_backend` (503), with per-candidate reasonsError
- 429The client key's concurrency limit (`[server] per_client_concurrency`), the daemon-wide limit (`[server] max_concurrent_requests`, with Retry-After: 1), too many failed authentication attempts from a remote peer, or the provider's rate limit when no other candidate could serve the requestError
- 502Upstream failureError
- 503`budget_exceeded` (402), `policy_denied` (403) or `no_backend` (503), with per-candidate reasonsError
Aliases and backend models (`alias` or `backend/model`)
Responses
- 200OpenAI model list
- 401A client key was sent but is unknown or revoked, or anonymous access is disabledError
OpenAI Responses API, translated to chat completions
Request body
object
Responses
- 200OpenAI response object, or an SSE stream of response events
- 400Invalid inputError
- 401A client key was sent but is unknown or revoked, or anonymous access is disabledError
- 402`budget_exceeded` (402), `policy_denied` (403) or `no_backend` (503), with per-candidate reasonsError
- 403`budget_exceeded` (402), `policy_denied` (403) or `no_backend` (503), with per-candidate reasonsError
- 429The client key's concurrency limit (`[server] per_client_concurrency`), the daemon-wide limit (`[server] max_concurrent_requests`, with Retry-After: 1), too many failed authentication attempts from a remote peer, or the provider's rate limit when no other candidate could serve the requestError
- 502Upstream failureError
- 503`budget_exceeded` (402), `policy_denied` (403) or `no_backend` (503), with per-candidate reasonsError
Schemas
Every object the API sends or accepts. Links in the reference above jump here.
Error1 field
| Field | Type | Description |
|---|---|---|
| errorRequired | object |
ValidationError1 field
| Field | Type | Description |
|---|---|---|
| errorRequired | object |
Locality
"local" | "cloud"
Price2 fields
| Field | Type | Description |
|---|---|---|
| input_per_mtok | number | |
| output_per_mtok | number |
BackendConfig28 fields
Which keys apply depends on `kind`.
| Field | Type | Description |
|---|---|---|
| idRequired | string | |
| kindRequired | "local" | "openai_compat" | "bedrock" | "ec2_vllm" | "azure_openai" | "vertex" | "mock" | |
| base_url | string | openai_compat, ec2_vllm: …/v1. azure_openai: the resource endpoint, https://<resource>.openai.azure.com |
| local | boolean | openai_compat, mock: runs on this machine |
| key_env | string | |
| key_ref | string | |
| models | string[] | Static model list (skips discovery). Required for azure_openai (deployment names) and vertex. |
| prices | map<string, Price> | |
| context_length | integer | |
| timeout_s | integer | |
| region | string | bedrock, ec2_vllm: AWS region (default us-east-1). vertex: location (default global) |
| project | string | vertex: Google Cloud project id |
| auth | "auto" | "api_key" | "sigv4" | |
| aws_profile | string | |
| endpoint | string | bedrock, ec2_vllm, vertex: API host override (VPC or Private Service Connect) |
| instance_id | string | |
| port | integer | |
| scheme | "http" | "https" | ec2_vllm without a published TLS pin. Plain HTTP is only used to loopback. |
| tunnel | "ssm" | "none" | ec2_vllm (default ssm). With none, Charon connects to the instance's address over HTTPS. |
| local_port | integer | |
| hourly_usd | number | |
| idle_stop_minutes | number | |
| stop_on_exit | boolean | |
| slots | integer | |
| latency_ms | integer | |
| tokens_per_sec | number | |
| fail_rate | number | |
| reply | string |
BackendModel5 fields
| Field | Type | Description |
|---|---|---|
| idRequired | string | |
| context_length | integer | |
| embedding | boolean | |
| loaded | boolean | |
| details | object |
Backend15 fields
| Field | Type | Description |
|---|---|---|
| idRequired | string | |
| kindRequired | string | |
| localityRequired | Locality | |
| healthyRequired | boolean | |
| stateRequired | string | ec2_vllm: the instance lifecycle (unknown | stopped | pending | starting | running | stopping | error). local: a summary such as "1 of 3 models loaded". Other kinds leave it empty and rely on `healthy`. |
| p50_ms | number | |
| error_rate | number | |
| consecutive_failures | integer | |
| last_error | string | |
| last_probe | string | null | |
| credential | object | null | |
| aws_credential | object | null | bedrock and ec2_vllm — where the AWS credentials for SigV4 come from |
| modelsRequired | BackendModel[] | |
| configRequired | BackendConfig | |
| instance | Ec2Instance |
ProbeResult4 fields
| Field | Type | Description |
|---|---|---|
| okRequired | boolean | |
| latency_ms | number | |
| models | string[] | |
| error | string |
Ec2Instance15 fields
| Field | Type | Description |
|---|---|---|
| instance_idRequired | string | |
| name | string | |
| stateRequired | string | |
| instance_type | string | |
| public_dns | string | |
| public_ip | string | |
| launch_time | string | |
| tagged | boolean | Has the charon=true tag |
| tls_pin | string | The charon:tls-pin tag ("sha256//…" of vLLM's self-signed public key), published by instances from the template; Charon then uses HTTPS and trusts exactly that key. |
| hourly_usd | number | |
| running_seconds | number | Time running since Charon first saw it running |
| ec2_state | string | Raw EC2 state (the lifecycle `state` adds starting and error) |
| idle_stop_minutes | number | |
| tunnel | "none" | "ssm" | |
| error | string |
Gpu10 fields
| Field | Type | Description |
|---|---|---|
| indexRequired | integer | |
| nameRequired | string | |
| vendorRequired | "nvidia" | "amd" | "apple" | "other" | |
| unified_memory | boolean | |
| vram_total_bytes | number | |
| vram_used_bytes | number | |
| vram_free_bytes | number | |
| utilization_pct | number | null | |
| temperature_c | number | null | |
| power_w | number | null |
Hardware11 fields
| Field | Type | Description |
|---|---|---|
| tsRequired | string | |
| gpusRequired | Gpu[] | |
| cpu_pctRequired | number | |
| cpu_cores | integer | |
| ram_total_bytesRequired | number | |
| ram_free_bytesRequired | number | |
| on_batteryRequired | boolean | |
| battery_pct | number | null | |
| ac_online | boolean | null | |
| temperature_c | number | null | |
| thermal_state | "nominal" | "fair" | "serious" | "critical" | null |
LocalModel20 fields
| Field | Type | Description |
|---|---|---|
| idRequired | string | |
| pathRequired | string | |
| file_size_bytes | number | |
| context_length | integer | |
| parallel_slots | integer | |
| quantization | string | |
| embedding | boolean | |
| stateRequired | "on_disk" | "downloading" | "loading" | "loaded" | "restarting" | "failed" | "missing" | |
| estimated_vram_bytes | number | |
| measured_vram_bytes | number | null | |
| n_gpu_layers | integer | null | |
| fits | boolean | |
| fit_note | string | |
| slots_busy | integer | |
| tokens_per_sec | number | null | |
| error | string | |
| progress | number | null | Download progress 0-1 |
| bytes | number | Bytes downloaded so far |
| total_bytes | number | |
| bytes_per_sec | number |
CatalogModel11 fields
| Field | Type | Description |
|---|---|---|
| idRequired | string | |
| nameRequired | string | |
| description | string | |
| size_bytesRequired | number | |
| urlRequired | string | |
| sha256 | string | |
| context_length | integer | |
| min_vram_bytes | number | |
| embedding | boolean | |
| installed | boolean | |
| recommended | boolean |
AddModel8 fields
| Field | Type | Description |
|---|---|---|
| id | string | |
| catalog_id | string | |
| url | string | https://huggingface.co/…/resolve/…/file.gguf |
| sha256 | string | |
| path | string | File name inside the models directory |
| context_length | integer | |
| parallel_slots | integer | |
| embedding | boolean |
PolicyDocument4 fields
| Field | Type | Description |
|---|---|---|
| yamlRequired | string | |
| path | string | |
| warnings | string[] | |
| policyRequired | object |
Weights4 fields
| Field | Type | Description |
|---|---|---|
| latency | number | |
| cost | number | |
| preference | number | |
| load | number |
ClientKey6 fields
| Field | Type | Description |
|---|---|---|
| idRequired | string | |
| nameRequired | string | |
| prefixRequired | string | |
| tagsRequired | string[] | |
| created_atRequired | string | |
| last_used_at | string | null |
RequestRecord18 fields
| Field | Type | Description |
|---|---|---|
| idRequired | string | |
| tsRequired | string | |
| endpoint | "chat" | "embeddings" | "responses" | |
| modelRequired | string | |
| backend | string | |
| backend_model | string | |
| locality | string | |
| reason | string | |
| prompt_tokens | integer | |
| completion_tokens | integer | |
| ttft_ms | number | |
| total_ms | number | |
| est_cost_usd | number | |
| statusRequired | "ok" | "error" | "cancelled" | |
| http_status | integer | |
| error | string | |
| stream | boolean | |
| client | string |
Candidate15 fields
| Field | Type | Description |
|---|---|---|
| backendRequired | string | |
| modelRequired | string | |
| locality | string | |
| position | integer | |
| survivedRequired | boolean | |
| filter | string | unknown_backend | paused | local_only | unhealthy | rate_limited | model | context | vram | saturated | budget | battery |
| reason | string | |
| expected_latency_ms | number | |
| latency_norm | number | |
| cost_usd | number | |
| cost_norm | number | |
| preference | number | |
| load | number | |
| score | number | |
| rank | integer | null |
RequestDetail
RequestRecord & object
Spend5 fields
| Field | Type | Description |
|---|---|---|
| periodRequired | "day" | "month" | |
| total_usdRequired | number | |
| budget_usd | number | null | |
| estimatedRequired | boolean | |
| by_backendRequired | object[] |
Settings9 fields
| Field | Type | Description |
|---|---|---|
| host | string | |
| port | integer | |
| allow_anonymous_local | boolean | |
| log_level | "trace" | "debug" | "info" | "warn" | "error" | "critical" | "off" | |
| log_bodies | boolean | |
| pause_cloud | boolean | |
| force_local_only | boolean | |
| idle_unload_minutes | integer | |
| vram_reserve_mb | integer |
SettingsPatch8 fields
| Field | Type | Description |
|---|---|---|
| port | integer | |
| allow_anonymous_local | boolean | |
| log_level | "trace" | "debug" | "info" | "warn" | "error" | "critical" | "off" | |
| log_bodies | boolean | |
| pause_cloud | boolean | |
| force_local_only | boolean | |
| idle_unload_minutes | integer | |
| vram_reserve_mb | integer |
Status16 fields
| Field | Type | Description |
|---|---|---|
| versionRequired | string | |
| git_sha | string | |
| pid | integer | |
| started_at | string | |
| uptime_sRequired | integer | |
| listenRequired | string | |
| config_dir | string | |
| data_dir | string | |
| draining | boolean | |
| routingRequired | object | |
| backendsRequired | Backend[] | |
| models | LocalModel[] | |
| hardware | Hardware | |
| requests | object | |
| spendRequired | object | |
| suggestions | object[] |
RoutedEvent10 fields
| Field | Type | Description |
|---|---|---|
| id | string | |
| ts | string | |
| model | string | |
| alias | string | |
| backend | string | |
| backend_model | string | |
| locality | string | |
| reason | string | |
| stream | boolean | |
| attempt | integer |
BackendHealthEvent8 fields
| Field | Type | Description |
|---|---|---|
| id | string | |
| kind | string | |
| locality | string | |
| healthy | boolean | |
| state | string | |
| p50_ms | number | |
| error_rate | number | |
| last_error | string |
ModelProgressEvent7 fields
| Field | Type | Description |
|---|---|---|
| id | string | |
| state | string | |
| progress | number | null | |
| bytes | number | |
| total_bytes | number | |
| bytes_per_sec | number | |
| error | string |