01Hardware
Knows your hardware
Charon watches your memory, load and battery, so it only runs what your computer can handle.
Charon runs your AI requests on your own computer, and only reaches for the cloud when it has to. Private by default, cheaper by design, and it always tells you why.
Free for Mac. No account needed.
OpenAI Python
base_url="…:7777/v1"OpenAI Node
baseURL: "…:7777/v1"LangChain
ChatOpenAI(base_url=…)LlamaIndex
OpenAILike(api_base=…)Responses API
POST /v1/responsescurl
curl …:7777/v1/…Works with your code
If your app talks to OpenAI, it already works with Charon. Point it at 127.0.0.1:7777/v1 and Charon picks where each request runs.
from openai import OpenAI
client = OpenAI(
base_url="https://api.openai.com/v1",
base_url="http://127.0.0.1:7777/v1",
api_key="charon",
)
reply = client.chat.completions.with_raw_response.create(
model="auto",
messages=[{"role": "user", "content": "Summarize…"}],
)
print(reply.headers["x-charon-backend"]) # local
from openai import OpenAI
client = OpenAI(
base_url="http://127.0.0.1:7777/v1",
api_key="charon",
)
reply = client.chat.completions.with_raw_response.create(
model="auto",
messages=[{"role": "user", "content": "Summarize…"}],
)
print(reply.headers["x-charon-backend"]) # local
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://api.openai.com/v1",
baseURL: "http://127.0.0.1:7777/v1",
apiKey: "charon",
});
const { data, response } = await client.chat.completions
.create({ model: "auto", messages: [{ role: "user", content: "Summarize…" }] })
.withResponse();
console.log(response.headers.get("x-charon-backend")); // local
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "http://127.0.0.1:7777/v1",
apiKey: "charon",
});
const { data, response } = await client.chat.completions
.create({ model: "auto", messages: [{ role: "user", content: "Summarize…" }] })
.withResponse();
console.log(response.headers.get("x-charon-backend")); // local
curl https://api.openai.com/v1/chat/completions \
curl http://127.0.0.1:7777/v1/chat/completions \
-H "Content-Type: application/json" \
-H "X-Charon-Policy: local-only" \
-d '{"model": "auto",
"messages": [{"role": "user", "content": "Summarize…"}]}' \
--include
curl http://127.0.0.1:7777/v1/chat/completions \
-H "Content-Type: application/json" \
-H "X-Charon-Policy: local-only" \
-d '{"model": "auto",
"messages": [{"role": "user", "content": "Summarize…"}]}' \
--include
How it works
Charon first rules out anywhere a request can't or shouldn't go: over budget, too big, or private. Then it picks the fastest, cheapest option that's left, and tells you why.
Step 01Filter
Too big, over budget, offline or private? Those options are out before anything else happens.
Step 02Score
Charon scores what's left on speed, cost and your preferences. By default, fast and free wins.
Step 03Fall back
If the first choice stalls before answering, the runner-up takes over. Your app never notices.
No black box
Each response notes where it ran and the reason, so nothing is a mystery. Try three examples.
Routing decision
model auto1,240 tokens
POST /v1/chat/completions
local/qwen2.5-7b-instructChosen
bedrock/amazon.nova-pro-v1:0
ec2-gpu/llama-3.1-70b
Filtered by unhealthy: Instance stopped
x-charon-backend: local
x-charon-reason:local: lowest latency (est 182 ms), cost 0
model auto96,210 tokens
POST /v1/chat/completions
local/qwen2.5-7b-instruct
Filtered by context: 96,210 + 1,024 > 32,768
bedrock/amazon.nova-pro-v1:0Chosen
azure-prod/gpt-4.1-mini
x-charon-backend: bedrock
x-charon-reason:bedrock: lowest latency (est 720 ms), cost $0.0781
model auto3,902 tokens
POST /v1/chat/completions
X-Charon-Policy: local-only
local/llama-3.1-8b-instructChosen
bedrock/amazon.nova-lite-v1:0
Filtered by local_only: Runs in the cloud
vertex/gemini-2.5-flash
Filtered by local_only: Runs in the cloud
x-charon-backend: local
x-charon-reason:local: only eligible candidate, est 410 ms, cost 0
Desktop app
Watch requests as they happen: where each one ran, what it cost, and why.
Features
01Hardware
Charon watches your memory, load and battery, so it only runs what your computer can handle.
02Privacy
Mark a request or an app as local-only and it never leaves your computer. Charon checks twice.
03Budgets
Set a daily or monthly cloud limit. Once you hit it, requests stay on your computer.
04Explainable
Each response says where it ran and why, and the app shows how every option scored.
05Backends
Bring Amazon Bedrock, your own cloud GPU, Azure OpenAI, Vertex AI or any OpenAI-compatible service.
06Failover
If a model stalls, Charon quietly hands the request to the next best option in under a second.
Benchmarks
Measured on an Apple M4 Pro. Charon adds less than a millisecond.
A coding agent's bursts of parallel tool calls, planning turns and files too long for a small local context, replayed under three policies. Local is Qwen2.5-0.5B on Metal; the cloud is simulated at Amazon Nova Pro prices and latency, so the costs are projected.
Still curious? The docs answer the rest in plain English.
Yes. Download the Mac app and use it, no account or sign-up. If you add a cloud provider, you pay that provider for what you use.
Only when Charon sends a request to a cloud you set up yourself, and never for requests you mark private. Charon has no telemetry and no account.
No. Any Mac with Apple Silicon (M1 or newer) works. Bigger requests can go to the cloud, and more stays local on faster machines.
Anything that talks to the OpenAI API: the official SDKs, LangChain, LlamaIndex, editors and agents. Change the address, keep your code.
Set a daily or monthly budget, pause the cloud from the menu bar, and let Charon stop an idle cloud GPU for you.
Coming soon. Today Charon runs on Macs with Apple Silicon.
Get started
Free, no account, and set up in about a minute.