Agent Fleet Console

At some point I stopped having “an AI assistant” and started having a fleet: a conversational assistant I message from my phone, a coding agent in the terminal, another one inside the editor, and a small model running locally on the same machine. All of them touching the same repos, at the same time, with no way to talk to each other.

Agent Fleet Console is what I built so that would be a system instead of a mess: how they coordinate, how work gets dispatched on its own, and — the part with the most engineering in it — how their consumption gets measured without the report lying to me.

What it does

It coordinates agents that can’t talk to each other. There’s no message channel between a Telegram agent and a terminal one, so coordination happens where they do overlap: shared files on disk. There’s a sync point every agent reads on startup and writes to when it finishes something relevant, plus an engineering guidelines contract they all load — only the sections relevant to their task — before writing code. The file is the source of truth; none of them depends on me relaying messages between windows.

It dispatches work on its own. Cron-scheduled tasks fire off a worker with nobody watching, and when they finish they report to Telegram with the result and the path to the log — not a blind “it’s done”, but something verifiable. The pattern repeats: a long task is dispatched to a headless worker, its output is checked against the actual repo, and only then is it considered closed.

It measures what the fleet consumes. This is where llm-agent-spend-manager lives: a Go CLI that gives spend and quota visibility across the agents sharing the machine. Its most important design decision is the unit. Since everything runs on flat-rate subscriptions, the dollar was never the scarce resource. What actually runs out — and strands you mid-task — is the provider’s quota window. So it reports how much of the window is left, who’s eating it, and which lever stretches it most, and it puts the dollar equivalent below, labeled as an equivalence rather than a charge.

It also classifies the shape of each workload, because two sessions that cost the same aren’t the same problem: a long conversation that dragged a growing context across thousands of turns gets cheaper by being cut; a mechanical burst gets cheaper by being routed to a cheaper agent. Different levers, identical numbers.

Under the hood

The project has one hard rule and everything else follows from it: no number is ever invented. Everything the report claims is derived from measured data, and anything that can’t be derived is said out loud — n/a, missing data, unclassified, with its reason — instead of being filled in with an estimate that would read exactly like a measurement. There’s no trained model and no state kept between runs: same input data, same report.

That sounds nice until it costs you figures. Twice it really bit, and those are my favorite examples in the whole project:

A 45× extrapolation the system caught on itself. The first run of the per-route savings plan claimed that moving 44,373 turns to a cheaper model would have saved $6,673 — having observed that model actually carry just 981 turns of that shape. That’s arithmetic dressed up as evidence. With the cap of “you may only claim what the cheap option has already been observed carrying”, the figure dropped to $147.21 over those 981 observed turns, and the report says out loud that it capped, and why. A smaller true result beats a big suggestive one.
A 38.7% drop it refused to claim as an improvement. Cost per turn went from $0.1635 to $0.1002 and the printed verdict is still “no level change”: measured against the daily dispersion of the series itself, that step is −0.9σ — it fits inside the noise. A report headlining “we cut costs 38.7%” would be selling variance as a result.

The same rule drives the rest: an explicit confidence hierarchy across agents (those exposing real token counts are reported as a single figure; those that only let you infer activity are reported as a range, flagged and one tier below, and never allowed downstream to compete against a measured number); a mandatory caveat on every attribution that temporal coincidence isn’t causality; and an explicit list of what the system cannot know — like how much each model weighs against the quota, which the provider doesn’t publish and which is printed here as “not derivable” with the reason, instead of an invented factor wearing the face of a finding.

Nobody publishes the quota window’s ceiling either, so it’s a calibrated estimate against exhaustions observed on the machine, reported as a range with its dispersion — below three observations, the command simply prints no ceiling at all.

The stack

Go — a single binary, no npm dependencies and no mandatory third-party services. status, quota, advise and outcome commands, all with --json so another agent can consume them.
A web dashboard served by the same binary: localhost-only by default; exposing it to the local network has to be asked for explicitly, and then it requires a random 128-bit token on every route. Installable as a PWA from a phone.
Deterministic analysis: JSONL transcript parsing, cost attribution per billable bucket, a per-session context curve, and CUSUM-style level-change detection over the daily series. No ML.
Orchestration: agents on OpenClaw (Telegram) plus terminal and editor agent CLIs, coordinated through files and dispatched by cron.
Every method — quota, workload classes, outcome log — documented in its own file, so the figures can be redone with a calculator.

Worth noting

This is the umbrella project over the other three on this list, and not by accident: llm-budget-cap and chatarmor cap the spend of a user-facing product, itza-rag keeps cheap and private what can be solved locally, and this measures and orders the fleet that built them.

If I had to keep one idea from all the agent work, it would be this: the hard part isn’t getting a model to produce a number, it’s getting it to stay quiet when it doesn’t have one. A dashboard that always has a nice figure for you is a dashboard you can’t base a decision on. That’s why the parts I’m proudest of here are the ones that print “not derivable”, “missing data” or “no level change” — each one is a place where the system could have told me what I wanted to hear and chose not to.