On this page
An AI platform pays a lot of vendors per unit of work: a model router bills per token, a voice platform bills per call minute, a telephony carrier bills per minute and per number, storage and email bill per whatever. Your customers, meanwhile, pay you in whatever unit you invented — credits, seats, minutes. Between those two ledgers is your margin, and if you can’t see both ledgers side by side, you don’t actually know what your margin is. You know what you designed it to be.
On an AI-agent platform I work on as fractional CTO, we built provider cost tracking into the admin panel — pulling spend from the LLM router, the voice platform, and the telephony carrier into one view, lined up against what customers were billed for the same activity. The point of this post is what that instrument immediately found, and the mechanics of fixing it.
The leak
Once provider spend sat next to billed usage, one line refused to reconcile: per-minute telephony costs were being paid on every call and charged to no one.
The platform billed customers for calls through the voice layer — that path worked. But underneath, the telephony carrier billed its own per-minute rate for the same calls, and that cost never entered the customer-facing billing pipeline at all. Every call quietly cost more than the billing system believed. Not dramatically more per call; steadily more, on every call, for as long as the platform had existed.
Nobody notices this kind of leak from the inside, because nothing is broken. Calls work. Invoices go out. The only symptom is that the provider bill grows slightly faster than revenue, which is indistinguishable from a dozen benign explanations — until the two ledgers are on one screen.
The fix shape: bill where the cost is born
The repair wasn’t “add a surcharge.” It was structural: charge at the moment the provider tells you the cost exists. The telephony carrier sends a webhook when a call completes, carrying its cost for that call. The voice platform and the carrier share a linkage — the carrier’s call SID is attached to the voice platform’s call record — so the webhook’s cost can be attributed to the right customer call, margin applied, and the charge written into the same billing pipeline as everything else.
The general principle: every provider that bills you per unit should have its cost captured at its own reporting boundary (usually a webhook or usage API), keyed by whatever ID links its records to yours. If a cost can’t be attributed to a customer action, that’s not a billing detail to defer — it’s a leak you haven’t found yet.
What review caught before production did
Two classes of bug showed up in review of the fix, and both generalize to any usage-billing pipeline:
No atomic billing claim. The first version could, under retried webhooks or concurrent processing, charge the same call twice — nothing claimed the cost record atomically before billing it. Usage billing needs the same discipline as payment processing: a single-writer claim (an atomic update that marks the record as being billed, succeeding for exactly one worker) before any charge is created. Webhooks are at-least-once; your billing must be exactly-once.
Reconciliation boundary mismatches. The reconciliation job compared windows of provider costs against windows of charges — but one side was bucketed by when the record was created and the other by when it was queued for billing. Records landing near a window edge fell into different buckets on each side, producing phantom discrepancies. Reconciliation windows must be defined on the same timestamp field on both sides, and the field should be the one that can’t be revised later.
Instrument first, optimize second
The order of operations matters. The temptation is to start with pricing changes — raise the unit price, trim the free allowance — because those are visible levers. But without the cost instrument, you’re adjusting margin you can’t measure, and the leaks stay leaks at any price point.
What the instrument gives you, concretely:
- Per-provider spend over time, from each provider’s own usage reporting — not from your assumptions about it.
- The same activity priced from both sides: what the call/message/generation cost you, next to what it earned you.
- A reconciliation job that flags activity with cost but no charge (leaks) and charges with no matching cost (billing bugs in the other direction — also real).
- Alerting on the ratio, not just the totals, so a new leak shows up as a trend break instead of a quarterly surprise.
The thesis
Unit economics is usually treated as a spreadsheet exercise you do for investors. It’s more useful treated as observability: live telemetry about whether the machine converts vendor costs into revenue the way you designed it to. Margins fail like software fails — silently, at the seams between systems, on the paths nobody instrumented. The fix is the same as for any other reliability problem: put a sensor on it, reconcile continuously, and treat every unexplained gap as a bug until proven otherwise.