Companion Chat App Conversation Cost
A companion app with very long conversations runs ~120-turn conversations (tune below). Here's what the re-sent transcript really costs — and how much caching claws back.
For teams running chatbots — computes how a conversation's cost grows quadratically as the transcript is re-sent each turn, and how much prompt caching claws back, not a 'turns × one message' estimate.
Model prices from OpenRouter · updated 2026-07-13
01 Your conversation
$2/M in · $10/M out · cache reads $0.2/M
02 Naive estimate vs real cost
A 120-turn chat costs 1.8× the naive “turns × one message” estimate — the transcript replay adds up.
Real conversation cost is 1.8× the naive estimate
Without caching this would be $2,491,200 (12.1×) — caching the transcript is your biggest lever on long chats.
Related cost calculators
Prices from OpenRouter, snapshot 2026-07-13, synced daily. Turn k re-sends system + prior turns + new user message; total input = T×(system+user) + (user+assistant)×T(T−1)/2. With caching, the re-read prefix bills at the cache-read rate. The first-turn cache-write premium and any summarization/truncation aren't modeled. All math runs in your browser.
How the math works
It's tempting to price a 120-turn conversation as 120 × one message. But every turn re-sends the whole transcript so far — turn 120 ships the system prompt plus all 119 previous turns before the model reads a single new word. So the input tokens grow with the SQUARE of the conversation length, not linearly.
Same baseline, one identity: the naive "one message × turns" estimate is $0.41 per conversation. Adding the re-sent history (2,284,800 tokens of replayed prior turns) brings the real, uncached cost to $4.98 — 12.1× the naive number. That gap is pure transcript replay, and it widens every turn.
Prompt caching is the fix, because the transcript is a stable growing prefix — exactly what caching is built for. With caching on, the replayed history re-reads at $0.2/M instead of $2/M, cutting the real cost to $0.74/conversation (1.8× naive) — $370,620/month at 500,000 conversations. Turn caching off and the same chats cost $2,491,200/month.
The levers all target the history term: a sliding window (keep only the last N turns), summarizing old turns into a short recap, and caching the transcript. Because the cost is quadratic, trimming the oldest turns of a long chat saves far more than trimming the same tokens from a short one — the tail of a long conversation is where the money is.
Not modeled: the one-time cache-write premium on the first turn (small), and any summarization/truncation you apply — this assumes the full transcript is re-sent. Inference prices sync daily from OpenRouter (updated 2026-07-13); this is a token-accounting comparison on the live catalog, not a separate price source. All math runs client-side with tested code.
Frequently asked questions
Why does a long conversation with a companion app with very long conversations cost more than turns × one message?
Because every turn re-sends the entire transcript. A 120-turn chat replays 2,284,800 tokens of prior turns on top of the new messages, so the uncached cost is 12.1× the naive estimate ($4.98 vs $0.41). The longer the chat, the wider the gap.
How much does prompt caching save on a chatbot?
A lot for long chats — the transcript is a stable prefix, so caching re-reads it at $0.2/M instead of $2/M. Here it cuts the bill from $2,491,200/month (uncached) to $370,620/month. Caching is close to mandatory once conversations run long.
Does the cost really grow quadratically?
Yes — turn k re-sends roughly k turns of history, so summing over a conversation gives a term proportional to turns². Doubling the conversation length nearly quadruples the history-replay tokens. That's why very long sessions get expensive fast, and why capping history matters.
What's the cheapest way to cut chatbot cost?
Cap the history: a sliding window that keeps only the last N turns turns the quadratic back into a linear cost. Summarizing old turns into a short recap does the same while preserving context. And enable prompt caching so whatever history you do keep re-reads cheaply. Trimming the oldest turns of long chats moves the number most.
Are these prices current?
Inference prices sync daily from OpenRouter (updated 2026-07-13). This mold adds no separate price source — it's a token-accounting model of transcript replay on top of the live catalog, so it stays accurate as prices change automatically.
Should a companion app with very long conversations worry about this?
Yes — at 120 turns the uncached cost is 12.1× the naive estimate, so history replay dominates. Caching and a history cap are the difference between $370,620 and $2,491,200/month here.