Customer Support RAG Cost
A typical customer support RAG bot retrieves top-4 chunks per query over a 1M-token corpus (tune below). Here's what it really costs, and whether long-context would beat it.
For engineers pricing a RAG pipeline — shows that generation, not embeddings, dominates the bill, and prices RAG against stuffing the whole corpus into a cached long context.
Model prices from OpenRouter · updated 2026-07-13
01 Your pipeline
02 Generation vs embedding
Embeddings are <1% of this bill — generation dominates; optimize chunk count and model choice, not embedding price.
$1,518 generation + $0.48 embedding = $1,518/mo RAG total
03 RAG vs long-context
RAG wins by $329,258/mo (100%)
Related cost calculators
Generation prices from OpenRouter, snapshot 2026-07-13, synced daily. Embedding prices hand-verified (checked 2026-07-13), re-audited quarterly. RAG monthly = generation(retrieved+question→output, uncached) + queries×embedding. Long-context monthly = generation(corpus+question→output, cached at the given share). All math runs in your browser.
How the math works
A RAG bill has two parts: embedding the query and generating the answer over retrieved chunks. At this page's defaults — top-4 chunks of 300 tokens each (1K retrieved context) — embedding costs $0.48/month against generation's $1,518: embeddings are under 1% of the bill. Same baseline, one identity: generation $1,518 + embedding $0.48 = $1,518/month total.
Optimizing embedding price is optimizing the wrong line item — retrieved-chunk tokens riding into Claude Sonnet 5 at $10/M output and $2/M input drive the bill. Fewer, better-targeted chunks (a smaller top-K, tighter chunking) cuts more cost than switching embedding providers ever will.
The other question this page answers: does RAG even save money over stuffing the whole 1.0M-token corpus into every query as a cached prefix? At these defaults, RAG wins — $329,258/month cheaper (100%) than long-context. This is a token-cost comparison only — retrieval quality and recall are separate, real reasons to use RAG regardless.
Prices sync daily from OpenRouter for the generation model; the embedding price is hand-verified against provider docs (embedding models aren't in the daily catalog). All math runs in your browser.
Frequently asked questions
How much does a customer support RAG bot cost per month?
At this page's defaults — 300,000 queries/month, top-4 retrieval — Claude Sonnet 5 generation costs $1,518 and text-embedding-3-small query embedding costs $0.48: $1,518/month total. Tune every input above for your real pipeline.
Do embeddings actually matter for RAG cost?
Rarely much. At this workload embeddings are under 1% of the bill ($0.48 vs generation's $1,518). Query embedding is a few dozen tokens; generation processes thousands of retrieved tokens per call at rates 100–1000× the embedding price. Shop for embedding quality, not embedding price.
Is RAG cheaper than just using a long context window?
At this page's defaults, yes: RAG costs $1,518/month vs $330,776 for stuffing the full 1.0M-token corpus into every query (even with 50% caching) — $329,258/month saved (100%). The gap widens as the corpus grows relative to what you retrieve.
What retrieval size should I use?
This page defaults to top-4 chunks of ~300 tokens (1K total retrieved context). More chunks improve recall but scale generation cost linearly — the ranking of cost impact is retrieval size first, model choice second, embedding model a distant third.
What formula does this calculator use?
RAG monthly = generation(retrievedTokens + question → output, uncached) + queries × queryTokens × embedPrice. Long-context monthly = generation(corpusTokens + question → output, cached at the given share). Savings = longContext − RAG. All generation costs use the same engine as the LLM API Cost Calculator.
Are these prices current?
Generation prices sync daily from OpenRouter. Embedding prices are hand-verified against provider docs and re-audited quarterly (they aren't in the daily-synced catalog). All math runs client-side with tested code.