Docs Q&A Cost
A typical documentation Q&A system retrieves top-6 chunks per query over a 3M-token corpus (tune below). Here's what it really costs, and whether long-context would beat it.
For engineers pricing a RAG pipeline — shows that generation, not embeddings, dominates the bill, and prices RAG against stuffing the whole corpus into a cached long context.
Model prices from OpenRouter · updated 2026-07-13
01 Your pipeline
02 Generation vs embedding
Embeddings are <1% of this bill — generation dominates; optimize chunk count and model choice, not embedding price.
$510 generation + $0.10 embedding = $510/mo RAG total
03 RAG vs long-context
RAG wins by $191,696/mo (100%)
Related cost calculators
Generation prices from OpenRouter, snapshot 2026-07-13, synced daily. Embedding prices hand-verified (checked 2026-07-13), re-audited quarterly. RAG monthly = generation(retrieved+question→output, uncached) + queries×embedding. Long-context monthly = generation(corpus+question→output, cached at the given share). All math runs in your browser.
How the math works
A RAG bill has two parts: embedding the query and generating the answer over retrieved chunks. At this page's defaults — top-6 chunks of 500 tokens each (3K retrieved context) — embedding costs $0.10/month against generation's $510: embeddings are under 1% of the bill. Same baseline, one identity: generation $510 + embedding $0.10 = $510/month total.
Optimizing embedding price is optimizing the wrong line item — retrieved-chunk tokens riding into Claude Sonnet 5 at $10/M output and $2/M input drive the bill. Fewer, better-targeted chunks (a smaller top-K, tighter chunking) cuts more cost than switching embedding providers ever will.
The other question this page answers: does RAG even save money over stuffing the whole 3.0M-token corpus into every query as a cached prefix? At these defaults, RAG wins — $191,696/month cheaper (100%) than long-context. This is a token-cost comparison only — retrieval quality and recall are separate, real reasons to use RAG regardless.
Prices sync daily from OpenRouter for the generation model; the embedding price is hand-verified against provider docs (embedding models aren't in the daily catalog). All math runs in your browser.
Frequently asked questions
How much does a documentation Q&A system cost per month?
At this page's defaults — 50,000 queries/month, top-6 retrieval — Claude Sonnet 5 generation costs $510 and text-embedding-3-small query embedding costs $0.10: $510/month total. Tune every input above for your real pipeline.
Do embeddings actually matter for RAG cost?
Rarely much. At this workload embeddings are under 1% of the bill ($0.10 vs generation's $510). Query embedding is a few dozen tokens; generation processes thousands of retrieved tokens per call at rates 100–1000× the embedding price. Shop for embedding quality, not embedding price.
Is RAG cheaper than just using a long context window?
At this page's defaults, yes: RAG costs $510/month vs $192,206 for stuffing the full 3.0M-token corpus into every query (even with 40% caching) — $191,696/month saved (100%). The gap widens as the corpus grows relative to what you retrieve.
What retrieval size should I use?
This page defaults to top-6 chunks of ~500 tokens (3K total retrieved context). More chunks improve recall but scale generation cost linearly — the ranking of cost impact is retrieval size first, model choice second, embedding model a distant third.
What formula does this calculator use?
RAG monthly = generation(retrievedTokens + question → output, uncached) + queries × queryTokens × embedPrice. Long-context monthly = generation(corpusTokens + question → output, cached at the given share). Savings = longContext − RAG. All generation costs use the same engine as the LLM API Cost Calculator.
Are these prices current?
Generation prices sync daily from OpenRouter. Embedding prices are hand-verified against provider docs and re-audited quarterly (they aren't in the daily-synced catalog). All math runs client-side with tested code.