LLM Token Cost Calculator › RAG cost

RAG Cost Calculator

By the TokenForge team · Last updated July 2026

Retrieval-augmented generation multiplies your input tokens: every request re-sends the system prompt plus thousands of tokens of retrieved context. This calculator prices that pipeline per month — broken down by component, so you can see exactly where the money goes before the first invoice does.

RAG cost calculator

Estimated monthly cost $49.50/mo $22.50 input + $27.00 output
Monthly token volume 36M tokens 30M input + 6M output across 10,000 requests

List prices for standard API calls — batch (−50%) and prompt-caching discounts can cut this further. † Anthropic's newest models include the ~30% tokenizer correction.

Prices updated July 10, 2026

The RAG cost formula

Worked example. 10,000 requests a month, each carrying a 300-token system prompt, 2,500 tokens of retrieved context and a 200-token user question, answered in 600 tokens. Input: 10,000 × 3,000 = 30M tokens; output: 10,000 × 600 = 6M tokens. On GPT-5.4 Mini that is 30 × $0.75 + 6 × $4.50 = $22.50 + $27.00 = $49.50 a month — and 83% of the input bill is retrieved context.

Where the money goes in a RAG pipeline

Two lines dominate. The first is retrieved context: those chunks are re-sent on every request, so at five 500-token chunks per query the worked example burns 25M of its 30M monthly input tokens on retrieval alone. The second is output: at $4.50 versus $0.75 per million, the 6M output tokens cost more ($27.00) than all 30M input tokens ($22.50) despite being a sixth of the volume. Everything else — embedding the query, the vector database lookup — is real infrastructure but rarely more than a few percent of the bill.

To compare providers on your RAG traffic, feed the totals into the LLM Token Cost Calculator: the example works out to 36M total tokens a month at a 5:1 input:output ratio, and the calculator prices that against every model — including the self-hosting comparison this page doesn't make.

Six ways to cut RAG costs

  1. Retrieve fewer, smaller chunks. Top-k and chunk size multiply: dropping from five 500-token chunks to four 400-token chunks cuts retrieved context from 2,500 to 1,600 tokens — a 36% reduction on the dominant input line.
  2. Rerank before you stuff. Fetch 20 candidates with cheap vector search, rerank, and send only the best 3–4 to the LLM. A reranking pass costs far less than the context tokens it saves.
  3. Cache the static prefix. The system prompt and tool definitions are identical on every request and cache at ~90% off; per-query retrieved chunks usually differ per request and don't. The prompt caching calculator puts a number on it.
  4. Route by difficulty. Grounded factual answers rarely need a frontier model. The worked example costs $49.50 on GPT-5.4 Mini and about $330 on GPT-5.6 Sol — route only the hard queries up.
  5. Cap output length. Trimming answers from 600 to 300 tokens saves $13.50 a month in the example — 27% of the total bill — and grounded answers usually read better shorter anyway.
  6. Measure real tokens per request. Conversation history, retries and prompt growth quietly inflate averages. Log actual input/output counts and reconcile monthly — the usage estimation guide shows what to watch.

Frequently asked questions

Related tools and guides