Is RAG the same as training a model on my data?

No, and the difference matters commercially. RAG shows the model your documents at question time and leaves the model untouched. Training changes the model itself. RAG is cheaper, updates the moment you edit a document, and can point at the source it used.

Does my data leave my systems with RAG?

Only the retrieved passages go to the model provider with each question, and you choose the provider and region. The document store itself can sit in your own infrastructure in the EU. For clients with strict rules we run both the store and the model inside their environment.

Why does a RAG chatbot still get things wrong?

Almost always because retrieval returned the wrong passage, not because the model was weak. Fixing it means better chunking, better metadata and a test set of real questions, which is where most of the build effort goes.

How many documents can RAG handle?

The document count is rarely the limit; the quality of the search over them is. We have run this over a few dozen policy pages and over tens of thousands of records. The second one needs a proper vector store and evaluation, not a bigger model.

What does RAG cost to run?

The running cost is the retrieved passages plus the question and answer, billed per token by whichever model you use, plus a small hosting cost for the document store. Long retrieved contexts are the expensive part, so tighter retrieval is also cheaper retrieval.

What Is RAG? Retrieval-Augmented Generation

What is RAG (retrieval-augmented generation)?

RAG means looking up relevant passages from your own documents and putting them in front of the model together with the question, so the answer comes from your material rather than the model's memory. Nothing is retrained. The documents stay yours, and updating them updates the answers immediately.

Reviewed: Jun 7, 2026

	RAG (retrieval)	Fine-tuning
What changes	What the model is shown	The model weights
Updating a fact	Edit the document	Retrain and redeploy
Can cite its source	Yes, the retrieved passage	No
Handles a document added today	Immediately	Only after the next training run
Good at	Facts, policies, product data	Tone, format, a narrow task
Cost driver	Tokens per question	Training runs, then hosting
Fails by	Retrieving the wrong passage	Confidently repeating stale training data

RAG (retrieval)

Fine-tuning

What changes

What the model is shown

The model weights

Updating a fact

Edit the document

Retrain and redeploy

Can cite its source

Yes, the retrieved passage

Handles a document added today

Immediately

Only after the next training run

Good at

Facts, policies, product data

Tone, format, a narrow task

Cost driver

Tokens per question

Training runs, then hosting

Fails by

Retrieving the wrong passage

Confidently repeating stale training data

Where it shows up in practice

A support bot over your help centre

Every answer is grounded in an article you control, with a link back to it. When the refund policy changes on Tuesday, the bot is correct on Tuesday. This is the case RAG was made for.

A sales assistant over a price list

Retrieval keeps the numbers current and quotable. We still put a hard rule in front of it: the assistant may quote a price only from a retrieved row, never from memory, and says it will check when nothing matches.

An internal search across contracts

People ask in plain language and get the clause plus the document it came from. The citation matters more than the fluency here, because the person is going to open the file anyway.

Where fine-tuning still wins

You want a fixed output shape, or a house tone of voice, on a task where the facts barely move. That is a training problem, not a retrieval one. In practice we combine them rarely and only after RAG alone has been measured.

How we use it

Retrieval is unglamorous and it wins most of the time. The reason is boring: businesses change their facts faster than anyone wants to retrain a model. A price list, a delivery policy, a list of who covers which region. Put those in a document store, retrieve the right rows at question time, and the system is correct the moment someone edits the document.

The part people underestimate is that RAG quality is a search problem, not a model problem. If retrieval hands over the wrong three paragraphs, the best model in the world writes a fluent wrong answer. So most of our build time on these projects goes into chunking, metadata and evaluation: does the right passage come back for the fifty questions customers actually ask? We measure that before anyone sees the bot. The technique traces back to a 2020 paper by Lewis and colleagues and has barely changed in shape since; what changed is that the surrounding tooling got good. It pairs naturally with AI agents, which use retrieval as one of their tools. See how we deploy it in AI chatbots and Voice AI.