What Is RAG? Retrieval-Augmented Generation Explained

What RAG is

A language model knows what was in its training data, up to its cutoff, and nothing about your internal docs, last week's release notes or a customer's contract. RAG closes that gap without retraining: at question time, look up the passages that are likely to contain the answer and put them in the prompt.

It has two advantages over fine-tuning for knowledge. The corpus can change every hour and the next answer reflects it, and every answer can cite the passages it was built from, which makes it checkable.

A RAG system cannot give a better answer than the passages it retrieved. Most RAG quality work is retrieval work.

How a RAG pipeline works

It splits into an ingest side that runs ahead of time and a query side that runs per question.

Ingest

Fetch pages and documents: crawl a site, pull files, read a database.
Parse them into clean text, ideally markdown so headings survive.
Chunk each document into self-contained passages.
Embed each chunk into a vector and index it, usually alongside a keyword index.

Query

Search the index with the user's question, ideally with hybrid search.
Take the top few chunks, with their source URLs.
Prompt the model with the question and those chunks, and instruct it to answer from them and cite them.

res = client.embed_search_kb(kb_id, q=question, size=5)

if res.low_confidence:
    answer = "I could not find that in the docs."
else:
    context = "\n\n".join(f"[{h.title}]({h.source_url})\n{h.text}" for h in res.hits)
    answer = llm(f"Answer only from these sources.\n\n{context}\n\nQ: {question}")

The two halves have different costs. Ingest is slow and asynchronous, because crawling and parsing are, and it runs once per document plus again whenever that document changes. Query runs on every question, so it has to be fast: one search call, then one model call.

The model never sees the whole corpus, only the handful of chunks retrieval chose. Every upstream stage therefore shows up directly in the answer. A wrong RAG answer is more often a retrieval miss than a model mistake, and the first debugging step is to read the chunks that were actually retrieved.

Where RAG goes wrong

Bad chunks. An answer split across two chunks, or a table cut in half, cannot be retrieved whole. See document chunking.
Vector-only retrieval. Embeddings miss literals like error codes and version numbers. A keyword leg fixes it.
No way to say no. Nearest-neighbour search always returns neighbours. Without a confidence signal the model answers off-topic questions from irrelevant chunks.
Junk in the index. Crawled error pages and boilerplate are short, generic and close to every query in vector space.
Stale corpus. A knowledge base built once answers confidently about last year's product. Re-crawl incrementally.
One source, five times. Several chunks from the same page look like corroboration and are not.

How SarvCrawl handles it

SarvCrawl runs the ingest half for you: crawl a site or upload files (PDF, Office, EPUB, images, audio), and the job parses to markdown, chunks on headings, embeds and indexes before it completes. The query half is one call to the search API, which fuses keyword and semantic results with reciprocal rank fusion, caps chunks per page, and sets low_confidence when nothing matches on keywords and the best cosine similarity is below 0.25.

The same retrieval is exposed through the Node and Python SDK sarvcrawl, the sarv-kb CLI, an n8n node and 21 MCP tools. Search is on paid plans. The end-to-end walkthrough is web scraping for AI in 2026, and a docs-specific version is build a RAG knowledge base from a documentation site.

Frequently asked questions

Is RAG better than fine-tuning?

For adding knowledge, usually yes. RAG reflects corpus changes immediately and can cite sources. Fine-tuning is better suited to changing a model's style, format or behaviour than to teaching it facts that change.

Do I need a vector database for RAG?

You need a vector index, which engines such as Elasticsearch, OpenSearch and Postgres with pgvector provide alongside a keyword index. Keeping both in one engine makes hybrid search a single round trip.

How many chunks should I give the model?

A handful, typically three to ten, from more than one source page. More context is not free: it costs tokens and dilutes the relevant passage.

Why does my RAG system answer questions it should refuse?

Because vector search always returns something. Add a confidence check, such as requiring a keyword match or a minimum similarity, and tell the model to decline when it fails.

What is retrieval-augmented generation (RAG)?