Retrieval-Augmented Generation (RAG) Explained

A general-purpose model knows a great deal about the world and nothing at all about your contract terms, your product catalogue or last quarter's board pack. Retrieval-augmented generation is the standard way to close that gap without retraining anything — and it is the architecture behind the large majority of enterprise AI assistants in production today.

This article assumes you know roughly what a language model is; if not, start with inside large language models, which covers the engine itself. Here we go one level down into the retrieval pipeline — the part that actually decides whether an enterprise RAG system is useful or embarrassing.

Why RAG exists

Three limitations of a bare model drive nearly every RAG project:

• It does not know your data. Private documents were never in the training set, and putting them there is expensive and slow.
• It goes stale. A model's knowledge stops at its training cut-off; retrieval reads whatever the source system says today.
• It hallucinates. Asked something it does not know, a model will often produce a fluent, confident, wrong answer. Grounding the response in retrieved text — and citing it — is the most practical mitigation available, because the reader can check the source.

The RAG pipeline, stage by stage

Every RAG system, however it is packaged, does the same two-phase job. Indexing happens ahead of time; retrieval and generation happen per question:

1. Ingest — collect the source documents (wikis, PDFs, tickets, databases) and extract clean text.
2. Chunk — split that text into passages small enough to retrieve precisely and large enough to make sense alone.
3. Embed — convert each chunk into a vector, a list of numbers capturing its meaning.
4. Store — load the vectors into a vector database or index, alongside metadata such as source, date and access permissions.
5. Retrieve — at question time, embed the question and find the closest chunks by similarity, often blended with keyword search.
6. Re-rank — score the candidates more carefully and keep only the best few.
7. Generate — put those passages in the prompt with the question, and have the model answer using them, with citations.

Chunking and embeddings

Chunking sounds trivial and is not. Chunks that are too large dilute the signal and waste context; too small and they lose the surrounding meaning — a paragraph that says "this does not apply to enterprise customers" is dangerous when retrieved without the clause it modifies. Sensible practice: split on natural boundaries (sections, headings), keep a little overlap between neighbours, and carry metadata on every chunk so you know where it came from.

Embeddings are what make meaning searchable. An embedding model maps text into a high-dimensional vector such that passages with similar meaning land near each other — so a question about "annual leave entitlement" can retrieve a passage titled "vacation policy" even with no words in common. That is semantic search, and it is why RAG finds things keyword search misses. Two practical notes: the same embedding model must be used for both indexing and querying, and changing that model means re-indexing the whole corpus.

Retrieval is where RAG succeeds or fails

The techniques that reliably improve retrieval quality:

• Hybrid search — combine semantic (vector) similarity with keyword search, so exact identifiers, part numbers and names are not lost in meaning-space.
• Re-ranking — retrieve a generous candidate set, then use a more precise model to reorder it and keep the top few.
• Metadata filtering — restrict retrieval by date, source, product or, critically, by the user's access permissions.
• Query rewriting — expand or rephrase a terse user question before searching.
• Source hygiene — retrieval cannot fix a corpus full of outdated, duplicated or contradictory documents. Curation is part of the system.

RAG vs fine-tuning vs long context

Three ways to make a model useful on your material — they solve different problems and are often combined:

The rule of thumb: RAG for knowledge, fine-tuning for behaviour. If the complaint is "it does not know our facts", that is retrieval. If it is "it knows, but answers in the wrong style or format", that is fine-tuning.

Evaluating a RAG system

The step most projects skip — and the reason so many pilots never become production systems. Evaluate the two halves separately, because they fail differently:

• Retrieval quality — for a set of real questions, did the correct passage appear in the retrieved set at all, and how highly was it ranked?
• Answer quality — is the answer faithful to the retrieved passages (not invented), relevant to the question, and complete?
• A regression set — a fixed list of question/expected-source pairs, re-run whenever you change chunking, the embedding model, or the prompt. Without it, every 'improvement' is a guess.
• Human review of failures — sample the bad answers regularly; the pattern is usually a corpus or chunking problem you can fix.

Private data, PDPA and AI governance

RAG is, by definition, a system that pipes your private documents into a language model — which makes it a data-protection and governance question as much as an engineering one. Under Singapore's PDPA, personal data inside a RAG corpus remains personal data: the protection obligation applies to the index and its backups, and if the model or vector store sits overseas, the transfer limitation obligation applies too.^[3] Knowing where the vectors and the inference run physically happen is a real requirement, not a detail.

On the AI side, IMDA and the AI Verify Foundation published the Model AI Governance Framework for Generative AI in May 2024, which addresses exactly the risks RAG is often deployed to manage — hallucination, content provenance, and security among them.^[2] It is a practical checklist for an enterprise deployment: know your data lineage, be able to explain and attribute an answer, and test the system before and after it goes live.

Getting it right

A sequence that avoids the common dead ends:

1. Pick one narrow, high-value question set — a single well-understood corpus beats an "ask us anything" bot that is vague at everything.
2. Clean the corpus first. Remove outdated and duplicate documents; retrieval amplifies whatever is in there.
3. Build the evaluation set before the demo. Real questions with known correct sources, so progress is measurable.
4. Enforce permissions at retrieval time, filtered by the asking user — not after generation.
5. Start simple, then tune. Basic chunking and hybrid search first; add re-ranking and query rewriting when evaluation shows where the gap is.
6. Decide residency deliberately — where the vector store lives and where inference runs, with PDPA in mind.

Building an enterprise AI assistant?

RAG lives or dies on data engineering, retrieval quality and governance. Compare Singapore AI and system-integration partners who build and run production RAG systems.

Browse AI & Systems Integrators in Singapore

Frequently asked questions

Sources

1. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Lewis et al., 2020) — the original RAG paper — arXiv, 2020 reported checked 2026-09-01
2. Model AI Governance Framework for Generative AI — IMDA and AI Verify Foundation — IMDA, 2024-05 official checked 2026-09-01
3. Overview of the PDPA — Personal Data Protection Commission (PDPC) — PDPC official checked 2026-09-01