RAG vs Fine-Tuning: Which Problem Are You Trying to Solve?
Retrieval-augmented generation supplies relevant information at answer time. Fine-tuning changes a model through additional training. Choosing between them starts with the problem: changing reference material, inconsistent output behaviour or a combination of the two. They can also be used together.
Why it matters
A team FAQ changes whenever its policies change. A formatting task may stay stable while requiring many examples of a specific style. Those are different maintenance problems and deserve different experiments.
A practical workflow
- Collect actual failure examples and classify each as missing knowledge, incorrect evidence or unwanted output behaviour.
- For changing facts, test retrieval over a small set of approved documents with visible citations.
- For repeated behaviour problems, first establish a strong prompt and examples before considering a training experiment.
- Compare both approaches on held-out questions, including updates, missing answers and contradictory documents.
An example
A support assistant may retrieve the latest return policy while following a standard response structure. If it answers from an old policy, fix document freshness and retrieval first. Training more examples of polite replies will not automatically repair that evidence problem.
A mistake to avoid
Retrieval does not guarantee that the chosen passage is relevant, and fine-tuning is not a reliable substitute for a frequently updated database. Both approaches require evaluation and a plan for corrections.
Try it and check the result
Prepare ten policy questions and revise one source document. Measure how quickly the system reflects the change, whether it cites the right version and whether unsupported answers are rejected.
Reference and further reading
Google's introduction to embeddings.
The workflows and examples in this guide are illustrative. Product controls and availability can change; check the linked official guidance for your setup.
Continue with What Is Generative AI? A Practical Guide for First-Time Users or explore all blog topics.