RAG is what its name says: retrieve the relevant data, then let the model generate an answer from it. The generate half we have. This post is about the retrieve half, which has more than one method.
Four families cover most of what an agent needs. Each is strong exactly where another is weak, so the skill is matching the method to the question.
Keyword search: exact words
Keyword search finds documents that contain the words you searched for. The classic algorithms are BM25 and TF-IDF, and they rank a document higher when your rare search terms appear in it often.
Keyword search is precise and fast, and it shines when the exact term matters. Searching a codebase for
get_user_profile, you want the lines with that exact function name, not lines that are vaguely about
users.
Its weakness is meaning. Search for "sofa" and a document that only says "couch" scores zero, because the words differ even though the meaning is identical. Keyword search cannot see that connection, because it matches strings, not sense.
Vector search: meaning
Vector search finds documents with a similar meaning, whether or not they share words. It is the method most people mean when they say RAG, and the next two posts build one from scratch.
The short version: an embedding model turns text into a point in a high-dimensional space, positioned so that similar meanings land close together. "Sofa" and "couch" sit near each other; "cat" and "car", similar in spelling, sit far apart. Search becomes finding the nearest points to your question.
Vector search shines on natural language and on the synonym problem that defeats keyword search. It finds "persist data to storage" when you asked about "saving to the database", because it matches on sense.
Its weakness is the mirror of keyword search's strength. When you need an exact token, a function name or an error code, vector search can drift to something merely similar and miss the precise match.
Graph search: relationships
Graph search represents data as entities (nodes) and relationships (edges), then answers by traversing the connections. "Which projects depend on a library maintained by someone who left last year?" is a graph question: it hops from person to library to project.
Graph search shines on multi-step relationship questions that the other methods answer clumsily. Its cost is up front: extracting reliable entities and relationships from raw text, and designing the schema, is real work, and getting it wrong gives a graph that answers confidently and incorrectly.
Structure-based search: the layout itself
Structure-based search uses the organisation of the data as the index. A codebase has folders, filenames and imports; you navigate them the way a developer navigates a new project. No embedding, no graph, just the structure that is already there.
Structure-based search shines when the data is already well organised and the layout carries meaning, a code repository being the clearest case. It has little to offer for a flat pile of unstructured text, where there is no structure to exploit.
Four families, each strong where the others are weak.
Choosing by the shape of the question
The common mistake is reaching for vector search every time, because it is the one everyone has heard of. The better habit is to read the question first.
| Method | Best when | Weak when | Reach for it for |
|---|---|---|---|
| Keyword | You need an exact word or code | Synonyms and paraphrase matter | A function name, an error code, an id |
| Vector | Meaning matters more than wording | You need a precise token | "How do we handle refunds?" over prose |
| Graph | The answer is a chain of relationships | The data has no clear entities | "What connects A to C through B?" |
| Structure | The layout already carries meaning | The data is a flat unstructured pile | Exploring a codebase or a file tree |
Real systems often combine them. Keyword and vector search together, one for precision and one for meaning, is a common and strong default, because each covers the other's blind spot.
What to take from this
- Retrieval is not one method. Keyword finds exact words, vector finds meaning, graph follows relationships, structure uses the layout.
- Each family is strong where another is weak, so choose by the shape of the question rather than by habit.
- Vector search is not the automatic answer. For an exact token, keyword search beats it, and the two together cover more than either alone.
Vector search is the one worth building by hand, because the ideas under it, turning text into points and measuring closeness, are the foundation of most retrieval you will meet. The next post starts there.