Why this matters
The model does not know your documents
A model knows a lot about the world and nothing about your returns policy as of last Tuesday. Asked about it, it will produce a fluent, confident answer that may be wrong. Retrieval fixes this by searching your documents first and giving the model only the passages it found.
This pattern is often called RAG, retrieval-augmented generation. In plain words: look it up, then answer from what you looked up. The answer is only as good as the search, so most of the work is in the search, not the prompt.
The second rule matters as much as the first. If the search finds nothing relevant, the right answer is to say so. A system that refuses when it has no basis is more useful than one that is always confident.
- QuestionWhat the user typed
- ChunksYour docs, split into passages
- RankerScores every chunk against the question
- ThresholdTop score too low? Refuse
- ModelAnswers from the top chunks only
- CitationsEach claim links to its chunk
Blue edge: model · orange: tool · green: person · grey: data · plain: your code
Build step
Chunk, rank, then cite or refuse
Chunking splits documents into passages small enough to rank. Splitting at headings keeps each chunk about one topic. Splitting every forty words is simpler but cuts sentences and mixes topics, which hurts ranking and makes citations hard to read.
Ranking scores each chunk against the question. This lesson uses BM25, a keyword scoring method that rewards chunks containing the question's rarer words. Production systems often mix keyword scores with embeddings, which match meaning rather than exact words. Start with keywords: you can see exactly why a chunk ranked where it did.
The threshold is the refusal rule. If the best chunk scores below it, the system answers that it does not have enough material instead of guessing. Set it too high and you refuse good questions. Set it too low and you answer from irrelevant text.
Citations close the loop. Every answer names the chunk it came from, so a reader can check it in one click and you can debug a bad answer by looking at what was retrieved.
export function answer(question: string, chunks: Chunk[], k = 3, threshold = 2.0) { const ranked = rankBM25(question, chunks).slice(0, k); if (!ranked.length || ranked[0].score < threshold) { return { answer: "I don't have enough material to answer that.", citations: [] }; } return { answer: compose(question, ranked), // the model sees only these chunks citations: ranked.map((r) => ({ id: r.chunk.id, title: r.chunk.title })), };}Try it
Ask the demo help centre
The documents below belong to Parcelo, a made-up courier used for this lesson. Ask a question, then change the chunking method, the number of chunks kept (k) and the refusal threshold. Every score is computed in your browser so you can see why each chunk ranked where it did.
Ranked chunks (top 1). Highlighted words matched your question.
- #1 Chunk 2 (starts in Returns)score 3.12working days after the sender receives the item. If your parcel arrived damaged, take photos of the box and the contents before you throw anything away. Report the damage within 7 days from the order page and attach the photos.
Test your result
Measure it: recall at k on six questions
Recall at k asks one question per test case: did the right document appear in the top k chunks? It is the simplest retrieval measure and the most useful one to start with, because if the right passage never reaches the model, no prompt can save the answer.
The sixth test question has no answer in the documents. It passes only if the system refuses. Find settings that pass all six, then switch to fixed-size chunks with the same settings and watch the score drop. The ranked list shows why: fixed chunks mix topics and dilute the scores.
What broke
What broke for other builders
- No refusal path. Every question got an answer, including ones the documents never covered. Users trusted the confident wrong ones.
- Chunks too big. Whole pages as chunks meant the right sentence was retrieved inside two thousand irrelevant words and the model missed it.
- Stale index. Documents changed but the index did not. Rebuild or update the index whenever a source document changes.
- Measured by feel. A prompt change felt better in three manual tries and lowered recall on the real test set. Measure before and after.
Next commit
Next commit: trace every answer
Once retrieval is measured, record it. Store which chunks were retrieved for each answer and how long each step took, so a bad answer can be traced to the search or the model. The agent trace lesson shows how.
Quick check
Ship the demo
- ○ Test your result: every case passing at least once
- ○ Quick check answered
Pass the tests and answer the quick check to unlock the merge.