Section 4.4 Studying and working with LLMs
Retrieval and retrieval-augmented generation
A retrieval-augmented generation system (RAG) first assembles a corpus, divides it into identifiable passages and indexes them. Given a question, a retriever selects passages. A generator sees the question and this evidence and proposes a response. Lewis et al. describe an influential formulation of this combination.
Evaluate retrieval and generation separately
Keyword search favors terms shared with the question. An encoder can bring paraphrases together through vector similarity. The first can be better at finding an exact identifier, the second at matching expressions that differ. For a retrieval test, prepare a list of expected passages. If a passage is missing from the top k results, inspect indexing or retrieval. If it appears and is correctly identified but mishandled, inspect the supplied context and generated reply. Hit@k is the share of questions for which at least one expected piece of evidence appears in the top k.
Choose a useful chunk size
Too little text can lose headers and exceptions. Too much can introduce competing statements. Retain a stable ID, section heading, version and link. With a small corpus, compare any enhancement against simple lexical search. Include questions the corpus cannot answer. The appropriate output is a reasoned abstention and an indication of what evidence is missing.