Glossary
Each term links to the section that explains it and puts it into practice.
- Inference
- Computing an output from available inputs and a model’s parameters. Section 1.1
- Label
- Reference value used to train or evaluate a supervised model. Section 1.3
- Gradient
- Vector of derivatives of the loss with respect to model parameters. Section 1.4
- Baseline
- A fully specified, typically simple procedure used for comparison. Section 1.5
- Self-supervision
- Learning with targets derived from the data itself. Section 2.1
- Data leakage
- Incorporating information that should have been unavailable at the relevant time. Section 2.2
- Calibration
- Agreement between stated probabilities and observed frequencies. Section 2.3
- Precision
- Share of flagged cases that really belong to the positive class. Section 2.3
- Recall
- Share of reference-positive cases that are flagged. Section 2.3
- Overfitting
- Fit to quirks of the training set at the expense of generalization. Section 2.4
- Embedding
- Numeric vector associated with an object such as a token or a document. Section 3.1
- Token
- Unit created by a tokenizer and represented by an ID. Section 3.1
- Causal attention
- Attention restricted to the current and earlier positions. Section 3.2
- Perplexity
- Exponential of the mean per-token log loss when using the natural logarithm. Section 3.3
- Logit
- Score before conversion to a probability. Section 3.4
- Context window
- Maximum number of tokens that can be used together for a request. Section 3.4
- Diffusion
- Family of generative methods that learn to reverse progressive noising. Section 3.5
- Transfer
- Ability to apply a principle to a problem that differs from the original exercise. Section 4.1
- RAG
- Generation that receives passages retrieved from a collection at request time. Section 4.4
- Provenance
- Information tracing the origin, version and handling of data. Section 5.2
- Coverage
- Share of requests that the system chooses to answer. Section 5.4