Section 1.2 From problem to model
Rules, prediction, generation and decisions
A single system can combine several kinds of computation. A “selected office” field with an agreed meaning can trigger a deterministic rule. A classifier can estimate the office from free text. A generative model can draft a summary for the person reviewing the message. Each output calls for a different test.
A taxonomy by output
| Task | Output | Possible criterion |
|---|---|---|
| Retrieval | Ranked list of passages | Relevant passage appears among the top results |
| Classification | Category or scores by category | Per-class recall and confusion matrix |
| Regression | Numerical quantity | Absolute error in minutes, dirhams or physical units |
| Generation | Text, image or another structure | Accuracy of claims and compliance with the request |
| Decision | An action chosen under constraints | Error cost, delay and access to recourse |
Prediction estimates an unknown quantity. Decision-making applies a policy to that estimate alongside additional information. A 0.70 probability that a ticket needs rapid handling can justify review. Queue capacity and the cost of a delay determine the policy.
Learning associations
A classifier can exploit the association between a phrase and a category without explaining why someone submitted a request. Whether changing that phrase would solve the underlying problem is a causal question. To evaluate an intervention, such as adding guidance to the form, we need a comparison that can distinguish its effect from the enrollment calendar and other changes. A prediction score alone cannot make that distinction.