Section 1.1 From problem to model
Specify the task before selecting a model
A university wants to route requests submitted through a form to the appropriate office. At submission time, it has the subject and message body. The desired output is a suggested category, with a referral to a human intake team when needed. The office that actually closes the request may only be known several days later. These two moments define the boundaries of the technical problem.
The prediction contract
Let x denote the information available at decision time, y the reference category and f(x) the model output. The unit of observation might be a complete request or the first message in a thread. Choosing between them changes what counts as a duplicate and what an error means. If one message combines an account problem with an enrollment request, single-label classification needs a priority rule. A multi-label system changes both annotation and evaluation.
Who bears the cost of an error?
A classifier score could become a suggestion, a priority queue entry or an automatic transfer. The same model carries different consequences in each setting. A wrong category in a review queue mainly costs reading time. Under automatic transfer, someone may wait for a response that never comes. The specification should therefore set permitted actions, say how ambiguous cases are handled and define how long a request can wait before reconsideration.
Method. Describe the task in one testable sentence before collecting data. In this case, “estimate the office needed for newly submitted first messages and measure transfers corrected by staff.” Naming first messages, unseen cases and an observable human decision lets us define an evaluation independently of the model.