Section 4.1 Studying and working with LLMs

Use an LLM to work on reasoning

A conversation can make reasoning easier to follow without improving your ability to reconstruct it. Start a study session by setting an observable outcome. In algorithms, it may be explaining an invariant, writing a trace or finding a counterexample. Immediate access to an answer changes the work required. The sequence of activities should preserve time for the student to answer independently.

Give the model a bounded task

After a first written attempt, a prompt can state the assistant’s role, the problem, what has been tried and the kind of help wanted. For binary search with an exclusive upper bound, ask “Review only the stated invariant. Give me a question about the empty case and wait for my reply.” The assistant can misread the problem. Keeping both the original problem and the attempt in view makes it possible to check the scope of the hint.

A cycle connects unaided attempt, hint, reasoned correction and a new transfer problem completed independently.
Figure 4.1. The transfer test measures new work by the student. The dialogue supplies feedback along the way.

Measure recall and transfer

For active recall, close the conversation and write the invariant conditions from memory. To test transfer, move from finding a first occurrence in a sorted array to finding the insertion point for an absent value. Write a justification before opening the solution. A useful hint helps you resume at the point of difficulty. One that gives away the final expression makes it harder to separate understanding from copying. Research on learning techniques distinguishes retrieval practice and spaced practice. See the review by Dunlosky et al..