University textbook · Computer science

Artificial
Intelligence

From data to language models.

A textbook on how AI systems learn, how to evaluate them and how to use them to study computer science.

First year · Morocco

Cover illustration. Paths cross three planes of computation from inputs to a prediction. Second edition
Inputs, representations and prediction
5 chapters 25 sections 25 figures 5 quizzes

Contents

Five chapters, from specifying a problem to evaluating a documentation assistant.

  1. From problem to model

    Define the inputs, decision and target. Compute a gradient and design a comparison that can fail.

    Show sections
    1. 1.1 Specify the task before selecting a model
    2. 1.2 Rules, prediction, generation and decisions
    3. 1.3 Measuring and labeling data
    4. 1.4 Loss functions and gradient descent
    5. 1.5 Design an experiment that can fail
    6. Quiz · From problem to model
    Open chapter
  2. Learning and evaluation

    Split the data, interpret errors, set a threshold and understand how a network generalizes.

    Show sections
    1. 2.1 Choose a learning regime
    2. 2.2 Split the data to reflect future use
    3. 2.3 Thresholds, errors and calibration
    4. 2.4 Regularization, overfitting and change
    5. 2.5 Deep networks and learned representations
    6. Quiz · Learning and evaluation
    Open chapter
  3. Generative model architecture

    Follow the computation from tokenizer to decoding. Examine causal attention, post-training and diffusion.

    Show sections
    1. 3.1 Tokenization and vector representations
    2. 3.2 Causal attention and the transformer
    3. 3.3 Pretraining, adaptation and post-training
    4. 3.4 Decoding, context and inference cost
    5. 3.5 Multimodal models and diffusion
    6. Quiz · Generative model architecture
    Open chapter
  4. Studying and working with LLMs

    Set boundaries for tutoring, check an answer against documentation and evaluate retrieval.

    Show sections
    1. 4.1 Use an LLM to work on reasoning
    2. 4.2 Read a source with an assistant
    3. 4.3 Review code with contracts and tests
    4. 4.4 Retrieval and retrieval-augmented generation
    5. 4.5 Compare two ways of studying
    6. Quiz · Studying and working with LLMs
    Open chapter
  5. Reliability and capstone

    Audit performance gaps, preserve provenance and compare two assistants on a technical corpus.

    Show sections
    1. 5.1 Audit performance by use case and language
    2. 5.2 Protect data and preserve provenance
    3. 5.3 Tools, agents and authority boundaries
    4. 5.4 Trade off reliability, cost and latency
    5. 5.5 Capstone · A verifiable documentation assistant
    6. Quiz · Reliability and capstone
    Open chapter