Deep Learning

A graduate textbook, not an introduction. It assumes comfort with linear algebra, probability, calculus and numerical computation, and it develops the material mathematically rather than through code.

There are no worked implementations and no notebooks. The book's purpose is to establish why the methods work and what they are doing, which is a different job from teaching someone to build one.

No code, no notebooks. It establishes why the methods work, which is a different job from teaching you to use them.

Devmark’s reading — not a quotation from the book

The opening section covers linear algebra, probability and information theory, numerical computation, and classical machine learning.

Its function is to establish shared notation and to make the book self-contained for a reader with the prerequisites but not the specific background. It is a compressed review rather than a teaching text, and readers who need to learn these topics should learn them elsewhere first.

The core of the book. Feedforward networks, regularisation, optimisation for training, convolutional networks, sequence modelling with recurrent and recursive nets, and practical methodology.

The chapter on practical methodology is the one most often recommended independently: it covers how to choose a baseline, how to decide whether to gather more data, and how to debug a model that is not learning — the judgement that separates practitioners from people who can describe architectures.

Their treatment of why deep networks generalise at all, and of the techniques that make training work: weight decay, dropout, early stopping, batch normalisation, parameter initialisation and adaptive learning rates.

This is the part of the book with the most enduring value, because the architectures have changed considerably since publication while the underlying optimisation problems have not.

The final section covers linear factor models, autoencoders, representation learning, structured probabilistic models, Monte Carlo methods, the partition function, approximate inference and deep generative models.

This is the most mathematically demanding material and the most dated — it predates the generative approaches that subsequently dominated, and reads now as a snapshot of where research stood.

The research section is a snapshot of 2016. The mathematics holds; the sense of what matters has moved.

Devmark’s reading — not a quotation from the book

Published in 2016, before the transformer architecture, before the scaling results that reshaped the field, and before large language models became the centre of attention.

Attention mechanisms appear only briefly. A reader coming to it now should understand it as the definitive reference for the foundations and not as a current account of the field.

Because the foundations did not move. Backpropagation, optimisation dynamics, regularisation theory, the bias-variance trade-off and the probabilistic framing are all unchanged, and they are treated here more carefully than anywhere else at comparable accessibility.

It is also freely available online in full, which removes the main barrier to using it as a reference.

Someone with the mathematical background who wants to understand rather than apply — graduate students, researchers, and practitioners who have been building models and want to know what they have been doing.

It is the wrong first book for almost everyone. Géron's practical text, or a course, should come first, with this used afterwards as the reference that explains the machinery.

Not linearly. Most readers use it as a reference, reading the chapter relevant to a current problem rather than working through it.

The practical methodology chapter and the regularisation and optimisation chapters are the ones with the widest applicability and are worth reading independently of any specific need.

The framing that deep learning is representation learning — that the depth is doing the work of building useful intermediate representations rather than merely adding capacity.

And the practical methodology discipline: establish a baseline, measure before changing, and determine whether your problem is underfitting or overfitting before choosing a remedy.