The Alignment Problem
Brian Christian's account of the effort to make machine learning systems do what their designers actually intended — structured as three problems, told through the researchers working on them, and connecting present harms to long-horizon concerns.
- First published
- 2020
- Publisher
- W. W. Norton & Company
- Pages
- 356
- Shelf
- Technology
- Level
- Core
- ISBN-13
- 9780393635829
Record checked against
Catalogue note
- Reads as
- TechnicalAssumes and builds craft knowledge.
- Level
- Core / BuildingAssumes general familiarity with the subject.
- Approach
- Applied
- Written for
- Engineers, Readers with some grounding
Saved in this browser only — no account, nothing sent anywhere. My library
Read this if
- You want to think seriously about what these systems can and cannot do.
- You want to think harder about what is actually right.
- You work in this area and want depth rather than an overview.
- You know roughly how the subject works and want the fuller account.
The structure
Christian organises the book in three parts: prophecy, agency and normativity — roughly, what goes wrong with systems that classify, systems that act, and systems that must infer what we value.
The structure is the argument. It shows that present-day fairness failures and long-horizon alignment concerns are not separate topics but the same specification problem appearing at different capabilities.
Present bias failures and long-horizon alignment are the same specification problem at different capability levels.
Devmark’s reading — not a quotation from the book
Representation and bias
His first section covers the well-documented failures: models trained on historical data reproducing historical patterns, word embeddings encoding occupational stereotypes, facial analysis systems performing far worse on darker-skinned faces.
His treatment is careful about mechanism. The systems were not malfunctioning; they were accurately learning a distribution that encoded decisions people had made, which is a harder problem than a bug.
Fairness has no single definition
His clearest technical explanation: several reasonable definitions of fairness — equal false positive rates, equal predictive value, calibration — are mathematically incompatible when base rates differ between groups.
This is not an engineering shortfall. It is an impossibility result, which means fairness in a deployed system is a choice among incompatible definitions rather than a property that can be achieved.
Reinforcement learning and reward
His second section covers agents that act, and the problem of specifying reward. His examples of reward hacking — agents finding unintended ways to score that defeat the purpose — are the most vivid demonstration available that optimisation is literal.
His treatment of sparse reward, curiosity and intrinsic motivation connects the engineering problem to developmental psychology in a way that is genuinely illuminating rather than decorative.
Imitation and inverse reinforcement learning
His third section covers learning what to want from observing behaviour: imitation learning, inverse reinforcement learning, and learning from human feedback.
His account of the difficulties is honest — humans are inconsistent demonstrators, optimal behaviour is not the same as the behaviour we would endorse, and inferring a goal from actions is underdetermined — and these are the techniques now central to how large models are tuned.
Inferring what someone wants from what they do is underdetermined. That is the technique everything now relies on.
Devmark’s reading — not a quotation from the book
Uncertainty and deference
His treatment of Russell's argument for systems uncertain about their objectives, presented alongside the research that developed it.
His contribution is showing the lineage: that the theoretical proposals and the practical techniques used in deployed systems come from the same body of work, which is not obvious from either literature alone.
The reporting
Christian is a writer rather than a researcher, and the book's method is extended interviews with the people doing the work.
This is its main strength. Ideas arrive attached to the person who had them and the problem that provoked them, which makes the technical material considerably more legible than a systematic treatment would be.
Where it is limited
Breadth costs depth. No topic is covered thoroughly enough to work from, and a reader wanting to implement anything will need the primary literature.
The book is also a snapshot of a fast-moving field, and the specific systems discussed have been substantially superseded — though the problems they illustrate have not.
What it gets right that others miss
The refusal to separate near-term harm from long-term risk is its most valuable editorial decision, because that split has made the public argument considerably less productive than it should be.
By treating both as the same specification failure, the book makes it possible to care about both without choosing a camp.
What to take from it
The fairness impossibility result, which changes how you read any claim that a system has been made fair: fair by which definition, and at whose expense?
And the reward-hacking examples, which are the cheapest available cure for the intuition that a system will do roughly what you meant.
What to take away
- 01Bias was not a malfunctionThe systems accurately learned a distribution encoding decisions people had already made.
- 02Fairness definitions are incompatibleAn impossibility result, not an engineering gap — so deployment means choosing which definition to satisfy.
- 03Optimisation is literalReward hacking is the clearest demonstration that a system pursues what you specified, not what you meant.
- 04Goal inference is underdeterminedWorking out what someone wants from what they do has no unique answer, and everything now relies on it.
- 05Near-term and long-term are one problemTreating them as separate camps has made the public argument much less useful than it could be.
