Technically Wrong
Sexist Apps, Biased Algorithms, and Other Threats of Toxic Tech Technically Wrong: Sexist Apps, Biased Algorithms, and Other Threats of Toxic Tech
Sara Wachter-Boettcher on how everyday software fails the people its designers did not picture — forms, defaults, algorithms and onboarding flows that assume one kind of user.
- First published
- 2017
- Publisher
- W.W. Norton & Company, independent publishers since 1923
- Pages
- 240
- Shelf
- Technology
- Level
- Entry
- ISBN-13
- 9780393634648
Record checked against
Only one source carried a publication date for this work, so the year is uncorroborated.
Catalogue note
- Reads as
- ReflectiveChanges how you see rather than what you do.
- Level
- Accessible / FoundationalNo prior reading assumed.
- Approach
- Theoretical
- Written for
- Engineers, Newcomers to the subject
Saved in this browser only — no account, nothing sent anywhere. My library
Read this if
- You care why some things are obvious to use and others are not.
- You want to think harder about what is actually right.
- You want to understand what technology is doing to how we live.
- You want a book that changes how you see things rather than what you do tomorrow.
- You are starting from scratch here and want a way in that assumes no prior reading.
The argument
Wachter-Boettcher works in content strategy, and the book's strength is that it stays at the level of ordinary product decisions: the name field that rejects real names, the gender dropdown with two options, the fitness app that cannot be told you had a miscarriage.
Her point is that these are not edge cases. They are the visible output of a homogeneous team treating its own experience as the default.
Most of these failures are not algorithmic. They are a dropdown menu that someone designed in ten minutes.
Devmark’s reading — not a quotation from the book
What it covers
Form design and identity, algorithmic filtering and its feedback loops, error states and the tone they take, and what the author calls delightful copy that becomes cruel in the wrong context.
The chapter on defaults is the most useful: a default is a decision about who is normal, and it is almost never recorded as one.
Who it is for
It is written for people who build software rather than for critics, and the examples are drawn from products the reader has used.
It predates the sharpest phase of the AI debate and is the better for it — the failures it documents are low-technology and therefore fixable by the people reading.
The problems are small, concrete and fixable, which is what makes the book usable.
Devmark’s reading — not a quotation from the book
The stress case
The idea she returns to is the stress case: the user encountering your product on the worst day of their life. Most design research recruits people in ordinary circumstances, so the flows that break are exactly the ones nobody tested.
A form that cheerfully asks about your relationship status is fine until someone is filling it in after a bereavement, and the book's practical argument is that designing for that edge improves the centre too.
Personas and the averaged user
She is sharp on the research artefacts teams build to stand in for their users. A persona assembled from demographic averages describes nobody, and its real function is to let a team feel it has consulted someone while continuing to design for itself. Her objection is not to the method but to its use as a substitute for contact with the people concerned.
The average user does not exist, and designing for them means designing for no one in particular.
Devmark’s reading — not a quotation from the book
Forms that will not accept a person
The chapters on data entry are the most immediately actionable: name fields that reject real names, gender options that force a false answer, required fields that assume a family structure, and address formats that assume a country. Each is a small decision that tells a user the product was not built with them in mind, and each is cheap to fix once noticed.
Delight at the wrong moment
She examines the industry habit of cheerful microcopy and celebratory animation, and the failure mode where a system congratulates someone on content that commemorates a death, or applies a playful tone to a distressing error. The underlying diagnosis is that tone was designed for a single imagined mood and applied unconditionally.
Who is in the room
A substantial thread connects the product failures to hiring, and in particular to the notion of culture fit as a criterion, which she argues reliably reproduces the existing team's assumptions and therefore its blind spots. The product problems and the workforce problems are presented as the same problem observed at two points.
Defaults are decisions
The recurring argument is that every default setting, preselected option and assumed value is a decision that the overwhelming majority of users will never revisit, because changing a default requires noticing it first. Treating those settings as neutral is precisely how one team's unexamined assumptions become the lived experience of millions of people who were never asked and have no practical way to object.
Where it dates
Published in 2017 and built from specific product failures, several of which have since been corrected. It is also light on implementation: it is persuasive that these are problems and comparatively thin on process, testing or organisational structure for preventing them. It diagnoses better than it prescribes.
What to take from it
It is aimed at people who build things rather than at critics, and that is how it is most useful. The practical residue is a checklist habit: before shipping, ask who this form will reject, who this tone will wound, and what this default assumes — three questions that take minutes and catch a surprising proportion of what the book documents.
What to take away
- 01Defaults are decisionsA default setting defines who counts as normal, usually without being noticed.
- 02Forms as exclusionName and gender fields reject real people routinely.
- 03Tone in error statesPlayful copy becomes cruel in circumstances nobody designed for.
- 04Homogeneous teamsThe failures track the narrowness of who was in the room.
- 05Fixable problemsLow-technology failures that the reader can actually correct.
