Noise and bias are mathematically independent components of error, and organizations badly underestimate how much noise costs them.
Kahneman defines bias as the average error in a system and noise as the variability (standard deviation) of error - completely separate quantities. In a real insurance company noise audit, executives predicted underwriters' premiums would differ by about 10 percent for the same risk; the actual difference was roughly 50 percent, five times larger than expected, and the company estimated the resulting cost at over a billion dollars a year.
noise-vs-bias
A noise audit - giving the same case to many professionals independently and comparing their answers - is the practical tool for surfacing hidden unreliability.
Kahneman and Levitt ran this method at an insurance company through their firm The Greatest Good: dozens of underwriters priced identical risk cases, and the spread between individual judgments (not the presence of bias) revealed the scale of the problem. Levitt notes it was the one case in years of consulting that produced real organizational change, because the company designed and ran the experiment itself and so accepted the results.
noise-vs-bias
Decision hygiene - decomposing a judgment into independent parts, evaluating each separately, and delaying intuition - reduces noise without adding bureaucracy.
Kahneman argues the intuitive approach (absorb a large file, then trust a holistic gut call) is not optimal; people do better with a plan that breaks the problem into pieces judged independently before combining them. Levitt connects this to a guesstimation technique from an MIT course: breaking a hard estimate (e.g., total mass of all trees on Earth) into independent sub-estimates whose individual errors tend to cancel out.
decision-hygiene
Independent judgment matters as much as decomposition, and people resist the practices that would reveal disagreement.
Kahneman cites psychologist Nathan Kuncel's account of university admissions readers who passed essays along with the prior reader's grade already visible - defeating the purpose of independent review. When advised to hide the first grade, staff admitted they used to do that but reverted because it exposed 'so much disagreement.' Kahneman frames this as a general pattern: people don't want to detect that a system is noisy.
decision-hygiene
Kahneman and Levitt sharply disagree on whether noise matters in criminal sentencing, exposing a real fault line in how to value fairness.
Levitt argues that if a noisy sentence (e.g., three to seven years around a five-year norm) is symmetric with no bias, he personally wouldn't pay much to remove the variance, since he'd be as happy with the low draw as unhappy with the high one. Kahneman counters that equal treatment of similarly situated people is a fairness principle independent of whether outcomes wash out on average, and that most people - unlike Levitt - would not accept a lottery added on top of a judge's sentence.
justice-and-fairness
What looks like noise in a system is often produced by the aggregation of individual biases.
Kahneman's rejoinder to Levitt is that each judge (or underwriter, or admissions reader) has a distinctive, consistent way of thinking that differs from colleagues' - so a system-level view sees 'noise,' but each individual within it is behaving in their own biased, non-random way. Noise is the complement of bias at the system level, not proof that individual judgment is random.
noise-vs-bias
Kahneman and Tversky generated their most influential findings by studying themselves and building simple 'riddle' questions, not by starting from economic theory.
Kahneman describes spending hours a day with Tversky inventing problems on which they, despite knowing the correct answer, felt tempted toward a wrong one - then testing whether a single, story-like question could reveal how people reasoned. He credits the riddle-like simplicity of these questions, more than any theoretical framework, with making the work land across disciplines that would not otherwise engage with psychological experiments.
behavioral-economics-origins
People are 'reasonable but not logical' - the core distinction that separates behavioral economics from classical rational-agent models.
Kahneman rejects the word 'irrational' for describing human judgment. Classical economics assumes people reason according to formal logic; psychological study of actual judgment and decision-making shows people don't follow logic, but they aren't stupid or irrational either - they follow different, more heuristic-driven patterns that formal models fail to capture.
behavioral-economics-origins
Behavioral economics has influenced psychology far less than psychology has influenced economics, aside from contributing methodological rigor.
Kahneman argues economics necessarily rests on psychological assumptions about the economic agent, while psychology does not depend on economic assumptions - making psychology the more foundational discipline. He credits economics with pushing psychology toward more rigorous methods, but says the substantive traffic runs mostly one way.
behavioral-economics-origins
The popular expectation that behavioral economics can reliably drive large behavior change has outpaced what the field can actually deliver.
Levitt notes he is approached roughly weekly by companies wanting to use behavioral economics to change customer behavior. Kahneman agrees the field's real successes are small changes that cost little; changing behavior is extremely difficult, and researchers within behavioral economics itself tend to be realistic about this even as public expectations, fueled partly by books like Thinking, Fast and Slow, have become inflated.
behavioral-economics-origins
Kahneman cooperated with The Undoing Project largely to correct a credit imbalance created by Tversky's early death.
Amos Tversky died in 1996; Kahneman says that afterward he alone received a disproportionate share of credit for their joint work, including the 2002 Nobel Prize, since Tversky was no longer alive to share it. He describes feeling 'duty bound' to participate in Michael Lewis's book because it brought Tversky back into the historical picture, even though the book dramatized and exaggerated some of the real differences between the two men.
behavioral-economics-origins
Continued intellectual engagement, not innate resistance to aging, is what Kahneman credits for staying sharp - and willingness to abandon failed ideas is his one piece of general advice.
At 86, Kahneman frames his motivation as simple curiosity about his own mistakes, and says changing his mind is a pleasant experience of having learned something, not an admission of failure. His advice to young people is narrow and specific: follow what you're inclined to do, but be willing to discard ideas that don't work - and if you can't do that, you're in the wrong profession.
decision-hygiene
Books referenced
Noise: A Flaw in Human Judgment - Daniel Kahneman, Olivier Sibony, and Cass Sunstein - Kahneman's then-new book, the central subject of the conversation; built on the 'noise audit' work he did with Levitt's consulting firm and later research with collaborators Sibony and Dan Lovallo
Thinking, Fast and Slow - Daniel Kahneman - Kahneman's 2011 bestseller, discussed as a lonelier, harder writing experience than Noise, and as the book that made behavioral economics broadly popular
The Undoing Project - Michael Lewis - 2016 book telling the story of Kahneman and Amos Tversky's collaboration; Kahneman discusses why he cooperated with Lewis despite reservations
Companies
The Greatest Good - consulting firm Levitt and Kahneman co-founded, where Kahneman first ran the insurance-company noise audit that became a seed of the book
McKinsey - the firm Olivier Sibony and Dan Lovallo were consulting for when the idea for Noise originated
Techniques and frameworks
Noise audit - Kahneman's method for measuring unwanted variability in a system: give the same case to many professionals independently and compare their judgments
Decision hygiene - Kahneman's term for breaking a judgment into independent sub-parts, evaluating each separately, and delaying intuition until enough information is gathered, rather than absorbing a whole case and blurting out a number
Fermi-izing / breaking up a problem - estimation technique named after physicist Enrico Fermi that Philip Tetlock teaches his superforecasters, of decomposing a hard estimate into independent smaller pieces whose individual errors tend to offset
Summary
This is Steve Levitt's 2021 conversation with Nobel laureate Daniel Kahneman, built around Kahneman's then-new book "Noise: A Flaw in Human Judgment," co-written with Olivier Sibony and Cass Sunstein. Levitt and Kahneman were previously business partners in a consulting firm called The Greatest Good, and the conversation opens with Kahneman explaining noise as the underappreciated "complement of bias": bias is the average error a system produces, noise is the variability of error across supposedly interchangeable decision-makers. The two revisit the noise audit they ran together at a large insurance company, where executives predicted underwriters' price quotes for identical risks would differ by roughly 10 percent and the real difference turned out to be about 50 percent - a gap the company itself estimated was costing over a billion dollars a year. Kahneman argues the fixes are not exotic: decompose judgments into independent parts, evaluate each separately before combining, and delay intuitive conclusions until enough information is in - a set of practices he calls decision hygiene.
A substantial middle section is a genuine disagreement rather than an interview. Levitt pushes back hard on Kahneman's use of criminal sentencing as a leading example of costly noise, arguing that if a sentence is unbiased and symmetric around a fair average (three years instead of five feels as good as seven feels bad), he personally wouldn't pay much to eliminate the variance. Kahneman holds that equal treatment of similarly situated people is a fairness principle independent of whether outcomes wash out on average, and that most people - unlike Levitt - would reject a system that literally added a random lottery on top of a judge's sentence. Kahneman adds a subtler point: what looks like system-level "noise" is often the aggregation of many individually consistent, individually biased styles of judgment (each judge reasons in their own particular way), rather than true randomness.
The conversation then turns to the origins of behavioral economics itself. Kahneman describes his decades of collaboration with Amos Tversky as built on studying themselves - deliberately inventing simple, riddle-like questions on which even they, knowing the right answer, felt pulled toward the wrong one - because a question with the character of a riddle could cross disciplinary lines that a formal psychological experiment could not. He draws a sharp line between "reasonable" and "logical": classical economic models assume people reason logically, and Kahneman's life's work shows they mostly don't, without being irrational either. He also argues, bluntly, that psychology has shaped economics far more than economics has shaped psychology, and that popular enthusiasm for behavioral economics as a lever for large behavior change has significantly outpaced what the field can actually deliver - its real successes tend to be small, low-cost nudges, not durable transformation.
The episode closes on a more personal register. Levitt asks about Michael Lewis's "The Undoing Project," and Kahneman explains that he cooperated with the book largely to correct a credit imbalance: after Tversky's death in 1996, Kahneman alone received public recognition - including the 2002 Nobel Prize - for work that was equally Tversky's, and he felt duty-bound to help restore Tversky to the story even though the book dramatized some real tensions between them. Kahneman also touches briefly and matter-of-factly on surviving as a Jewish child fleeing the Nazi regime in Europe, saying he doesn't attribute anything he has done to that difficulty. He ends by framing his continued sharpness at 86 as a product of active use ("use it or lose it") and describing his only real piece of general advice as the willingness to discard ideas that don't work, however painful that is in the moment.
Notable Quotes
"Noise is unreliability... this is really the complement of bias." - Daniel Kahneman
"They realized that noise was costing them probably over a billion dollars a year." - Daniel Kahneman
"People are quite reasonable, but they are not logical." - Daniel Kahneman
"When I change my mind is the pure experience of having learned something. Yesterday, I was stupid and now I've seen the light." - Daniel Kahneman
"It was the luckiest thing that ever happened to me." - Daniel Kahneman, on his collaboration with Amos Tversky