The Bitter Lesson
| The Bitter Lesson |
| Author: Richard S. Sutton Published: March 13, 2019 Original Link: Incomplete Ideas |
Core Philosophy
The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin.
The ultimate reason for this is Moore's law, or rather its generalization of continued exponentially falling cost of computation per unit time. Most AI research has been conducted as if the computation available to the agent were constant (in which case leveraging human knowledge would be one of the only ways to improve performance) but, over a slightly longer time than a typical research project, vastly more computation inevitably becomes available. Seeking short-term improvements, researchers try to leverage their human knowledge of the domain, but the only thing that matters in the long run is the leveraging of computation.
These two need not be mutually exclusive, but in practice they tend to be. Time spent on one is time not spent on the other. There are psychological commitments to investment in one approach or another. Here are a few prominent examples from the history of AI.
Historical Examples
Computer Chess
In chess, the breakthrough that defeated the world champion Kasparov in 1997 (IBM's Deep Blue) was based on massive, deep **search methods**.
- At the time, this was looked upon with dismay by researchers who had tried to leverage human understanding of chess structure to make a master player.
- But it turned out that all the effort and knowledge of human strategies were defeated by a brute-force, massive distributed search over specialized hardware.
Computer Go
Twenty years later, the same story repeated itself in the game of Go.
- Early research was heavily invested in putting human knowledge of patterns and influence into Go-playing programs.
- However, when AlphaGo appeared—leveraging deep learning and **massive reinforcement learning (search and learning combined)**—all those personal rules and database of human games became irrelevant.
- Learning by playing against itself completely surpassed any human expertise.
Speech Recognition
In the 1970s, a speech recognition competition funded by DARPA showcased a clear divergence in methodologies.
- The traditional linguistics-based school argued for leveraging human knowledge like phonemes, syllables, and grammatical rules.
- The other school applied statistical methods like Hidden Markov Models (HMMs) which were much more mathematical and computational.
- Eventually, the statistical methods that leveraged computation and big data won completely. This led to Fred Jelinek's famous quote: "Every time I fire a linguist, the performance of the speech recognizer goes up."
Computer Vision
Computer vision has recently followed a similar trajectory.
- Early vision research relied on hand-crafted features (such as SIFT) or human-engineered systems based on lines, edges, and optical flow.
- All of this was overturned when Deep Convolutional Neural Networks (CNNs) were introduced, powered by GPU computing and a massive dataset (ImageNet).
- Today, networks **learn** features by themselves from vast amounts of data, achieving results far beyond human intuition.
The Bitter Lesson & Implications
Two key takeaways must be learned from these historical patterns:
1. Embrace Scalability
We must realize that **general methods**—specifically search and learning—are the ones that scale effortlessly with the growth of computation.
- As Moore's Law delivers more computing power, search and learning methods become exponentially more effective.
- Conversely, any method that relies on hard-coding human knowledge hits a performance ceiling and fails to improve automatically with more hardware.
2. Avoid Anthropomorphism
Human beings often overestimate their own understanding of how the mind works, mistakenly believing that its inner mechanisms match our conscious intuition.
- The human mind is incredibly complex; we should stop trying to hard-code the structures of our perception (like language syntax or visual lines) into AI agents.
- Instead, we should build the fundamental building blocks that allow AI to **discover** and **understand** the world on its own, much like a human child.