Reward, error and self-play: what machine learning shows about human learning
Michael Littman's history of reinforcement learning, Terry Sejnowski's account of the brain's reward and value signals and Fei-Fei Li on how AI learns and where minds differ describe learning in machines and people in overlapping terms. The comparison clarifies which lessons from AI transfer to human learners and which do not.
What learning rule does Terry Sejnowski say the brain and AlphaGo share? Show answer
Predict the next reward, compare what you got with what you expected, and update. In AI it is called reinforcement learning.
What carries the reward signal in the human brain? Show answer
Dopamine neurons in the brainstem, which send a global signal that modulates activity across the cortex.
What is the value function? Show answer
An accumulated sense of what is good or bad for you, built up from every action and its outcome.
How do rewards and punishments differ in their effect, according to Sejnowski? Show answer
Small rewards move you gradually, while punishment can teach in a single trial and leave a lasting mark.
What is the limit of self-play, according to Michael Littman? Show answer
A system practicing against itself gets better at that one game and nothing else, and improvement may level off.
How did the early game-playing systems begin before playing themselves? Show answer
With expert moves or human data as a kick-start, which Littman says shows the human skill behind the results.
What does Littman say a language system needs beyond reading our writing? Show answer
To be pushed back at: to argue with people and lose a few times in order to discover nuance.
What did Fei-Fei Li notice about how children learn objects? Show answer
By age six they learn tens of thousands of object categories, and can learn a cat from only a handful of examples.
Why does Li say a machine's objective is not a motive? Show answer
Its modes are just objectives for an algorithm, not feelings. A friend's sympathy comes from care and a memory of pain.
Why must learning be active? Show answer
Wrestling with a problem engages trial and error, while being told the answer stores a fact that will not surface for a different problem.
What did the dermatology study Sejnowski cites find? Show answer
Doctors and an AI each scored about 90 percent on skin lesions, and doctors using the AI reached 98 percent.
What should you do with a hard problem before bed? Show answer
Think about it before you go to sleep, since sleep helps consolidate the day's experience.