Richard Sutton Argues Trial-And-Error Drives Learning Over Imitation

Sutton is the father of reinforcement learning - listen to my full interview with him here: https://www.dwarkesh.com/p/richard-sutton

@dwarkesh_sp This guy just wants attention.

@dwarkesh_sp Your pushback here was very good, imitation is incredibly important in humans, as you say in learning to communicate and forage being two clear examples.

@dwarkesh_sp Not sure I buy into not learning by imitation. Why do stories of people dominate all literature from ancient to now? Is it because story-telling is a form of experiential learning, the type of learning that is primary to adult learning? Is this not a form of imitation?

@dwarkesh_sp My toddler imitates words that I say all the time as she is learning to speak. She also babbles quite a bit. Maybe Richard's point is that there is an RL cycle happening internally (brain to mouth to ears to brain again) regardless of whether and imitation is successful or not.

@dwarkesh_sp imitation clearly has value, but it limits the student to what already exists. there must be another thing that drives the process of discovery. it's this other thing that Sutton considers fundamental to "learning".

@dwarkesh_sp Probably the interview where I felt you were struggling a bit more. I think his argument is neat, and makes sense if you have kids ? They don’t necessarily imitate all the time ?

i train animals (horses, dogs) to do sophisticated things, and there is no imitation involved. i also take music instruction, and where there is imitation, you can only imitate what you can already do- so this is not the transmission mechanism either. it's purely reward and relief. there isn't even punishment, as that is based on a flawed theory of mind. to teach you need to act through the student, and all learning and training is being acted-through.

@dwarkesh_sp He is wrong.

If you have a well trained dog and get a puppy you do not need to train it, it copies the other dog.

If only have a puppy you need to train it using rewards for good behaviour....oh wait.

We also see cultural transmission such as chimpanzee termite fishing.

@dwarkesh_sp Absolutely wrong. This is ver ver very very well studied.

@dwarkesh_sp Forgive me a novice question here, but isn’t a lot of AI trial and error? The AI says this is a motorcycle, I say it isn’t, AI updates.

@dwarkesh_sp Richard Sutton would love talking to Rene Girard

@dwarkesh_sp Isn't this one of David Deutsch's point about Popperiqn conjecture?

Statement that imitation has "almost no role" in human learning is contradicted by the evidence from isolation and feral cases. Many cases show children adopting the specific behaviors of their animal "family" such as walking on all fours, barking/howling, eating raw meat/bones, sniffing food, etc. the specificity of adopted behaviors is hard to explain without some capacity for observational matching

@dwarkesh_sp wtf i'm sure there must be more to this because it seems real dumb

@dwarkesh_sp How does the child know what 'error' is unless they have something to compare it to (ie the thing they trying to 'imitate'?)

@dwarkesh_sp Despite the evidence...

@dwarkesh_sp RL learning gives the agent in most situation too few information to adapt its behavior. After decades of unsuccessful RL research he should accept that humans are intelligent because they have language which children do not learn by pure rewards or cost signalsl

@dwarkesh_sp Sutton is, in many respects, the godfather of RSI in the sense that Hinton is a godfather of transformers.

@dwarkesh_sp 👇

https://www.reddit.com/r/Cervantes_AI/comments/1u3yc2t/the_quiet_collapse_of_men/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button