What is dopamine doing in model-based reinforcement learning?

Akam T., Walton ME.

Experiments have implicated dopamine in model-based reinforcement learning (RL). These findings are unexpected as dopamine is thought to encode a reward prediction error (RPE), which is the key teaching signal in model-free RL. Here we examine two possible accounts for dopamine's involvement in model-based RL: the first that dopamine neurons carry a prediction error used to update a type of predictive state representation called a successor representation, the second that two well established aspects of dopaminergic activity, RPEs and surprise signals, can together explain dopamine's involvement in model-based RL.

DOI

10.1016/j.cobeha.2020.10.010

Type

Journal article

Journal

Curr Opin Behav Sci

Publication Date

04/2021

Volume

38

Pages

74 - 82

Permalink Original publication