When humans and other animals assess that they have little control over outcomes, their motivation and capacity for learning change. However, how we first learn what the control level is and how it then influences learning remain unclear. Using functional magnetic resonance imaging (fMRI) and transcranial magnetic stimulation (TMS) in human participants, we provide both recording and transient disruption evidence for an anatomically grounded model of control learning, in which dorsomedial prefrontal cortex (dmPFC) tracks confidence in each individual decision in order to then estimate controllability and changes in controllability via interactions with the dorsal raphe nucleus. Disrupting activity in this circuit, via dmPFC manipulation, impaired control learning. In addition, another mechanism was identified through which control estimates influenced learning. Participants' estimates of controllability were associated with widespread changes in outcome signals, including in dopamine-associated midbrain nuclei and in credit-assignment-linked signals in a distinct prefrontal cortex subregion.