Focusing on relevant contents to guide adaptive behavior is a core property of the brain. For decades, scientists have investigated mechanisms to anticipate, select, prioritize, and prepare sensory signals according to goals, memories, and salient events. More recently, researchers have considered how these attention functions operate within internal representations. However, neither external nor internal attention in isolation captures everyday behavior. The brain frequently and seamlessly shifts between contents from the sensory stream and those held in mind. In this perspective, we ask how the brain shifts between external and internal attention. We describe similarities and differences between selective external and internal attention, present competing hypotheses regarding the operating principles of between-domain shifts, and highlight putative brain areas and mechanisms. We discuss the scarce experimental forays comparing attention shifts between vs. within domains and contemplate how these constrain theoretical and computational models. We conclude by suggesting open questions to guide investigation.