Autism and ADHD are characterised by atypical sensory responsivity, and this may be driven by alterations in the balance of cortical excitation to inhibition (E/I). Studies early in development are required to establish when sensory responsivity differences emerge and whether they predict later neurodevelopmental condition outcomes. We utilised data from a prospective longitudinal cohort of infants with and without a family history (FH) of autism and/or ADHD (N = 151; 55% male, 83% white). We extracted electroencephalography (EEG) metrics of E/I balance at 5, 10 and 14 months; the aperiodic exponent of the slope of the power spectrum ('1/f'). Models estimated latent growth curves of parent-reported hyper and hypo-responsivity between 10 - 36 months. Analyses tested associations between developmental trajectories of FH, sensory responsivity, parent-rated neurodevelopmental traits at 3 years and E/I balance. We coded and entered binary variables indexing FH-autism and FH-ADHD in the same model, which allowed us to test for effects of one form of FH whilst adjusting for the impact of the other. Results showed that FH-autism was associated with greater increases in parent-reported hyper-responsivity between 10-36 months (over and above the effects of FH-ADHD), and in univariate models for hyper-responsivity only, the intercept and the slope of hyper-responsivity were positively associated with both autistic and ADHD traits at age 3 years. However, in joint models which included hypo-responsivity, associations between hyper-responsivity and autistic and ADHD traits became non-significant. In these joint models, FH-ADHD was associated with steeper increases in hypo-responsivity (over and above the effects of FH-autism). Higher hypo-responsivity at 10-month baseline was associated with both autistic and ADHD traits at 3 years. A steeper slope of hypo-responsivity predicted ADHD traits at 3 years. Males displayed higher baseline hypo-responsivity. Aperiodic exponent values at 5 and 10, but not 14 months, were associated with hyper-responsivity. Results suggest a dissociation in the type of sensory responsivity associated with a family history of autism as compared to a family of ADHD, and that hypo-responsivity in infancy may be an indicator of later autism and ADHD outcomes. However, better measurement of each domain is required to draw strong conclusions as many hypo-responsivity items overlapped with autistic and ADHD traits. Alterations in E/I balance may contribute to early differences in sensory responsivity but further research is required to determine the directionality of effects.