Attention and memory may be impaired in individuals at-risk for Alzheimer's disease (AD), though standard cognitive assessments typically study the two in parallel. In reality, attention and memory interact to facilitate information processing, and thus a more integrative approach is required. Here, we used a novel auditory paradigm to assess how long-term memory for auditory scenes facilitates detection of an auditory target in asymptomatic carriers of Apolipoprotein E4 (APOE4), the principle risk gene for late-onset AD. We tested 60 healthy middle-aged adults with varying doses of APOE4 - 20 APOE4 homozygotes (E4/E4), 20 heterozygotes (E3/E4) and 20 non-carriers (E3/E3) - to determine effect on memory-guided attention. While explicit memory was unaffected by genotype, APOE4 dose significantly impaired memory-guided attention. A relationship between explicit memory and memory-guided attention was observed in non-carriers, but this correlation was not significant in E3/E4 and E4/E4 carriers, suggesting that APOE4 carriers rely less on explicit memory to facilitate attention. Since memory-guided attention declined with age in APOE4 homozygotes, this impairment may reflect early disease rather than being a life-long trait. In sum, asymptomatic individuals at increased genetic risk of AD show an age-dependent decline in attention-memory interaction when memory alone is not impaired.