Extensive evidence supports the role of the immune system in modulating brain function and behaviour. However, past studies have revealed striking heterogeneity in behavioural phenotypes produced from immune system dysfunction. Using magnetic resonance imaging, we studied the neuroanatomical differences among 11 distinct genetically modified mouse lines (n = 371), each deficient in a different element of the immune system. We found a significant and heterogeneous effect of immune dysfunction on the brains of both male and female mice. However, by imaging the whole brain and using Bayesian hierarchical modelling, we were able to identify patterns within the heterogeneous phenotype. Certain structures-such as the corpus callosum, midbrain, and thalamus-were more likely to be affected by immune dysfunction. A notable brain-behaviour relationship was identified with neuroanatomy endophenotypes across mouse models clustering according to anxiety-like behaviour phenotypes reported in literature, such as altered volume in brains regions associated with promoting fear response (e.g., the lateral septum and cerebellum). Interestingly, genes with preferential spatial expression in the most commonly affected regions are also associated with multiple sclerosis and other immune-mediated diseases. In total, our data suggest that the immune system modulates anxiety behaviour through well-established brain networks.