This paper presents a new framework for the analysis of anatomical connectivity derived from diffusion tensor MRI. The framework has been applied to estimate whole brain structural networks using diffusion data from 174 adult subjects. In the proposed approach, each brain is first segmented into 83 anatomical regions via label propagation of multiple atlases and subsequent decision fusion. For each pair of anatomical regions the probability of connection and its strength is then estimated using a modified version of probabilistic tractography. The resulting brain networks have been classified according to age and gender using non-linear support vector machines with GentleBoost feature extraction. Classification performance was tested using a leave-one-out approach and the mean accuracy obtained was 85.4%.