We develop a probabilistic framework to model connectivity patterns in the brain as a latent network graph. In particular, we model the interaction between latent anatomical and functional connectivity and present an intuitive extension to population studies. The method simultaneously infers the templates of latent connectivity for each population and the differences in connectivity between the groups. We also develop an approach to identify foci of a neurological disorder based on anatomical and functional connectivity information. Specifically, we formulate a generative model that characterizes the network of abnormal functional connectivity emanating from the affected nodes. We demonstrate our methods on a schizophrenia study. Our model identifies significant increases in functional connectivity between the parietal/posterior cingulate region and the frontal lobe and reduced functional connectivity between the parietal/posterior cingulate region and the temporal lobe in schizophrenia.