Many inverse problems may involve a large number of observations. Yet these observations are seldom equally informative; moreover, practical constraints on storage, communication, and computational costs may limit the number of observations that one wishes to employ. We introduce strategies for selecting subsets of the data that yield accurate approximations of the inverse solution. This goal can also be understood in terms of optimal experimental design. Our strategies exploit the structure of inverse problems in the Bayesian statistical setting, and are based on optimal low-rank approximations to the posterior distribution---extended to the observation space. This is joint work with Jayanth Jagalur-Mohan.