Contrasting the widespread application of data science methods and ever increasing volumes of data, human supervision capacities remain limited. Thus, the efficient allocation of limited resources is required, for example by selection of data for inspection, annotation, or processing. In this talk, we study active sampling approaches, which provide techniques for determining and querying the (expectedly) most valuable information. We review common active sampling approaches, and demonstrate the use of decision-theoretic and probabilistic approaches for this problem. We then discuss the close relationship to questions of uncertainty quantification, performance estimation, and control of a predictor's learning process.