Adaptive sampling, also called ``Active Learning'', uses information gleaned from previous measurements (e.g., feedback) to guide and focus the sampling process. Theoretical and experimental results have shown that adaptive sampling can dramatically outperform conventional non-adaptive sampling schemes. I will review some of the most encouraging theoretical results to date, and focus on new results regarding the capabilities of adaptive sampling methods for learning piecewise smooth functions. I will also contrast adaptive sampling with a new approach known as compressive sampling. Compressive sampling, or ``Compressed Sensing'', has generated a tremendous amount of excitement in the signal processing community and is seen as a strong competitor of adaptive procedures. Compressive sampling involves taking a relatively small number of non-traditional samples in the form of non-adaptive randomized projections that are capable of capturing the most salient information in a signal. I will compare adaptive and compressive sampling in noisy conditions, and show that in certain interesting cases both schemes are near-optimal. This result is remarkable since it is the first evidence of cases in which compressive sampling, which is non-adaptive, cannot be significantly outperformed by adaptive procedures, even in presence of noise. This is joint work with Rui Castro and Jarvis Haupt.