In medical imaging, the true underlying property of interest is unknown. A single image provides little to no insight into the impact of confounding factors such as: statistical noise, biological variability, scattered radiation, patient motion, deadtime in detectors and electronics, detector resolution, etc. Some of these physical factors can be quantified by scanning various configurations of test objects, often called phantoms. Physical phantoms, however, can not capture variability due to patient physiology and offer only mean performance characteristics of a limited set of objects. Forward modeling of imaging systems provides an unparalleled window for examining and improving performance. For example, simulations using Monte Carlo photon tracking can be used to isolate a single factor of interest, for instance multiple i.i.d. realizations of the same imaging scenario can determine the effect of quantum noise or biological variability. Likewise, faster analytical models can be used to elucidate non-linear effects in the imaging chain. Accurate forward models also play a key role in improving iterative estimation of optimal images. Finally, forward models can be integrated into a feedback loop for optimization of a medical imaging tasks (not just images), that are not predictable a priori. We will give examples of the value of forward modeling in which an accurate model of the physics of the medical imaging system is an essential component to solving challenges in imaging research and healthcare.