In this talk we consider problems of parameter identification for models defined by dynamical systems with structured uncertainty. This type of problem often occurs in science and engineering and we show how parameter estimation can be improved by taking advantage of the specific structure of the model form and by using physics to define constraints on the corresponding optimization problem. The results are applicable to problems where, in the process of modeling a complex system, one ignores “small” parameters to obtain simplified models and when infinite dimensional systems are approximated to obtain finite dimensional models. We show for these common sources of model discrepancies, hierarchical modeling can be employed to aid in the development of prior knowledge about the model form caused by uncertain or ignored parameters. We focus on a special class of problems arising from parameter estimation and illustrate how one can use information about the structure of the dynamical system to help deal with model discrepancy. Examples from population dynamics and thermal fluid systems are given to illustrate the ideas.