Inverse problems are often formulated as constrained optimization problems. The constraint is in the form of a "forward" problem. The forward model is repeatedly solved and iteratively updated in order to bring its predictions in conformity with a set of data, or observations. We identify two distinct classes of deficiencies present in standard forward solvers (i.e. variational formulations) when used in this way. These deficiencies has motivated the development of two new variational formulations to address them. We will present the Coupled Adjoint-State Equation formulation, (or CASE) formulation, which is suitable when the forward problem is itself ill defined. We will also present a set of Augmented Lagrangian Stabilized (ALS) variational formulations. The use of ALS stabilization ensures that the discretized inverse problem inherits the stability of the original continuous inverse problem. Indeed, in certain inverse problems, ALS can fill the role that is often accomplished through regularization. These two novel formulations show that having a stable and convergent forward problem solver is neither necessary nor sufficient to have good inverse problem solutions.