Shape optimization plays a central role in engineering and biological design. However, numerical optimization of complex systems that involve coupling of fluid mechanics to rigid or flexible bodies can be prohibitively expensive (to implement and/or run). A great deal of insight can often be gained by optimizing a reduced model such as Reynolds' lubrication approximation, but optimization within such a model can sometimes lead to geometric singularities that drive the solution out of its realm of validity. We present new rigorous error estimates for Reynolds' approximation and its higher order corrections that reveal how the validity of these reduced models depend on the geometry. We use this insight to study the problem of shape optimization of a sheet swimming over a thin layer of viscous fluid.