Earth System Models are large, complex, and continually evolving. How do we know the model's results can still be trusted after a minor change, such as a new hardware platform, a compiler update, or a refactored subroutine? Earth System Models are valuable for understanding past and present climates and are vital for addressing otherwise intractable questions about future climate scenarios. Because the simulation output may affect societal responses to a changing climate, maintaining model confidence and reliability is critical for institutions such as the National Center for Atmospheric Research, which leads the development of the popular Community Earth System Model (CESM). CESM models the Earth system by simulating the major Earth system components (e.g., atmosphere, ocean, land, river, ice, etc.) and the interactions between them. These processes result in a complex multi-physics model that is inherently chaotic, where even seemingly inconsequential factors, such as changes to the hardware or software stack, can cause large effects. This sensitivity, compounded by a vast, complex, and continually evolving code base, means that ensuring quality assurance is non-trivial, as requiring bitwise identical results is not feasible. This talk will overview our multi-year effort to evaluate "correctness" in CESM, including the challenges encountered and the effectiveness in practice of an ensemble-based testing approach that casts correctness in terms of statistical distinguishability.