We introduce Quantum Utility, an approach to quantum performance evaluation that aims to capture the user experience by considering overhead costs associated with the quantum computation. We define Milestones, which may be thought of as demonstrations of quantum utility in restricted contexts associated with subsets of overheads. We evaluate performance of a D-Wave Advantage system with respect to three Milestones, and discuss these results in the context of technological progress of annealing based QPUs.