Traditional exact solvers and purely end-to-end machine learning (ML) models often face steep bottlenecks in scalability, data efficiency, and generalization. We present a unifying hybrid ML–algorithmic framework rooted in the Learning to Prune (LTP) paradigm. Instead of relying on black-box networks to output solutions directly, our framework leverages lightweight, highly interpretable ML models to predict and prune components by fixing variables, thereby isolating the difficult core of a problem instance to be processed by the state-of-the-art exact and heuristic solvers. We showcase the versatility and theoretical grounding of this framework across three distinct problem domains: k-median and Facility Location: We illustrate how quantities derived from classical approximation algorithms can serve as highly effective features for the k-median, facility location, and set cover problems. Remarkably, these features generalize across problem variants, allowing classifiers to identify the core of the problem while training on minimal data. Routing and Matheuristics: We extend LTP to complex electric vehicle routing problems with non-linear charging functions. We demonstrate how the framework operates as a variable sparsification tool integrated within an exact branch-and-bound matheuristic, even when exact training labels are unavailable. Independent Set and Vertex Cover Variants: We demonstrate how adapting the multiplicative weights method yields fast, high-quality surrogate features for problems like maximum independent set and minimum vertex cover variants. Collectively, these results highlight a powerful blueprint for the future of discrete algorithms: combining domain-specific algorithmic insights with data-driven pruning to achieve drastic runtime reductions while maintaining robust solution quality.