Structural break time series models, which are commonly used in macroeconomics and finance, capture unknown structural changes by allowing for abrupt changes to model parameters. However, most specifications suffer from an over-parametrization issue, since all parameters have to change when a break occurs. We introduce a sparse change-point model to detect which parameters change over time. We propose a shrinkage prior distribution, which controls model parsimony by limiting the number of parameters that change from one structural break to another. We develop a Bayesian sampler for inference on the sparse change-point model. An extensive simulation study based on AR, ARMA and GARCH processes highlights the excellent performance of the sampler. We provide three empirical applications. First, we revisit the US three-month Treasury bill time series to emphasize that many of the detected breaks are due to a change in only a subset of the model parameters. A second application consists of studying often used macroeconomic time series for breaks in parameters. Finally, we summarize GARCH parameter breaks detected in financial returns for companies in the S&P 500 index.