Statistical mechanics models undergoing a phase transition often exhibit rich, fractal-like behaviour at their critical points, which are described in part by critical exponents, the indices governing the power-law growth or decay of various quantities of interest. These exponents are expected to depend on the dimension but not on the microscopic details of the model such as the choice of lattice. After much progress over the last 30 years, we now understand two-dimensional and high-dimensional models rather well, but intermediate dimensions such as three remain mysterious. I will discuss these issues in the context of long-range and hierarchical percolation, and in particular how we can now compute some critical exponents for the hierarchical model in all dimensions.