Stochastic differential equations (SDEs) are an important class of time-series models, used to describe systems evolving stochastically in continuous time. Simulating paths from these processes, particularly after conditioning on noisy observations of the latent path, however remains challenging. Existing methods often introduce bias through time-discretization, involve complicated rejection sampling schemes or are restricted to a narrow family of diffusions (Wang et al. (2020), constraining their applicability. In this work, we propose an exact Markov chain Monte Carlo (MCMC) sampling algorithm that broadens the applicability of Wang et al. (2020). Building on the Gibbs sampler framework from that paper, we now allow exact MCMC for diffusions belonging to the so-called classes EA2 and EA3. Our methodology is thus applicable to essentially any SDE with unit diffusion coefficient (and through a variance-stabilizing transform, to essentially any 1-d SDE). We demonstrate how our MCMC methodology allows us to order computations to need only fairly straightforward simulation steps. Our framework also allows tools from the Gaussian process literature to be straightforwardly applied. We evaluate our method on both synthetic and real datasets, demonstrating superior performance compared to a number of baselines.