Langevin diffusions are continuous-time stochastic processes that are based on the gradient of a potential function. As such they have many connections---some known and many still to be explored---to gradient-based machine learning. I'll discuss several recent results in this vein: (1) the use of Langevin-based algorithms in bandit problems; (2) the acceleration of Langevin diffusions; (3) how to use Langevin Monte Carlo without making smoothness assumptions. I'll present these results in the context of a general argument about the virtues of continuous-time perspectives in the analyis of discrete-time optimization and Monte Carlo algorithms.