Stephen Wright University of Wisconsin-Madison Computer Science Optimization techniques have become vital tools in machine learning and data analysis. We review here some fundamentals of relevant optimization formulations and techniques, with a focus on the most widely used methods. The discussion will include stochastic gradient methods, coordinate descent methods, methods for nonsmooth-regularized problems, and augmented Lagrangian methods including ADMM.