Optimal configurations can typically be realized as minimizers of some cost function. While in general the function can have many local minima, making it challenging to search for minimizers numerically, some cost functions which have natural interpretations in geometric invariant theory and symplectic geometry are surprisingly simple to optimize despite being non-convex. The goal in this talk is to explain some of the geometric context, and then to illustrate this approach with some applications, including to equal-norm Parseval frames, tight fusion frames, and normal matrices. This is joint work with Tom Needham and partially with Dustin Mixon and Soledad Villar.