Wasserstein gradient flows often arise from mean-field interactions among exchangeable particles. In many interesting applications however, the ``particles’’ are edge weights in a graph whose vertex labels are exchangeable but not the edges themselves. We investigate the question of optimization of functions over this different class of symmetries. This leads us to gradient flows or curves of maximal slopes on the metric space of continuum graphs called graphons. We will also mention recent progress on limits of stochastic gradient descents on large dense graphs.