I will discuss recent results on metastable dynamics in cortical circuits, characterized by seemingly random switching among a finite number of discrete states. Single states and their metastable dynamics can reflect abstract features of external stimuli as well as internal deliberations, and have been proposed as supporting a role in a variety of functions including sensory coding, expectation, decision making and behavioral accuracy. Many results in this context have been captured by spiking network models with a clustered architecture. I will review data and models while trying to provide a model-inspired unitary view of the phenomena discussed. If time permits, I will present a model of how this type of dynamics can emerge from (and coexist with) experience-dependent plasticity in a network of spiking neurons.