A bar-joint framework in d-dimensional space is rigid if it cannot be deformed without changing at least one of its edge's length. A rigid bar-joint framework can be incarnated into a structure using physical materials. Sometimes the physical structure, though technically rigid, will still be easy to deform it by adding very little energy to the structure. To this end we define a new energy-based notion of the ``order of rigidity'' a structure. We explore both bar-energies, where each member is at its prefered length and pre-stressed energies, where the structure contains members in tension and compression. We present universality results that explain that most of the material properties are irrelevant for this order. With no-prestress, the order only depends on the geometry. With a prestress the order only depends on the geometry and the stress-coefficients. Next we compare how rigidity orders change when going between un-stressed and prestressed structures. In nice cases, the addition of a prestress will halve the rigidity order. In nice cases, these orders can be efficiently computed. Joint with He, Holmes-Cerfon and Theran