When a ring is Gorenstein, certain modules over it often exhibit nice duality properties. One such classic duality, due to Herzog, holds for the Koszul homologies of an ideal under certain conditions. We show that a slightly weaker version that holds for all ideals (still over a Gorenstein ring); this was proved in the graded case by Chardin via a different spectral sequence. In a different direction, we give a set of results in which duality holding forces Cohen-Macaulayness if a certain amount of local depth is present. The first requires the Koszul homologies to satisfy Serre's condition for index at least half the dimension of the ring. The second requires them to satisfy some sort of weakened sliding depth and a bound on the local number of generators of the ideal, yielding an extension of a result of Herzog, Vasconcelos, and Villareal. Each of these results is curious in that it appears to display the phenomenon that (in these cases) the heuristic from an old result of Hartshorne and Ogus for dual modules can be strengthened for the particular case of Koszul homology modules. The key to all of the results is a repeated careful analysis of a certain spectral sequence, hence also giving a new approach to the original sliding depth result. This is joint work with Hamid Rahmati and Janet Striuli.