Finite sample properties of random covariance-type matrices have been the subject of much research. In this paper we focus on the "lower tail" of such a matrix, and prove that it is subgaussian under a simple fourth moment assumption on the one-dimensional marginals of the random vectors. A similar result holds for more general sums of random positive semidefinite matrices and the (relatively simple) proof uses a variant of the so-called PAC-Bayesian method for bounding empirical processes. Some applications of the main result to high dimensional problems will be briefly sketched.