It is well-known that for any finite group $G$, there exists a closed $3$-manifold $M$ with $G$ as a quotient of the fundamental group of $M$. However, we can ask more detailed questions about the possible finite quotients of $3$-manifold groups, e.g. for $G$ and $H_1, \ldots ,H_n$ finite groups, does there exist a $3$-manifold group with $G$ as a quotient but no $H_i$ as a quotient?  We give an answer to all such questions, and explain how this is related to our work on the analogous problem of whether there exist number fields or function fields with an unramified $G$ extension but no unramified $H_i$ extension. This talk is on joint work with Will Sawin, and also includes joint work with Yuan Liu and David Zureick-Brown.