We attach Galois representations to automorphic representations on unitary groups whose weight (=component at infinity) is a holomorphic limit of discrete series. The main innovation is a new construction of congruences, using the Hasse Invariant, which avoids q-expansions and so is applicable in much greater generality than previous methods. Our result is a natural generalization of the classical Deligne-Serre Theorem on weight one modular forms and work of Taylor on GSp(4).