Kudla and Millson proved in the 80's that the generating series of special cycles in orthogonal and unitary Shimura varieties are modular forms and a well-established conjecture of Kudla asks about extensions to toroidal compactifications. In this talk, I will answer Kudla's conjecture for divisors in O(p,2) and cycles in U(n,1) up to the middle degree, in cohomology. Joint work with François Greer and Philip Engel.