In this talk we will consider recent advances on the approximation of second order elliptic problems with interfaces that have poor, non-standard stability, or are ill-posed. Such problems arise in a multitude of applications for example in seismic inversion problems or the design of meta materials. As a model problem we will consider the classical ill-posed problem of unique continuation in a heterogeneous environment. First we will discuss primal-dual stabilized finite elements for the homogeneous case and recall recent results on the accuracy and optimality of such methods. Then we will show how the method can be modified to handle internal interfaces using an unfitted finite element method. We will report error estimates for this method and discuss how to handle the destabilizing effect of error in the geometrical data. Finally we will show how the ideas can be applied to so-called sign changing materials, where the coefficient of the diffusion operator is of different sign in different subdomain. The accurate approximation of wave propagation in such materials are important for the design of meta-materials.