We are interested in the task of estimating an unknown function from data, given as a set of point evaluations. In this context, Gaussian process regression is often used as a Bayesian inference procedure, and we are interested in the convergence as the number of data points goes to infinity. Using results from scattered data approximation, we provide a convergence analysis of the method applied to a given, unknown function of interest. We are particularly interested in the case of non-stationary covariance kernels, and the extension of the results to deep Gaussian processes.