In this lecture, Ernest Fokoue presents a general tour of some of the most commonly used kernel methods in statistical machine learning and data mining. He touches on elements of artificial neural networks and then highlight their intricate connections to some general purpose kernel methods like Gaussian process learning machines. He also resurrects the famous universal approximation theorem and will most likely ignite a [controversial] debate around the theme: could it be that [shallow] networks like radial basis function networks or Gaussian processes are all we need for well-behaved functions? Do we really need many hidden layers as the hype around Deep Neural Network architectures seem to suggest or should we heed Ockham’s principle of parsimony, namely “Entities should not be multiplied beyond necessity.” (“Entia non sunt multiplicanda praeter necessitatem.”) he spends the last 15 minutes of this lecture sharing his personal tips and suggestions with our precious postdoctoral fellows on how to make the most of their experience.