Computational anatomy is an emerging discipline at the interface of geometry, statistics, image analysis and medicine that aims at analysing and modelling the biological variability of the organs shapes at the population level. The goal is to model the mean anatomy and its normal variation among a population and to discover morphological differences between normal and pathological populations. For instance, the analysis of population-wise structural brain changes with aging in Alzheimer's disease requires first the analysis of longitudinal morphological changes for a specific subject. This can be evaluated through the non-rigid registration. Second, To perform a longitudinal group-wise analysis, the subject-specific longitudinal trajectories need to be transported in a common reference (using some parallel transport). To reach this goal, one needs to design a consistent statistical framework on manifolds and Lie groups. The geometric structure considered so far was that of metric and more specially Riemannian geometry. Roughly speaking, the main steps are to redefine the mean as the minimizer of an intrinsic quantity: the Riemannian squared distance to the data points. When the Fréchet mean is determined, one can pull back the distribution on the tangent space at the mean to define higher order moments like the covariance matrix. In the context of medical shape analysis, the powerful framework of Riemannian (right) invariant metric on groups of diffeomorphisms (aka LDDMM) has often been investigated for such analyses in computational anatomy. In parallel, efficient image registration methods and discrete parallel transport methods based on diffeomorphisms parameterized by stationary velocity fields (SVF) (DARTEL, log-demons, Schild's ladder etc) have been developed with a great success from the practical point of view but with less theoretical support. In this talk, I will detail the Riemannian framework for geometric statistics and partially extend if to affine connection spaces and more particularly to Lie groups provided with the canonical Cartan-Schouten connection (a non-metric connection). In finite dimension, this provides strong theoretical bases for the use of one-parameter subgroups. The generalization to infinite dimensions would grounds the SVF-framework. From the practical point of view, we show that it leads to quite simple and very efficient models of atrophy of the brain in Alzheimer's disease. Learning what are the topological invariants of these noisy deformations fields is now the next step where computational topology has a role to play.