Tensor networks and tensor decompositions enable efficient simulation of quantum systems. We describe advances in methods and software for these problems and their application to approximate modelling of quantum circuits and electronic structure in quantum chemistry. On the algorithms side, we propose schemes that use perturbative approximation and randomization to accelerate solution of quadratic optimization subproblems in common alternating optimization algorithms. On the software side, we introduce two Python libraries, Koala and AutoHOOT, which achieve distributed-memory parallelism via the Cyclops tensor library. Koala uses 2D tensor network states (projected entangled pair states) to approximately simulate quantum circuits or perform general time-evolution. AutoHOOT provides efficient high-order automatic differentiation for tensor optimization problems.