We will consider the problem of distributed cooperative non-Bayesian learning in a network of agents, where the agents are repeatedly gaining partial information about an unknown random variable whose distribution is to be jointly estimated. The joint objective of the agent system is to globally agree on a hypothesis (distribution) that best describes the observed data by all agents in the network. Interactions between agents occur according to an unknown sequence of time-varying graphs. We highlight some interesting aspects of Bayesian learning and stochastic approximation approach for the case of a single agent, which has not been observed before and it allows for a new connection between optimization and statistical learning. Then, we discuss and analyze the general case where subsets of agents have conflicting hypothesis models, in the sense that the optimal solutions are different if the subset of agents were isolated. Additionally, we provide a new non-Bayesian learning protocol that converges an order of magnitude faster than the learning protocols currently available in the literature for arbitrary fixed undirected graphs. Our results establish consistency and a non-asymptotic, explicit, geometric convergence rate for the learning dynamics.