Videos

Extreme Data Management Analysis and Visualization for Climate Data Exploration

Presenter
November 19, 2013
Keywords:
  • Data smoothing
MSC:
  • 93E14
Abstract
Effective use of data management techniques for massive climate models is a crucial ingredient for the success of modern environmental research. Developing such techniques involves a number of major challenges such as the real-time management of massive data, or the quantitative analysis of scientific features of unprecedented complexity. The Center for Extreme Data Management Analysis and Visualization (CEDMAV) addresses these challenges with and interdisciplinary research in diverse topics including the mathematical foundations of data representations, the design of robust, efficient algorithms, and the integration with relevant applications. In this talk, I will discuss one approach developed for dealing with massive amount of information via a framework for processing large scale scientific data with high performance selective queries on multiple terabytes of raw data. The combination of this data model with progressive streaming techniques allows achieving interactive processing rates on a variety of computing devices ranging from handheld devices like an iPhone, to simple workstations, to the I/O of parallel supercomputers. With this framework we demonstrated how one can enable the real time streaming of massive simulations from DOE platforms such as Hopper2 at LBNL and Intrepid at ANL. I will also present the application of a discrete topological framework for the representation and analysis of the same large scale scientific data. Due to the combinatorial nature of this framework, we can implement the core constructs of Morse theory without the approximations and instabilities of classical numerical techniques. The inherent robustness of the combinatorial algorithms allows us to address the high complexity of the feature extraction problem for high resolution scientific data such as eddies ocean simulations. During the talk, I will provide a live demonstration of the effectiveness of some software tools developed in CEDMAV and discuss the deployment strategies in an increasing heterogeneous computing environment.