Knowledge discovery in science and engineering has been driven by theory, experiments and more recently by large-scale simulations using high-performance computers. Modern experiments and simulations involving satellites, telescopes, high-throughput instruments, imaging devices, sensor networks, accelerators, and supercomputers yield massive amounts of data. At the same time, the world, including social communities is creating massive amounts of data at an astonishing pace. Just consider Facebook, Google, Articles, Papers, Images, Videos and others. But, even more complex is the network that connects the creators of data. There is knowledge to be discovered in both. This represents a significant and interesting challenge for HPC and opens opportunities for accelerating knowledge discovery.

In this talk, followed by an introduction to high-end data mining and the basic knowledge discovery paradigm, we present the process, challenges and potential for this approach. We will present many cases, examples, results and future directions including (1) mining sentiments from massive datasets on the web, (2) Real-time stream mining of text from millions of posts and tweets to identify influencers and sentiments of people; (3) Discovering knowledge from massive social networks containing millions of nodes and hundreds of billions of edges from real world Facebook, twitter and other social network data (E.g., Can anyone follow Presidential campaigns in real-time?) and (4) Discovering knowledge from massive datasets from science applications including climate, medicine, biology and sensors. The talk will be illustrative and example driven and may include 1-2 live demonstrations.

We are entering an exciting era where information technology is being used in an increasingly instrumented world to make it run better. The inefficiencies in how the world operates today (e.g. in industries like transportation, energy and healthcare) have been estimated to be around $15 Trillion annually. We describe how some of these inefficiencies can be eliminated by applying analytics over data being collected in real time We introduce several unique challenges faced in emerging markets like India, and present examples of innovation in dealing with those challenges, such as handling a much larger scale, dealing with noisy data, and leveraging mobile devices for novel solutions. We describe areas of success as well as outstanding issues that will require further advances in computing and business model innovations to achieve truly transformational impact.

Most stencil applications such as CFD and structure analysis are memory-bound problems. GPU has high performances in both computation and memory bandwidth suitable for them. The TSUBAME 2.0 supercomputer, which is equipped with 4224 NVIDIA Tesla M2050 GPUs, has started the operation since November 2010 at the Tokyo Institute of Technology. For GPU computing, we have rewritten the entire code of a high resolution meso-scale atmosphere model ASUCA that is being developed by the Japan Meteorological Agency for the purpose of the next-generation weather forecasting service. Using 3996 GPUs on TSUBAME 2.0, we achieved extremely high performance of 145 TFLOPS in single precision for 14368x14284x48 mesh. We also show a metal dendritic solidification by solving the phase-field model. Recently, we have achieved 1.017 PFLOPS for 4096x6 500x10400 mesh on 4000 GPUs and the paper has been nominated as one of 5 Gordon Bell Award finalists in SC'11 conference in November 2011.