Yahoo! Enables Cutting Edge Research Projects With Donation of Cloud Supercomputing Cluster

Posted December 9th, 2009 at 2:02 pm by Yahoo!, Blog Editors

Number of Comments 2 Comments / Filed in: Uncategorized

Today, Yahoo! is proud to announce the donation of a supercomputing cluster with 2,000-processor-cores to the Open Cloud Consortium (OCC), a cloud computing testbed for research on large-scale data clouds. Yahoo! is a founding member of the OCC, and its participants include a diverse group of universities and research institutions which develop standards, benchmarks, and interoperability frameworks for cloud computing.

At Yahoo!, massive-scale data-crunching is our bread and butter. We are the leader in use and development of Apache Hadoop and other open-source cloud computing technology to process many petabytes of data, billions of Web images and objects, and hundreds of thousands of Web requests per second. With cloud computing and the careful analysis of data, we are able to deliver highly relevant content and improve the quality of online experiences across the Yahoo! network – from the homepage to Yahoo! Sports, Yahoo! Finance, Yahoo! Mail, and more.

Academic researchers exploring everything from astronomy to genomics have the same need to analyze massive amounts of data. Yahoo!’s cloud is intended to free up engineering resources to focus more on innovation by creating highly scalable applications. We hope that our technologies can be used in a similar way by research institutions, so that scientists can sift through massive amounts of data in a smart, scalable way and focus their energy on finding real breakthroughs. The Yahoo!-donated cluster will be part of the OCC’s Open Science Data Cloud. It will be used by a variety of scientists, including world-renowned researchers such as Alex Szalay at Johns Hopkins University to study the birth of the universe and Kevin White at Chicago’s Institute for Genomics and Systems Biology to study the structure of the human genome.

This donation will enable OCC to scale their benchmarking and performance analysis efforts, significantly enhancing their capabilities to improve systems software for cloud computing. The OCC plans to distribute the processing power between data centers at the University of Ilinois at Chicago and Johns Hopkins University. The data centers are connected by a very high speed (10 Gigabit per second) network link, enabling researchers to test distributed virtualization technologies.

Our participation in the OCC is a continuation of the cloud computing research projects we began supporting a few years ago. Top research universities such as Carnegie Mellon, the University of California at Berkeley, Cornell University, and the University of Massachusetts at Amherst use Hadoop on our M45 supercomputer for a broad range of computer science research, and are progressing with some ground breaking studies. Similarly, we are a founding sponsor of the Open Cirrus™ Cloud Computing Testbed, along with HP and Intel, which is advancing cloud computing research on an international scale.

We’re excited to play an important role in this interesting intersection between technology and academic research, and we look forward to working with our counterparts as part of the Open Cloud Consortium.

- Shelton Shugar, senior vice president, cloud computing, Yahoo!.