Cloud, HPC And Open Technologies Converge To Fuel Research, Innovation

Filed in Cloud Industry Insights by Paul Rad | September 4, 2013 12:00 pm

With a mission-based focus on scientific research, universities and research institutes have long been at the forefront of technology innovation that embraces open standards and ease of collaboration. By nature, the academy well understands that when hundreds of researchers contribute to a shared purpose and solve a shared problem in open and transparent ways, everyone benefits. The pace of innovation is accelerated and the diversity of solutions and approaches ensures that good solutions persist and not so good ones are quickly identified. Some might argue that the relative success of open-sourced platforms suggests that proprietary technologies often preclude necessary innovation. The growth of cloud has further challenged both researchers and industry. The scale economies of cloud-based solutions brings with it attendant challenges for the research community – beginning with the question of how to scale a conversation that brings both sides of an open community together to ensure that the full benefits of the cloud can be realized.

Recognizing the potential role of cloud-based open computing technologies to the research community, a group of 30 key stakeholders and decision-makers from academia and industry got together last week to share their views on how open computing solutions can best support existing and emerging use cases in a range of research disciplines and high performance workloads. The event was hosted by Argonne National Labs and jointly sponsored by Notre Dame, Internet2 and Rackspace; with attendees representing 20 organizations including participants from top research institutes, major research universities and key industry partners that provide technology supporting the research community.

The group discussed how big data and high-performance computing can introduce new challenges and new frustrations. Say you’re an academic or researcher who needs time on a supercomputer; oftentimes you’ll have to wait months to get approved, and even then you likely only get a limited window. So if something with your software is not working at that time, you’re out of luck.

The cloud changes the computing equation and redefines the experience and service by adding on-demand, utility and self-service capabilities to computing infrastructure. The cloud is quickly evolving into a premium model for scientific computation and big data, and the face of high-performance computing is changing faster than ever. We’ve seen the change happen over the past couple of years, as open technologies like the Open Compute Project[1] and OpenStack, in particular, democratize access to mass commodity hardware and software.  Now, top research institutes such as CERN[2], Argonne National Laboratory[3], Notre Dame, University of Texas at San Antonio[4] and MIT[5] have chosen to build their high-performance clouds on OpenStack. By embracing open standards and collaboration, university researchers are at the forefront of innovation and contribute to a shared purpose that benefits everyone.

The event organizers – Narayan Desai (Argonne National Lab) , Paul Brenner (Notre Dame University), Khalil Yazdi (Internet2)  and Paul Rad (Rackspace) – welcomed participants and laid out a vision and an agenda for the workshop. Here are some of the key points that emerged during discussions in the morning:

[6]

Click image to enlarge

In the afternoon sessions, as part of a “Cloud Best Practices” discussion, Notre Dame’s Brenner, Argonne’s Desai and UTSA’s Raj Boppana delivered in-depth presentations on a wide range of research and university OpenStack use cases for High Performance Cloud.

They shared lessons learned and presented the findings and gaps that point the way forward for compute- and data-intensive applications.

Internet2 CTO Stephen Wolff said, “Workshops like the one held at ANL play an important role in bringing together stakeholders to explore technology trends and to cooperatively accelerate their development.  Internet2 is excited to be involved in this cutting edge work.

At the end of the session, the community identified two immediate incubation projects (with several other possible projects noted):

The teams are planning to develop blue prints, detailed service descriptions and plans for a continuing collaborative effort and identifying regular communication channels for these projects. They will likely get back together at supercomputing 2013[7] and WCSC 2013[8] in San Antonio Texas.

Endnotes:
  1. Open Compute Project: http://www.rackspace.com/blog/open-compute-gains-more-awareness-at-openstack-summit/
  2. CERN: http://www.rackspace.com/blog/were-collaborating-with-cern-openlab-for-hybrid-cloud-powered-research/
  3. Argonne National Laboratory: http://www.google.com/url?sa=t&rct=j&q=&esrc=s&frm=1&source=web&cd=1&cad=rja&ved=0CCoQFjAA&url=http%3A%2F%2Fwww.youtube.com%2Fuser%2FArgonneNationalLab&ei=yHQfUuClJMmtsQSGxIGABw&usg=AFQjCNEDS8muaMU-6_L0VSDwiAN2VVkblg&sig2=1SXefYhX9GhD2NeKXN61GA
  4. University of Texas at San Antonio: http://www.rackspace.com/blog/utsa-uses-open-hybrid-cloud-for-research-and-academic/
  5. MIT: http://www.rackspace.com/blog/rackspace-training-for-openstack-heads-to-mit/
  6. [Image]: http://ddf912383141a8d7bbe4-e053e711fc85de3290f121ef0f0e3a1f.r87.cf1.rackcdn.com/ANL-headshots-large.png
  7. supercomputing 2013: http://sc13.supercomputing.org/
  8. WCSC 2013: http://www.wacong.org/wcsc2013/

Source URL: http://www.rackspace.com/blog/cloud-hpc-and-open-technologies-converge-to-fuel-research-innovation/