Leveraging the Benefits of Hybrid HPC Computing Sponsored Content by Penguin Computing
Hybrid HPC computing combines public and on-premise compute resources to offer organizations a flexible, cost effective approach to meeting their users' requirements. To fully augment on-premise HPC systems requires cloud platforms that can scale-out as bare-metal environments. While moving all compute workloads from on-premise to cloud and back may be problematic for some, dividing workflows into on-premise “core” compute while using cloud resources for “project” workloads can be helpful. Leveraging an HPC cloud can provide organizations with a variety of benefits, ranging from economic ones to offering users advanced technologies or temporarily increased capacity. An HPC cloud as part of your infrastructure might also make sense even if the economics don’t.
The Value of a Hybrid Approach
HPC clouds provide access to newer technologies than a traditional three year hardware refresh cycle would. Additionally, the right cloud provider can grant access to HPC expertise that will augment your existing IT team. To make the most out of hybrid computing, balancing the size of your on-premise systems to manage “core” workloads while using an on-demand HPC cloud to run schedule-sensitive “project” workloads can be the most economically optimized approach.
The Importance of Singularity and Virtual Workstations
Two key technologies for hybrid HPC computing are Singularity and virtual workstations.
Singularity is an HPC Linux container solution that can be used to package entire workflows, software and libraries, and even data into a portable file. This enables users to encapsulate workloads into a consistent and reproducible format that can be moved to and from the cloud. To provide a consistent user experience between on-premise and HPC cloud environments requires exporting a Linux or Windows desktop from the cloud to the user. A consistent configuration of the exported desktop coupled with 3D graphics acceleration to support the graphics intensive applications typically used in the pre and post process of HPC data provides a seamless experience for the user.
The Economics
Comparing the cost of on-premise and HPC cloud computing involves a number of factors - but one of the most significant is utilization. Utilization is the percent of time a compute resource (core, processor, server) is in use by the application. Cost-effective HPC clouds only charge for the core-seconds that an application uses, and the cost to store user data. On-premise costs include the capital expense of the cluster, network, and storage divided by the utilization plus operating costs (IT staff, power, cooling, facility). Designing an on-premise cluster to meet peak demands without incurring application wait times for the users can mean that utilization will fall - often to the point where an HPC cloud will be more cost-effective. Balancing core compute requirements and flexible project-based compute requirements leads to an optimal solution.
Why Hybrid HPC Computing Makes Sense When the Economics Do not
HPC clouds can cost more than on-premise systems, but they may offer benefits that justify the increased cost. These benefits include immediate access to current technologies, minimizing time to deployment, dynamically increasing the capabilities of limited HPC IT resources, bridging a gap to a upcoming technology refresh, and mitigating restraints imposed by CapEx limits. The most common reason that many organizations prefer HPC cloud solutions is that a dynamic, HPC-as-a-Service environment removes the burden of supplying an HPC cluster and storage solution with internal resources.
Register for Penguin Computing’s upcoming whitepaper on leveraging the benefits of hybrid HPC computing.
