Wondering about AWS and HPC? We explain what HPC is and discuss the pros and cons of using AWS services with your HPC applications.

What is AWS HPC? High-performance computing aggregates computer power to process data and perform complex calculations at higher speeds than a singular computer could do. Amazon Web Services offers a suite of products to optimize the performance of the complex workloads used in HPC.

What Is High-Performance Computing?

In the earliest days of enterprise and government computing, high-powered devices were massive, often taking entire rooms or even entire floors. The evolution of processing hardware and software has continually reduced the size and complexity of high-performance machines. Supercomputers were large beasts of devices, often connected to several forms of storage media and accessible through work terminals located on premises.

This was the only way to access high-performance computing resources for a long time. The evolution of hardware, the internet, and cloud computing services have fundamentally reshaped how we think of high-performance computing.

Generally speaking, high-performance computing has a few core characteristics:

  • Remote, Clustered Devices: Rather than running centralized supercomputers, modern HPC uses decentralized collections of servers, called clusters, connected via cloud software. This approach allows administrators to parallel harness multiple systems’ storage and computational capabilities.
  • Specialized Hardware: Traditional supercomputers would use top-of-the-line but traditional hardware, including CPUs and RAM. Modern HPC clusters will often deploy systems built with purpose-built field programmable gate arrays and GPUs that can more readily handle parallel processing jobs with increased data throughput.
  • Remote Services: Traditional supercomputers would allow shared access to system resources through remote terminals—essentially, the user would log in to the terminal and execute commands on the supercomputer. Modern HPC setups will often provide even more control with a host of managed applications, services, and infrastructure to support users attempting to build complex systems.

Because of modern HPC systems’ speed, availability, and flexibility, enterprise and scientific users receive cost-effective and powerful computational resources, often without implementing and maintaining them independently.

What Are Some Use Cases for High-Performance Computing?

While HPC isn’t available for general use by consumers and may not even be of much use for small- or mid-sized businesses, it is powering some of the most critical and cutting-edge industrial processes and consumer development projects on the market.

Some of the uses for HPC architecture include the following:

  • Chemical Engineering: Many industrial operations involve storing, transporting, and mixing chemicals and other liquids. Even the most negligible inefficiency can have a huge impact on the costs and effectiveness of chemical processes. HPC platforms are often used to power intelligent systems as part of Computational Fluid Dynamics.
  • Genomic Sequencing: The human genome’s scanning, cataloging, and sequencing is incredibly complex and takes massive processing power (and time) to complete. Advanced in cloud HPC is helping medical researchers better understand challenging diseases and hereditary conditions.
  • Predictive Analytics: HPC platforms serve enterprise users in retail, insurance provision and financial services model trends and patterns in petabytes of data to make informed decisions based on amazing insights. These analytics can drive business operations, manage risk or help executives prepare for different potential organizational outcomes.
  • Ecological Simulations: Mapping geological changes, weather patterns and seismic events operate much like powerful analytics–taking massive amounts of data and processing it through HPC clusters to derive insights about future behaviors of complex systems.

What Are the Pros and Cons of AWS HPC?

Amazon Web Services, or AWS, are Amazon’s suite of HPC cloud infrastructure. One of its major selling points is that it provides several cloud features and partner integrations to support enterprise clients who want to build apps and services of their own without deploying their infrastructure.

Like most things Amazon, AWS HPC has quickly taken the lead in terms of public awareness around cloud infrastructure. They make it relatively easy to deploy containerized operating systems, applications or storage to support Software-as-a-Service offerings with a clear understanding of pricing, scale and features.

Some of the benefits of using AWS HPC include the following:

  • Ease of Use: One of AWS’s more important steps toward opening up HPC to business users is removing barriers between these organizations and the platform. Amazon has emphasized making onboarding easier for organizations, providing easy-to-understand pricing and maintaining a comprehensive support documents and videos library.
  • Scalability: AWS is mainly a public cloud (with private and hybrid options) which means that resources, operating system types and virtual machines are readily implementable and scalable.
  • Flexibility: AWS includes leveraging various machines, supporting Windows and Linux systems, various kinds of cloud applications and programming language environments.

WEKA: Specialized HPC Cloud for Heavy-Duty Workloads

For organizations needing specialized, high-performance hybrid cloud infrastructure for the most challenging workloads out there, consider WEKA built on AWS hybrid cloud infrastructure. WEKA provides the hardware, software and expertise to empower researchers and engineers to build the most complex systems in health services, genomics, life sciences, machine learning and analytics.

With WEKA, you get the following features:

  • Streamlined and fast cloud file systems to combine multiple sources into a single high-performance computing system
  • Industry-best, GPUDirect performance (113 Gbps for a single DGX-2 and 162 Gbps for a single DGX A100)
  • In-flight and at-rest encryption for governance, risk, and compliance requirements
  • Agile access and management for edge, core, and cloud development
  • Scalability up to exabytes of storage across billions of files

Contact the WEKA team if you want an HPC platform that can fit your exact needs.

Additional Resources