WEKA on AWS Customers
Win the AI Race with Optimized AWS Infrastructure
Accelerate Distributed Model Training
WEKA provides a high-performance data platform for SageMaker HyperPod optimized for every phase of FM model training across data loading, pre-processing, model training, checkpointing, verification, tuning, and data set archiving.
Increase AI Developer Productivity
WEKA software accelerates model data load times by 50%, accelerating model training times and improving data scientist and developer productivity.
Increase Cluster Resilience
WEKA reduces FM model checkpoint times by 90%, enabling faster training times and increasing the resilience of HyperPod deployments.
Increase GPU Cluster Utilization
High-performance storage from WEKA eliminates data bottlenecks driving up GPU infrastructure utilization above 90% (from an avg 30%) and ensuring your HyperPod infrastructure is never starved for training data.
Reduce epoch times from
Weeks to Hours
Amazon SageMaker HyperPod
With WEKA support for Amazon SageMaker HyperPod, customers can build a high-performance data platform for distributed model training that scales massively, increases GPU infrastructure utilization, and reduces infrastructure costs.
Use AWS for more of your workloads
Accelerate AI and HPC applications on AWS
Deliver performance for your most demanding applications running in AWS with the world’s fastest cloud native data platform supporting high I/O, low latency, small files, and mixed workloads with zero tuning.
Seamlessly tier data in AWS
Intelligent tiering automatically moves data between high performance flash-based storage on Amazon EC2 instances to low cost, massively scalable object storage in Amazon S3, all in a single namespace for the best performance, scale, and economics.
Scale your data in AWS
Autoscaling enables you to add and remove high performance storage capacity on the fly to meet the needs of your most demanding applications without paying for resources you don’t use.
Move data to AWS
Send snapshots of a filesystem to any Amazon S3 object store for backup and disaster recovery. Full and incremental snapshots include metadata to enable seamless data portability between on-prem and AWS.
Burst data analysis to AWS
Maintain a usable copy of your on-premises data in your AWS environment, where you can use elastic compute resources to run calculations and analyses and gain new insights.
Build agile data pipelines in AWS
Your researchers, data scientists, creative teams, and more can collaborate faster by using a single copy of data optimized to meet the performance needs of every step in your workflow.
How it works
WEKA is deployed on Amazon EC2 I3en instances with local NVMe storage to form a high-performance storage layer. The software extends the namespace to an Amazon S3 bucket for large scale capacity and optimal cost. The entire data set is available to the applications without the need to move or copy data. The namespace can extend from Terabytes to Petabytes. The same data can be accessed by multiple protocols – S3, NFS, SMB, and POSIX.
WEKA is in the AWS Builder Studio Melbourne
The AWS Builder Studio allows you to get hands-on with AWS technology while learning our unique methodologies to invent, honing your own culture of experimentation, and discovering the ‘Art of the Possible’ for your organization. Visit WEKA at the AWS Builder’s Studio Melbourne!
Dive a little deeper
