Last June, we launched the fourth generation of the WEKA® Data Platform, which among other things, added native support for Google Cloud. Since then, we’ve been excited to see how customers are using WEKA in Google Cloud to help enable some of their most demanding and performance-intensive workloads, or what we like to call  “impossible workloads”. From the outset, customers have been looking for simpler ways to move and run these workloads in the cloud to untether and fast-track innovation.

So today, we’re excited to announce that customers can easily now deploy the WEKA platform directly from Google Cloud Marketplace. Think of it like an “easy button” for getting started with WEKA in Google Cloud.

To celebrate this milestone, we wanted to share a few ways Google Cloud customers are benefitting from WEKA’s combination of cloud-level scalability and simplicity, with on-premises performance to run their ‘impossible workloads’ in the cloud.

Accelerating Generative AI

The industry’s eyes are now wide open to AI’s potential for making applications and services better, faster, and smarter. In just the past few weeks alone, Google Cloud has unleashed a bevy of innovations to help customers capitalize on AI, like adding generative AI capabilities to their cybersecurity suite, and much more on the horizon. The large language models (LLMs) behind these next-generation AI offerings require larger and larger datasets for training, driving the need for flexible, scalable data management and storage. At the same time, the need to bring to market newer, more accurate models necessitates faster GPU-based compute capabilities, low latency networks, and high I/O, high bandwidth data storage.

The WEKA Data Platform delivers the exabyte scale and high performance required to continuously feed and train LLMs being built by some of the most advanced generative AI companies in the world. For example, one leading generative AI company is building a service that generates images based on simple natural language prompts and plans to use the WEKA Data Platform running in Google Cloud to train their next-generation large language model.

Building a VFX Studio in the Cloud

Creative agencies are turning to the scale and agility of the cloud to accelerate project times, and enable collaboration between globally distributed artists. These organizations rely on a diverse set of specialized applications to generate award-winning videos, ads, feature films, shorts, and more. The cloud is enabling a rapid transformation in the business model agencies use to source creative talent, build amazing content, and bring it to market. The combination of performance, scale, and efficiency that WEKA provides in the cloud is enabling creative agencies to successfully transform their businesses using cloud infrastructure and is one reason why WEKA was just recognized with a NAB Show Product of the Year in the Cloud Computing and Storage category.

For example, when one leading creative studio began its journey to the cloud, it found existing cloud native storage offerings could not deliver the performance needed to render visual effects with the responsiveness artists expect. So critical workloads remained in the office, with the team shipping content stored in hard drives between offices in Los Angeles and New York to complete development projects. With WEKA running in Google Cloud, the agency now retains a single copy of data in their WEKA environment in Google Cloud. Artists access the master copy with the performance and responsiveness they are used to on-premises, even delivering rendering at 120 frames per second directly in the cloud. Because WEKA seamlessly scales up and down alongside the creative applications, it can now stand up projects much faster without overprovisioning storage, and respond to last-minute changes from clients. By the end of 2022, the agency had more than tripled the number of projects it can support within its existing footprint and rapidly respond to late changes.

Running Genomics Sequencing 10x Faster with Hybrid Cloud Agility

In life sciences, labs are taking advantage of the agility of the cloud and access to the latest high-performance networks and compute types to drive computational biology workflows for projects ranging from genomic sequencing to drug discovery. For example, whole genome processing is transforming how researchers accomplish a range of public health tasks like tracking foodborne illnesses, identifying viral outbreaks, and developing new cancer treatment therapies. The genomic processing pipeline involves multiple steps to capture the raw genome, compare it to a reference, identify variants, locate anomalies, and render visualizations of the analyzed genome for added insights. This is a computationally intensive process, requiring fast networks, accelerated computing, and highly responsive data storage – traditionally done on specialized computers in-house.

The agility of the cloud means researchers can now spin up a high-performance cluster and run sequencing analysis in just a few hours. However, as researchers take advantage of the latest GPUs and network enhancements from the cloud providers, they are finding the existing data management offerings can’t keep up, either in terms of scale or analytics performance. Consider this: a single human genome sequence generates 200 gigabytes of raw data. With increasing goals of the 100K Genome Project and even the promise of a 1 Million Genome Project on the horizon, genomic sequencing is already generating a flood of data – much of it with wide variations in file sizes and types for use in different stages of the analytics pipeline.

Researchers using WEKA are able to eliminate bottlenecks in the data pipeline to successfully build an entire genomic sequencing pipeline in the cloud. Using a single WEKA environment,  they are able to process millions of tiny files in a sequencing pipeline, stream 100s of TBs of data for the final genomic renderings, and store exabytes of capacity in an efficient and economical way.

Learn more about WEKA on Google Cloud Platform