The media and entertainment (M&E) sector has gone through a huge evolution over the last 10 years, with a shift from large studios with total in-house control of media assets to an outsourced model. In particular, the area of digital content creation and post-production has seen the fastest transition to a cloud model.

Media & Entertainment Market and Cloud Storage

The M&E space continues to see an exponential growth in data creation as we move to a visual world with more content consumed in video format. The challenge is further exacerbated by the development of 4K and 8K video resolution that drives much higher storage demands. Total M&E storage is anticipated to grow to 241 exabytes1 by 2025, 122 exabytes dedicated to non-archival storage. This massive growth in data generation means more demand on budget resources to meet the unending need for storage capacity. The storage challenge is exacerbated by the need to ever higher performance – which drives up cost – balanced with the need to improve storage costs per terabyte.

According to Coughlin and Associates2, cloud storage is growing by a factor of 12x between 2018 and 2024, while the broad market is growing by 2.2x, highlighting the rapid adoption of cloud. Post-production and visual effects comprise 28% of the growth while 52% is archival storage. Both of these categories are highly aligned with the cloud value proposition.

In addition to growing storage demands, the turnaround time on video production has shrunk by orders of magnitude, while viewers demand ever increasing high quality and resolution. Post production steps that improve color, audio, overlay music, and touch-up significantly impact the final product, and artistic touch-up is in high demand.

Cloud Computing in Media & Entertainment Market

The AWS public cloud has rapidly gained traction as a way to allow creative teams to quickly bring new products to market, and many studios are adopting a cloud-first strategy.
Several factors explain this rapid shift:

  • Creative artists have embraced the model of a distributed virtual team with total online collaboration long before the COVID effect. The cloud is an ideal platform for hosting collaborative workflows as artists can be on any continent or in any time-zone and have a consistent view of the project. This was unthinkable prior to the cloud evolution.
  • On-premises technology investment in compute rendering farms, high speed networks, and fast storage created a significant barrier to entry for independent studios, whichwas removed with the evolution of cloud.
  • The use of GPU-accelerated computing, in which only the most well-funded studios could invest, is now available in the cloud to every artist.
  • Cloud is on-demand, so artists have to pay only for what they use, and don’t have to cover the fixed cost of infrastructure when not in use.
  • The cloud scan scales infinitely. Compute resources can be scaled up or down as needed. Storage consumption is immediate: if a studio wants to take on a 500-terabyte project it can do so in a frictionless manner. It would take weeks or months to get a project implemented on premises starting from scratch.
  • Artists can bid on projects before they have to invest in the infrastructure to execute on the job. If they don’t win the contract, they are not out of pocket for equipment that will not get used.

Popular M&E Workflows and Use Cases in the Cloud

Animation
The industry has seen a shift away from 2D to 3D animation, and the workflow has increased in complexity along with this. A typical 3D animation workflow will involve creating the 3D model, texturing, rigging and skinning, animation, rendering, compositing and special effects, finishing with music overlay, editing, and final output. Each of these steps requires skilled artist input, and typically many animators collaborate on the same project to create the animation. The ability to share projects in a collaborative manner with simplicity is essential to meeting tight production deadlines.

Rendering and Special Effects
Rendering is the process of displaying computer code into a visual format. Data has to be coded quickly enough to display the images in an acceptable manner. For professional quality video the rendering process is done in advance so it can be viewed in real-time in high quality. Once rendered, final layering of details such as light and shading are added to enhance the images. These processes tend to be the most demanding computationally, and they take place at the end of the production cycle, so it is not unusual to have significant stress on this stage of production as deadlines need to be met. Studios are increasingly moving to GPU-based rendering, which can significantly speed up the rendering process. GPUs are expensive, and the cloud provides an excellent way to utilize this technology on demand.

Transcoding
A final step in the media workflow is transcoding from one digital encoding format to another so that it can be displayed on multiple devices. For movies, this requires translating all elements of the video at the same time – the file format, video, and audio. Different devices have different video file format demands, for example MOV or MPEG-4.

Cloud-based Media and Entertainment Storage Workflow Solution

The cloud is an ideal platform for independent artists and creative designers to collaborate in a virtual fashion; however, the process of getting to the cloud can be very daunting. Virtual workstations have to be created, applications loaded, storage assigned, and it all has to come together. With hundreds of compute instances and different storage options to choose from, even the most technical users can become overwhelmed.

Amazon has collaborated with vendors to create a cloud native “studio-in-the-cloud” managed service that provides workstations on-demand to allow artists to quickly and easily spin up a production environment, all without having to become expert on cloud deployment. Under the hood, AWS manages the physical infrastructure, artist tools, security, compliance, networking, and storage needs, delivering a real-time feel from a virtual workstation. Artists can collaborate on shared projects across the globe, from London to New York to Toronto to LA to India, without the friction of on-premises environments. The Cloud provides near limitless scale in compute and storage infrastructure on-demand based on a consumption model, so you pay only for the resources used.

Weka has teamed with Nimble Studio to offer WekaFS, the fastest POSIX compliant file system, on the AWS cloud. WekaFS delivers very high streaming bandwidth to the applications, while maintaining low latency across the storage cluster. The Weka native client delivers very high single-client streaming performance and can fully saturate the client’s network link.

Benefits of WekaFS on AWS for Media & Entertainment

WekaIO on AWS delivers the ideal cloud-based solution for M&E. To improve rendering efficiency and cost-effectiveness, Amazon Elastic Compute Cloud (Amazon EC2) compute instances and Amazon Simple Storage Service (Amazon S3) object storage provide on-demand resources that can scale up or down as dynamic rendering needs dictate. The WekaFS data platform leverages these dynamic resources to deliver the simplicity and sharing capabilities of NAS at significantly higher parallel performance.

Customers benefit from:

  • Scalability and mixed workload performance
  • Enhanced collaboration, productivity, and business agility
  • Flexibility in implementation

Accelerating 3D-Animation & VFX Workflows in the Cloud

WekaFS offers a multi-protocol POSIX-compliant file system with full file shareability and file locking across different operating systems. Windows, Mac, and Linux users can access and share a single project set allowing for easy collaboration among creative teams. Additionally, WekaFS has a rich feature set including snapshots, encryption, quotas, and much more. Its non-volatile memory express (NVMe) flash tier tightly integrates with Amazon Simple Storage Service (Amazon S3) for high performance and cost optimization in a single namespace. Weka’s unique snap-to-object feature allows users to “park” user projects as well as the file system and metadata in Amazon S3 when not in use to save on cost. When needed again, the snapshot can be re-hydrated in minutes, and artists can resume work.

The infrastructure required to create realistic 3D-animation or visual effects can vary dramatically from project to project, and studios frequently struggle to meet infrastructure demands between peak and off-peak work periods. Amazon Nimble Studio with Weka storage allows creative studios to take on bigger projects with more aggressive timelines without the limitations of physical hardware. By eliminating high up-front CAPEX costs and depreciating them over several years, studios can fund OPEX project expenses from the project contract. WekaFS with Nimble Studio allows a studio to scale its storage infrastructure and storage bandwidth to meet the needs of the project without the cost of underutilized hardware capacity. Studios pay for only the resources they need when they need them. An on-premises solution might take weeks to add more storage, costing artists time and money – or worse, never winning the contract. With WekaFS, artists can instantly add storage capacity to the studio resources, so no project is “too-big.” With features like auto scaling groups, users can increase the storage cluster for peak demand and shrink back down when the render project is complete. Artists no longer have to pay for weekend time when projects are mid-stream. Artists can add a project snapshot to Amazon S3 on Friday and spin it back up on Monday when work resumes. When the project is done, WekaFS can also store it in S3 for long-term archive.

1https://www.forbes.com/sites/tomcoughlin/2020/09/01/media-and-entertainment-storage-tam-to-exceed-16b-by-20205/?sh=3d6f9a207e80
2 2019 Digital Storage for Media and Entertainment Report, Thomas Coughlin

Additional Helpful Resources

Cloud File System

Solution Brief: Cloud-Based Post-Production Infrastructures with WekaIO and Amazon Web Services (AWS)
Build a digital studio-in-the-cloud with Amazon Nimble Studio and WekaIO
Amazon Nimble Studio