Data tiering technology has been around as long as humans have existed on earth. The human brain can store a near infinite quantity of information it processes in memory, but only a small portion is stored in the “tip of our fingers” part — which explains why we remember some details faster than others. In most instances, we apply contextual filters, keys that allow us to search metadata about information stored in our memory to recall events that may have happened recently, or even years back.

Computer systems also employ data tiering utilizing different levels of processor caches and RAM to store and access data. An advanced storage platform like WEKA is no different. The WEKA® Data Platform stores hundreds of petabytes of data in object storage while keeping the most recent version of the working set of data needed for data-hungry applications in NVMe storage. It leverages tiering technology to efficiently and optimally place data in NVMe storage to drive high performance when applications need it. In this blog, we will highlight why tiering technology matters in modern day architectures and how DataOps teams can benefit from tiering to drive efficient operations while accelerating business outcomes in their environments.

Traditionally, data tiering in file systems has been driven solely by data-aging policies. Information lifecycle policies that are set in the system determine the value of data based on its accessed frequency. If data has not been accessed for a while, its value is deemed low and it gets moved to a low-cost, low-performance storage tier thereby lowering Total Cost of Ownership (TCO). A data-aging policy recalls tiered data when required; this initiates data migrations between high-performance and low-performance storage tiers in the background with the goal of avoiding any impact to application performance. In many legacy storage systems, tiering is performed using hierarchical storage management (HSM) software that moves the data between different secondary or tertiary storage solutions out of band, meaning that in many cases the HSM application needs to know what data it requires and then ask for it to be moved to the appropriate storage tier for access. In other words, the application asks for data to be moved from a backup tier to a faster tier to be processed.

However, DataOps teams who are tasked with feeding large quantities of data to performance-sensitive applications want more than just a policy-based approach to tiering. A VFX studio editor may want to change the background color to a few minutes of video footage in a video that may be hours long. This could force the application to populate a significant amount of data in the high-performance storage tier for a small period of time only to be moved back to cold storage after the minor edits are completed. On another occasion, the VFX edits may be more sequential in nature and applying a first-in-first-out (FIFO) kind of data migration policy is sufficient. A data tiering policy based on data aging alone or data access frequency may be insufficient to meet next generation application performance needs and availability.

A variety of next generation workloads such as AI/ML modeling, Genomic Sequencing, VFX editing, DevOps, EDA design, HPC, and Financial risk analysis, just to name a few, ingest, transform, process, analyze and report on large quantities of data in different phases. In some phases of this data pipeline, such as during ingestion and reporting, all of the workload data, say 1PB or 2PB of data, resides in high performance NVMe storage for only a short period of time. Other stages of the data pipeline act upon portions of the entire dataset requiring fast access to the data in order to reduce application cycle time.

In our experience enterprise DataOps teams want to leverage the power of their storage systems that span multiple compute nodes, NVMe storage servers, and high speed connectivity to object storage to tier data in parallel. This enables them to perform operations efficiently, sustainably, and without compromising performance. These DataOps teams depend on data tiering for mobilizing data at scale (without making a second copy) across the different data pipelines and eliminating infrastructure clutter and expensive storage costs. The WEKA Data Platform is a single hybrid system with a fully distributed parallel file system with inbuilt tiering at its core. It combines commercial off-the-shelf (COTS) NVMe Flash with object storage in a single namespace to provide both high-performance and low latency for multiple workloads through different protocols such as POSIX, GPU Direct Storage, NFS, SMB, and S3 allowing access to all of the data through all of these protocols.

The WEKA system distributes the metadata responsibility between thousands of virtual metadata processes that are running (containerized) on all of the storage nodes. Each is responsible for a certain part of the namespace (e.g. file A is handled by virtual metadata process 1 while file B is handled by virtual metadata process 2) and even different block ranges in a single file are managed by different virtual metadata processes. This allows for a very fine-grained control of metadata handling and removes any possible metadata bottlenecks, which means that for any IO pattern the load on the virtual metadata processes across the system will be equal. When tiering data to different object storage buckets for multiple file systems these virtual metadata processes will each be responsible for writing and reading their own data in parallel which maximizes the object storage performance capabilities. By contrast, legacy solutions are using a gateway node that brings data in on the fly or on demand only to/from the object storage and becomes a performance bottleneck. The virtual metadata processes will utilize objects that are 64MB in size to maximize object storage performance; this is achieved by splitting large files into 64MB chunks or by amortizing multiple smaller files into 64MB objects. When reading from the object storage, the virtual metadata servers will only read the relevant byte range that is required from the object (e.g. if a 4KB file is required then they will perform a 4KB byte range read from the 64MB object instead of bringing the entire object). This allows DataOps teams to tier data more granularly allowing for better utilization of resources.

Another capability the WEKA platform provides is the ability for the user to provide hints for the tiering of the data. One such hint is the duration of time the application may require the data. For example, a set of data may be required to be kept in NVMe storage for up to a month for reruns of a machine learning model that predicts the chances of developing certain health conditions with different data sets. Not all of the data created or accessed in that period will be copied to the object store in the order it was written. This allows the DataOps team to efficiently feed data-hungry applications that perform reruns, modifications and overwrites at scale. Additionally, the system reduces the number of transfers to and from object storage by marking data for tiering based on their creation, access and modification times.

The WEKA system will try to abide by these hints as much as possible but in certain scenarios in order to protect the user it will need to “break policy” and evict data from NVMe tier without abiding by these hints. One such scenario is when the NVMe tier of a file system gets above 95% utilization in this case the system will “release” capacity from this NVMe tier down to 90%. In fact the tiering is actually in a sub file granularity which means that as files are created and accessed the WEKA system may intelligently make the decision to have parts of the files in NVMe and parts tiered in the object storage.

As files are written to the WEKA system their metadata and data is placed on the NVMe tier. As part of the tiering process, data is tiered to object store while metadata only resides on the NVMe tier. Since all metadata is always on the NVMe tier, and due to the metadata heavy nature of POSIX operations, the performance of a file system will still be flash performance for the majority of its IOPs.

Additionally, the WEKA system is designed to make use of data that is tiered to an object store for other data services that are able to leverage the granularity at which data is laid out on the object store. Data resiliency and snapshot copy are two core capabilities in the WEKA architecture that leverage this to their advantage. In contrast to legacy storage systems, WEKA leverages data tiered to object storage efficiently while rebuilding the cluster after two concurrent failures. If required, the tiered data chunks are used by the nodes participating in the rebuild process providing an additional layer of data protection.

WEKA Snap-To-Object (S2O) is a capability that layers on top of the WEKA filesystem (WekaFS) and moves WekaFS snapshot data and metadata to an object store. S2O and tiering can use the same object storage attached to the file system to perform their operations efficiently. In such cases, S2O points to blocks that are already tiered and avoids rewriting the same blocks to the object storage. This allows S2O and tiering to consume less storage space making DataOps operations fast and efficient.

TL;DR

Key takeaways:

  • DataOps teams should leverage data tiering to efficiently provide data to multiple data-hungry applications without compromising application performance or downtime.
  • Metadata is processed on all the nodes of the system and data is tiered granularly to scale from multiple filesystems to multiple object stores.
  • WEKA’s data tiering is performance-conscious and avoids frequent reads and writes to tiered storage.
  • Data that is tiered to object storage is made useable for data protection scenarios rather than being idle cold storage.
  • WEKA’s tiering framework is more comprehensive than data age-based tiering policy and tiers data only after applications have derived full value from being in the NVMe storage tier.

Learn More About the WEKA Data Platform