The FrostByte storage solution featuring the WekaIO File System breaks 8 benchmark records
CAMPBELL, Calif. – June 6, 2019 – WekaIO, the innovation leader in high-performance, scalable file storage for data-intensive applications, today announced record-breaking performance on the STAC-M3™ Benchmark from a new integrated solution with Penguin Computing—the FrostByte™ storage solution featuring the WekaIO File System. The STAC-M3 Antuco and Kanaga Benchmark Suites are the industry standard for testing solutions that enable high-speed analytics on time series data, such as tick database stacks. The WekaIO File System on Penguin Computing® Relion® servers and FrostByte storage system delivered record-breaking performance at scale for the STAC-M3 “Tick Analytics” Benchmark, affirming that the integrated solution is the ideal choice for algorithmic trading and quantitative analysis workloads common in financial services.
The STAC-M3 benchmark specifications are maintained by the STAC Benchmark Council, which consists of over 300 financial institutions and more than 50 vendor organizations whose purpose is to discuss technical challenges and solutions in financial services and to develop technology benchmark standards that are useful to financial organizations. User firms include the largest global banks, brokerage houses, exchanges, hedge funds, proprietary trading shops, and other market participants. The STAC-M3 results were audited by the Securities Technology Analysis Center (STAC®), which facilitates the Council.
The system under test was a Kx kdb+ 3.6 database system distributed across 7 Penguin Computing Relion XO1132g servers with dual Intel® Xeon® Gold 6140 CPU @ 2.30 GHz and data stored on Penguin FrostByte with the WekaIO File System (v3.2.2) Software running on 8-1U RXE1112 server nodes with 9 NVMe SSDs per server, using Mellanox SB7780 36-port Non-blocking Managed EDR 100Gb/s InfiniBand Router and InfiniBand EDR/ Ethernet 100Gb 1-port 840QSFP28 Adapters. The tests were run on the baseline benchmark suite (“Antuco”) and the scaling suite (“Kanaga”), a test suite with five years of simulated stock market data. The test system had a combined performance density of up to 87.5 GB/sec available bandwidth to the client nodes, 40.5 GB/sec of actual throughput*, and 2.5M 4K IOPS in 8U, with the ability to fully distribute data, metadata and system services, as well as super low file system latency that leverages high-speed networks and NVMe-optimized storage.
The test system delivered record-breaking performance, breaking 8 STAC-M3 world records for mean query-response times and 4 world records for throughput.
Kx’s time-series database kdb+ is commonly used for large scale complex analytics on streaming, real time, and historical datasets in industries ranging from financial services to marketing analytics, and for Industrial IoT. In the financial industry, typical kdb+ applications are for algorithmic trading, back testing (trading algorithm validation), surveillance, regulatory and compliance reporting and research environments. These workloads can demand a lot from the file system.
Better than block-based solutions for Financial Services, the WekaIO File System is the only NVMe-optimized, shared, coherent, POSIX-compliant file system with a fully distributed architecture for data resiliency. The WekaIO Software eliminates islands of storage, providing all applications access to a shared data pool with record-breaking performance. The WekaIO File System delivers 10x better performance than a local file system by bending the rules on data locality, distributing the data across the cluster over low latency InfiniBand or Ethernet networks.
WekaIO is a firm believer in the value of benchmark testing as a means to guide and inform development in the industry and frame results in a manner that delivers actionable intelligence to customers. The WekaIO File System currently holds one of the top positions on the independent SPEC SFS 2014 benchmark and the #1 position on VI4IO’s 10 Node Challenge list.
For the full STAC Report, which contains an explanation of the benchmark specifications and the results for this solution, see www.STACresearch.com/news/2019/06/03/KDB190501.
Supporting quotes:
“Traditionally, IT organizations have used block-based storage solutions for their trading databases, rather than compromise performance due to network and file system latencies. The result has been large investments in specialized hardware and software to run different applications,” said David Hiatt, Director of Business Development and Alliances at WekaIO. “Now with the STAC-M3 Benchmark results, we have proven that with the WekaIO File System on Penguin Computing Relion servers and FrostByte delivers better application performance for Kx kdb+ by running in distributed mode over a network than with traditional block storage, with the additional benefit of simultaneously supporting other applications on the same platform. That’s a huge leap in application acceleration and simplifying the storage infrastructure for financial services.”
“The Penguin Computing FrostByte integrated solution featuring the WekaIO File System is designed, engineered and vetted to deliver a high-performance storage solution for Kx kdb+,” added Dr. Kevin Tubbs, Sr. Director of Technology, Advanced Solutions Group at Penguin Computing. “To break eight STAC-M3 performance records is an incredible achievement and a testament to the innovative architecture of the WekaIO File System and leading HPC and AI solutions from Penguin Computing. For IT supporting different applications and Kx kdb+ time-series databases, this platform is an unbeatable combination for performance, scale, and ease of management.”
Glenn Wright, Systems Architecture expert at Kx said: “Kx customers demand both high performance and low latency for compute and storage. The ability to combine both a distributed kdb+ implementation running against a high-performance shared file system can be attractive to our customers. With WekaIO as this file system, our customers are able to choose how and when they distribute their historical data workload to be accessible by one or more of their analytics workloads.”
“It’s great to see another successful benchmark test outcome when the WekaIO File System is paired with Mellanox’s network adapters. We have successfully collaborated with WekaIO on many benchmarks and together, we continue to set benchmark records,” contributed Gilad Shainer, Senior Vice President Marketing at Mellanox Technologies. “The Mellanox InfiniBand EDR/Ethernet 100Gb 1-port 840QSFP28 Adapters provide the lowest latency available, and that coupled with the WekaIO distributed and parallel file system satisfies the performance of even the most demanding I/O intensive application requirements.”
“There’s now a proliferation of software and hardware configurations aiming to support I/O intensive workloads,” commented David Floyer, Co-founder and CTO at Wikibon. “But to meet the performance demands of real-time and historical analytics in kdb+ environments, storage systems must offer both high performance and low latency. The results from the STAC-M3 Benchmark should help IT in making an informed decision when architecting storage systems to support algorithmic trading and quantitative analytics workloads.”
*As per the benchmark STAC-M3.ß1.1T.5YRHIBID.MAX
About WekaIO
WekaIO helps companies manage, scale and futureproof their data center so they can solve big problems. WekaIO built the world’s fastest shared and parallel file system on new architecture optimized for flash that leapfrogs legacy storage infrastructures by delivering the best performance density at any scale, for a fraction of the cost. With the most flexible deployment models, the storage software is a modern file system designed for modern I/O intensive and extreme technical computing workloads. For more information go to, https://www.weka.io.
Media Contact
WEKA Communications
media.relations@weka.io