Liran Zvibel, Co-Founder and CEO of WekaIO, shares his reflections in response to the question “Is Weka the iPhone of Storage?”

Notable business writer Jim Collins, who is the author of “Good to Great” and “Great by Choice” and other outstanding books, talks about the fundamental principles that make a good company. I would like to talk about one of his principles that can also be applied to the creation and design of revolutionary products. That principle is to reject the “Tyranny of the OR” and embrace the “Genius of the AND” (click here for more details).

The following paragraph is taken verbatim from the web page referenced above that describes this principle.

“Genius of the AND” is a concept developed in the book Built to Last. Builders of greatness reject the ‘Tyranny of the ’ and embrace the ‘Genius of the AND.’ They embrace both extremes across a number of dimensions at the same time—purpose AND profit, continuity AND change, freedom AND responsibility, discipline AND creativity, humility AND will, empirical analysis AND decisive action, etc.”

The most profound day-to-day product that followed this concept is the iPhone. When Steve Jobs announced the iPhone in 2007 it was a true example of this concept. From the original announcement:

“Apple® today introduced iPhone, combining three products—a revolutionary mobile phone, a widescreen iPod® with touch controls, and a breakthrough Internet communications device with desktop-class email, web browsing, searching and maps—into one small and lightweight handheld device.”

Later Apple also added a camera to the iPhone making it a combination of four products that work extremely well together. The iPhone took some long introduction time into the market and a few years to get all the kinks out of it, but it created a new category of products and changed the lives of all of us.

What would be the equivalent of the iPhone in the storage market?

Unfortunately, the storage market is filled with either-OR decisions that limit the functionality of the data center. Some of those trade-off decisions are:

  • simplicity OR performance
  • shared data OR performance
  • public cloud OR on-premises
  • scale OR usability
  • speed OR shareability
  • and so on…

Due to decades-long technical limitations, the storage market has segmented itself to different product categories that force customers to make hard compromising decisions on how to implement solutions to their problems. In many cases, customers are forced to choose between several storage categories and to integrate them with some middleware.

DAS vs. SAN vs. NAS vs. Object Storage

  • DAS (Direct Attached Storage) — currently these are NVMe devices similar to the devices FusionIO, Violin, or Smart Modular offered not long ago. There is a belief that running a local file system over Direct Attached devices provides the highest level of performance. However, this approach fails to provide a way to manage a large cluster of servers as data is not shared and cannot be backed up. In many cases, the DAS approach wastes a lot of capacity because organizations buy the biggest devices they can, and many servers have unused capacity.
  • SAN (Storage Area Network, or “block”) — this is the most popular form of storage, as for a long time, it has provided a good mix of manageability, features, and performance. The protocols employed used to be SCSI-based (for example, FC) and are now being displaced by NVMe-over Fabrics (NVME-oF). SAN performance is not as good as that of DAS, but it has been sufficient in most cases. The ability to manage storage has made SAN the most popular category. The big downside of the SAN systems is that data can only be accessed from one server at a time, so data cannot be shared. New enterprise performance workloads require more servers to access the same data set, making this tradeoff a big problem.
  • NAS (Network Attached Storage, or “filer”) — this is also quite a popular form of storage, where the most established product options have been NetApp FAS and Isilon. Because of the historically lower speeds of Ethernet and the NFS protocol, the performance of filers has never even come close to that of DAS and SAN. But, these storage systems provide easy sharing of data and convenient management. Another downside of NAS is metadata scalability, as it is limited to relatively small directories of 100s of thousands of elements and the associated capacity of a few PBs.
  • Object Storage — this kind of storage has made a recent entry (in the last 20 years) to the market as scalability has become a significant issue. These systems give up the standard File System interface in order to scale both capacity and metadata. The main downside of object storage is that the users have to re-write their applications to support the object protocol and performance is the lowest of the 4 options.

You can Learn more about NAS vs. SAN vs. DAS storage comparison

In addition to choosing from these options, customers have to decide whether they will keep running their current data centers (“on-premises”) or will migrate to the public cloud. The public cloud has merely adopted the on-premises model and features the same four storage categories; for AWS, the i3 instances with local NVMe devices are the DAS solution, and EBS, EFS, and S3 are SAN, NAS, and Object Storage, respectively. As the figure below shows, the same tradeoffs of these storage categories exist in the cloud, so the public cloud did not solve the problem.

If customers want to view the public cloud as a replacement for their data centers, they would want to keep managing, running, and using the same tools. Thus, switching to yet another set of tools (with little competition), which is not what customers are looking for.

So, can there be an iPhone of storage? Is there a solution that embraces the “Genius of the AND” rather the “Tyranny of the OR” that I described above?

Well, there is indeed, and this is exactly the reason we created Weka.

At our previous company, XIV, we created a better version of SAN (over a decade ago), and then IBM acquired us and created Spectrum Accelerate from our innovation. Working at IBM allowed us to get to know the “Tyranny of the OR” of the storage world from up close. Hence, when starting Weka, we decided to look at storage from a different angle and embrace the “Genius of the AND”.

At Weka, we created a parallel file system system. We are faster than DAS (breaking the decades-old myths that this can’t be done), with the reliability of SAN (+2 protection, network HA, end-to-end data protection, etc.), the data shareability of filers, and the scalability and economics of Object Storage. We have a full suite of enterprise features allowing for manageability, and we run the same product on-premises or in the public cloud.

Another form of the “Tyranny of the OR” in storage has been the AFA (All-Flash Array or SSA – Solid State Array) market. When Pure Storage was telling the world about the “All-Flash Data Center” in 2012, it was the most extreme example of that. At Weka, we embrace HDDs with our ability to tier seldom accessed data to hard drives and also automatically save snapshots (for backup, archive, or workload migration) to S3 Object Storage as part of our product design. Again, the perfect storage embraces the “Genius of the AND” rather than the “Tyranny of the OR”.

Going back to Steve Jobs and his introduction of the iPhone in 2007, at that time, neither Apple nor the market was able to foresee the profound changes that would result from embracing the “Genius of the AND”. Similarly, although it is still too early to envision how profoundly Weka is going to change the data center and expand the realm of what is possible, I am confident that Weka will turn out to be the iPhone of storage.

For more information on WekaIO’s approach to storage, read the following architecture white paper here.