Some years ago – never mind how long precisely – having little or no choice, and nothing particular to point me to one or the other, I would submit compute jobs to one supercomputer or another. It was a way I had of not caring about the hardware, the implementations, networks and so on, but with my only interest being in knowing the physics of a handful of atoms. The IBM SP2 or the Cray T3D were my choices with little to decide between them other than the length of the queue and my allocated resource quota.

Well, we have lived to see the end of that era of limited proprietary environments. By the time I had completed my PhD (not _too_ long after I started), a new ecosystem of open off the shelf was entering, and many of my last works were performed on a compact pile of metal and wires in the form of a Beowulf cluster. I moved to Sun Microsystems – cue more open-ness, new networks, more processors. Then three years later, I moved to a small HPC cluster company – we combined Linux software and other open source components, the grid engine scheduler, various compilers, with off the shelf components and screwed them into racks, plugged them in and tried not to trip over the Myrinet cables on our way out.

Whilst I was completing my PhD, DDN launched and delivered their first system to NASA – a specialized, high performance storage system that could handle the toughest data problems with what seemed to be effortless ease. And here we are twenty years later, and DDN has become the largest private storage system in the world, with over 2,500 customers, more than 10 Exabytes of storage deployed, more than a hundred patents, solving highly complex data at scale challenges in thousands of the most demanding environments in the world.

DDN has also developed some of the most innovative and the most performant storage platforms, file systems, IO technologies, object stores, and comprehensive sets of software tools, not to mention successfully conducted a number of company and technology acquisitions to further enhance the capabilities it delivers to its customers.

In short DDN holds a very special place in our industry as the original innovator for data at scale, with a boundless passion and steadfast dedication to innovate, create, architect and build radically enabling and easy to use technologies to solve the increasingly complex challenges of AI, Big Data and HPC in Multicloud environments.

With the consumption of SGI and now Cray by HPe, we are left with two behemoths, HPe and IBM in our industry. For both of these organizations, financial efficiency is paramount and HPC is a rounding off error in the spreadsheets of their financial organizations. We all know that cutting costs and slashing everything that does not contribute to this quarter’s revenue works wonders in the world of Wall Street, but not so in the world of HPC. Unfortunately, financial engineering kills innovation (R&D is unacceptably expensive in HPC) and destroys customer satisfaction (HPC is a bunch of rocket scientists who know exactly what they are talking about and demand true value, also very expensive).

More than ever before, the need for innovation and long term and sustained investments in R&D is very, very strong. The AI, Big Data and HPC landscapes (and cloudscapes) are changing very rapidly, from compute diversification, stronger cloud traction, new data-centric S/W frameworks, flexibility and vendor lock in removal, blending of seamless networking, GPU, processor and storage layers, AI enablement, it is all evolving and transforming at an incredible pace.Big Data and HPC are an indispensable part of the evolution of AI into the next set of frameworks that enable new capabilities in research, healthcare and business.  EXA5 was developed to enable these transformative solutions by streamline and optimizing the management of data, for the extraction of maximum value.

Will we see shrunken and ineffective innovation aimed at the lowest common denominator of the consumer base? Maybe from others, but absolutely not if DDN has anything to do with it. DDN has the means, resolve, experience, track record and vision to address all current and future AI, Big Data and HPC needs – today, in the future, on prem, in hybrid and multicloud. We love dinosaurs, but not in our data centers…

  • Dr. James Coomer
  • Dr. James Coomer
  • Sr. Vice President of Products
  • Date: June 16, 2019