DDN BLOG

Delivering Data When It Counts

Why is scalable storage management essential for AI, Analytics, Big Data, HPC, objects and cloud and how does DDN’s EXAScaler produce results?

With the richness and variety of current data generation technology, Enterprises and Research Organizations alike have a greater opportunity than ever to create value from their data. The combined software ecosystem across the fields of Big Data, Large Scale Numerical computing (HPC), Analytics and AI bring unprecedented capability into the hands of those with large data sets. However, the associated challenges in data management are large as well. The diversity and sheer volume of these new data intensive workloads accentuates the need for a new way to manage your data at scale; stronger and more comprehensive data security, better reliability and consistent high performance, but also a storage environment with the capability and flexibility to cater for cloud, multiple protocols, objects and files and integrates into a comprehensive data lifecycle approach. DDN’s EXAScaler meets these challenges by giving you a completely new approach to data at scale.

What is EXAScaler, where did it come from and how does it really help?

Firstly, EXAScaler is highly scalable software for data storage and analysis and management, developed by DDN. EXAScaler runs in DDN appliances and in the cloud. At its core, is the fastest, most scalable open parallel file system, Lustre. Lustre itself is the most popular filesystem for scale-out computing and has been proven and hardened in the most demanding HPC environments for over 15 years. Lustre and EXAScaler are developed by a very active, dedicated and talented team, most of whom now work at DDN. DDN supports the open source community with Lustre through our Whamcloud organization and also develops EXAScaler which builds on Lustre’s innate performance and scalability with a host of additional major features to greatly simplify deployment, storage and management at scale with differentiated integrations into emerging workloads. EXAScaler really helps when you need full spectrum performance across workloads from deep learning through to Big Data and analytics. With special connections into NVIDIA DGX systems and other NVIDIA GPU based architectures, environments running on AWS, Azure and Google Clouds EXAScaler supports complete full range of modern workloads

Major EXAScaler Features

  • Extreme Scalability and Performance: A highly parallel architecture for low latency workloads and high bandwidth applications alike
  • Data Aware Intelligence: EXAScaler grows with your data, allowing limitless scale with the good economics by intelligently management your data across tiers
  • Data Security and Integrity: Secure Multi-Tenancy, Encryption, End-to-End data protection, Replication all from a filesystem hardened at the largest scales
  • Flexible access from any application, any architecture, anywhere: Integration with cloud, GPU, AI frameworks, containers, file and object

Simplicity is difficult to deliver at scale. At DDN, we’ve heard many troublesome tales of data storage systems that don’t cope when presented with the onslaught of corner cases which are created when clients are in the tens of thousands, data is over 100PB and IO rates are measured in Terabytes per second. Much of the hard work of delivering strong RAS (Reliability, Availability, Serviceability) is down to our engineers’ experience in developing and testing at the largest scales.

DDN EXAScaler appliances bring all the sophistication required to deliver fast, secure, stable, and simple storage at scale. EXAScaler can be delivered as all-NVMe and Flash-Hybrid options starting as small as 2 rack units and growing up to solutions that provide over 10PB per rack. No need for the complexities and headaches of Ceph and Big Data solutions that required multiple data copies, and deep domain knowledge to keep stable and performant. No complicated user experience in understanding your data architecture. No limitations on performance. Full Flexibility in building a data architecture that matches your business which grows and adapts alongside your data and workloads.

Continuing Innovation to Address a Range of Requirements

DDN’s EXAScaler is deployed by customers across the range of modern data-intensive businesses from autonomous driving at massive scale, real-time fraud detection, gene sequencing, image analysis .. the list is long and comprehensive.

At DDN we are dedicated to innovating around our EXAScaler range. 2019 was an amazing year for EXAScaler with the introduction of a native data management engine, Stratagem and great performance results and customer wins in conjunction with AI frameworks and NVIDIA GPU systems. There is more to come in 2019, but 2020 will be even more spectacular and we’d like to talk with you about how EXAScaler can revolutionize your approach to data at scale and take your performance, data growth and management concerns off the table.

  • Dr. James Coomer
  • Dr. James Coomer
  • Sr. Vice President of Products
  • Date: October 10, 2019