SCALE-OUT, FLASH-NATIVE
DATA CACHE COST-EFFECTIVELY
SOLVES I/O SCALE &
BOTTLENECK CHALLENGES
PRODUCT FAMILY
Infinite Memory Engine®
BREAK FREE FROM THE CHALLENGES AND INEFFICIENCIES CAUSED BY I/O BOTTLENECKS
DDN’s Infinite Memory Engine® (IME®) is a scale-out, software-defined, flash storage platform that streamlines the data path for application I/O. IME interfaces directly to applications and secures I/O via a data path that eliminates file system bottlenecks. With IME, architects can realize true flash-cache economics with a storage architecture that separates capacity from performance.
THE NEED FOR A NEW APPROACH TO FAST I/O
Several key factors, both technological and commercial are creating demand for a new approach to high performance I/O. New non-volatile memory (NVM) device technologies are proliferating and media capacities are increasing rapidly. Diverse many-core processor strategies are pushing higher I/O volumes and more demanding I/O profiles. A new generation of high-business-value markets are taking advantage of analytics and machine learning and further stressing performance boundaries. Traditional file systems only crudely manage flash at scale whereas HDD performance degrades as concurrency increases. IME introduces a new layer, above the file system that:
- Maximizes flash media performance and longevity
- Improves the efficiency of network and storage through adaptive I/O and data coalescing
- Efficiently feeds the diverse I/O requirements of modern workloads and mixed CPU/GPU environments
- Delivers the data rates needed across commercial analytics in machine learning
IME delivers predictable job performance, provides faster computation against data sets too large to fit in memory, accelerates I/O-intensive applications and provides a safe, cost and space-efficient landing space for bursty data.
DDN Infinite Memory Engine IME Overview
At SC17, Kurt Kuckein of DDN provides an overview of DDN IME and how it improves scale-out application performance, eliminates file system bottlenecks, and delivers true flash-cache economics with a storage architecture that separates capacity from performance.
IME in 2017 by James Coomer - HPCXXL in Bologna (CINECA)
IME is well suited for a variety of use cases
- Machine Learning and Analytics: Massive, flash cache with over 600K read IOPs in 2U and 20GB/s to support large scale data sets at speed
- Application Ensembles: Multiple, simultaneous applications use cache to support communication throughout the job
- Application Workflows: Cache stores a common dataset used by a succession of independent applications
- In-situ Analysis: Real-time output of an application’s data in cache analyzed in-situ
- Application Pre-Processing: Pre-processing job places output in cache, ready for main run
- Application Post-Processing: Main run outputs to cache, post-processing commences in-situ
- Checkpointing: Stores checkpoints faster than a parallel file system and keeps them in cache to support restarts
IME Test Beds and Major Deployments
- Software-Defined Flash Platform
- Scale-Out Cache from TBs to PBs
- Reduces Performance Infrastructure Footprint
- Removes File System Bottlenecks
- Delivers Predictable Run Times
- Accelerate One Application or all I/O
- Compute, Storage, and Network Agnostic
- Supports Infiniband, Omni-Path, and Ethernet
- Flexible Data Protection Options
USE CASES
IME DEPLOYMENT OPTIONS
IME SOFTWARE-ONLY
For environments that want the maximum deployment flexibility, IME is available as software-only. IME software-only is ideal for those environments with IT-based limitations on supported hardware; that want the flexibility to repurpose existing hardware; and that have open-platform requirements. Most importantly, whether you decide to use IME240, or software-only, IME is the industry’s only flash-native cache that is not locked into a specific server or storage hardware vendor. With DDN’s IME, the choice is yours.
I/O PERFORMANCE-OPTIMIZED COMMODITY SERVERs
For an integrated appliance, DDN offers the IME140™ and IME240™. The IME240 is built on a best-of-breed 2U commodity storage server that delivers over 20GB/s of bandwidth and over 1M IOPs direct to applications. IME240 has up to 23 NVMe SSD drives for data and flexible N+K erasure coding. The IME140 supports extreme file performance in a small 1U flash data appliance. Each appliance can deliver more than 11GB/s write and 20GB/s read throughputs and more than 1M file IOPs (read and write). Starting with a resilient solution as small as 4 units, the IME140 allows organizations to cost-effectively scale performance independent of amount of capacity required.
SPECIFICATIONS
IME140 |
IME240 |
|
SYSTEM FEATURES |
Dual redundant power and cooling |
|
FABRIC PORTS |
Up to 2x InfiniBandTM EDR/FDR; or 2x 10/40/100GbE Ethernet; or 2x Intel® Omni-Path |
2x InfiniBandTM EDR/FDR; or 2x 10/40/100GbE Ethernet; or 2x Intel® Omni-Path |
WEIGHT |
Maximum weight: 48lbs / 21.8KG |
Maximum weight: 55lbs / 24KG |
DIMENSIONS |
1RU; (WxHxD) 17.2 x 1.7 x 27.82 in. (437 x 43 x 707 mm) |
2RU; (WxHxD) 17.2 x 3.5 x 27.76 in. (437 x 89 x 705 mm) |
POWER |
800W total system power consumption (AC input requirement, max. config. with 8 data drives) |
796W total system power consumption (AC input requirement, max. config. with 23 data drives) |
DATA DRIVES |
NVMe data drives up to 8 per system. Capacities from 1.2TB per NVMe SSD |
NVMe data drives up to 23 per system. Capacities from 1.2TB per NVMe SSD |
SYSTEM DRIVE |
2x 2.5”, 1TB, SAS3.0 12GB/S, 7.2K RPM |
2x 2.5”, 1TB, SAS3.0 12GB/S, 7.2K RPM |
CPU, MEMORY |
Dual Intel® Xeon® Scalable Processors (skylake), 8x 8GB DDR4-2400 RAM |
Dual Intel® E5 Series Processor, 8x 16GB |
OUT-OF-BAND MANAGEMENT |
Dedicated IPMI LAN RJ45 |
|
DRIVE BAYS |
10, tool-less 2.5” drive bays |
24, tool-less 2.5” drive bays |
SEQUENTIAL READ AND WRITE PERFORMANCE |
Up to 20GB/s (EDR/OPA) |
Up to 23GB/s (EDR/OPA) |
ERASURE CODING |
N (data)+K (parity) Flexible Erasure Coding options with K=1, 2 or 3. Supports (3+1)...(15+1), (6+2)... (15+2), (9+3)...(15+3) |
|
*Specifications are subject to change.