Oak Ridge National Laboratory Storage System Delivers Over One Terabyte Per Second in Throughput to Drive Radical Advances in Science and Big Data Analysis, Essential to DOE and Office of Science Missions

ORNL is a national multi-program research and development facility managed by UT-Battelle for the U.S. Department of Energy. The Oak Ridge Leadership Computing Facility (OLCF) was established at Oak Ridge National Laboratory in 2004 with the mission of providing leadership computing for scientists working on some of the world’s most pressing problems.

In support of its new Titan supercomputer, Oak Ridge National Laboratory (ORNL) has selected DataDirect Networks (DDN) to build the world’s fastest storage system to power the fastest supercomputer in the world. Titan is designed to deliver a peak capability of over 27,000 trillion calculations per second, or 27 petaflops, a system that is over ten times more powerful than previous generations of ORNL computers.

DDN IME: Early Analysis at OLCF - Video
In this video from the DDN User Group at SC16, Jesse Hanley HPC Storage System Administrator, Oak Ridge National Lab, presents "DDN IME: Early Analysis at OLCF"
DDN IME: Early Analysis at OLCF

Buddy Bland, project director for the Oak Ridge Leadership Computing Facility at Oak Ridge National Laboratory:

“When building the world’s fastest system for data intensive computing, we carefully considered all aspects of high-throughput I/O infrastructure and how efficient storage platforms can complement our supercomputer’s efficiency. The ORNL and DDN teams have worked together to architect a file system designed to enhance the performance of our Titan supercomputer and enable our users to achieve unprecedented simulations and big data insights through massively scalable computing.”

For the growing number of problems where experiments are impossible, dangerous, or inordinately costly, advances of this compute magnitude offer the benefit of immediate and transformative insights in energy, national security, the environment and the economy, as well as to answer fundamental scientific questions. Using DDN’s SFA12K-40 storage systems as the backbone for Spider II, this new file storage system is designed with 40 petabytes of raw capacity and is capable of ingesting, storing, processing and distributing research data at unprecedented speed. This amount of storage capacity is equivalent to more than 227,000 miles of stacked books – or the distance from ORNL’s facility in Oak Ridge, TN to the moon – and enables ORNL to dramatically increase Titan’s computational efficiency and deliver vastly more accurate predictive models than ever before. As the de facto standard in storage for the world’s leading supercomputers, DDN continues to push the frontiers of science and technology from laptop to petaflop, building on its $100M investment in extreme scale computing and commitment to the DOE’s FastForward program to pave the road to exascale.

DDN Sets Standard for High Performance Computing

  • ORNL selected the DDN SFA12K-40 as the high-throughput building block for its Lustre* parallel file system. The platform delivers performance in excess of 10x what is achievable with contemporary scale-out NAS systems.
  • Building on a decade of ORNL and DDN optimizations for the Lustre file system, the DDN system is configured with Lustre performance of over one terabyte per second to meet the demands of Titan’s 299,008 CPU cores.
  • The ORNL Spider II configuration from DDN includes:
    • 36 DDN SFA12K-40 systems, each with 1.12PB of raw storage capacity;
    • Over 40PB of raw capacity in only 36 data center racks;
    • A combined 20,000 disk drives in a single system.
  • The combination of DDN’s and ORNL’s expertise in scaling Lustre in production environments will enable Titan to perform approximately 6x faster with 3x the capacity of its predecessor, Spider.
  • Architecturally unique in many ways, Titan’s power, scalability and efficiency serve as a showcase for the requirements of tomorrow as high performance computing (HPC) technologies continue to be adopted across the enterprise for Big Data computing.