DataDirect Networks Annual High Performance Computing Trends Survey Underscores that Storage I/O Performance is The Number One Issue in Designing HPC Systems for Big Data
HPC End Users Confirm that Not Only is Storage I/O a Challenge for Today’s Petascale Environments, But Fundamental Architectural Changes are Required for I/O Support for Exascale-Level Compute
- HPC end users are elevating the discussion about data storage and recognizing the importance of storage I/O – not just compute – in an effort to hit the exascale threshold by the end of the decade according to the results of a DataDirect Networks (DDN) High Performance Computing (HPC) Trends survey, released today.
- The survey polled a cross-section of 60 DDN HPC end users representing hundreds of petabytes of storage investment, which included individuals responsible for HPC allocations encompassing high availability storage and file systems across a variety of industries. Of those surveyed, 60 percent manage or use at least one petabyte of storage, and over 20 percent manage or use 10 PBs or more.
- According to the survey results, more than two-thirds (68%) of those polled agree that data and data storage has become the most strategic part of the HPC datacenter. Moreover, by a margin of two to one respondents, the survey also reveals that today’s storage technologies will need to undergo massive change to hit exascale proportions.
- In addressing this challenge, the overwhelming majority of respondents (78%) agree that hybrid storage is the evolutionary next step for HPC storage, combining both the performance and cost efficiency benefits required of storage at exascale.
Future Advances in Storage Performance to Deliver Massive I/O Gains and Reduction in Storage Costs for HPC, Analytics and Cloud Computing Environments
- With a nod to future technology requirements, the survey also shines light on the biggest storage and/or Big Data challenges for HPC users today, with nearly half of respondents citing mixed I/O performance the primary issue (47%), followed by managing rapid data growth (22%), performance (13%) and multi-site replication or collaboration (5%). Thirteen percent of those polled name ‘other’ concerns, including the need to manage the overall total cost of storage without compromising storage scalability and performance.
- With storage performance a critical requirement especially at today’s petascale levels, of those respondents with an opinion, 85 percent agree that site-wide file systems like those at Oakridge National Laboratory (ORNL), National Energy Research Scientific Computing Center (NERSC) and Texas Advanced Computing Center (TACC) are the next infrastructure trend in HPC environments.
- With HPC no longer limited to data centers at research institutions of the Top500 Supercomputing sites, the survey also examines end users opinions on analytics and integrating computing workloads into large scale HPC cloud environments. More than two-thirds (70%) of those polled point to storage I/O as the biggest bottleneck in analytics workflows, suggesting that compute is no longer the main challenge for analytics platforms. While multi-site HPC collaboration may be on the rise, security and data sharing complexity remain the biggest impediments according to 75 percent of those surveyed.
- Finally, HPC organizations continue to find themselves under increasing funding scrutiny and according to nearly three quarters of the survey respondents, it has become more difficult for HPC sites to obtain funding for compute infrastructure without demonstrating a clear return on investment (ROI).
Jeff Denworth, Vice President of Marketing (read blog)
- "The results of DDN’s annual HPC Trends Survey reinforce what we hear from customers on a daily basis. Users are identifying storage I/O as the primary challenge in computing and the industry conversation needs to shift to reflect that. As the evolutionary gap between compute and storage media grows, DDN is focused on conquering the I/O performance problem and providing new approaches and innovative solutions to help our customers reach any level of computing scale faster, more efficiently and most cost effectively."
- Survey results available upon request at email@example.com
- Connect with DDN at SC13
- More on DDN HPC Storage Products
- Follow DDN via Blog and Twitter
DataDirect Networks (DDN) is a leader in data center infrastructure for Big Data and Cloud Storage applications. Our data storage and processing solutions and professional services enable content-rich and high growth IT environments to achieve the highest levels of systems scalability, efficiency and simplicity. DDN enables enterprises to extract value and deliver results from their information. Our customers include the world's leading online content and social networking providers, high performance cloud and grid computing, life sciences, financial services, media production, and security and intelligence organizations. Deployed in thousands of mission critical environments worldwide, DDN's solutions have been designed, engineered and proven in the world's most scalable data centers to ensure competitive business advantage for today's information powered enterprise. For more information, go to datadirect.wpengine.com or call 1-800-837-2298.
©2013 All rights reserved. DDN, Storage Fusion Architecture, SFA12K, EXAScaler and GRIDScaler are trademarks owned by DataDirect Networks. All other trademarks are the property of their respective owners. All rights reserved.