As 2016 winds down, it’s only natural to reflect on the successes of the passing year and the challenges for the year to come. In that vein, DDN recently announced the results of its fourth-annual High Performance Computing (HPC) Trends Survey which polled a cross-section of 143 end-users who manage data-intensive infrastructures worldwide and who represent hundreds of petabytes of storage investment across a diverse range of industries. You can link to the official survey results at the end of this blog, but I thought I’d take a closer look at what I found to be four of the most noteworthy concerns from the survey—and how DDN is uniquely positioned to address these concerns.
#4. The biggest challenges for HPC organizations continue to be managing mixed I/O performance and rapid data growth. Clearly, performance issues have escalated as Big Data environments contend with a proliferation of diverse applications, creating mixed I/O patterns and stifling the performance of their storage infrastructure.
DDN’s new GS14KX™ and ES14KX™ were developed precisely to overcome the toughest storage and data management challenges in extreme, data-intensive environments. These solutions scale up and/or out, leveraging a hyper-converged, parallel architecture with the options of SSD acceleration to deliver industry-leading performance and efficiency for HPC Big Data workflows.
#3. Concerns about security, privacy, and data sharing have greatly increased. A strong majority of those surveyed (70 percent) view security and data-sharing complexity as the biggest impediments to multi-site collaborations.
DDN’s new Multi-Level Security (MLS) Lustre* Solution provides the performance, scale, and openness users need while also allowing them to meet the highest multi-level security requirements. Built on DDN’s EXAScaler 3.0, the MLS Lustre Solution incorporates community-based Lustre and embedded DDN software code. Simpler to administer and more cost effective than maintaining a siloed infrastructure for sensitive data, the MLS delivers up to 2x the performance of competitive offerings.
#2. End users are increasingly choosing to upgrade specific parts of their environments with flash as they modernize their data centers.
More than 90 percent of respondents are currently using flash storage in their data centers. What’s truly surprising, however, is that even though industry perception is that all-flash arrays are the fastest storage available in today’s market, only 10 percent of these data-intense environments are using the technology! (The other 80 percent are using hybrid flash arrays as an extension to storage-level cache, to accelerate metadata, or to accelerate data sets associated with key or problem I/O applications.)
Burst buffers are considered by most respondents to be the technology most likely to take storage to the next level, and DDN is ahead of the curve. DDN has built its own SSD-based application acceleration software, the infinite memory engine (IME), that can act as a burst buffer and is available in three different versions to allow user flexibility: 1) IME14KX™ can scale up to 48NVMe SSDs, delivering an expected performance starting at 10GB/sec for entry performance and scaling up to 50GB/sec in each 4U appliance; 2) IME240™ realizes the same benefits but in a smaller 20GB/sec bandwidth building block; and 3) IME is also available as software-only, ideal for repurposing existing hardware and open-platform environments. Most importantly, IME is the industry’s only application acceleration software with burst buffer capabilities that is not locked into a specific server or storage hardware vendor.
#1. End users in the world’s most data-intense environments, like those in many general IT environments, are increasing their use of cloud. However, unlike general IT environments, the HPC sector is overwhelmingly opting for private and hybrid clouds instead of the all public cloud option.
Organizations planning to leverage a cloud for at least part of their data in 2017 rose to 37 percent, and of those, more than 80 percent are choosing private or hybrid clouds versus an all public cloud option. The reasoning behind this move seems to be based primarily on three reasons: cost, poor latency, and sheer data immobility issues. DDN’s approach to dealing with these issues to assist in the building of private clouds with our WOS object storage solution and in easy, seamless data migration solutions between tier one storage like our GS14KX and your private cloud and /or public cloud service. Our data management solutions automatically migrate data from a file-based solution to an object-based solution or public cloud while being transparent to users, tackling the speed and data locality concerns at a reasonable price.