“Security in High-Performance Computing Environments”
Communications of the ACM, September 2017, Vol. 60 No. 9, Pages 72-80
Review Articles
By Sean Peisert
“Historically, security for HPC systems has not necessarily been treated as distinct from general-purpose computing, except, typically, making sure that security does not get in the way of performance or usability. While laudable, this article argues that this assessment of HPC’s distinctiveness is incomplete.”
How is computer security different in a high-performance computing (HPC) context from a typical IT context? On the surface, a tongue-in-cheek answer might be, “just the same, only faster.” After all, HPC facilities are connected to networks the same way any other computer is, often run the same, typically Linux-based operating systems as are many other common computers, and have long been subject to many of the same styles of attacks, be they compromised credentials, system misconfiguration, or software flaws. Such attacks have ranged from the “wily hacker” who broke into U.S. Department of Energy (DOE) and U.S.
Department of Defense (DOD) computing systems in the mid-1980s, to the “Stakkato” attacks against NCAR, DOE, and NSF-funded supercomputing centers in the mid-2000s, to the thousands of probes, scans, brute-force login attempts, and buffer overflow vulnerabilities that continue to plague high-performance computing facilities today.
On the other hand, some HPC systems run highly exotic hardware and software stacks. In addition, HPC systems have very different purposes and modes of use than most general-purpose computing systems, of either the desktop or server variety. This fact means that aside from all of the normal reasons that any network-connected computer might be attacked, HPC computers have their own distinct systems, resources, and assets that an attacker might target, as well as their own distinctive attributes that make securing such systems somewhat distinct from securing other types of computing systems.
The fact that computer security is context- and mission-dependent should not be surprising to security professionals “security policy is a statement of what is, and what is not, allowed,” and each organization, will therefore have a somewhat distinctive security policy. For example, a mechanism designed to enforce a particular policy considered essential for security by one site might be considered a denial of service to legitimate users of another site, or how a smartphone is protected is distinct from a desktop computer. Thus, for HPC systems, we must ask what is the desired functioning of the system so that we can establish what the security policies are and better understand the mechanisms with which those policies can be enforced.
On the other hand, historically, security for HPC systems has not necessarily been treated as distinct from general-purpose computing, except, typically, making sure that security does not get in the way of performance or usability. While laudable, this article argues that this assessment of HPC’s distinctiveness is incomplete.
This article focuses on four key themes surrounding this issue:
The first theme is that HPC systems are optimized for high performance by definition. Further, they tend to be used for very distinctive purposes, notably mathematical computations.
The second theme is that HPC systems tend to have very distinctive modes of operation. For example, compute nodes in an HPC system may be accessed exclusively through some kind of scheduling system on a login node in which it is typical for a single program or common set of programs to run in sequence. And, even on that login node, from which the computation is submitted to the scheduler, it may be the case that an extremely narrow range of programs exist compared to those commonly found on general-use computing systems.
The third theme is that while some HPC systems use standard operating systems, some use highly exotic stacks. And even the ones that use standard operating systems, very often have custom aspects to their software stacks, particularly at the I/O and network driver levels, and also at the application layer. And, of course, while the systems may use commodity CPUs, the CPUs and other hardware system components are often integrated in HPC systems in a way (for example, by Cray or IBM) that may well exist nowhere else in the world.
The fourth theme, which follows from the first three themes, is that HPC systems tend to have a much more regular and predictable mode of operation, which changes the way security can be enforced.
As a final aside, many, but by no means all HPC systems are often extremely open systems from a security standpoint, and may be used by scientists worldwide whose identities have never been validated. Increasingly, we are also starting to see HPC systems in which computation and visualization are more tightly coupled and, a human manipulates the inputs to the computation itself in near-real time.
This distinctiveness presents both opportunities and challenges. This article discusses the basis for these themes and the conclusions for security for these systems.
About the Author:
Sean Peisert is Staff Scientist at Lawrence Berkeley National Laboratory, Chief Cybersecurity Strategist at CENIC, and an associate adjunct professor at the University of California, Davis.
See also:
