Attaining High Performance Communications: A Vertical Approach

Technological Advances and Problems of High Performance CommunicationsAn ecosystem of solutions along a stack of technology layers
Cohesively collecting state-of-the-art contributions from leading researchers in industry, national laboratories, and academia, Attaining High Performance Communications: A Vertical Approach discusses various issues pertaining to high performance communications in a particular layer of a vertical stack. It explores efficient interconnection hardware, the architectural aspects of network adapters and their integration with processor cores, the design of scalable and robust high performance end-to-end communications services and protocols, and system services and tools for new multi-core environments.

No single solution applied at one particular layer can help applications solve all performance-related issues with communication services. Instead, this book shows that a coordinated effort is needed among the layers. It covers many different types of technologies and layers across the stack, from the architectural features of the hardware, through the protocols and their implementation in operating system kernels, to the manner in which application services and middleware are using underlying platforms. The book also describes key developments in high-end platforms, high performance interconnection fabrics and communication libraries, and multi- and many-core systems.

This volume addresses the challenges involved in emerging types of communications applications, platforms, and services. Examining each layer in the vertical stack, it illustrates how to eliminate bottlenecks and provide optimization opportunities.

Ada Gavrilovska is a Research Scientist in the College of Computing and the Center for Experimental Research in Computer Systems (CERCS) at Georgia Tech.

High Performance Interconnects for Massively Parallel Systems
Introduction
Performance
Network Topology
Network Features
Future Directions
Summary
Commodity High Performance Interconnects
Introduction
Overview of Past Commodity Interconnects, Features, and Trends
InfiniBand Architecture
Existing InfiniBand Adapters and Switches
Existing InfiniBand Software Stacks
Designing High-End Systems with InfiniBand: Case Studies
Current and Future Trends of InfiniBand
Ethernet vs. EtherNOT
Overview
Introduction
Background
Ethernet vs. EtherNOT?
Commercial Perspective
Concluding Remarks
System Impact of Integrated Interconnects
Introduction
Technology Trends
Integrated Interconnects
Case Study: Implementation of Global Address Spaces
Future Trends and Expectations
Network Interfaces for High Performance Computing
Introduction
Network Interface Design Issues
Current Approaches to Network Interface Design Issues
Research Directions
Summary
Network Programming Interfaces for High Performance Computing
Introduction
The Evolution of High Performance Computing (HPC) Network Programming Interfaces
Low-Level Network Programming Interfaces
Distinguishing Characteristics
Supporting Message Passing Interface (MPI)
Supporting Shared Memory (SHMEM) and Partitioned Global Address Space (PGAS)
Portals 4.0
High Performance IP-Based Transports
Introduction
Transmission Control Protocol (TCP)
TCP Performance Tuning
User Datagram Protocol (UDP)-Based Transport Protocols
Stream Control Transmission Protocol (SCTP)
Summary
Remote Direct Memory Access and iWARP
Introduction
Remote Direct Memory Access (RDMA)
iWARP
Summary
Accelerating Communication Services on Multi-Core Platforms
Introduction
The "Simple" Onload Approach
Partitioned Communication Stacks
Specialized Network Multi-Cores
Toward Heterogeneous Multi-Cores
Summary
Virtualized I/O
Introduction
Split Device Driver Model
Direct Device Access Model
Opportunities and Tradeoffs
Summary
The Message Passing Interface
Introduction
MPI’s Layer in the Network Stack
Threading and MPI
Point-to-Point Communications
Collective Operations
Implementation Strategies
Summary
High Performance Event Communication
Introduction
Design Points
The EVPath Architecture
Performance Microbenchmarks
Usage Scenarios
Summary
The Case of the Fast Financial Feed
Introduction
Market Data Processing Systems
Performance Requirements
The OPRA Case Study
Summary
Data-Movement Approaches for High Performance Computing Storage Systems
Introduction
Lustre
Panasas
Parallel Virtual File System 2 (PVFS2)
Lightweight File Systems
Other Microsoft Project Plan (MPP) File Systems
Summary
Network Simulation
Introduction
Discrete Event Simulation
Maintaining the Event List
Modeling Routers, Links, and End-Systems
Modeling Network Packets
Modeling the Network Applications
Visualizing the Simulation
Distributed Simulation
Summary
References