An Introduction to the SPEC High Performance Group and their HPC Benchmark Suites

In this video from the 2018 Swiss HPC Conference, Robert Henschel from Indiana University presents: An Introduction to the SPEC High Performance Group and their HPC Benchmark Suites.

"The High Performance Group (HPG) of the Standard Performance Evaluation Corporation (SPEC) is a forum for discussing and developing benchmark methodologies for High Performance Computing (HPC) systems. The group is comprised of HPC vendors and research institutions from all over the world and has been developing production quality benchmark suites, like SPEC MPI2007, SPEC OMP2012 and SPEC ACCEL, for over 20 years. SPEC HPG benchmark suites are based on real world parallel scientific applications and go beyond synthetic kernel benchmarks to allow users to better understand real world system performance. An important part of the work is peer reviewing the results and publishing them in a repository on the SPEC web page. This curated result repository is freely available and can be used to model and estimate performance of a wide range of HPC systems.

In this talk, I will present an overview of the High Performance Group as well as SPEC’s benchmarking philosophy in general. Most everyone knows SPEC for the SPEC CPU benchmarks that are heavily used when comparing processor performance, but the High Performance Group specifically focusses on whole system benchmarking utilizing the parallelization paradigms common in HPC, like MPI, OpenMP and OpenACC. I will present use cases for the benchmarks like comparing compiler performance and performance of programming paradigms like OpenMP and OpenACC. I will show how the benchmarks are used at Indiana University as a selection criteria during procurement of large HPC systems, to track system performance over time and evaluate overhead of various virtualization solutions. At the end of my talk I will invite the audience to participate in the development of HPG’s next benchmark suite, a hybrid benchmark that will utilize multiple data sets to scale from a single compute node all the way to multiple thousand compute nodes with accelerators."

Learn more: https://eth-cscs.github.io/reframe/about
and
http://www.hpcadvisorycouncil.com/eve...

Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter