Performance Analysis is Data Science (Todd Gamblin, LLNL)

(part of series of talks at HPC-UGent on Feb 2nd 2018, see https://www.ugent.be/hpc/en/training/...)

Understanding the performance of a large HPC facility is very complex. Job runtimes can vary from run to run, and performance depends on many factors: network congestion, filesystem contention, and application input parameters. Traditional performance tools allow easy analysis of a single run, but tools that can look deeply into performance of applications across the center are rare. This talk will cover a number of efforts at Livermore Computing to do HPC center-wide performance analysis. We will discuss LLNL’s ongoing Sonar project, which aims to set up a performance cluster, several efforts to analyze and tune applications using performance data from the center, and the recent center-wide deployment of JupyterHub for data analysis, and how we plan to use it for HPC Center performance data.