Team „CC HPC“

CC HPC covers the following areas of responsibility and
services for locally operated HPC clusters:

• Operation and Monitoring: Monitoring the entire cluster, login servers/gateways, compute nodes, network, storage, and schedulers;
• System Maintenance: Applying patches to operating systems, updating firmware, and installing security updates;
• Troubleshooting: Analyzing and resolving outages, slow jobs, faulty compute nodes, and network issues;
• User support: Assisting researchers with job submission, software, debugging, and performance optimization;
• Software maintenance: Install and update scientific applications, compilers, and software libraries (such as MPI and CUDA) used by larger user groups;
• Resource management: Configuring CPU, GPU, RAM, queuing, and priorities in the scheduler; Account and access management: Managing user accounts, groups, permissions, and quotas;
• Performance tuning: Identify bottlenecks and optimize cluster configuration
• Data management: Oversee scratch, home, and project partitions, as well as backup and restore processes
• Documentation and Processes: Maintain operational documentation, standards, and incident runbooks
• Consulting on the migration of use cases with exceptionally high computational time requirements (several hundred thousand CPU or GPU hours) to systems described in the HPC supply pyramid at Tier 1 (national high-performance computing) and Tier 2 (national high-performance computing and thematically specialized high-performance computing centers) as described in the HPC supply pyramid.

In addition, CC HPC is responsible for tasks related to the procurement of HPC resources, e.g.,

• assessing resource needs in collaboration with the TU Dortmund University Center for Data Science and Simulation (DoDaS) and the Lamarr Institute for Machine Learning and Artificial Intelligence;
• preparing major equipment grant proposals for the German Research Foundation (DFG), including reporting requirements;
• Drafting and finalizing requests for proposals for HPC clusters, conducting discussions with manufacturers, managing negotiation processes, and accepting delivery of the systems;
• Planning and advising on expansions of existing HPC clusters using the departments’ and chairs’ own funds and third-party funding.

The time commitment required for the HPC.nrw competence network is becoming increasingly significant: The State of North Rhine-Westphalia’s HPC Concept 2025 no longer provides for separate HPC clusters at universities funded by state resources. Instead, funding will be limited to HPC clusters that are jointly used by researchers from multiple universities. To ensure an adequate supply of HPC resources for researchers at TU Dortmund University in the future, CC HPC—in collaboration with the HPC teams at Ruhr University Bochum and the University of Duisburg-Essen —will operate an HPC cluster at the Duisburg-Wedau site (hpcFachCluster.nrw@UARuhr).

An HPC cluster is not a single, exceptionally powerful, and well-equipped server with thousands of processing cores, petabytes of hard-disk storage, and terabytes of main memory. It is not a highly scaled-up version, but rather a horizontally scaled version of a computer: it typically consists

• of several hundred individual servers (usually called compute nodes) with standard components that can also be found in PCs;
• supplemented by shared hard-drive storage on central servers, which each of these compute nodes can access simultaneously,
• connected via a specialized, high-speed network with low latency,
• cooled in a special way to dissipate the heat generated during operation despite the compact design of the compute nodes (in the past, mostly through cooling doors with cold water; nowadays, mostly direct hot-water cooling)
• Interactive or automated, exclusive access to some or all of these compute nodes is controlled by a resource management system (a so-called job scheduler).
• User applications run faster on an HPC cluster than on a commercially available computer only if they can utilize the resources of multiple compute nodes simultaneously: either because multiple independent instances of the user application run simultaneously, thereby processing numerous different problems at the same time (with different input parameters); Or because a user application is able to break down the specific simulation task into subproblems and solve these subproblems simultaneously across many processing cores or even compute nodes.