HPC
Thread Usage Best Practices
This document serves as a knowledge base for properly managing and diagnosing threading issues in user jobs. It focuses on OpenMP, Intel Math Kernel Library (MKL), and common thread-related misuse at OSC.
Understanding Threading with OpenMP and MKL
Intel MKL is widely used in HPC for linear algebra, FFTs, and statistical routines. MKL is multithreaded by default, which can significantly improve performance but only when correctly configured.
osc-seff
Introduction
osc-seff is a command developed at OSC for use on OSC's systems and provides a the CPU resource data of the seff command with the GPU resource data of gpu-seff.
gpu-seff
Introduction
gpu-seff is a command developed at OSC for use on OSC's systems and is similar providing GPU resource data, similar to the CPU resource data reported by the seff command.
Batch Limit Rules
Memory limit
It is strongly suggested to consider the memory use to the available per-core memory when users request OSC resources for their jobs.
Technical Specifications
The following are technical specifications for Cardinal.
- Number of Nodes
-
378 nodes
- Number of CPU Sockets
-
756 (2 sockets/node for all nodes)
- Number of CPU Cores
-
39,312
- Cores Per Node
-
104 cores/node for all nodes (96 usable)
Migrating jobs from other clusters
This page includes a summary of differences to keep in mind when migrating jobs from other clusters to Ascend.
Guidance for Pitzer Users
Hardware Specifications
Ascend (PER NODE) | Pitzer (PER NODE) | ||
---|---|---|---|
Regular compute node | n/a |
40 cores and 192GB of RAM 48 cores and 192GB of RAM |
Batch Limit Rules
Memory limit
2022 Storage Service Upgrades
In October 2022, OSC retires the Data Direct Networks (DDN) GRIDScaler system deployed in 2016 and expands the IBM Elastic Storage System (ESS) for both Project and global Scratch services. This expands the total capacity of Project and Scratch storage at OSC to ~16 petabytes with better performance.