OpenMPI 4 and NVHPC MPI Compatibility Issues with SLURM HWLOC
A pure MPI application using mpirun or mpiexec with more ranks than the number of NUMA nodes may encounter an error similar to the following:
Cardinal
BWA 0.7.17 vulnerability
Cardinal hosted a version of bwa that had an unpatched vulnerability, 0.7.17.
This version has been removed from Cardinal in favor of 0.7.18
LS-DYNA mpp-dyna Cardinal: Remote access error on mlx5_0:1, RDMA_READ
You may encounter the following error while running mpp-dyna jobs with multiple nodes:
Ansys OMP: System error #22: Invalid argument
You may encounter the following error while running Ansys on Cardinal:
OMP: Error #100: Fatal system error detected. OMP: System error #22: Invalid argument forrtl: error (76): Abort trap signal
Workaround
Set the environment variable KMP_AFFINITY=disabled before running Ansys
Abaqus Parallel Job Failure with PMPI Due to Out-of-Memory (OOM) Error
You may encounter the following error while running an Abaqus parallel job with PMPI:
Handling full-node MPI warnings with MVAPICH 3.0
When running a full-node MPI job with MVAPICH 3.0 , you may encounter the following warning message:
HWloc warning: Failed with: intersection without inclusion
When running MPI+OpenMP hybrid code with the Intel Classic Compiler and MVAPICH 3.0, you may encounter the following warning message from hwloc:
GCC 13 compilation errors due to missing headers
Users may encounter the following errors when compiling a C++ program with GCC 13:
error: 'uint64_t' in namespace 'std' does not name a type
or
HCOLL-related failures in OpenMPI applications
Several applications using OpenMPI, including HDF5, Boost, Rmpi, ORCA, and CP2K, may fail with errors such as
mca_coll_hcoll_module_enable() coll_hcol: mca_coll_hcoll_save_coll_handlers failed
or
Caught signal 11: segmentation fault
We have identified that the issue is related to HCOLL (Hierarchical Collectives) being enabled in OpenMPI.
STAR-CCM+ MPI job failure
STAR-CCM+ encounters errors when running MPI jobs using Intel MPI or Open MPI, displaying the following message:
ib_iface.c:1139 UCX ERROR Invalid active_speed on mlx5_0:1: 128
This issue occurs because the UCX library (v1.8) bundled with STAR-CCM+ only supports Mellanox InfiniBand EDR, whereas the Cardinal cluster utilizes newer Mellanox InfiniBand NDR (Next Data Rate) fabric. Because UCX v1.8 cannot recognize the NDR speed, STAR-CCM+ fails to communicate across the network.
