We are actively investigating intermittent slowness on the shared /fs/scratch filesystem.

OSC systems staff will execute a rolling reboot across all active compute clusters (Pitzer, Ascend, and Cardinal) beginning Thursday, November 05.

Known issues

Unresolved known issues

Known issue with an Unresolved Resolution state is an active problem under investigation; a temporary workaround may be available.

Resolved known issues

A known issue with a Resolved (workaround) Resolution state is an ongoing problem; a permanent workaround is available which may include using different software or hardware.

A known issue with Resolved Resolution state has been corrected.

Known Issues

Title Category Resolutionsort descending Description Posted Updated
HCOLL-related failures in OpenMPI applications Cardinal, Software Resolved
(workaround)

Several applications using OpenMPI, including HDF5, Boost, Rmpi, ORCA, and CP2K, may fail with errors such as

mca_coll_hcoll_module_enable() coll_hcol: mca_coll_hcoll_save_coll_handlers...
Read more
2 years 3 weeks ago 1 year 6 months ago
Stale File Handles on GPFS clients filesystem Resolved

OSC is experiencing some problems with the Project and Scratch filesystems that are resulting in some jobs seeing "stale file handles". We are investigating the problem and will provide updates as...

Read more
7 years 10 months ago 7 years 10 months ago
Correction on OSC project restriction email Account Management Resolved

Updated:

OSC has resolved this morning's issue and reverted impacted projects back to an ACTIVE status. Queued jobs under those projects will be able to start once today's...

Read more
5 years 4 months ago 5 years 4 months ago
Problems with MVAPICH2 Owens, Ruby, Software Resolved

Some MVAPICH2 MPI installations on Oakley, Ruby, and Owens, such as the default module mvapich2/2.2 as well as mvapich2/2.1, appear to have a bug that is triggered by certain programs.  The...

Read more
10 years 9 months ago 4 years 6 months ago
Nvidia drivers on Oakley GPU Resolved

We upgraded the drivers for the Nvidia GPUs on all of our clusters during the downtime this week. Unfortunately, we are noticing some subtle problems with the GPUs on Oakley. We will be rolling...

Read more
10 years 1 month ago 8 years 5 months ago
MPI fails with UCX 1.18 Software Resolved
(workaround)

After the downtime on August 19, 2025, users may encounter UCX errors such as:

UCX ERROR no active messages transport to <no debug data>: self/memory -...
Read more
1 year 2 months ago 1 year 1 month ago
Brief disruption to external network, 2013/11/27 Connectivity Resolved

This maintenence was cancelled, to be rescheduled at some undetermined point in the future.

Between 12:01AM and 2:00AM Eastern on Wednesday, November 27th 2013, OARnet will be...

Read more
13 years 1 day ago 12 years 11 months ago
MOE license server down Licensing Resolved

The MOE license server is experiencing an unknown issue and potentially down.  We are working to resolve the issue.

3 years 1 month ago 3 years 1 month ago
Occasional failures in file permissions filesystem Resolved

Users may experience occasional failures in file permissions with our filesystem. We've opened a case with the vendor for further investigations. If you get 'permission denied' message when you...

Read more
8 years 7 months ago 4 years 11 months ago
vLLM versions prior to 0.14.1 are deprecated Software Resolved

vLLM versions prior to 0.14.1 are deprecated due to security issue CVE-2026-22778 which can allow remote code execution.  Clients are advised to use versions 0....

Read more
9 months 2 days ago 9 months 2 days ago

Pages