Issue with GPFS on Owens since April 14, 2017

3:10PM 4/18/2017 Update: Rolling reboots on Owens have started to address this GPFS issue. 

We have had issues with GPFS mounts on Owens Cluster since Friday afternoon, April 14, 2017. The affected nodes have been marked offline to be restarted or rebooted to fix this issue. Jobs may have been negatively impacted by this issue since April 14. If you experience any 'stale file handle' or file not found errors, please let us know.

Rolling reboot of all clusters, starting from Wednesday morning, April 19, 2017

1:40PM 4/27/2017 Update: Rolling reboots are completed. 

3:10PM 4/18/2017 Update: Rolling reboots on Owens have started to address GPFS errors occured late Friday. 

Rolling reboot of Owens, Oakley, and Ruby clusters is scheduled to start from Wednesday morning, April 19, 2017. Highlights of the rolling reboot activities:

Scratch and Project are hung; schedulings have been paused

1:00PM 4/6/2017 Update:  The Scratch and Project file systems are back to normal service. Scheduling on systems are resumed. We are still investigating the causes to this problem and will keep you updated when we know more. 


The Scratch and Project file systems are currently hung. Schedulings on all three clusters (Owens, Ruby, and Oakley) have been paused for investigating this problem. We will update this page when we know more. 

Critical change about using $PFSDIR directory at OSC

Starting from Thursday, Feb 2nd, the $PFSDIR directory on scratch (/fs/scratch) won’t be created by job prologue. For example, if you simply use the command cd $PFSDIR, you will get an error indicating that this directory does not exist. The reason we are making this change is to address recent problems with the batch environments on OSC’s clusters. You will have to create the $PFSDIR directory by yourself if you use this directory. Please include the following additional lines in the job script.

If you use bash:


Subscribe to Batch