filesystem

Stale File Handles on GPFS clients

OSC is experiencing some problems with the Project and Scratch filesystems that are resulting in some jobs seeing "stale file handles". We are investigating the problem and will provide updates as we have them. Job scheduling is currently paused on all filesystems.

We are working with vendor engineers to resolve the issue, but you can currently still log in via SSH and access your files in your home directories. OnDemand logins will likely hang, and attempts to access files on Project or Scratch will likely fail.

Issue with GPFS on Owens since April 14, 2017

3:10PM 4/18/2017 Update: Rolling reboots on Owens have started to address this GPFS issue. 

We have had issues with GPFS mounts on Owens Cluster since Friday afternoon, April 14, 2017. The affected nodes have been marked offline to be restarted or rebooted to fix this issue. Jobs may have been negatively impacted by this issue since April 14. If you experience any 'stale file handle' or file not found errors, please let us know.

Scratch and Project are hung; schedulings have been paused

1:00PM 4/6/2017 Update:  The Scratch and Project file systems are back to normal service. Scheduling on systems are resumed. We are still investigating the causes to this problem and will keep you updated when we know more. 

 

The Scratch and Project file systems are currently hung. Schedulings on all three clusters (Owens, Ruby, and Oakley) have been paused for investigating this problem. We will update this page when we know more. 

Project space giving errors "No space left on device"

11/01/2016 11:52AM Update: This issue has been fixed. 

We have become aware of a problem with the Project storage space that gives errors "No space left on device". The cause is that the metadata subsystem is full. This issue might result in failures of users' jobs. 

We are working hard to fix this issue as soon as possible and really apologize for any inconveniences this may cause you. 

Pages

Subscribe to filesystem