Users may experience performance issues in home directory. It is recommended to use temporary directory ($TMPDIR, or scratch) or project storage to minimize the impact on your jobs.
OSC is currently troubleshooting the cause. Contact oschelp@osc.edu if there are questions.
Users may receive an error when using the chgrp command on data in /fs/ess/ locations.
$ chgrp -v PEX1234 my-file.txt chgrp: changing group of 'my-file.txt': Disk quota exceeded failed to change group of 'my-file.txt' from PEX0001 to PEX1234
This is a very infrequent issue and OSC have identified a workaround.
Please contact oschelp@osc.edu for assistance with changing group ownership of data.
Symlinks to /fs/project directories were missed for a short period of time both on Tuesday afternoon October 11(right after downtime) and Wedenday morning October 12. It might result in job failures. We have fixed it.
Sorry for the disruption. Please contact OSC Help if you have any questions.
There was an issue with GPFS clients that affected /fs/project and possibly /fs/scratch between around 3:30AM and 8:30AM on Sunday September 4th. Some jobs from clients were also impacted.
The systems are working properly now. Sorry for the inconvenience. Please contact OSC Help if you have any questions.
Updates on May 20 2023:
verbsRDMA is enabled on Pitzer.
Updates on Dec 14 2022:
verbsRDMA is enabled on Owens during December 13 downtime. verbsRDMA remains disabled on Pitzer.
Original Post:
To prepare for expansion of the IBM Elastic Storage System (ESS), the data transfer is switched from InfiniBand (IB) verbsRDMA to IP over IB (IPoIB) after August 9th's downtime. This may cause some impact to the job performance.
We have been experiencing inconsistent performance degradation of ESS filesystem (/fs/ess), starting from 12/22/2021.
The systems are working properly now.
Updated 15:30 October 19:
The issue of little space for metadata on scratch storage is resolved. If you have any questions, please contact OSC Help.
Original Post:
The scratch storage /fs/scratch has very little free space for metadata starting from Saturday, October 16. This may result in the failure of file or directory creations on /fs/scratch.
We had several auto-reboots with one of our home directory servers. Jobs might be impacted.