Pitzer

Pitzer SSH key fingerprints

These are the public key fingerprints for Pitzer:
pitzer: ssh_host_rsa_key.pub = 8c:8a:1f:67:a0:e8:77:d5:4e:3b:79:5e:e8:43:49:0e 
pitzer: ssh_host_ed25519_key.pub = 6d:19:73:8e:b4:61:09:a9:e6:0f:e5:0d:e5:cb:59:0b 
pitzer: ssh_host_ecdsa_key.pub = 6f:c7:d0:f9:08:78:97:b8:23:2e:0d:e2:63:e7:ac:93 

Services have been restored after switch failure

At about 1:50 am on November 14th, 4:05 am on November 17th, and 5:00 am on November 18th, OSC experienced three separate major switch failures. We restored all the services after each outage, and have completed the update to the NetApp appliance that provides the home directory service to address a separate bug triggered by the outage. We are still working with the vendor for the network switches on a permanent resolution to the bug that has caused these interruptions. We will continue to keep you informed.

Switch failure on Nov 17 2018

At about 4:05 am on November 17th, OSC experienced a major switch failure which resulted in the home directory service and GPFS file systems being disrupted. Most services were back up around 10 am, but some users may still be seeing stale file handles on GPFS. We are still working on recovering GPFS clients. For more updates, see: https://bit.ly/2DIXr1G

Reboot of NetApp as part of an upgrade on November 19

We will have a reboot of the NetApp as part of an upgrade, starting from 9:30 AM on Monday, November 19, 2018, to address a bug that causes NetApp issues caused by the network switch outage we had on Nov 14, 2018. Any cluster nodes, OnDemand service, and all filesystems won't be impacted by the reboot. We also do not expect any disruptions to users' jobs due to this reboot.

Major network switch outage on November 14, 2018

At about 1:50 AM on November 14th, OSC experienced a major switch failure which resulted in the home directory service being disrupted. As a result, the home directories were offline and all logins were failing to all clusters. All user-facing issues have been resolved and the services are back. Running jobs may recover, but please look at job output to verify correctness. Some jobs experienced failures and will need to be resubmitted. For more information, see: https://bit.ly/2FlCZFD

Pages