Pitzer

Pitzer Production Deployment December 4

Pitzer, OSC's latest cluster, will be deployed to full production status on Tuesday, December 4. All users will have access to the cluster and will be able to submit jobs. For details on how to modify your jobs to run on Pitzer, please see https://bit.ly/2P7G4Zz For general information about the new cluster, please visit osc.edu and see our Cluster Computing pages. If you have any questions, please contact OSC help https://bit.ly/29AXmdf

CCAPP Condo on Pitzer Cluster

Condo model refers to that the participants (condo owners) lease one or more compute nodes for the shared cluster while OSC provides all infrastructure, as well as maintenance and services. CCAPP Condo on Pitzer cluster is owned by the Center for Cosmology and AstroParticle Physics, at OSU. Prof. Annika Peter has been heavily involved in specifying requirements.

Hardware

Detailed system specifications:

  • 13 total nodes

    • 40 cores per node

Pitzer SSH key fingerprints

These are the public key fingerprints for Pitzer:
pitzer: ssh_host_rsa_key.pub = 8c:8a:1f:67:a0:e8:77:d5:4e:3b:79:5e:e8:43:49:0e 
pitzer: ssh_host_ed25519_key.pub = 6d:19:73:8e:b4:61:09:a9:e6:0f:e5:0d:e5:cb:59:0b 
pitzer: ssh_host_ecdsa_key.pub = 6f:c7:d0:f9:08:78:97:b8:23:2e:0d:e2:63:e7:ac:93 

Services have been restored after switch failure

At about 1:50 am on November 14th, 4:05 am on November 17th, and 5:00 am on November 18th, OSC experienced three separate major switch failures. We restored all the services after each outage, and have completed the update to the NetApp appliance that provides the home directory service to address a separate bug triggered by the outage. We are still working with the vendor for the network switches on a permanent resolution to the bug that has caused these interruptions. We will continue to keep you informed.

Switch failure on Nov 17 2018

At about 4:05 am on November 17th, OSC experienced a major switch failure which resulted in the home directory service and GPFS file systems being disrupted. Most services were back up around 10 am, but some users may still be seeing stale file handles on GPFS. We are still working on recovering GPFS clients. For more updates, see: https://bit.ly/2DIXr1G

Reboot of NetApp as part of an upgrade on November 19

We will have a reboot of the NetApp as part of an upgrade, starting from 9:30 AM on Monday, November 19, 2018, to address a bug that causes NetApp issues caused by the network switch outage we had on Nov 14, 2018. Any cluster nodes, OnDemand service, and all filesystems won't be impacted by the reboot. We also do not expect any disruptions to users' jobs due to this reboot.

Major network switch outage on November 14, 2018

At about 1:50 AM on November 14th, OSC experienced a major switch failure which resulted in the home directory service being disrupted. As a result, the home directories were offline and all logins were failing to all clusters. All user-facing issues have been resolved and the services are back. Running jobs may recover, but please look at job output to verify correctness. Some jobs experienced failures and will need to be resubmitted. For more information, see: https://bit.ly/2FlCZFD

Pages