OSC's Owens cluster being installed in 2016 is a Dell-built, Intel® Xeon® processor-based supercomputer.
Detailed system specifications:
- 824 Dell Nodes
- Dense Compute
648 compute nodes (Dell PowerEdge C6320 two-socket servers with Intel Xeon E5-2680 v4 (Broadwell, 14 cores, 2.40GHz) processors, 128GB memory)
1 60 ‘GPU ready’ compute nodes -- Dell PowerEdge R730 two-socket servers with Intel Xeon E5-2680 v4 (Broadwell, 14 cores, 2.40GHz) processors, 128GB memory
NVIDIA Tesla P100 (Pascal) GPUs -- 5.3TF peak (double precision), 16GB memory
16 huge memory nodes (Dell PowerEdge R930 four-socket server with Intel Xeon E5-4830 v3 (Haswell 12 core, 2.10GHz) processors, 1,536GB memory, 12 x 2TB drives)
- 23,392 total cores
- 28 cores/node & 128GB of memory/node
- Mellanox EDR (100Gbps) Infiniband networking
- Theoretical system peak performance
- ~750 teraflops (CPU only)
- 4 login nodes:
- Intel Xeon E5-2680 (Broadwell) CPUs
- 28 cores/node and 256GB of memory/node
How to Connect
To login to Owens at OSC, ssh to the following hostname:
You can either use an ssh client application or execute ssh on the command line in a terminal window as follows:
You may see warning message including SSH key fingerprint. Verify that the fingerprint in the message matches one of the SSH key fingerprint listed here, then type yes.
From there, you are connected to Owens login node and have access to the compilers and other software development tools. You can run programs interactively or through batch requests. We use control groups on login nodes to keep the login nodes stable. Please use batch jobs for any compute-intensive or memory-intensive work. See the following sections for details.
You can also login to Owens at OSC with our OnDemand tool. The first step is to login to OnDemand. Then once logged in you can access Owens by clicking on "Clusters", and then selecting ">_Owens Shell Access".
Instructions on how to connect to OnDemand can be found at the OnDemand documention page.
Owens accesses the same OSC mass storage environment as our other clusters. Therefore, users have the same home directory as on the Oakley and Ruby clusters. Full details of the storage environment are available in our storage environment guide.
Home directories should be accessed through either the
$HOME environment variable or the tilde notation (
~username ). Project directories are located at
/fs/project . Scratch storage is located at
The module system on Owens is the same as on the Oakley and Ruby systems. Use
module load <package> to add a software package to your environment. Use
module list to see what modules are currently loaded and
module avail to see the modules that are available to load. To search for modules that may not be visible due to dependencies or conflicts, use
module spider . By default, you will have the batch scheduling software modules, the Intel compiler and an appropriate version of mvapich2 loaded.
You can keep up to on the software packages that have been made available on Owens by viewing the Software by System page and selecting the Owens system.
Compiling Code to Use Advanced Vector Extensions (AVX2)
The Haswell and Broadwell processors that make up Owens support the Advanced Vector Extensions (AVX2) instruction set, but you must set the correct compiler flags to take advantage of it. AVX2 has the potential to speed up your code by a factor of 4 or more, depending on the compiler and options you would otherwise use.
In our experience, the Intel and PGI compilers do a much better job than the gnu compilers at optimizing HPC code.
With the Intel compilers, use
-O2 or higher. With the gnu compilers, use
-O3 . The PGI compilers by default use the highest available instruction set, so no additional flags are necessary.
This advice assumes that you are building and running your code on Owens. The executables will not be portable. Of course, any highly optimized builds, such as those employing the options above, should be thoroughly validated for correctness.
See the Owens Programming Environment page for details.
Refer to the documentation for our batch environment to understand how to use PBS on OSC hardware. Some specifics you will need to know to create well-formed batch scripts:
- The qsub syntax for node requests is the same on Owens as on Ruby and Oakley
- Most compute nodes on Owens have 28 cores/processors per node (ppn). Huge-memory (analytics) nodes have 48 cores/processors per node.
- Jobs on Owens may request partial nodes. This is in contrast to Ruby but similar to Oakley.
- Owens has 6 debug nodes which are specifically configured for short (< 1 hour) debugging type work. These nodes have a walltime limit of 1 hour.
- To schedule a debug node:
#PBS -l nodes=1:ppn=28 -q debug
- To schedule a debug node:
Using OSC Resources
For more information about how to use OSC resources, please see our guide on batch processing at OSC. For specific information about modules and file storage, please see the Batch Execution Environment page.