Intel MPI

Certain MPI-IO operations with intelmpi/2019.3 may crash, fail or proceed with errors on the home directory. We do not expect the same issue on our GPFS file system, such as the project space and the scratch space. The problem might be related to the known issue reported by HDF5 group. Please read the section "Problem Reading A Collectively Written Dataset in Parallel" from HDF5 Known Issues for more detail.

Set-up on Ruby

To configure your environment for the default version of Intel MPI, use module load intelmpi. To configure your environment for a specific version of Intel MPI, use module load intelmpi/version. For example, use  module load intelmpi/4.1.3.049  to load Intel MPI version 4.1.3.049 on Ruby.

You can use module spider intelmpi  to view available modules on Ruby.

Using Intel MPI

Software compiled against this module will use the libraries at runtime.

Building With Intel MPI

On Ruby, we have defined several environment variables to make it easier to build and link with the Intel MPI libraries.

Variable	Use
$MPI_CFLAGS	Use during your compilation step for C programs.
$MPI_CXXFLAGS	Use during your compilation step for C++ programs.
$MPI_FFLAGS	Use during your compilation step for Fortran programs.
$MPI_F90FLAGS	Use during your compilation step for Fortran 90 programs.
$MPI_LIBS	Use when linking your program to Intel MPI.

In general, for any application already set up to use  mpicccompilation should be fairly straightforward. 

Batch Usage on Ruby

When you log into ruby.osc.edu you are actually logged into a Linux box referred to as the login node. To gain access to the multiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request multiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. 

Non-interactive Batch Job (Parallel Run)

A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. The following example batch script file will run a program compiled against Intel MPI (called my-impi-application) for five hours on Ruby:

#PBS -N MyIntelMPIJob
#PBS -l nodes=4:ppn=20
#PBS -l walltime=5:00:00
module swap mvapich2 intelmpi
cd $PBS_O_WORKDIR
mpiexec my-impi-application

Set-up on Owens

To configure your environment for the default version of Intel MPI, use module load intelmpi. To configure your environment for a specific version of Intel MPI, use module load intelmpi/version. For example, use   module load intelmpi/5.1.3 to load Intel MPI version 5.1.3 on Owens.

You can use module spider intelmpi to view available modules on Owens.

Using Intel MPI

Software compiled against this module will use the libraries at runtime.

Building With Intel MPI

On Ruby, we have defined several environment variables to make it easier to build and link with the Intel MPI libraries.

VARIABLE	USE
$MPI_CFLAGS	Use during your compilation step for C programs.
$MPI_CXXFLAGS	Use during your compilation step for C++ programs.
$MPI_FFLAGS	Use during your compilation step for Fortran programs.
$MPI_F90FLAGS	Use during your compilation step for Fortran 90 programs.
$MPI_LIBS	Use when linking your program to Intel MPI.

In general, for any application already set up to use  mpicc, (or similar), compilation should be fairly straightforward. 

Batch Usage on Owens

When you log into owens.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the multiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request multiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. 

Non-interactive Batch Job (Parallel Run)

A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. The following example batch script file will run a program compiled against Intel MPI (called my-impi-application) for five hours on Owens:

#PBS -N MyIntelMPIJob
#PBS -l nodes=4:ppn=28
#PBS -l walltime=5:00:00
module swap mvapich2 intelmpi
cd $PBS_O_WORKDIR
mpiexec my-impi-application

Set-up on Pitzer

To configure your environment for the default version of Intel MPI, use module load intelmpi. 

Using Intel MPI

Software compiled against this module will use the libraries at runtime.

Building With Intel MPI

On Oakley, we have defined several environment variables to make it easier to build and link with the Intel MPI libraries.

VARIABLE	USE
$MPI_CFLAGS	Use during your compilation step for C programs.
$MPI_CXXFLAGS	Use during your compilation step for C++ programs.
$MPI_FFLAGS	Use during your compilation step for Fortran programs.
$MPI_F90FLAGS	Use during your compilation step for Fortran 90 programs.
$MPI_LIBS	Use when linking your program to Intel MPI.

In general, for any application already set up to use mpicc compilation should be fairly straightforward. 

Batch Usage on Pitzer

When you log into pitzer.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the multiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request multiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. 

Non-interactive Batch Job (Parallel Run)

A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. The following example batch script file will run a program compiled against Intel MPI (called my-impi-application) for five hours on Pitzer:

#PBS -N MyIntelMPIJob
#PBS -l nodes=4:ppn=28
#PBS -l walltime=5:00:00
module swap mvapich2 intelmpi
cd $PBS_O_WORKDIR
mpiexec my-impi-application