ParMETIS / METIS

ParMETIS (Parallel Graph Partitioning and Fill-reducing Matrix Ordering) is an MPI-based parallel library that implements a variety of algorithms for partitioning unstructured graphs, meshes, and for computing fill-reducing orderings of sparse matrices. ParMETIS extends the functionality provided by METIS and includes routines that are especially suited for parallel AMR computations and large scale numerical simulations. The algorithms implemented in ParMETIS are based on the parallel multilevel k-way graph-partitioning, adaptive repartitioning, and parallel multi-constrained partitioning schemes developed in Karypis lab.

METIS (Serial Graph Partitioning and Fill-reducing Matrix Ordering) is a set of serial programs for partitioning graphs, partitioning finite element meshes, and producing fill reducing orderings for sparse matrices. The algorithms implemented in METIS are based on the multilevel recursive-bisection, multilevel k-way, and multi-constraint partitioning schemes developed in Karypis lab.

Availability and Restrictions

ParMETIS is available on Oakley, Ruby, and Owens Clusters. The versions currently available at OSC are:

Version Oakley Ruby owens
4.0.3 X* X* X*
* : Current Default Version                                                        X: available for gnu and intel compilers

METIS is available on Oakley, Ruby, and Owens Clusters. The versions currently available at OSC are:

Version Oakley Ruby Owens
5.1.0 X* X* X*
* : Current Default Version                                                       X: available for gnu, intel, and pgi compilers

You can use module -r spider '.*metis.*'  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

ParMETIS / METIS is available without restriction to all OSC users.

Usage on Oakley

Set-up

To load ParMETIS, run the following command: module load parmetis . To use the serial implementation, METIS, run the following command instead: module load metis . You can use module spider metis   and module spider parmetis to view available modules. Use module spider metis/version and module spider parmetis/version to check what modules should be loaded before load ParMETIS / METIS.

Building With ParMETIS / METIS

With the ParMETIS library loaded, the following environment variables will be available for use:

Variable Use
$PARMETIS_CFLAGS Use during your compilation step for C or C++ programs.
$PARMETIS_LIBS

Use when linking your program to ParMETIS.

 

Similarly, when the METIS module is loaded, the following environment variables will be available:

VARIABLE USE
$METIS_CFLAGS Use during your compilation step for C programs.
$METIS_LIBS Use when linking your program to METIS.

 

For example, to build the code myprog.cc with the METIS library you would use:

g++ -o myprog myprog.cc $METIS_LIBS

Batch Usage

When you log into oakley.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info.  Batch jobs run on the compute nodes of the system and not on the login node. It is desirable for big problems since more resources can be used.

Non-interactive Batch Job (Serial Run)
A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. You must load the ParMETIS / METIS module in your batch script before executing a program which is built with the ParMETIS / METIS library. Below is the example batch script that executes a program built with ParMETIS:
#PBS -N myprogJob
#PBS -l nodes=1:ppn=12
module load gnu/4.8.4
module load parmetis
cd $PBS_O_WORKDIR
cp foo.dat $TMPDIR
cd $TMPDIR
myprog < foo.dat > foo.out
cp foo.out $PBS_O_WORKDIR

Usage on Ruby

Set-up

To load ParMETIS, run the following command: module load parmetis . To use the serial implementation, METIS, run the following command instead: module load metis . You can use module spider metis   and module spider parmetis to view available modules. Use module spider metis/version and module spider parmetis/version to check what modules should be loaded before load ParMETIS / METIS.

Building With ParMETIS / METIS

With the ParMETIS library loaded, the following environment variables will be available for use:

Variable Use
$PARMETIS_CFLAGS Use during your compilation step for C or C++ programs.
$PARMETIS_LIBS

Use when linking your program to ParMETIS.

 

Similarly, when the METIS module is loaded, the following environment variables will be available:

VARIABLE USE
$METIS_CFLAGS Use during your compilation step for C programs.
$METIS_LIBS Use when linking your program to METIS.

 

For example, to build the code myprog.cc with the METIS library you would use:

g++ -o myprog myprog.cc $METIS_LIBS

Batch Usage

When you log into ruby.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info.  Batch jobs run on the compute nodes of the system and not on the login node. It is desirable for big problems since more resources can be used.

Non-interactive Batch Job (Serial Run)
A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. You must load the ParMETIS / METIS module in your batch script before executing a program which is built with the ParMETIS / METIS library. Below is the example batch script that executes a program built with ParMETIS:
#PBS -N myprogJob
#PBS -l nodes=1:ppn=20
module load gnu/4.8.4
module load parmetis
cd $PBS_O_WORKDIR
cp foo.dat $TMPDIR
cd $TMPDIR
myprog < foo.dat > foo.out
cp foo.out $PBS_O_WORKDIR

Usage on Owens

Set-up

To load ParMETIS, run the following command: module load parmetis . To use the serial implementation, METIS, run the following command instead: module load metis . You can use module spider metis   and module spider parmetis to view available modules. Use module spider metis/version and module spider parmetis/version to check what modules should be loaded before load ParMETIS / METIS.

Building With ParMETIS / METIS

With the ParMETIS library loaded, the following environment variables will be available for use:

Variable Use
$PARMETIS_CFLAGS Use during your compilation step for C or C++ programs.
$PARMETIS_LIBS

Use when linking your program to ParMETIS.

 

Similarly, when the METIS module is loaded, the following environment variables will be available:

VARIABLE USE
$METIS_CFLAGS Use during your compilation step for C programs.
$METIS_LIBS Use when linking your program to METIS.

 

For example, to build the code myprog.cc with the METIS library you would use:

g++ -o myprog myprog.cc $METIS_LIBS

Batch Usage

When you log into owens.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. Batch jobs run on the compute nodes of the system and not on the login node. It is desirable for big problems since more resources can be used.

Non-interactive Batch Job (Serial Run)
A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. You must load the ParMETIS / METIS module in your batch script before executing a program which is built with the ParMETIS / METIS library. Below is the example batch script that executes a program built with ParMETIS:
#PBS -N myprogJob
#PBS -l nodes=1:ppn=28
module load gnu/4.8.5
module load parmetis
cd $PBS_O_WORKDIR
cp foo.dat $TMPDIR
cd $TMPDIR
myprog < foo.dat > foo.out
cp foo.out $PBS_O_WORKDIR

Further Reading

Tag: 
Service: