Software List

Ohio Supercomputer Center (OSC) has a variety of software applications to support all aspects of scientific research. We are actively updating this documentation to ensure it matches the state of the supercomputers. This page is currently missing some content; use module avail on each system for a comprehensive list of available software.

Supercomputer: 
Service: 

ABAQUS

ABAQUS is a finite element analysis program owned and supported by SIMULIA, the Dassault Systèmes brand for Realistic Simulation.

Availability and Restrictions

ABAQUS is available on Ruby, Oakley, and Glenn Clusters. The available programs are ABAQUS/CAE, ABAQUS/Standard and ABAQUS/Explicit. The versions currently available at OSC are:

Version Glenn Oakley Ruby Notes
6.8-2 X      
6.8-4 X      
6.9-2 X*      
6.11-1 X      
6.11-2 X X   Default version on Oakley prior to 09/15/2015
6.12-pr3   X    
6.12 X X    
6.13     X  
6.14   X* X*  
6.14-5   X    
2016   X   Version scheme has been changed. 
*: Current default version

You can use  module avail abaqus  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access for Academic Users

OSC's ABAQUS license can only be used for educational, institutional, instructional, and/or research purposes. Only users who are faculty, research staff or students at the following institutions are permitted to utilized OSC's license:

  • The Ohio State University
  • University of Toledo
  • University of Cincinnati
  • Miami University of Ohio
  • Wright State University
  • University of Akron

Users from additional degree granting academic institutions may request to be added to this list per a cost by contacting OSC Help.

The use of ABAQUS for academic purposes requires validation. In order to obtain validation, please contact OSC Help for further instruction. 

If you are a SIMCenter user, please include your SIMCenter ID when you contact OSC Help for the license validation.

 

Access for Commerical Users

Contact OSC Help for getting access to ABAQUS if you are a commercial user.

Usage

Token Usage

ABAQUS software usage is monitored though a token-based license manager. This means very time you run a ABAQUS job, tokens are checked out from our pool for your tasks usage. To ensure your job is only started when its required ABAQUS tokens are available it is important to include a software flag within your job script's PBS directives.  A minimum of 5 tokens are required per a job, so a 1 node, 1 processor ABAQUS job would need the following PBS software flag: #PBS -l software=abaqus+5 . Jobs requiring more cores will need to request more tokens as calculated with the formula: M = int(5 x N^0.422) , where N is the total number of cores.  For common requests, you can refer to the following table:

         Cores            (nodes x ppn each):

1 2 3 4 6 8 12 16 24 32 48
Tokens needed: 5 6 7 8 10 12 14 16 19 21 25

Usage on Glenn

Set-up on Glenn

To load the default version of ABAQUS module, use  module load abaqus . To select a particular software version, use   module load abaqusversion . For example, use  module load abaqus6.12  to load ABAQUS version 6.12 on Glenn. 

Using ABAQUS

Example input data files are available with the ABAQUS release. The abaqus fetch utility is used to extract these input files for use. For example, to fetch input file for one of the sample problems including 4 input files, type:

abaqus fetch job=knee_bolster

abaqus fetch job=knee_bolster_ef1

abaqus fetch job=knee_bolster_ef2

abaqus fetch job=knee_bolster_ef3

Also, use the abaqus help  utility is to list all the abaqus execution procedures.

Batch Usage on Glenn

When you log into glenn.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your ABAQUS analysis to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. 

Interactive Batch Session
For an interactive batch session on Glenn, one can run the following command:
qsub -I -l nodes=1:ppn=8 -l walltime=1:00:00
which gives you 8 cores ( -l nodes=1:ppn=8 ) with 1 hour ( -l walltime=1:00:00 ). You may adjust the numbers per your need.
Non-interactive Batch Job (Serial Run)

batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice.

Assume that you have fetched the four input files above into your work directory (where you submit your job, represented by  $PBS_O_WORKDIR ). Below is the example batch script ( job.txt ) for a serial run:

#PBS -N knee
#PBS -l walltime=1:00:00
#PBS -l nodes=1:ppn=1
#PBS -l software=abaqus+5
#PBS -j oe
#
# The following lines set up the ABAQUS environment
#
module load abaqus
#
# Move to the directory where the job was submitted
#
cd $PBS_O_WORKDIR
cp *.inp $TMPDIR
cd $TMPDIR
#
# Run ABAQUS
#
abaqus job=knee_bolster interactive
#
# Now, copy data (or move) back once the simulation has completed
#
cp * $PBS_O_WORKDIR

In order to run it via the batch system, submit the job.txt  file with the command: qsub job.txt  

NOTE:

  • Make sure to copy all the files needed (input files, restart files, user subroutines, python scripts etc.) from your work directory ( $PBS_O_WORKDIR ) to  $TMPDIR , and copy your results back at the end of your script. Running your job on  $TMPDIR  ensures maximum efficiency.
  • The keyword interactive  is required in the execution line abaqus job=knee_bolster interactive for the following reason: If left off, ABAQUS will background the simulation process. Backgrounding a process in the OSC environment will place it outside of the batch job and it will receive the default 1 hour of CPU time and corresponding default memory limits. The keyword interactive in this case simply tells ABAQUS not to return until the simulation has completed.
  • The name of the input file is sometimes omitted in the execution line, which may work fine if you've copied only the input files for one specific model. Although, it is better practice to designate the main input file explicitly by adding input=<my_input_file_name>.inp to the execution line so that it looks like abaqus job=knee_bolster input=<my_input_file_name>.inp interactive
  • Define nodes=1 (1<=ppn<=8 for Glenn) for a serial run.
  • If ppn>1, add cpus=<n> to the execution line, where n=ppn. This is how it should look like: abaqus job=test input=<my_input_file_name1>.inp cpus=<n> interactive
Non-interactive Batch Job (Parallel Run)

Below is the example batch script ( job.txt ) for a parallel run:

#PBS -l walltime=1:00:00
#PBS -l nodes=2:ppn=8
#PBS -N my_abaqus_job
#PBS -l software=abaqus+16
#PBS -j oe
#
# The following lines set up the ABAQUS environment
#
module load abaqus
#
# Move to the directory where the job was submitted
#
cd $PBS_O_WORKDIR
cp *.inp $TMPDIR/
cd $TMPDIR
#
# Run ABAQUS, note that in this case we have provided the names of the input files explicitly
#
abaqus job=test input=<my_input_file_name1>.inp cpus=16 interactive
#
# Now, move data back once the simulation has completed
#
mv * $PBS_O_WORKDIR

NOTE:

  • Define nodes>1 for a parallel run. 1<=ppn<=8 for Glenn.
  • Specify  cpus=<n> in the execution line, where n=nodes*ppn.
  • Everything else is similar to the serial script above.

Usage on Oakley

Set-up on Oakley

To load the default version of ABAQUS module, use  module load abaqus . To select a particular software version, use   module load abaqus/version . For example, use  module load abaqus/6.12  to load ABAQUS version 6.12 on Oakley. 

Using ABAQUS

Example input data files are available with the ABAQUS release. The abaqus fetch utility is used to extract these input files for use. For example, to fetch input file for one of the sample problems including 4 input files, type:

abaqus fetch job=knee_bolster

abaqus fetch job=knee_bolster_ef1

abaqus fetch job=knee_bolster_ef2

abaqus fetch job=knee_bolster_ef3

Also, use the abaqus help  utility is to list all the abaqus execution procedures.

Batch Usage on Oakley

When you log into oakley.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your ABAQUS analysis to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. 

Interactive Batch Session
For an interactive batch session on Oakley, one can run the following command:
qsub -I -l nodes=1:ppn=12 -l walltime=1:00:00
which gives you 12 cores ( -l nodes=1:ppn=12 ) with 1 hour ( -l walltime=1:00:00 ). You may adjust the numbers per your need.
Non-interactive Batch Job (Serial Run)

batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice.

Assume that you have fetched the four input files above into your work directory (where you submit your job, represented by  $PBS_O_WORKDIR ). Below is the example batch script ( job.txt ) for a serial run:

#PBS -N knee
#PBS -l walltime=1:00:00
#PBS -l nodes=1:ppn=1
#PBS -l software=abaqus+5
#PBS -j oe
#
# The following lines set up the ABAQUS environment
#
module load abaqus
#
# Move to the directory where the job was submitted
#
cd $PBS_O_WORKDIR
cp *.inp $TMPDIR
cd $TMPDIR
#
# Run ABAQUS
#
abaqus job=knee_bolster interactive
#
# Now, copy data (or move) back once the simulation has completed
#
cp * $PBS_O_WORKDIR

In order to run it via the batch system, submit the job.txt  file with the command: qsub job.txt

NOTE:

  • Make sure to copy all the files needed (input files, restart files, user subroutines, python scripts etc.) from your work directory ( $PBS_O_WORKDIR ) to  $TMPDIR , and copy your results back at the end of your script. Running your job on  $TMPDIR  ensures maximum efficiency.
  • The keyword interactive  is required in the execution line abaqus job=knee_bolster interactive for the following reason: If left off, ABAQUS will background the simulation process. Backgrounding a process in the OSC environment will place it outside of the batch job and it will receive the default 1 hour of CPU time and corresponding default memory limits. The keyword interactive in this case simply tells ABAQUS not to return until the simulation has completed.
  • The name of the input file is sometimes omitted in the execution line, which may work fine if you've copied only the input files for one specific model. Although, it is better practice to designate the main input file explicitly by adding input=<my_input_file_name>.inp to the execution line so that it looks like abaqus job=knee_bolster input=<my_input_file_name>.inp interactive
  • Define nodes=1 (1<=ppn<=12 for Oakley) for a serial run.
  • If ppn>1, add cpus=<n> to the execution line, where n=ppn. This is how it should look like: abaqus job=test input=<my_input_file_name1>.inp cpus=<n> interactive
Non-interactive Batch Job (Parallel Run)

Below is the example batch script ( job.txt ) for a parallel run:

#PBS -l walltime=1:00:00
#PBS -l nodes=2:ppn=12
#PBS -N my_abaqus_job
#PBS -l software=abaqus+19
#PBS -j oe
#
# The following lines set up the ABAQUS environment
#
module load abaqus
#
# Move to the directory where the job was submitted
#
cd $PBS_O_WORKDIR
cp *.inp $TMPDIR/
cd $TMPDIR
#
# Run ABAQUS, note that in this case we have provided the names of the input files explicitly
#
abaqus job=test input=<my_input_file_name1>.inp cpus=24 interactive
#
# Now, move data back once the simulation has completed
#
mv * $PBS_O_WORKDIR

NOTE:

  • Define nodes>1 for a parallel run. 1<=ppn<=12 for Oakley.
  • Specify  cpus=<n> in the execution line, where n=nodes*ppn.
  • Everything else is similar to the serial script above.

Usage on Ruby

Set-up on Ruby

To load the default version of ABAQUS module, use  module load abaqus . To select a particular software version, use   module load abaqus/version . For example, use  module load abaqus/6.13  to load ABAQUS version 6.13 on Ruby. 

Using ABAQUS

Example input data files are available with the ABAQUS release. The abaqus fetch utility is used to extract these input files for use. For example, to fetch input file for one of the sample problems including 4 input files, type:

abaqus fetch job=knee_bolster

abaqus fetch job=knee_bolster_ef1

abaqus fetch job=knee_bolster_ef2

abaqus fetch job=knee_bolster_ef3

Also, use the abaqus help  utility is to list all the abaqus execution procedures.

Batch Usage on Ruby

When you log into ruby.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your ABAQUS analysis to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. 

Interactive Batch Session
For an interactive batch session on Ruby, one can run the following command:
qsub -I -l nodes=1:ppn=20 -l walltime=1:00:00
which gives you 20 cores ( -l nodes=1:ppn=20 ) with 1 hour ( -l walltime=1:00:00 ). You may adjust the wall time per your need.
Non-interactive Batch Job (Serial Run)

batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice.

Assume that you have fetched the four input files above into your work directory (where you submit your job, represented by  $PBS_O_WORKDIR ). Below is the example batch script ( job.txt ) for a serial run:

#PBS -N knee
#PBS -l walltime=1:00:00
#PBS -l nodes=1:ppn=20
#PBS -l software=abaqus+17
#PBS -j oe
#
# The following lines set up the ABAQUS environment
#
module load abaqus
#
# Move to the directory where the job was submitted
#
cd $PBS_O_WORKDIR
cp *.inp $TMPDIR
cd $TMPDIR
#
# Run ABAQUS
#
abaqus job=knee_bolster cpus=20 interactive
#
# Now, copy data (or move) back once the simulation has completed
#
cp * $PBS_O_WORKDIR

In order to run it via the batch system, submit the job.txt  file with the command: qsub job.txt

NOTE:

  • Make sure to copy all the files needed (input files, restart files, user subroutines, python scripts etc.) from your work directory ( $PBS_O_WORKDIR ) to  $TMPDIR , and copy your results back at the end of your script. Running your job on  $TMPDIR  ensures maximum efficiency.
  • The keyword interactive  is required in the execution line abaqus job=knee_bolster interactive for the following reason: If left off, ABAQUS will background the simulation process. Backgrounding a process in the OSC environment will place it outside of the batch job and it will receive the default 1 hour of CPU time and corresponding default memory limits. The keyword interactive in this case simply tells ABAQUS not to return until the simulation has completed.
  • The name of the input file is sometimes omitted in the execution line, which may work fine if you've copied only the input files for one specific model. Although, it is better practice to designate the main input file explicitly by adding input=<my_input_file_name>.inp to the execution line so that it looks like abaqus job=knee_bolster input=<my_input_file_name>.inp interactive
  • Ruby has full node charging policy. Define nodes=1:ppn=20   for a serial run
Non-interactive Batch Job (Parallel Run)

Below is the example batch script ( job.txt ) for a parallel run:

#PBS -l walltime=1:00:00
#PBS -l nodes=2:ppn=20
#PBS -N my_abaqus_job
#PBS -l software=abaqus+23
#PBS -j oe
#
# The following lines set up the ABAQUS environment
#
module load abaqus
#
# Move to the directory where the job was submitted
#
cd $PBS_O_WORKDIR
cp *.inp $TMPDIR/
cd $TMPDIR
#
# Run ABAQUS, note that in this case we have provided the names of the input files explicitly
#
abaqus job=test input=<my_input_file_name1>.inp cpus=40 interactive
#
# Now, move data back once the simulation has completed
#
mv * $PBS_O_WORKDIR

NOTE:

  • Define nodes>1 for a parallel run. 
  • Specify  cpus=<n> in the execution line, where n=nodes*ppn.
  • Everything else is similar to the serial script above.

Further Reading

See Also

Supercomputer: 
Service: 
Fields of Science: 

ABINIT

ABINIT is a package whose main program allows one to find the total energy, charge density and electronic structure of systems made of electrons and nuclei (molecules and periodic solids) within Density Functional Theory (DFT), using pseudopotentials and a planewave or wavelet basis. ABINIT also includes options to optimize the geometry according to the DFT forces and stresses, or to perform molecular dynamics simulations using these forces, or to generate dynamical matrices, Born effective charges, and dielectric tensors, based on Density-Functional Perturbation Theory, and many more properties.

Availability and Restrictions

The following versions of ABINIT are available on OSC clusters:

Version Oakley
7.10.5 X*
*: Current default version

You can use  module avail abinit  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

ABINIT is available to all OSC users without restriction.

Usage on Oakley

Set-up

To configure your environment for use of ABINIT on Oakley, run the following command:   module load abinit. The default version will be loaded. To select a particular ABINIT version, use   module load abinit/version  . For example, use  module load abinit/7.10.5 to load ABINIT 7.10.5 on Oakley.

Further Reading

Supercomputer: 
Service: 

ACML - AMD Core Math Library

The AMD Core Math Library (ACML) is a set of numerical routines tuned specifically for AMD64 platform processors (including Opteron and Athlon64 ). The routines, which are available via both FORTRAN 77 and C interfaces, include BLAS, LAPACK, FFT, RND, and many others.

Availability & Restrictions

OSC supports use of ACML on the Glenn cluster.

VERSION GLENN OAKLEY
3.6.0 X  
3.6.1 X  
4.0.1 X  
4.3.0 X  
4.4.0 X  
There are no restrictions on the use of the ACML library.

Usage

Set-up

To configure your environment for use of ACML load the “acml” module appropriate for your compiler:

Compiler Module Command
GNU Fortran module load acml-gfortran
Intel module load acml-intel
Portland Group module load acml-pgi
Portland Group, Multi-Core module load acml-pgimp

This step is required for both building and running ACML applications. The default version is 4.0.1.

Building With ACML

The Glenn ACML modules will automatically configure your environment to locate appropriate include files and libraries. When the ACML modules are loaded the following environment variables will be set:

Variable Use
$OMP_NUM_THREADS Number of OpenMP threads to be used. Set to 1 in all ACML modules except acml-pgimp, which sets it to 4. Should be set by the user AFTER loading the module if a different value is needed.
$ACML_CFLAGS Compiler flags: include path to be used
$ACML Linker flags: libraries to be used

Usage with the Portland Group Compiler, C code

To build a sample C-code that uses the ACML library with the Portland Group Compiler on Glenn, follow the example steps below after logging into the system:

$ cp ~support/examples/ACML/example.c .
$ module load acml-pgi
$ pgcc $ACML_CFLAGS $ACML -lm -lpgftnrtl –lrt example.c
$ ./a.out

Usage with the Portland Group Compiler, FORTRAN code

To build a sample FORTRAN-code that uses the ACML library with the Portland Group Compiler on Glenn, follow the example steps below after logging into the system:

$ cp ~support/examples/ACML/example.f .
$ module load acml-pgi
$ pgf77 example.f $ACML
$ ./a.out

Further Reading

See Also

  • Intel MKL
  • Armstrong Libraries and Compilers Group
Tag: 
Supercomputer: 
Service: 

AMBER

The Assisted Model Building with Energy Refinement (AMBER) package contains many molecular simulation programs targeted at biomolecular systems. A wide variety of modelling techniques are available. It generally scales well on modest numbers of processors, and the GPU enabled CUDA programs are very efficient.

Availability and Restrictions

AMBER is available on Oakley, Ruby, and Owens Clusters. The following versions are currently available at OSC (S means serial executables, P means parallel, C means CUDA, i.e., GPU enabled, and M means MIC, i.e., Xeon Phi enabled):

Version Oakley ruby Owens notes
11 SPC     Default version on Oakley prior to 09/15/2015
12 SPC      
14 SPC* SPC*    
16 SPC SPCM SP  
*: Current default version

You can use module avail amber to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access for Academic Users

OSC's Amber is available to not-for-profit OSC users; simply contact OSC Help to request the appropriate form for access.

Access for Commerical Users

For-profit OSC users must obtain their own Amber license. 

Usage on Oakley

Set-up

To load the default version of AMBER module, use  module load amber  . To select a particular software version, use   module load amber/version . For example, use  module load amber/12   to load AMBER version 12. 

Using AMBER

A serial Amber program in a short duration run can be executed interactively on the command line, e.g.:

tleap

Parallel Amber programs must be run in a batch environment with mpiexec, e.g.:

mpiexec pmemd.MPI

The CUDA Amber programs, which must be run in a batch environment, are recommended due to their computational performance; e.g.:

pmemd.cuda

For supported features of the GPU accelerated version of PMEMD see Amber GPU Support.

In Amber14 the CUDA programs no longer support Glenn's hardware, and thus cannot be installed.

Batch Usage

When you log into oakley.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your AMBER analysis to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. 

Interactive Batch Session
For an interactive batch session, one can run the following command:
qsub -I -l nodes=1:ppn=12 -l walltime=1:00:00
which gives you 12 cores ( -l nodes=1:ppn=12 ) with 1 hour ( -l walltime=1:00:00 ). You may adjust the numbers per your need.
Non-interactive Batch Job (Serial Run)

batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. Sample batch scripts and Amber input files are available here:

~srb/workshops/compchem/amber/

Below is the example batch script (  job.txt  ) for a serial run:

# AMBER Example Batch Script for the Basic Tutorial in the Amber manual
#PBS -N 6pti
#PBS -l nodes=1:ppn=1
#PBS -l walltime=0:20:00
module load amber
# Use TMPDIR for best performance.
cd $TMPDIR
# PBS_O_WORKDIR refers to the directory from which the job was submitted.
cp -p $PBS_O_WORKDIR/6pti.prmtop .
cp -p $PBS_O_WORKDIR/6pti.prmcrd .
# Running minimization for BPTI
cat << eof > min.in
# 200 steps of minimization, generalized Born solvent model
&cntrl
maxcyc=200, imin=1, cut=12.0, igb=1, ntb=0, ntpr=10,
/
eof
sander -i min.in -o 6pti.min1.out -p 6pti.prmtop -c 6pti.prmcrd -r 6pti.min1.xyz
cp -p min.in 6pti.min1.out 6pti.min1.xyz $PBS_O_WORKDIR

In order to run it via the batch system, submit the job.txt  file with the command: qsub job.txt  

Usage on Ruby

Set-up

To load the default version of AMBER module, use  module load amber  . To select a particular software version, use   module load amber/version . For example, use  module load amber/14   to load AMBER version 14. 

Using AMBER

A serial Amber program in a short duration run can be executed interactively on the command line, e.g.:

tleap

Parallel Amber programs must be run in a batch environment with mpiexec, e.g.:

mpiexec pmemd.MPI

The CUDA Amber programs, which must be run in a batch environment, are recommended due to their computational performance; e.g.:

pmemd.cuda

For supported features of the GPU accelerated version of PMEMD see Amber GPU Support.

In Amber14 the CUDA programs no longer support Glenn's hardware, and thus cannot be installed.

Batch Usage

When you log into ruby.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your AMBER analysis to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. 

Interactive Batch Session
For an interactive batch session, one can run the following command:
qsub -I -l nodes=1:ppn=20 -l walltime=1:00:00
which gives you 20 cores ( -l nodes=1:ppn=20 ) with 1 hour ( -l walltime=1:00:00 ). You may adjust the numbers per your need.
Non-interactive Batch Job (Serial Run)

batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. Sample batch scripts and Amber input files are available here:

~srb/workshops/compchem/amber/

Below is the example batch script (  job.txt  ) for a serial run:

# AMBER Example Batch Script for the Basic Tutorial in the Amber manual
#PBS -N 6pti
#PBS -l nodes=1:ppn=20
#PBS -l walltime=0:20:00
module load amber
# Use TMPDIR for best performance.
cd $TMPDIR
# PBS_O_WORKDIR refers to the directory from which the job was submitted.
cp -p $PBS_O_WORKDIR/6pti.prmtop .
cp -p $PBS_O_WORKDIR/6pti.prmcrd .
# Running minimization for BPTI
cat << eof > min.in
# 200 steps of minimization, generalized Born solvent model
&cntrl
maxcyc=200, imin=1, cut=12.0, igb=1, ntb=0, ntpr=10,
/
eof
sander -i min.in -o 6pti.min1.out -p 6pti.prmtop -c 6pti.prmcrd -r 6pti.min1.xyz
cp -p min.in 6pti.min1.out 6pti.min1.xyz $PBS_O_WORKDIR

In order to run it via the batch system, submit the job.txt  file with the command: qsub job.txt  

Usage on Owens

Set-up

To load the default version of AMBER module, use  module load amber  . To select a particular software version, use   module load amber/version . For example, use  module load amber/16   to load AMBER version 16. 

Using AMBER

A serial Amber program in a short duration run can be executed interactively on the command line, e.g.:

tleap

Parallel Amber programs must be run in a batch environment with mpiexec, e.g.:

mpiexec pmemd.MPI

For supported features of the GPU accelerated version of PMEMD see Amber GPU Support.

Batch Usage

When you log into owens.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your AMBER analysis to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. 

Interactive Batch Session
For an interactive batch session, one can run the following command:
qsub -I -l nodes=1:ppn=28 -l walltime=1:00:00
which gives you 28 cores ( -l nodes=1:ppn=28 ) with 1 hour ( -l walltime=1:00:00 ). You may adjust the numbers per your need.
Non-interactive Batch Job (Serial Run)

batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. Sample batch scripts and Amber input files are available here:

~srb/workshops/compchem/amber/

Below is the example batch script (  job.txt) for a serial run:

# AMBER Example Batch Script for the Basic Tutorial in the Amber manual
#PBS -N 6pti
#PBS -l nodes=1:ppn=28
#PBS -l walltime=0:20:00
module load amber
# Use TMPDIR for best performance.
cd $TMPDIR
# PBS_O_WORKDIR refers to the directory from which the job was submitted.
cp -p $PBS_O_WORKDIR/6pti.prmtop .
cp -p $PBS_O_WORKDIR/6pti.prmcrd .
# Running minimization for BPTI
cat << eof > min.in
# 200 steps of minimization, generalized Born solvent model
&cntrl
maxcyc=200, imin=1, cut=12.0, igb=1, ntb=0, ntpr=10,
/
eof
sander -i min.in -o 6pti.min1.out -p 6pti.prmtop -c 6pti.prmcrd -r 6pti.min1.xyz
cp -p min.in 6pti.min1.out 6pti.min1.xyz $PBS_O_WORKDIR

In order to run it via the batch system, submit the job.txt  file with the command: qsub job.txt  

Further Reading

Supercomputer: 
Service: 

ANSYS

ANSYS offers a comprehensive software suite that spans the entire range of physics, providing access to virtually any field of engineering simulation that a design process requires. Supports are provided by ANSYS, Inc

Availability and Restrictions

OSC has academic license of structural-fluid dynamics academic products, which offer structural mechanics, explicit dynamics, fluid dynamics and thermal simulation capabilities. These bundles also include ANSYS Workbench, relevant CAD import tools, solid modeling and meshing, and High Performance Computing (HPC) capability. See "Academic Research -> ANSYS Academic Research Mechanical and CFD" in this table for all available products at OSC.

Access for Academic Users

OSC has an "Academic Research " license for ANSYS. This allows for academic use of the software by Ohio faculty and students, with some restrictions. To view current ANSYS node restrictions, please see ANSYS's Terms of Use.

Use of ANSYS products at OSC for academic purposes requires validation. Please contact OSC Help for further instruction.

Access for Commerical Users

Contact OSC Help for getting access to ANSYS if you are a commerical user.

Usage

For more information on how to use each ANSYS product at OSC systems, refer to its documentation page provided at the end of this page.

Note

Due to the way our Fluent and ANSYS modules are configured, simultaneously loading multiple of either module will cause a cryptic error. The most common case of this happening is when multiple of a users jobs are started at the same time and all load the module at once. In order for this error to manifest, the modules have to be loaded at precisely the same time; a rare occurrence, but a probable occurrence over the long term.

If you encounter this error you are not at fault. Please resubmit the failed job(s).

If you frequently submit large amounts of Fluent or ANSYS jobs, we recommend you stagger your job submit times to lower the chances of two jobs starting at the same time, and hence loading the module at the same time. Another solution is to establish job dependencies between jobs, so jobs will only start one after another. To do this, you would add the PBS directive:

#PBS -W after:jobid

To jobs you want to only start after another job has started. You would replace jobid with the job ID of the job to wait for. If you have additional questions, please contact OSC Help.

Further Reading

See Also

Supercomputer: 
Service: 

ANSYS Mechanical

ANSYS Mechanical is a finite element analysis (FEA) tool that enables you to analyze complex product architectures and solve difficult mechanical problems. You can use ANSYS Mechanical to simulate real world behavior of components and sub-systems, and customize it to test design variations quickly and accurately.

Availability and Restrictions

ANSYS Mechanical is available on Oakley and Glenn Clusters. The versions currently available at OSC are:

Version Glenn Oakley Notes
11.0 X    
11.0  update X    
13.0 X    
14.0   X  
14.5 X*    
14.5.7   X Default verion on Oakley prior to 09/15/2015
16.0   X*  
*: Current default version

You can use module avail ansys to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access for Academic Users

Use of ANSYS for academic purposes requires validation. In order to obtain validation, please contact OSC Help for further instruction.

Access for Commerical Users

Contact OSC Help for getting access to ANSYS if you are a commerical user.

Usage

Usage on Glenn

Set-up on Glenn

To load the default version of ANSYS module, use  module load ansys . To select a particular software version, use   module load ansysversion . For example, use  module load ansys130  to load ANSYS version 13.0 on Glenn. 

Using ANSYS Mechanical

Following a successful loading of the ANSYS module, you can access the ANSYS Mechanical commands and utility programs located in your execution path:

ansys <switch options> <file>

The ANSYS Mechanical command takes a number of Unix-style switches and parameters.

The -j Switch

The command accepts a -j switch. It specifies the "job id," which determines the naming of output files. The default is the name of the input file.

The -d Switch

The command accepts a -d switch. It specifies the device type. The value can be X11, x11, X11C, x11c, or 3D.

The -m Switch

The command accepts a -m switch. It specifies the amount of working storage obtained from the system. The units are megawords.

The memory requirement for the entire execution will be approximately 5300000 words more than the -m specification. This is calculated for you if you use ansnqs to construct an NQS request.

The -b [nolist] Switch

The command accepts a -b switch. It specifies that no user input is expected (batch execution).

The -s [noread] Switch

The command accepts a -s switch. By default, the start-up file is read during an interactive session and not read during batch execution. These defaults may be changed with the -s command line argument. The noread option of the -s argument specifies that the start-up file is not to be read, even during an interactive session. Conversely, the -s argument with the -b batch argument forces the reading of the start-up file during batch execution.

The -g [off] Switch

The command accepts a -g switch. It specifies that the ANSYS graphical user interface started automatically.

ANSYS Mechanical parameters

ANSYS Mechanical parameters may be assigned values on the command. The parameter must be at least two characters long and must be a legal parameter name. The ANSYS Mechanical parameter that is to be assigned a value should be given on the command line with a preceding dash (-), a space immediately after, and the value immediately after the space:

module load ansys
ansys -pval1 -10.2 -EEE .1e6
sets pval1 to -10.2 and EEE to 100000

Batch Usage on Glenn

When you log into glenn.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your ANSYS Mechanical analysis to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time, up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. Batch jobs run on the compute nodes of the system and not on the login node. It is desirable for big problems since more resources can be used.

Interactive Batch Session

Interactive mode is similar to running ANSYS Mechanical on a desktop machine in that the graphical user interface will be sent from OSC and displayed on the local machine. Interactive jobs are run on compute nodes of the cluster, by turning on X11 forwarding. The intention is that users can run ANSYS Mechanical interactively for the purpose of building their model and preparing their input file. Once developed this input file can then be run in no-interactive batch mode.

To run interactive ANSYS Mechanical , a batch job need to be submitted from the login node, to request necessary compute resources, with X11 forwarding. For example, the following command requests one whole node with 8 cores ( -l nodes=1:ppn=8 ), for a walltime of 1 hour ( -l walltime=1:00:00 ), with one ANSYS license:

qsub -I -X -l nodes=1:ppn=8 -l walltime=1:00:00 -l software=ansys+1

You may adjust the numbers per your need. This job will queue until resources becomes available. Once the job is started, you're automatically logged in on the compute node; and you can launch ANSYS Mechanical and start the graphic interface with the following commands:

module load ansys
ansys -g
Non-interactive Batch Job (Serial Run)

A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice.

Starting from September 15, 2015, Glenn only accepts ANSYS jobs with ANSYS base license ( software=ansys+1 ); and a job using HPC tokens (with "ansyspar" flag) should be submitted to Oakley clusters due to scheduler issue.

For a given model, prepare the input file with ANSYS Mechanical commands (named ansys.in  for example) for the batch run. Below is the example batch script (  job.txt ) for a serial run:

#PBS -N ansys_test  
#PBS -l walltime=30:00:00  
#PBS -l nodes=1:ppn=1
#PBS -l software=ansys+1  
#PBS -j oe
cd $TMPDIR  
cp $PBS_O_WORKDIR/ansys.in .    
module load ansys  
ansys < ansys.in   
cp * $PBS_O_WORKDIR

In order to run it via the batch system, submit the job.txt  file with the command: qsub job.txt  

Usage on Oakley

Set-up on Oakley

To load the default version of ANSYS module, use  module load ansys . To select a particular software version, use   module load ansys/version . For example, use  module load ansys/13  to load ANSYS version 13.0 on Oakley. 

Using ANSYS Mechanical

Following a successful loading of the ANSYS module, you can access the ANSYS Mechanical commands and utility programs located in your execution path:

ansys <switch options> <file>

The ANSYS Mechanical command takes a number of Unix-style switches and parameters.

The -j Switch

The command accepts a -j switch. It specifies the "job id," which determines the naming of output files. The default is the name of the input file.

The -d Switch

The command accepts a -d switch. It specifies the device type. The value can be X11, x11, X11C, x11c, or 3D.

The -m Switch

The command accepts a -m switch. It specifies the amount of working storage obtained from the system. The units are megawords.

The memory requirement for the entire execution will be approximately 5300000 words more than the -m specification. This is calculated for you if you use ansnqs to construct an NQS request.

The -b [nolist] Switch

The command accepts a -b switch. It specifies that no user input is expected (batch execution).

The -s [noread] Switch

The command accepts a -s switch. By default, the start-up file is read during an interactive session and not read during batch execution. These defaults may be changed with the -s command line argument. The noread option of the -s argument specifies that the start-up file is not to be read, even during an interactive session. Conversely, the -s argument with the -b batch argument forces the reading of the start-up file during batch execution.

The -g [off] Switch

The command accepts a -g switch. It specifies that the ANSYS graphical user interface started automatically.

ANSYS Mechanical parameters

ANSYS Mechanical parameters may be assigned values on the command. The parameter must be at least two characters long and must be a legal parameter name. The ANSYS Mechanical parameter that is to be assigned a value should be given on the command line with a preceding dash (-), a space immediately after, and the value immediately after the space:

module load ansys
ansys -pval1 -10.2 -EEE .1e6
sets pval1 to -10.2 and EEE to 100000

Batch Usage on Oakley

When you log into oakley.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your ANSYS Mechanical analysis to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info.  Batch jobs run on the compute nodes of the system and not on the login node. It is desirable for big problems since more resources can be used.

Interactive Batch Session

Interactive mode is similar to running ANSYS Mechanical on a desktop machine in that the graphical user interface will be sent from OSC and displayed on the local machine. Interactive jobs are run on compute nodes of the cluster, by turning on X11 forwarding. The intention is that users can run ANSYS Mechanical interactively for the purpose of building their model and preparing their input file. Once developed this input file can then be run in no-interactive batch mode.

To run interactive ANSYS Mechanical, a batch job need to be submitted from the login node, to request necessary compute resources, with X11 forwarding. For example, the following command requests one whole node with 12 cores ( -l nodes=1:ppn=12 ), for a walltime of 1 hour (  -l walltime=1:00:00 ), with one ANSYS license:

qsub -I -X -l nodes=1:ppn=12 -l walltime=1:00:00 -l software=ansys+1

You may adjust the numbers per your need. This job will queue until resources becomes available. Once the job is started, you're automatically logged in on the compute node; and you can launch ANSYS Mechanical and start the graphic interface with the following commands:

module load ansys
ansys -g
Non-interactive Batch Job (Serial Run)

A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. For a given model, prepare the input file with ANSYS Mechanical commands (named  ansys.in  for example) for the batch run. Below is the example batch script (   job.txt ) for a serial run:

#PBS -N ansys_test  
#PBS -l walltime=30:00:00  
#PBS -l nodes=1:ppn=1
#PBS -l software=ansys+1  
#PBS -j oe
cd $TMPDIR  
cp $PBS_O_WORKDIR/ansys.in .    
module load ansys  
ansys < ansys.in   
cp * $PBS_O_WORKDIR

In order to run it via the batch system, submit the  job.txt  file with the command:  qsub job.txt .

Non-interactive Batch Job (Parallel Run)

To take advantage of the powerful compute resources at OSC, you may choose to run distributed ANSYS Mechanical for large problems. Multiple nodes and cores can be requested to accelerate the solution time. Note that you'll need to change your batch script slightly for distributed runs.

Starting from September 15, 2015, a job using HPC tokens (with "ansyspar" flag) should be submitted to Oakley clusters due to scheduler issue.

For distributed ANSYS Mechanical jobs using one node (nodes=1), the number of processors needs to be specified in the command line with options '-dis -np':

#PBS -N ansys_test 
#PBS -l walltime=3:00:00 
#PBS -l nodes=1:ppn=12
#PBS -W x=GRES:ansys+1%ansyspar+8
...
ansys -dis -np 12 < ansys.in  
...

Notice that in the script above, the ansys parallel license is requested as well as ansys license in the format of

#PBS -W x=GRES:ansys+1%ansyspar+n

where n=m-4, with m being the total cpus called for this job. This line is necessary when the total cpus called is greater than 4 (m>4), which applies for the parallel example below as well.

For distributed jobs requesting multiple nodes, you need to specify the number of processors for each node in the command line. This information can be obtained from $PBS_NODEFILE. The following shows changes in the batch script if 2 nodes on Oakley are requested for a parallel ANSYS Mechanical job:

#PBS -N ansys_test 
#PBS -l walltime=3:00:00 
#PBS -l nodes=2:ppn=12
#PBS -W x=GRES:ansys+1%ansyspar+20
...
export MPI_WORKDIR=$PWD
machines=`uniq -c ${PBS_NODEFILE} | awk '{print $2 ":" $1}' | paste -s -d ':'`
ansys -dis -machines $machines < ansys.in  
...
pbsdcp -g '*' $PBS_O_WORKDIR

The 'pbsdcp -g' command in the last line in the script above makes sure that all result files generated by different compute nodes are copied back to the work directory.

Further Reading

See Also

Supercomputer: 
Service: 

CFX

ANSYS CFX (called CFX hereafter) is a computational fluid dynamics (CFD) program for modeling fluid flow and heat transfer in a variety of applications.

Availability and Restrictions

CFX is available on Oakley and Glenn Clusters. The versions currently available at OSC are:

VERSION GLENN OAKLEY notes
14.5 X*    
14.5.7   X  
15.0.7   X*  
16.0   X  
*: Current default version

You can use module avail fluent  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access for Academic Users

Use of ANSYS products for academic purposes requires validation. In order to obtain validation, please contact OSC Help for further instruction.

Currently, there are in total 25 ANSYS CFD base license tokens and 68 HPC tokens for academic users. These base tokens are shared by available ANSYS CFD related projusts (see "Academic Research -> ANSYS Academic Research CFD" in this table for details). These HPC tokens are shared with all ANSYS products we have at OSC. A base license token will allow CFX to use up to 4 cores without any additional tokens. If you want to use more than 4 cores, you will need an additional "HPC" token per core. For instance, a serial CFX job with 1 core will need 1 base license token while a parallel CFX job with 12 cores will need 1 base license token and 8 HPC tokens.

A job using a base license token can be submitted to either Glenn or Oakley cluster. A parallel job using HPC tokens (with "ansyspar" flag) however can only be submitted to Oakley cluster due to scheduler limitations. 

Access for Commerical Users

Contact OSC Help for getting access to CFX if you are a commerical user.

Usage

Usage on Glenn

Set-up on Glenn

CFX can only be run on the compute nodes. Therefore, all CFX jobs are run via the batch scheduling system, either as interactive or unattended jobs. To load the default version, use  module load fluent . To select a particular software version, use   module load fluentversion . For example, use  module load fluent14.5   to load CFX version 14.5 on Glenn. 

Batch Usage on Glenn

When you log into glenn.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your analysis to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time, up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. Batch jobs run on the compute nodes of the system and not on the login node. It is desirable for big problems since more resources can be used.

Interactive Batch Session

Interactive mode is similar to running CFX on a desktop machine in that the graphical user interface will be sent from OSC and displayed on the local machine. Interactive jobs are run on compute nodes of the cluster, by turning on X11 forwarding. The intention is that users can run CFX interactively for the purpose of building their model and preparing their input file. Once developed this input file can then be run in no-interactive batch mode.

To run interactive CFX GUI, a batch job need to be submitted from the login node, to request necessary compute resources, with X11 forwarding. Please follwoing the steps below to use CFX GUI interactivly:

  1. Ensure that your SSH client software has X11 forwarding enabled
  2. Connect to Glenn system
  3. Request an interactive job. The command below will request one whole node with 8 cores (  -l nodes=1:ppn=8 ), for a walltime of one hour ( -l walltime=1:00:00 ), with one ANSYS CFD license (modify as per your own needs):
    qsub -I -X -l nodes=1:ppn=8 -l walltime=1:00:00 -l software=fluent+1
    
  4. Once the interactive job has started, run the following commands to setup and start the CFX GUI:

    module load fluent
    cfx5
    
Non-interactive Batch Job (Serial Run Using 1 Base Token)

A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice.

Starting from September 15, 2015, Glenn only accepts CFX jobs with base license ( software=fluent+1 ); and a job using HPC tokens (with "ansyspar" flag) should be submitted to Oakley cluster due to scheduler issue.

Below is the example batch script (  job.txt ) for a serial run with an input file test.def ) on Glenn:

#PBS -N serialjob_cfx
#PBS -l walltime=1:00:00
#PBS -l software=fluent+1
#PBS -l nodes=1:ppn=1
#PBS -j oe
#PBS -S /bin/bash
#Set up CFX environment.
module load fluent
#'cd' directly to your working directory
cd $PBS_O_WORKDIR
#Copy CFX files like .def to $TMPDIR and move there to execute the program
cp test.def $TMPDIR/
cd $TMPDIR
#Run CFX in serial with test.def as input file
cfx5solve -batch -def test.def 
#Finally, copy files back to your home directory
cp  * $PBS_O_WORKDIR

In order to run it via the batch system, submit the job.txt  file with the command: qsub job.txt  

Usage on Oakley

Set-up on Oakley

To load the default version, use  module load fluent . To select a particular software version, use   module load fluent/version . For example, use  module load fluent/16.0  to load CFX version 16.0 on Oakley. 

Batch Usage on Oakley

When you log into oakley.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your analysis to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info.  Batch jobs run on the compute nodes of the system and not on the login node. It is desirable for big problems since more resources can be used.

Interactive Batch Session

Interactive mode is similar to running CFX on a desktop machine in that the graphical user interface will be sent from OSC and displayed on the local machine. Interactive jobs are run on compute nodes of the cluster, by turning on X11 forwarding. The intention is that users can run CFX interactively for the purpose of building their model and preparing their input file. Once developed this input file can then be run in no-interactive batch mode.

To run interactive CFX GUI, a batch job need to be submitted from the login node, to request necessary compute resources, with X11 forwarding. Please follwoing the steps below to use CFX GUI interactivly:

  1. Ensure that your SSH client software has X11 forwarding enabled
  2. Connect to Oakley system
  3. Request an interactive job. The command below will request one whole node with 12 cores (  -l nodes=1:ppn=12 ), for a walltime of one hour ( -l walltime=1:00:00 ), with one ANSYS CFD license (modify as per your own needs):
    qsub -I -X -l nodes=1:ppn=12 -l walltime=1:00:00 -l software=fluent+1
    
  4. Once the interactive job has started, run the following commands to setup and start the CFX GUI:

    module load fluent
    cfx5 
    
Non-interactive Batch Job (Serial Run Using 1 Base Token)

A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice.

Starting from September 15, 2015, Glenn only accepts CFX jobs with base license ( software=fluent+1 ); and a job using HPC tokens (with "ansyspar" flag) should be submitted to Oakley cluster due to scheduler issue.

Below is the example batch script (  job.txt ) for a serial run with an input file test.def ) on Glenn:

#PBS -N serialjob_cfx
#PBS -l walltime=1:00:00
#PBS -l software=fluent+1
#PBS -l nodes=1:ppn=1
#PBS -j oe
#PBS -S /bin/bash
#Set up CFX environment.
module load fluent
#'cd' directly to your working directory
cd $PBS_O_WORKDIR
#Copy CFX files like .def to $TMPDIR and move there to execute the program
cp test.def $TMPDIR/
cd $TMPDIR
#Run CFX in serial with test.def as input file
cfx5solve -batch -def test.def 
#Finally, copy files back to your home directory
cp  * $PBS_O_WORKDIR

In order to run it via the batch system, submit the job.txt  file with the command: qsub job.txt  

Non-interactive Batch Job (Parallel Execution using HPC token)

CFX can be run in parallel, but it is very important that you read the documentation in the CFX Manual on the details of how this works.

In addition to requesting the base license token ( -l software=fluent+1 ), you need to request copies of the ansyspar license, i.e., HPC tokens. However the scheduler cannot handle two "software" flags simultaneously, so the syntax changes. The new option is  -W x=GRES:fluent+1%ansyspar+[n] , where [n] is equal to the number of cores you requested minus 4.

Parallel jobs have to be submitted on Oakley via the batch system. An example of the batch script follows:

#PBS -N paralleljob_cfx
#PBS -l walltime=10:00:00
#PBS -l nodes=2:ppn=12
#PBS -W x=GRES:fluent+1%ansyspar+20
#PBS -j oe
#PBS -S /bin/bash
#Set up CFX environment.
module load fluent
#'cd' directly to your working directory
cd $PBS_O_WORKDIR
#Copy CFX files like .def to $TMPDIR and move there to execute the program
cp test.def $TMPDIR/
cd $TMPDIR
#Convert PBS_NODEFILE information into format for CFX host list
nodes=`cat $PBS_NODEFILE`
nodes=`echo $nodes | sed -e 's/ /,/g'`
#Run CFX in parallel with new.def as input file
#if multiple nodes
cfx5solve -batch -def test.def  -par-dist $nodes -start-method "Platform MPI Distributed Parallel"
#if one node
#cfx5solve -batch -def test.def -par-dist $nodes -start-method "Platform MPI Local Parallel"
#Finally, copy files back to your home directory
cp  * $PBS_O_WORKDIR

Further Reading

Supercomputer: 
Service: 

FLUENT

ANSYS FLUENT (called FLUENT hereafter) is a state-of-the-art computer program for modeling fluid flow and heat transfer in complex geometries.

Availability and Restrictions

FLUENT is available on Oakley and Glenn Clusters. The versions currently available at OSC are:

Version GLENN Oakley NOTEs
13 X    
14   X  
14.5 X*    
14.5.7   X  
15.0.7   X*  
16.0   X  
*: Current default version

You can use module avail fluent  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access for Academic Users

Use of ANSYS products for academic purposes requires validation. In order to obtain validation, please contact OSC Help for further instruction.

Currently, there are in total 25 ANSYS CFD base license tokens and 68 HPC tokens for academic users. These base tokens are shared by available ANSYS CFD related projusts (see "Academic Research -> ANSYS Academic Research CFD" in this table for details). These HPC tokens are shared with all ANSYS products we have at OSC. A base license token will allow FLUENT to use up to 4 cores without any additional tokens. If you want to use more than 4 cores, you will need an additional "HPC" token per core. For instance, a serial FLUENT job with 1 core will need 1 base license token while a parallel FLUENT job with 12 cores will need 1 base license token and 8 HPC tokens.

A job using a base license token can be submitted to either Glenn or Oakley cluster. A parallel job using HPC tokens (with "ansyspar" flag) however can only be submitted to Oakley cluster due to scheduler limitations. 

Access for Commerical Users

Contact OSC Help for getting access to FLUENT if you are a commerical user.

Usage

Usage on Glenn

Set-up on Glenn

FLUENT can only be run on the compute nodes. Therefore, all FLUENT jobs are run via the batch scheduling system, either as interactive or unattended jobs. To load the default version of FLUENT module, use  module load fluent . To select a particular software version, use   module load fluentversion . For example, use  module load fluent130  to load FLUENT version 13.0 on Glenn. 

Batch Usage on Glenn

When you log into glenn.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your FLUENT analysis to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time, up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. Batch jobs run on the compute nodes of the system and not on the login node. It is desirable for big problems since more resources can be used.

Interactive Batch Session

Interactive mode is similar to running FLUENT on a desktop machine in that the graphical user interface will be sent from OSC and displayed on the local machine. Interactive jobs are run on compute nodes of the cluster, by turning on X11 forwarding. The intention is that users can run FLUENT interactively for the purpose of building their model and preparing their input file. Once developed this input file can then be run in no-interactive batch mode.

To run interactive FLUENT GUI, a batch job need to be submitted from the login node, to request necessary compute resources, with X11 forwarding. Please follwoing the steps below to use FLUENT GUI interactivly:

  1. Ensure that your SSH client software has X11 forwarding enabled
  2. Connect to Glenn system
  3. Request an interactive job. The command below will request one whole node with 8 cores (  -l nodes=1:ppn=8 ), for a walltime of one hour ( -l walltime=1:00:00 ), with one FLUENT license (modify as per your own needs):
    qsub -I -X -l nodes=1:ppn=8 -l walltime=1:00:00 -l software=fluent+1
    
  4. Once the interactive job has started, run the following commands to setup and start the FLUENT GUI:

    module load fluent
    fluent 
    
Non-interactive Batch Job (Serial Run Using 1 Base Token)

A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice.

Starting from September 15, 2015, Glenn only accepts FLUENT jobs with base license ( software=fluent+1 ); and a job using HPC tokens (with "ansyspar" flag) should be submitted to Oakley cluster due to scheduler issue.

Below is the example batch script (  job.txt ) for a serial run with an input file run.input ) on Glenn:

#PBS -N serial_fluent
#PBS -l walltime=5:00:00 
#PBS -l nodes=1:ppn=1
#PBS -l software=fluent+1
#PBS -j oe
#
# The following lines set up the FLUENT environment
#
module load fluent
#
# Move to the directory where the job was submitted from
# You could also 'cd' directly to your working directory
cd $PBS_O_WORKDIR
#
# Copy files to $TMPDIR and move there to execute the program
#
cp test_input_file.cas test_input_file.dat run.input $TMPDIR
cd $TMPDIR
#
# Run fluent
fluent 3d -g < run.input  
#
# Where the file 'run.input' contains the commands you would normally
# type in at the Fluent command prompt.
# Finally, copy files back to your home directory
cp *   $PBS_O_WORKDIR  

As an example, your run.input file might contain:

file/read-case-date test_input_file.cas 
solve/iterate 100
file/write-case-data test_result.cas
file/confirm-overwrite yes    
exit  
yes  

In order to run it via the batch system, submit the job.txt  file with the command: qsub job.txt  

Usage on Oakley

Set-up on Oakley

To load the default version of FLUENT module, use  module load fluent . To select a particular software version, use   module load fluent/version . For example, use  module load fluent/16.0  to load FLUENT version 16.0 on Oakley. 

Batch Usage on Oakley

When you log into oakley.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your FLUENT analysis to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info.  Batch jobs run on the compute nodes of the system and not on the login node. It is desirable for big problems since more resources can be used.

Interactive Batch Session

Interactive mode is similar to running FLUENT on a desktop machine in that the graphical user interface will be sent from OSC and displayed on the local machine. Interactive jobs are run on compute nodes of the cluster, by turning on X11 forwarding. The intention is that users can run FLUENT interactively for the purpose of building their model and preparing their input file. Once developed this input file can then be run in no-interactive batch mode.

To run interactive FLUENT GUI, a batch job need to be submitted from the login node, to request necessary compute resources, with X11 forwarding. Please follwoing the steps below to use FLUENT GUI interactivly:

  1. Ensure that your SSH client software has X11 forwarding enabled
  2. Connect to Oakley system
  3. Request an interactive job. The command below will request one whole node with 12 cores (  -l nodes=1:ppn=12 ), for a walltime of one hour ( -l walltime=1:00:00 ), with one FLUENT license (modify as per your own needs):
    qsub -I -X -l nodes=1:ppn=12 -l walltime=1:00:00 -l software=fluent+1
    
  4. Once the interactive job has started, run the following commands to setup and start the FLUENT GUI:

    module load fluent
    fluent 
    
Non-interactive Batch Job (Serial Run Using 1 Base Token)

A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice.

Starting from September 15, 2015, Glenn only accepts FLUENT jobs with base license ( software=fluent+1 ); and a job using HPC tokens (with "ansyspar" flag) should be submitted to Oakley cluster due to scheduler issue.

Below is the example batch script (  job.txt ) for a serial run with an input file run.input ) on Oakley:

#PBS -N serial_fluent
#PBS -l walltime=5:00:00 
#PBS -l nodes=1:ppn=1
#PBS -l software=fluent+1
#PBS -j oe
#
# The following lines set up the FLUENT environment
#
module load fluent
#
# Move to the directory where the job was submitted from
# You could also 'cd' directly to your working directory
cd $PBS_O_WORKDIR
#
# Copy files to $TMPDIR and move there to execute the program
#
cp test_input_file.cas test_input_file.dat run.input $TMPDIR
cd $TMPDIR
#
# Run fluent
fluent 3d -g < run.input  
#
# Where the file 'run.input' contains the commands you would normally
# type in at the Fluent command prompt.
# Finally, copy files back to your home directory
cp *   $PBS_O_WORKDIR  

As an example, your run.input file might contain:

file/read-case-date test_input_file.cas 
solve/iterate 100
file/write-case-data test_result.cas
file/confirm-overwrite yes    
exit  
yes  

In order to run it via the batch system, submit the job.txt  file with the command: qsub job.txt  

Non-interactive Batch Job (Parallel Execution using HPC token)

FLUENT can be run in parallel, but it is very important that you read the documentation in the FLUENT Manual on the details of how this works.

In addition to requesting the FLUENT base license token ( -l software=fluent+1 ), you need to request copies of the ansyspar license, i.e., HPC tokens. However the scheduler cannot handle two "software" flags simultaneously, so the syntax changes. The new option is  -W x=GRES:fluent+1%ansyspar+[n] , where [n] is equal to the number of cores you requested minus 4.

Parallel jobs have to be submitted on Oakley via the batch system. An example of the batch script follows:

#PBS -N parallel_fluent   
#PBS -l walltime=1:00:00   
#PBS -l nodes=2:ppn=12
#PBS -j oe
#PBS -W x=GRES:fluent+1%ansyspar+20
#PBS -S /bin/bash
set echo on   
hostname   
#   
# The following lines set up the FLUENT environment   
#   
module load fluent
#   
# Move to the directory where the job was submitted from and   
# create the config file for socket communication library   
#   
cd $PBS_O_WORKDIR   
#   
# Create list of nodes to launch job on   
rm -f pnodes   
cat  $PBS_NODEFILE | sort > pnodes   
export ncpus=`cat pnodes | wc -l`   
#   
#   Run fluent   
fluent 3d -t$ncpus -pinfiniband.ofed -cnf=pnodes -g < run.input 

Further Reading

See Also

Supercomputer: 
Service: 

Workbench Platform

ANSYS Workbench platform is the backbone for delivering a comprehensive and integrated simulation system to users. See ANSYS Workbench platform for more information. 

Availability and Restrictions

ANSYS Workbench is available on Oakley Cluster. The versions currently available at OSC are:

Version Glenn Oakley Notes
14.5.7 SF   X  
CFD X
15.0.7 SF   X  
CFD X*
16.0 SF   X*  
CFD X
*: Current default version

Note:

  • SF: Structural-Fluid dynamics related applications. See "Academic Research -> ANSYS Academic Research Mechanical and CFD" in this table for all available products
  • CFD: CFD related applications. See "Academic Research -> ANSYS Academic Research CFD" in this table for all available products

You can use module avail ansys  to view available modules for a given machine if you want to use structural-fluid dynamics related applications or module avail fluent  to view available modules for a given machine if you want to use CFD related applications. Feel free to contact OSC Help if you need other versions for your work.

Access for Academic Users

Use of ANSYS products for academic purposes requires validation. In order to obtain validation, please contact OSC Help for further instruction.

Access for Commerical Users

Contact OSC Help for getting access to ANSYS if you are a commerical user.

Usage

Usage on Oakley

Set-up for Structural-Fluid dynamics related applications

To load the default version , use  module load ansys . To select a particular software version, use   module load ansys/version . For example, use  module load ansys/15.0.7   to load version 15.0.7 on Oakley. After the module is loaded, use the following command to open Workbench GUI:

runwb2

Set-up for CFD related applications

To load the default version , use  module load fluent  . To select a particular software version, use   module load ansys/fluent  . For example, use  module load fluent/15.0.7   to load version 15.0.7 on Oakley. After the module is loaded, use the following command to open Workbench GUI:

runwb2

Further Reading

See Also

Supercomputer: 
Service: 

Allinea

OSC provides Allinea Performance Reports and Allinea MAP to all OSC users. 

  • Allinea Performance Reports: very simple tool to generate one single-page HTML report that presents overall performance characteristics of HPC applications
  • Allinea MAP: HPC application profiler with easy-to-use GUI environment.

Versions

The following versions of Allinea are available on OSC clusters:

version oakley Ruby Owens
5.1 X X  
6.0.1 X X  
6.0.6     X

Allinea Performance Reports

We recommend Allinea Performance Reports to all OSC users who like to improve their code. With a simple command, It produces an informative one page long overall performance report in HTML or pdf format. A typical report looks like

Example report

One can find more examples of the report from here. It simply shows that the HPC software is whether CPU, MPI, I/O, or Memory bound, so that one can get a clear direction to improve their code.  For the usage instruction on OSC, please see the Usage section below.

Allinea MAP

After reviewing reports from Allinea Performance Reports, one can proceed with Allinea MAP for more detail profiling. For more features and benefits, please look at here. For the usage instruction on OSC, please see the Usage section below.

Usage

For more information, please read following pages:

Troubleshooting

Using Allinea Performance Reports with MVAPICH2

This is from Allinea's Getting Started Guide:

Some MPIs, most notably MVAPICH, are not yet supported by Allinea's Express Launch mode
(in which you can just put “perf-report” in front of an existing mpirun/mpiexec line). These can
still be measured using the Compatibility Launch mode.

Instead of this Express Launch command:

perf-report mpiexec <mpi args> <program> <program args> # BAD

Use the compatibility launch version instead:

perf-report -n <num procs> --mpiargs=”<mpi args>” <program> <program args>

Allinea Map Timeout Errors with an MPI-based Application under Batch Job Submission

One user noticed that using Allinea Map had no issues in an interactive job, but was problematic inside of a batch job submission.  Additionally, the error appears to occur in combination with using Intel MPI.  The messages looked like the following:

MAP: Timeout when running the build_wrapper script in 
    '/usr/local/allinea/forge-6.0.1/map/wrapper'.
MAP:
MAP: This script is needed to compile an MPI wrapper library for your system

While initially it was suspected that one could increase or turn off the timeout by setting

export ALLINEA_NO_TIMEOUT=1, this was not the solution; instead one should elect to pre-build the MPI wrapper libraries for Allinea Map:

module load allinea
mkdir -p $HOME/.allinea/mpiwrapper # you can do this anywhere, but it is a logical place
cd $HOME/.allinea/mpiwrapper
make-profiler-libraries # be patient - builds 'libmap-sampler-pmpi.so' and other files
export ALLINEA_MPI_WRAPPER=$HOME/.allinea/mpiwrapper/libmap-sampler-pmpi.so

Be sure to apply the last line in your shell profile or job submission script prior to running you MPI-based application with Allinea Map.

Further Reading

See Also

Documentation Attachment: 
Supercomputer: 
Service: 
Fields of Science: 

Allinea Performance Reports

Allinea Performance Reports is a simple tool to generate one single-page HTML report that presents overall performance characteristics of HPC applications. It supports pthreads, OpenMP, or MPI code on CPU, GPU, or MIC based architecture.

Availability and Restrictions

Allinea Performance Reports is available on the Oakley and Ruby clusters. The versions currently available at OSC are

Version OAKLEY RUBY notes
6.0 X X  
6.0.1 X* X*  
*: Current default version

You can use module avail allinea to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Allinea Performance Reports is available to all OSC users. We have 64 seats with 64 HPC tokens, and users can monitor the license status from here.

Usage

Usage on Oakley

Set-up on Oakley

To load the default version of Allinea Performance Reports module, use  module load allinea. To select a particular software version, use   module load allinea/version. For example, use  module load allinea/6.0 to load Allinea Performance Reports version 6.0 on Oakley. 

Usage on Oakley

When you log into oakley.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the multiple processors in the computing environment, you must submit your analysis to the batch system for execution. Batch jobs can request multiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. 

You can use your regular executables to generate the performance report. First, you need to grab nodes for interactive mode or prepare for a batch script for non-interactive mode. After you get into the interactive mode, if your executable is wavec, for example, then

module load allinea
perf-report -np 12 wavec

will generate a html and plain text file for the report, like wavec_12p_2016-02-05_12-46.html . You can open the report in html format by

firefox wavec_12p_2016-02-05_12-46.html

 In general, you can use:

perf-report -np <num procs> --mpiargs="<mpi args>" <program> <program args>

For more details, you can download the full manual from the link at the bottom of this page.  

 

Usage on Ruby

Set-up on Ruby

To load the default version of Allinea Performance Reports module, use   module load allinea. To select a particular software version, use    module load allinea/version.  For example, use   module load allinea/6.0 to load allinea Performance Reports version 6.0 on Ruby. 

Usage on Ruby

When you log into ruby.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the multiple processors in the computing environment, you must submit your analysis to the batch system for execution. Batch jobs can request multiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. 

You can use your regular executables to generate the performance report. First of all, you need to grab nodes for interactive mode or prepare for a batch script for non-interactive mode. After you get into the interactive mode, if your executable is  wavec for example, then

module load allinea
perf-report -np 20 wavec

will generate a html and plain text file for the report, like  wavec_20p_2016-02-05_12-46.html. You can open the report in html format by

firefox wavec_20p_2016-02-05_12-46.html

 In general, you can use

perf-report -np <num procs> --mpiargs="<mpi args>" <program> <program args>

For more details, you can download the full manual from the link at the bottom of this page.  

 

Further Reading

See Also

Documentation Attachment: 
Supercomputer: 
Service: 

Allinea MAP

Allinea MAP is a full scale profiler for HPC programs. We recommend to use Allinea MAP after reviewing reports from Allinea Performance Reports. It supports pthreads, OpenMP, or MPI code on CPU, GPU, or MIC based architecture.

Availability and Restrictions

Allinea MAP is available on the Oakley and Ruby clusters. The versions currently available at OSC are

Version OAKLEY RUBY notes
6.0 X X  
6.0.1 X* X*  
*: Current default version

You can use module avail allinea to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Allinea Performance Reports is available to all OSC users. We have 64 seats with 80 HPC tokens, and users can monitor the license status from here.

Usage

Usage on Oakley

Set-up on Oakley

To load the default version of Allinea MAP module, use  module load allinea . To select a particular software version, use   module load allinea/version . For example, use  module load allinea/6.0 to load allinea MAP version 6.0 on Oakley. 

Usage on Oakley

When you log into oakley.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the multiple processors in the computing environment, you must submit your analysis to the batch system for execution. Batch jobs can request multiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. 

There are three steps to profile a HPC code using Allinea MAP. 

1. Prepare executable for the profiling

 You cannot use your regular executables for Allinea MAP. You need executables with the debug flags, thus re-compile your code with   -g option with appropriate options that you have used. For example, you can use

mpicc -o wave_profile -g -O2 wave.c

This executable with the debug flags can be used for Allinea Performance Reports as well.

2. Get the profile data file (.map file)

Using the executable with the debug flags, you are ready to get the profile data file. You need to grab nodes for interactive mode or prepare for a batch script for non-interactive mode. After you get into the interactive mode, you can use

module load allinea
map --profile -n 12 wave_profile

As a result of this step, you get a .map file that is the profile data file. For more details, you can download the full manual from the link at the bottom of this page.  

3. Open the profile data file using a client
You can open the profile data file using a client from your local desktop computer. For installation and setting of the client, please look at the next section, Client Download and Setting .

 

Usage on Ruby (Not done yet. will be the same as Oakley.)

Set-up on Ruby

To load the default version of Allinea MAP module, use   module load allinea . To select a particular software version, use    module load allinea/version . For example, use   module load allinea/6.0 to load allinea MAP version 6.0 on Ruby. 

Usage on Ruby

When you log into ruby.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the multiple processors in the computing environment, you must submit your analysis to the batch system for execution. Batch jobs can request multiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. 

There are three steps to profile a HPC code using Allinea MAP. 

1. Prepare executable for the profiling

 You cannot use your regular executables for Allinea MAP. You need executables with the debug flags, thus re-compile your code with   -g option with appropriate options that you have used. For example, you can use

mpicc -o wave_profile -g -O2 wave.c

This executable with the debug flags can be used for Allinea Performance Reports as well.

2. Get the profile data file (.map file)

Using the executable with the debug flags, you are ready to get the profile data file. You need to grab nodes for interactive mode or prepare for a batch script for non-interactive mode. After you get into the interactive mode, you can use

module load allinea
map --profile -n 20 wave_profile

As a result of this step, you get a .map file that is the profile data file. For more details, you can download the full manual from the link at the bottom of this page.  

3. Open the profile data file using a client
You can open the profile data file using a client from your local desktop computer. For installation and setting of the client, please look at the next section, Client Download and Setting .

Client Download and Setting

1. Download the client

You can download the client from /usr/local/allinea/forge-6.0.1/clients/ for Windows, Mac, or Linux on Oakley or Ruby. For Windows and Mac, just double click on the downloaded file. For Linux, type ./install after extracting the tar file.

2. Configure the client

After installation, you can configure the client as followings.

  • Open the client program
  • Select Allinea MAP in the left column
  • Under Remote Launch in the center, select "Configure..."
  • Click Add to a new profile for your login
  • In Host Name, type your ssh connection, for example "username@ruby.osc.edu"
  • In Remote Installation Directory, type /usr/local/allinea/forge-6.0.1
  • You can click Test Remote Launch. It will ask your password. You can use the same password for the cluster login.
  • Close the Configure window. You will see a new option under Remote Launch. Select your profile and login with your password. This login configuration is needed for the first time of use. In the next times, you can just select your profile.
  • If the login was successful, then you should see License Serial:XXX on the left bottom of the window.
3. Open the profile data file

After login, click on LOAD PROFILE DATA FILE to open the profile data file. This click opens a file browser. What you see is your home directory of Ruby or Oakley, so you can navigate to the directory that you were working and click on the .map file to open it. Once it is open, you can navigate the source code line-by-line and investigate what the performance characteristics are in detail. During the client is open, it doesn't require any license, so you can take as long as you like. If you like you can skip 2. Configure the client step, then just download the profile data file to your desktop and just open it from LOAD PROFILE DATA FILE.

Note that the client is actually Allinea Forge client that contains Allinea MAP and Allinea DDT. Allinea DDT is a debugger, and OSC has license only for Allinea MAP. If you need a debugger, you can use Totalview instead.

Further Reading

See Also

Documentation Attachment: 
Supercomputer: 
Service: 

Altair HyperWorks

HyperWorks is a high-performance, comprehensive toolbox of CAE software for engineering design and simulation.

Availability & Restrictions

HyperWorks is available to all academic clients.

The following version of Altair Hyperworks can be found for the following environments:

Version Oakley Statewide
10.0   X
11.0 X X
12.0 X X
13.0 X X
14.0 X X

Usage

Using HyperWorks through OSC installation

To use HyperWorks on the OSC clusters, first ensure that X11 forwarding is enabled as the HyperWorks workbench is a graphical application. Then, load the hyperworks module:

module load hyperworks

The HyperWorks GUI can be launched then with the following command:

hw

State-wide access for HyperWorks

For information on downloading and installing a local copy through the state-wide license, follow the steps below. 

NOTE: To run Altair HyperWorks, your computer must have access to the internet. The software contacts the license server at OSC to check out a license when it starts and periodically during execution. The amount of data transferred is small, so network connections over modems are acceptable.

 

Usage of HyperWorks on a local machine using the statewide license will vary from installation to installation.

  1. Go to http://www.altairhyperworks.com/ 

  2. Click on "Login" in the upper right hand corner of the page. 

  3. If you have already registered with the Altair web site, enter the e-mail address that you registered with and your password and skip to step #5.

  4. If you have not registered yet, click the link that says "Click here to register now". You will be prompted for some contact information and an e-mail address which will be your unique identifier.

    • IMPORTANT: The e-mail address you give must be from your academic institution. Under the statewide license agreement, registration from Ohio universities is allowed on the Altair web site. Trying to log in with a yahoo or hotmail e-mail account will not work. If you enter your university e-mail and the system will not register you, please contact OSChelp at oschelp@osc.edu.

  5. Once you have logged in, click on "SUPPORT" and then "SOFTWARE DOWNLOADS"

  6. In addition to downloading the software, download the "Installation Guide and Release Notes" for instructions on how to install the software.

    • IMPORTANT: If you have any questions or problems, please contact OSChelp at oschelp@osc.edu, rather than HyperWorks support. The software agreements outlines that problems should first be sent to OSC. If the OSC support line cannot answer or resolve the question, they have the ability to raise the problem to Altair support.

  7. ​Please contact OSC Help for further instruction and license server information. In order to be added to the allowed list for the state-wide software access, we will need your IP address/range of machine that will be running this software. 

  8. You need to set an environment variable (ALTAIR_LICENSE_PATH) on your local machine to point at our license server (7790@license2.osc.edu). See this link for instructions if necessary.

Further Reading

For more information about HyperWorks, see the following:

See Also

Supercomputer: 
Service: 
Fields of Science: 

BLAST

The BLAST programs are widely used tools for searching DNA and protein databases for sequence similarity to identify homologs to a query sequence. While often referred to as just "BLAST", this can really be thought of as a set of programs: blastp, blastn, blastx, tblastn, and tblastx.

Availability & Restrictions

BLAST is available without restriction to all OSC users.

The following versions of BLAST are available on OSC systems:

Version Oakley Owens
2.2.24+ X  
2.2.25+ X  
2.2.26 X  
2.2.31+ X  
2.4.0+   X*

 

If you need to use blastx, you will need to load one of the C++ implimenations modules of blast (any version with a "+").

Set-up

To load BLAST, type the following into the command line:

module load blast

Then create a resource file .ncbirc, and put it under your home directory.


If you are using the legacy blast program, the contents of the file contains at least two variables DATA and BLASTDB:

[NCBI]
DATA="/usr/local/biosoftw/blast-2.2.17/data/"
[BLAST]
BLASTDB="/nfs/gpfs/PZS0002/biosoftw/db/"

 

If you are using the C++ implementation of blast program, the contents of the file contains at least one variable BLASTDB:

[BLAST]
BLASTDB="/nfs/gpfs/PZS0002/biosoftw/db/"

 

Upon start, BLAST will read this file to get the path information it needs during BLAST searches. Without this file, BLAST will search the working directory, or whichever directory the command is issued from.

Using BLAST

The five flavors of BLAST mentioned above perform the following tasks:

blastp: compares an amino acid query sequence against a protein sequence database
blastn: compares a nucleotide query sequence against a nucleotide sequence database
blastx: compares the six-frame conceptual translation products of a nucleotide query sequence (both strands) against a protein sequence database
tblastn: compares a protein query sequence against a nucleotide sequence database dynamically translated in all six reading frames (both strands).
tblastx: compares the six-frame translations of a nucleotide query sequence against the six-frame translations of a nucleotide sequence database. (Due to the nature of tblastx, gapped alignments are not available with this option)

We provide local access to nr and swissprot databases. Other databases are available upon request.

Batch Usage

A sample batch script is below:

#PBS -l nodes=1:ppn=1
#PBS -l walltime=10:00
#PBS -N Blast
#PBS -S /bin/bash
#PBS -j oe

module load blast
set -x

cd $PBS_O_WORKDIR
mkdir $PBS_JOBID

cp 100.fasta $TMPDIR
cd $TMPDIR
/usr/bin/time blastn -db nt -query 100.fasta -out test.out

cp test.out $PBS_O_WORKDIR/$PBS_JOBID

Further Reading

Supercomputer: 
Service: 
Fields of Science: 

BLAT

BLAT is a sequence analysis tool which performs rapid mRNA/DNA and cross-species protein alignments. BLAT is more accurate and 500 times faster than popular existing tools for mRNA/DNA alignments and 50 times faster for protein alignments at sensitivity settings typically used when comparing vertebrate sequences.

BLAT is not BLAST. DNA BLAT works by keeping an index of the entire genome (but not the genome itself) in memory. Since the index takes up a bit less than a gigabyte of RAM, BLAT can deliver high performance on a reasonably priced Linux box. The index is used to find areas of probable homology, which are then loaded into memory for a detailed alignment. Protein BLAT works in a similar manner, except with 4-mers rather than 11-mers. The protein index takes a little more than 2 gigabytes.

Availability & Restrictions

BLAT is available without restriction to all OSC users.

The following versions of BLAT are available at OSC:

Version Glenn Oakley
34 X X

Usage

Set-up

To initalize the Glenn system prior to using BLAT, run the following commands:

module load biosoftw
module load blat

To initalize the Oakley system prior to using BLAT, run the following commands:

module load blat

Using BLAT

The main programs in the blat suite are:

gfServer – a server that maintains an index of the genome in memory and uses the index to quickly find regions with high levels of sequence similarity to a query sequence.
gfClient – a program that queries gfServer over the network, and then does a detailed alignment of the query sequence with regions found by gfServer.
blat –combines client and server into a single program, first building the index, then using the index, and then exiting. 
webBlat – a web based version of gfClient that presents the alignments in an interactive fashion. (not included on OSC server)

Building an index of the genome typically takes 10 or 15 minutes.  Typically for interactive applications one uses gfServer to build a whole genome index.  At that point gfClient or webBlat can align a single query within few seconds.  If one is aligning a lot of sequences in a batch mode then blat can be more efficient, particularly if run on a cluster of computers.  Each blat run is typically done against a single chromosome, but with a large number of query sequences.

Other programs in the blat suite are:

pslSort – combines and sorts the output of multiple blat runs.  (The blat default output format is .psl).
pslReps – selects the best alignments for a particular query sequence, using a ‘near best in genome’ approach.
pslPretty – converts alignments from the psl format, which is tab-delimited format and does not include the bases themselves, to a more readable alignment format.
faToTwoBit – convert Fasta format sequence files to a dense randomly-accessable  .2bit format that gfClient can use.
twoBitToFa – convert from the .2bit format back to fasta
faToNib – convert from Fasta to a somewhat less dense randomly accessible format that predates .2bit.  Note each .nib file can only contain a single sequence.
nibFrag – convert portions of a nib file back to fasta.

The command line options of each of the programs is described below. Similar summaries of usage are printed when a command is run with no arguments.

Batch Usage

A sample batch script is as below:

#PBS -N blat
#PBS -j oe
#PBS -l nodes=1:ppn=1
#PBS -S /bin/bash

module load blat 

cd $PBS_O_WORKDIR
cp $BLAT_TEST_DIR/gfServer-tests/input/creaGeno.2bit .
cp $BLAT_TEST_DIR/gfServer-tests/input/crea.mrna .

blat -stepSize=5 -repMatch=2253 -minScore=0 -minIdentity=0 creaGeno.2bit crea.mrna output.psl 

 

Further Reading

See Also

Supercomputer: 
Service: 
Fields of Science: 

BWA

BWA is a software package for mapping low-divergent sequences against a large reference genome, such as the human genome. It consists of three algorithms: BWA-backtrack, BWA-SW and BWA-MEM.

Availability and Restrictions

The following versions of BWA are available on OSC clusters:

Version Oakley owens
0.7.13 X* X*
*: Current default version

You can use  module avail bwa to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

BWA is available to all OSC users without restriction.

Usage on Oakley

Set-up

To configure your environment for use of BWA, run the following command:   module load bwa. The default version will be loaded. To select a particular BWA version, use   module load bwa/version. For example, use  module load bwa/0.7.13 to load BWA 0.7.13.

Usage on Owens

Set-up

To configure your environment for use of BWA, run the following command:   module load bwa. The default version will be loaded. To select a particular BWA version, use   module load bwa/version. For example, use module load bwa/0.7.13 to load BWA 0.7.13.

Further Reading

Supercomputer: 
Service: 
Fields of Science: 

BamTools

BamTools provides both a programmer's API and an end-user's toolkit for handling BAM files.

Availability and Restrictions

The following versions of BamTools are available on OSC clusters:

Version Oakley Owens
2.2.2 X* X*
*: Current default version

You can use  module avail bamtools to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

BamTools is available to all OSC users without restriction.

Usage on Oakley

Set-up

To configure your environment for use of BamTools, run the following command:   module load bamtools . The default version will be loaded. To select a particular BamTools version, use  module load bamtools/version . For example, use  module load bamtools/2.2.2 to load BamTools 2.2.2.

Usage on Owens

Set-up

To configure your environment for use of BamTools, run the following command:   module load bamtools . The default version will be loaded. To select a particular BamTools version, use  module load bamtools/version . For example, use  module load bamtools/2.2.2 to load BamTools 2.2.2.

Further Reading

Supercomputer: 
Service: 
Fields of Science: 

Bioperl

Bioperl offers a set of perl modules which can be used for sequence manipulation. Knowledge of PERL programming is required.

Availability & Restrictions

Bioperl is available without restriction to all OSC users.

The following versions of Bioperl are available:

Version Glenn Oakley
1.5.1 X  
1.6.1 X X

Usage

Using Bioperl

This is an example of how to use bioperl and access the sequence package on the Oakley Cluster.

use lib '/usr/local/bioperl/1.6.1/lib/perl5/'
use Bio::Seq;

This is an example of how to use bioperl and access the sequence package on the Glenn Cluster.

use lib '/usr/local/biosoftw/bioperl-1.6.1/lib/perl5/'
use Bio::Seq;

Further Reading

See Also

Supercomputer: 
Service: 
Fields of Science: 

Boost

Boost is a set of C++ libraries that provide helpful data structures and numerous support functions in a wide range of aspects of programming, such as, image processing, gpu programming, concurrent programming, along with many algorithms.  Boost is portable and performs well on a wide variety of platforms.

Availability & Restrictions

Boost is available without restriction to all OSC Users

The following version of Boost are available on OSC systems:

Version Oakley Ruby owens Notes
1.53.0     System Install No Module Needed
1.56.0 X(GI) X(GI)    
1.58.0 X(G)      
1.60.0 X(G)*      

 

NOTE: G = available with gnu; I = available with intel; * means default version

Usage on Oakley

Set-up

Initalizing the system for use of the Boost library is independent of the compiler you are using.  To load the boost module run the following command:

module load boost

Building With Boost

The following environment variables are setup when the Boost library is loaded:

Variable Use
$BOOST_CFLAGS Use during your compilation step for C++ programs.
$BOOST_LIBS Use during your link step.

 

Below is a set of example commands used to build and run a file called example2.cpp . First copy the example2.cpp and jayne.txt with the following commands.

cp /usr/local/src/boost/boost-1.56.0/test.osc/example2.cpp .
cp /usr/local/src/boost/boost-1.56.0/test.osc/jayne.txt .
module load boost
g++ $BOOST_CFLAGS example2.cpp -o boostTest $BOOST_LIBS -lboost_regex
./boostTest < jayne.txt

Usage on Ruby

Set-up

Initalizing the system for use of the Boost library is independent of the compiler you are using.  To load the boost module run the following command:

module load boost

Building With Boost

The following environment variables are setup when the Boost library is loaded:

VARIABLE USE
$BOOST_CFLAGS Use during your compilation step for C++ programs.
$BOOST_LIBS Use during your link step.

 

Below is a set of example commands used to build and run a file called  example2.cpp  . First copy the example2.cpp and jayne.txt from Oakley into your home directory with the following commands:

cp /usr/local/src/boost/boost-1_56_0/test.osc/example2.cpp ~
cp /usr/local/src/boost/boost-1_56_0/test.osc/jayne.txt ~
Then compile and test the program with the folllowing commands:
module load boost
g++ $BOOST_CFLAGS example2.cpp -o boostTest $BOOST_LIBS -lboost_regex
./boostTest < jayne.txt

Usage on Owens

Set-up

Since Boost is a System Install, no module is needed to run the application.

Building With Boost

Below is a set of example commands used to build and run a file called  example2.cpp  . First copy the example2.cpp and jayne.txt from Oakley into your home directory with the following commands:

cp /usr/local/src/boost/boost-1_56_0/test.osc/example2.cpp ~
cp /usr/local/src/boost/boost-1_56_0/test.osc/jayne.txt ~
Then compile and test the program with the folllowing commands:
g++ example2.cpp -o boostTest -lboost_regex
./boostTest < jayne.txt

Further Reading

 

Supercomputer: 
Service: 
Fields of Science: 

Bowtie

"Bowtie is an ultrafast, memory-efficient short read aligner geared toward quickly aligning large sets of short DNA sequences (reads) to large genomes. It aligns 35-base-pair reads to the human genome at a rate of 25 million reads per hour on a typical workstation. Bowtie indexes the genome with a Burrows-Wheeler index to keep its memory footprint small: for the human genome, the index is typically about 2.2 GB (for unpaired alignment) or 2.9 GB (for paired-end or colorspace alignment). Multiple processors can be used simultaneously to achieve greater alignment speed. Bowtie can also output alignments in the standard SAM format, allowing Bowtie to interoperate with other tools supporting SAM, including the SAMtools consensus, SNP, and indel callers. Bowtie runs on the command line." (http://bowtie-bio.sourceforge.net/manual.shtml)

Availability & Restrictions

Bowtie is available to all OSC users without restriction.

The following version of Bowtie are available on OSC systems:

Version Glenn Oakley
0.12.7 X  

Usage

Set-up

To configure your environment for using PAUP, run the following command:

module load paup

Using Bowtie

On the Glenn Cluster bowtie is accessed by executing the following commands:

module load biosoftw
module load bowtie

bowtie will be added to the users PATH and can then be run with the command:

bowtie [options]* <ebwt> {-1 <m1> -2 <m2 | --12 <r> | <s>} [<hit]

Below are definitions for some of the main optional arguments:

<m1>    Comma-separated list of files containing upstream mates (or the sequences themselves, if -c is set) paired with mates in <m2>
<m2>    Comma-separated list of files containing downstream mates (or the sequences themselves if -c is set) paired with mates in <m1>
<r>     Comma-separated list of files containing Crossbow-style reads.  Can be a mixture of paired and unpaired.  Specify "-" for stdin.
<s>     Comma-separated list of files containing unpaired reads, or the sequences themselves, if -c is set.  Specify "-" for stdin.
<hit>   File to write hits to (default: stdout)

Options

Input:

-q                       query input files are FASTQ .fq/.fastq (default)
-f                       query input files are (multi-)FASTA .fa/.mfa
-r                       query input files are raw one-sequence-per-line
-c                       query sequences given on cmd line (as <mates>, <singles>)
-C                       reads and index are in colorspace
-Q/--quals <file>        QV file(s) corresponding to CSFASTA inputs; use with -f -C
--Q1/--Q2 <file>         same as -Q, but for mate files 1 and 2 respectively
-s/--skip <int>          skip the first <int> reads/pairs in the input
-u/--qupto <int>         stop after first <int> reads/pairs (excl. skipped reads)
-5/--trim5 <int>         trim <int> bases from 5' (left) end of reads
-3/--trim3 <int>         trim <int> bases from 3' (right) end of reads
--phred33-quals          input quals are Phred+33 (default)
--phred64-quals          input quals are Phred+64 (same as --solexa1.3-quals)
--solexa-quals           input quals are from GA Pipeline ver. < 1.3
--solexa1.3-quals        input quals are from GA Pipeline ver. >= 1.3
--integer-quals          qualities are given as space-separated integers (not ASCII)

Alignment:

-v <int>                 report end-to-end hits w/ <=v mismatches; ignore qualities or
-n/--seedmms <int>       max mismatches in seed (can be 0-3, default: -n 2)
-e/--maqerr <int>        max sum of mismatch quals across alignment for -n (def: 70)
-l/--seedlen <int>       seed length for -n (default: 28)
--nomaqround             disable Maq-like quality rounding for -n (nearest 10 <= 30)
-I/--minins <int>        minimum insert size for paired-end alignment (default: 0)
-X/--maxins <int>        maximum insert size for paired-end alignment (default: 250)
--fr/--rf/--ff           -1, -2 mates align fw/rev, rev/fw, fw/fw (default: --fr)
--nofw/--norc            do not align to forward/reverse-complement reference strand
--maxbts <int>           max # backtracks for -n 2/3 (default: 125, 800 for --best)
--pairtries <int>        max # attempts to find mate for anchor hit (default: 100)
-y/--tryhard             try hard to find valid alignments, at the expense of speed
--chunkmbs <int>         max megabytes of RAM for best-first search frames (def: 64)

Reporting:

-k <int>                 report up to <int> good alignments per read (default: 1)
-a/--all                 report all alignments per read (much slower than low -k)
-m <int>                 suppress all alignments if > <int> exist (def: no limit)
-M <int>                 like -m, but reports 1 random hit (MAPQ=0); requires --best
--best                   hits guaranteed best stratum; ties broken by quality
--strata                 hits in sub-optimal strata aren't reported (requires --best)

Output:

-t/--time                print wall-clock time taken by search phases
-B/--offbase <int>       leftmost ref offset = <int> in bowtie output (default: 0)
--quiet                  print nothing but the alignments
--refout                 write alignments to files refXXXXX.map, 1 map per reference
--refidx                 refer to ref. seqs by 0-based index rather than name
--al <fname>             write aligned reads/pairs to file(s) <fname>
--un <fname>             write unaligned reads/pairs to file(s) <fname>
--max <fname>            write reads/pairs over -m limit to file(s) <fname>
--suppress <cols>        suppresses given columns (comma-delim'ed) in default output
--fullref                write entire ref name (default: only up to 1st space)

Colorspace:

--snpphred <int>         Phred penalty for SNP when decoding colorspace (def: 30) or
--snpfrac <dec>          approx. fraction of SNP bases (e.g. 0.001); sets --snpphred
--col-cseq               print aligned colorspace seqs as colors, not decoded bases
--col-cqual              print original colorspace quals, not decoded quals
--col-keepends           keep nucleotides at extreme ends of decoded alignment

SAM:

-S/--sam                 write hits in SAM format
--mapq <int>             default mapping quality (MAPQ) to print for SAM alignments
--sam-nohead             supppress header lines (starting with @) for SAM output
--sam-nosq               supppress @SQ header lines for SAM output
--sam-RG <text>          add <text> (usually "lab=value") to @RG line of SAM header

Performance:

-o/--offrate <int>       override offrate of index; must be >= index's offrate
-p/--threads <int>       number of alignment threads to launch (default: 1)
--mm                     use memory-mapped I/O for index; many 'bowtie's can share
--shmem                  use shared mem for index; many 'bowtie's can share

Other:

--seed <int>             seed for random number generator
--verbose                verbose output (for debugging)
--version                print version information and quit
-h/--help                print this usage message

 

Batch Usage

The following is an example batch script file.

#PBS -n bowtie_test
#PBS -l walltime=00:05:00
#PBS -l nodes=1:ppn=4

module load biosoftw
module load bowtie-0.12.7
cd $PBS_O_WORKDIR
cp /usr/local/biosoftw/bowtie-$BOWTIE_VERSION/genomes/NC_008253.fna .
bowtie-build NC_008253.fna e_coli
bowtie –p 4 e_coli -c ATGCATCATGCGCCAT

Errors

The following scripts fail due to an ftp error: make_e_coli.sh, make_a_thaliana_tair.sh, and make_c_elegans_ws200.sh.  The following scripts fail to obtain all of the fasta format files prior to bowtie conversion and fail: make_galGal3.sh, make_hg18.sh, make_h_sapiens_ncbi36.sh, make_h_sapiens_ncbi37.sh, make_mm9.sh, make_m_musculus_ncbi37.sh.  The follow script does not work properly on the Glenn Cluster: gen_dnamasks2colormask.pl.

Further Reading

Supercomputer: 
Service: 
Fields of Science: 

Bowtie1

Bowtie1 is an ultrafast, memory-efficient short read aligner. It aligns short DNA sequences (reads) to the human genome at a rate of over 25 million 35-bp reads per hour. Bowtie indexes the genome with a Burrows-Wheeler index to keep its memory footprint small: typically about 2.2 GB for the human genome (2.9 GB for paired-end).

Availability and Restrictions

The following versions of Bowtie1 are available on OSC clusters:

Version Oakley Owens
1.1.2 X* X*
*: Current default version

You can use  module avail bowtie1  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Bowtie1 is available to all OSC users without restriction.

Usage on Oakley

Set-up

To configure your environment for use of Bowtie1, run the following command:   module load bowtie1. The default version will be loaded. To select a particular Bowtie1 version, use   module load bowtie/version. For example, use  module load bowtie1/1.1.2to load Bowtie1 1.1.2.

Usage on Owens

Set-up

To configure your environment for use of Bowtie1, run the following command:   module load bowtie1. The default version will be loaded. To select a particular Bowtie1 version, use   module load bowtie/version. For example, use module load bowtie1/1.1.2to load Bowtie1 1.1.2.

Further Reading

Supercomputer: 
Service: 
Fields of Science: 

Bowtie2

Bowtie 2 is an ultrafast and memory-efficient tool for aligning sequencing reads to long reference sequences. It is particularly good at aligning reads of about 50 up to 100s or 1,000s of characters, and particularly good at aligning to relatively long (e.g. mammalian) genomes. Bowtie 2 indexes the genome with an FM Index to keep its memory footprint small: for the human genome, its memory footprint is typically around 3.2 GB. Bowtie 2 supports gapped, local, and paired-end alignment modes.

Availability and Restrictions

The following versions of Bowtie2 are available on OSC clusters:

Version Oakley Owens
2.2.8 X*  
2.2.9   X*
*: Current default version

You can use  module avail bowtie2  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Bowtie2 is available to all OSC users without restriction.

Usage on Oakley

Set-up

To configure your environment for use of Bowtie2, run the following command:   module load bowtie2  . The default version will be loaded. To select a particular Bowtie2 version, use   module load bowtie2/version . For example, use  module load bowtie2/2.2.8  to load Bowtie2 2.2.8.

Usage on Owens

Set-up

To configure your environment for use of Bowtie2, run the following command:    module load bowtie2  . The default version will be loaded. To select a particular Bowtie2 version, use    module load bowtie2/version . For example, use  module load bowtie2/2.2.9  to load Bowtie2 2.2.9.

Further Reading

Supercomputer: 
Service: 
Fields of Science: 

CDO

CDO (Climate Data Operators) is a collection of command line operator to manipulate and analyse climate and NWP model data. It is open source and released under the terms of the GNU General Public License v2 (GPL).

Availability and Restrictions

Versions

CDO is available on Oakley Cluster. The versions currently available at OSC are:

version oakley
1.5.4 X
1.6.8 X*
* : Default Version

You can use module avail cdo to view available modules on Oakley. Feel free to contact OSC Help if you need other versions for your work.

Access 

CDO is available for use by all OSC users.

Usage

Usage on Oakley

Set-up on Oakley

To load the default version of CDO module, use  module load cdo. To select a particular software version, use  module load cdo/version. For example, use  module load cdo/1.5.4 to load CDO version 1.5.4 on Oakley. 

Batch Usauge on Oakley

Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations for Oakley and Scheduling Policies and Limits for more info. 
Interactive Batch Session
For an interactive batch session on Oakley, one can run the following command:
qsub -I -l nodes=1:ppn=12 -l walltime=1:00:00
which gives you 12 cores ( -l nodes=1:ppn=12 ) with 1 hour ( -l walltime=1:00:00 ). You may adjust the numbers per your need.
Once your job is ready, you can start to use CDO. For example, use the following command to display general information about a data file named infile:
cdo info infile

Use cdo -h [operator] for operator descriptions. 

Further Reading

Official documentation can be obtained from https://code.zmaw.de/projects/cdo/wiki/Cdo#Documentation

Supercomputer: 
Service: 
Fields of Science: 

COMSOL

COMSOL Multiphysics (formerly FEMLAB) is a finite element analysis and solver software package for various physics and engineering applications, especially coupled phenomena, or multiphysics. owned and supported by COMSOL, Inc.

Availability and Restrictions

COMSOL is available on the Oakley and Ruby clusters. The versions currently available at OSC are

Version oakley Ruby notes
4.3 X    
4.3a X   Default version on Oakley prior to 09/15/2015
4.3b X    
4.4 X    
5.1 X*    
5.2 X X*  
5.2a X    
*: Current default version

You can use module avail comsol  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access for Academic Users

COMSOL is for academic use, available to all Ohio State University users. OSC does not provide COMSOL licenses for academic use to students and faculty outside of Ohio State due to licensing restrictions. If you are at a different academic institution and would like to discuss access to COMSOL, please contact OSC Help .

To use COMSOL you will have to be added to the license server first.  Please contact OSC Help to be added.

Access for Commerical Users

Contact OSC Help for getting access to COMSOL if you are a commercial user. 

Usage on Oakley

Set-up

To load the default version of COMSOL module, use  module load comsol . To select a particular software version, use   module load comsol/version . For example, use  module load comsol/44  to load COMSOL version 4.4. 

Batch Usage

When you log into oakley.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your analysis to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. 

Interactive Batch Session
For an interactive batch session, one can run the following command:
qsub -I -l nodes=1:ppn=12 -l walltime=1:00:00 -l software=comsolscript
which gives you 12 cores ( -l nodes=1:ppn=12 ) with 1 hour ( -l walltime=1:00:00 ). You may adjust the numbers per your need.
Non-interactive Batch Job (Serial Run)

Assume that you have had a comsol script file mycomsol.m  in your working direcory ( $PBS_O_WORKDIR ). Below is the example batch script ( job.txt ) for a serial run: 

#PBS -N COMSOL
#PBS -l walltime=1:00:00
#PBS -l nodes=1:ppn=1
#PBS -j oe
#PBS -l software=comsolscript
#
# The following lines set up the COMSOL environment
#
module load comsol
#
# Move to the directory where the job was submitted
#
cd $PBS_O_WORKDIR
cp *.m $TMPDIR
cd $TMPDIR
#
# Run COMSOL
#
comsol batch mycomsol
#
# Now, copy data (or move) back once the simulation has completed
#
cp * $PBS_O_WORKDIR
Non-interactive Batch Job (Parallel Run for COMSOL 4.3 and Later)

As of version 4.3, it is not necessary to start up MPD before launching a COMSOL job. Below is the example batch script ( job.txt ) for a parallel run using COMSOL 4.3 or later versions:

#PBS -l walltime=01:00:00
#PBS -l nodes=2:ppn=12
#PBS -N COMSOL
#PBS -j oe
#PBS -r n
#PBS -l software=comsolscript
cd ${PBS_O_WORKDIR}
module load comsol
echo "--- Copy Input Files to TMPDIR and Change Disk to TMPDIR"
cp input_cluster.mph $TMPDIR
cd $TMPDIR
np=12
echo "--- Running on ${np} processes (cores) on the following nodes:"
cat $PBS_NODEFILE | uniq > hostfile
echo "--- COMSOL run"
comsol -nn 2 -np ${np} batch -f hostfile -mpirsh rsh -inputfile input_cluster.mph -outputfile output_cluster.mph
echo "--- Copy files back"
cp output_cluster.mph output_cluster.mph.status ${PBS_O_WORKDIR}
echo "---Job finished at: 'date'"
echo "---------------------------------------------"

Note:

  • Set nodes to 2 and ppn to 12 ( -l nodes=2:ppn=12  ). You can change the values per your need.
  • Copy files from your directory to $TMPDIR.
  • Provide the name of the input file and output file.

Usage on Ruby

Set-up

To load the default version of COMSOL module, use   module load comsol  . To select a particular software version, use    module load comsol/version  . For example, use   module load comsol/52   to load COMSOL version 5.2. 

Batch Usage

When you log into ruby.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your analysis to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. 

Interactive Batch Session
For an interactive batch session, one can run the following command:
qsub -I -l nodes=1:ppn=20 -l walltime=1:00:00 -l software=comsolscript
which gives you 20 cores (  -l nodes=1:ppn=20  ) with 1 hour (  -l walltime=1:00:00  ). You may adjust the numbers per your need.
Non-interactive Batch Job (Serial Run)

Assume that you have had a comsol script file  mycomsol.m   in your working direcory (  $PBS_O_WORKDIR  ). Below is the example batch script (  job.txt  ) for a serial run: 

#PBS -N COMSOL
#PBS -l walltime=1:00:00
#PBS -l nodes=1:ppn=1
#PBS -j oe
#PBS -l software=comsolscript
#
# The following lines set up the COMSOL environment
#
module load comsol
#
# Move to the directory where the job was submitted
#
cd $PBS_O_WORKDIR
cp *.m $TMPDIR
cd $TMPDIR
#
# Run COMSOL
#
comsol batch mycomsol
#
# Now, copy data (or move) back once the simulation has completed
#
cp * $PBS_O_WORKDIR
Non-interactive Batch Job (Parallel Run for COMSOL 4.3 and Later)

As of version 4.3, it is not necessary to start up MPD before launching a COMSOL job. Below is the example batch script (  job.txt  ) for a parallel run using COMSOL 4.3 or later versions:

#PBS -l walltime=01:00:00
#PBS -l nodes=2:ppn=20
#PBS -N COMSOL
#PBS -j oe
#PBS -r n
#PBS -l software=comsolscript
cd ${PBS_O_WORKDIR}
module load comsol
echo "--- Copy Input Files to TMPDIR and Change Disk to TMPDIR"
cp input_cluster.mph $TMPDIR
cd $TMPDIR
np=12
echo "--- Running on ${np} processes (cores) on the following nodes:"
cat $PBS_NODEFILE | uniq > hostfile
echo "--- COMSOL run"
comsol -nn 2 -np ${np} batch -f hostfile -mpirsh rsh -inputfile input_cluster.mph -outputfile output_cluster.mph
echo "--- Copy files back"
cp output_cluster.mph output_cluster.mph.status ${PBS_O_WORKDIR}
echo "---Job finished at: 'date'"
echo "---------------------------------------------"

Note:

  • Set nodes to 2 and ppn to 20 (  -l nodes=2:ppn=20 ). You can change the values per your need.
  • Copy files from your directory to $TMPDIR.
  • Provide the name of the input file and output file.

Further Reading

Supercomputer: 
Service: 

CUDA

CUDA™ (Compute Unified Device Architecture) is a parallel computing platform and programming model developed by Nvidia that enables dramatic increases in computing performance by harnessing the power of the graphics processing unit (GPU).

Availability and Restrictions

CUDA is available on Ruby, Oakley, and Glenn Clusters. The versions currently available at OSC are

Version Glenn Oakley Ruby notes
2.3 X      
3.0 X      
3.1 X*      
4.0 X      
5.0.35 X X X Default version on Oakley prior to 09/15/2015
5.5 X X X  
6.0.37   X X  
6.5.14 X X* X*  
7.0.28   X X  
7.5.18     X  
*: Current default version

You can use module avail cuda to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

CUDA is available for use by all OSC users.

Usage

Usage on Glenn

Set-up on Glenn

To load the default version of CUDA module, use  module load cuda. To select a particular software version, use   module load cuda-version. For example, use  module load cuda-4.0 to load CUDA version 4.0 on Glenn. 

GPU Computing SDK

The NVIDIA GPU Computing SDK provides hundreds of code samples and covers a wide range of applications/techniques to help you get started on the path of writing software with CUDA C/C++ or DirectCompute. 

Programming in CUDA

Please visit the following link to learn programming in CUDA, http://developer.nvidia.com/cuda-education-training. The link also contains tutorials on Optimizing CUDA codes to obtain greater SpeedUp. One can also refer to the following webpage for some more CUDA optimization techniques, http://www.cs.berkeley.edu/~volkov/

Compiling CUDA Code

One can type module show cuda-version-number to view the list of environment variables.
To compile a cuda code contained in a file, let say mycudaApp.cu, the following could be done after loading the appropriate CUDA module: nvcc -o mycudaApp mycudaApp.cu This will create an executable by name mycudaApp .

Important: The devices are configured in exclusive mode. This means that 'cudaSetDevice' should NOT be used if requesting one GPU resource. Once the first call to CUDA is executed, the system will figure out which device it is using. If both cards per node is in use by a single application, please use 'cudaSetDevice'.

Debugging CUDA code

cuda-gdb can be used to debug CUDA codes. module load cuda will make it available to you. For more information on how to use the CUDA-GDB please visit http://developer.nvidia.com/cuda-gdb.

Detecting memory access errors

CUDA-MEMCHECK could be used for detecting the source and cause of memory access errors in your program. For more information on how to use CUDA-MEMCHECK please visit http://developer.nvidia.com/cuda-memcheck.

Batch Usage on Glenn

When you log into glenn.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time, up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. Batch jobs run on the compute nodes of the system and not on the login node. It is desirable for big problems since more resources can be used.

Interactive Batch Session

For an interactive batch session one can run the following command:

qsub -I -l nodes=1:ppn=8:gpu -l walltime=00:20:00 
which requests one whole node with 8 cores ( -l nodes=1:ppn=8), for a walltime of 20 minutes ( -l walltime=00:20:00). You may adjust the numbers per your need.
Non-interactive Batch Job (Serial Run)

A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. Below is the example batch script (  job.txt) for a serial run:

#PBS -l walltime=01:00:00
#PBS -l nodes=1:ppn=8:gpu
#PBS -N compute
#PBS -j oe
module load cuda
cd $HOME/cuda
cp mycudaApp $TMPDIR
cd $TMPDIR
./mycudaApp

Usage on Oakley

Set-up on Oakley

To load the default version of CUDA module, use  module load cuda. To select a particular software version, use   module load cuda/version. For example, use  module load cuda/7.0.28 to load CUDA version 7.0.28 on Oakley. 

GPU Computing SDK

The NVIDIA GPU Computing SDK provides hundreds of code samples and covers a wide range of applications/techniques to help you get started on the path of writing software with CUDA C/C++ or DirectCompute. On Oakley, the SDK binaries are located in $CUDA_HOME/bin/linux/release ( $CUDA_HOME is an environment variable set when you load the module).

Programming in CUDA

Please visit the following link to learn programming in CUDA, http://developer.nvidia.com/cuda-education-training. The link also contains tutorials on Optimizing CUDA codes to obtain greater SpeedUp. One can also refer to the following webpage for some more CUDA optimization techniques, http://www.cs.berkeley.edu/~volkov/

Compiling CUDA Code

One can type module show cuda-version-number to view the list of environment variables.
To compile a cuda code contained in a file, let say mycudaApp.cu, the following could be done after loading the appropriate CUDA module: nvcc -o mycudaApp mycudaApp.cu This will create an executable by name mycudaApp.

Important: The devices are configured in exclusive mode. This means that 'cudaSetDevice' should NOT be used if requesting one GPU resource. Once the first call to CUDA is executed, the system will figure out which device it is using. If both cards per node is in use by a single application, please use 'cudaSetDevice'.

Debugging CUDA code

cuda-gdb can be used to debug CUDA codes. module load cuda will make it available to you. For more information on how to use the CUDA-GDB please visit http://developer.nvidia.com/cuda-gdb.

Detecting memory access errors

CUDA-MEMCHECK could be used for detecting the source and cause of memory access errors in your program. For more information on how to use CUDA-MEMCHECK please visit http://developer.nvidia.com/cuda-memcheck.

Batch Usage on Oakley

When you log into oakley.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info.  Batch jobs run on the compute nodes of the system and not on the login node. It is desirable for big problems since more resources can be used.

Interactive Batch Session

For an interactive batch session one can run the following command:

qsub -I -l nodes=1:ppn=1:gpus=1 -l walltime=00:20:00

which requests one core ( -l nodes=1:ppn=1), for a walltime of 20 minutes ( -l walltime=00:20:00). You may adjust the numbers per your need.

Please note that on Oakley, you can request any mix of ppn and gpus you need; please see the Batch Limit Rules and Job Scripts page in our batch guide for more information.
Non-interactive Batch Job (Serial Run)

A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. Below is the example batch script (  job.txt) for a serial run:

#PBS -l walltime=01:00:00
#PBS -l nodes=1:ppn=1:gpus=1
#PBS -N compute
#PBS -j oe
module load cuda
cd $HOME/cuda
cp mycudaApp $TMPDIR
cd $TMPDIR
./mycudaApp

Usage on Ruby

Set-up on Ruby

To load the default version of CUDA module, use  module load cuda. To select a particular software version, use   module load cuda/version. For example, use  module load cuda/7.0.28 to load CUDA version 7.0.28 on Ruby. 

GPU Computing SDK

The NVIDIA GPU Computing SDK provides hundreds of code samples and covers a wide range of applications/techniques to help you get started on the path of writing software with CUDA C/C++ or DirectCompute. 

Programming in CUDA

Please visit the following link to learn programming in CUDA, http://developer.nvidia.com/cuda-education-training. The link also contains tutorials on Optimizing CUDA codes to obtain greater SpeedUp. One can also refer to the following webpage for some more CUDA optimization techniques, http://www.cs.berkeley.edu/~volkov/

Compiling CUDA Code

One can type module show cuda-version-number to view the list of environment variables.
To compile a cuda code contained in a file, let say mycudaApp.cu, the following could be done after loading the appropriate CUDA module: nvcc -o mycudaApp mycudaApp.cu This will create an executable by name mycudaApp.

Important: The devices are configured in exclusive mode. This means that 'cudaSetDevice' should NOT be used if requesting one GPU resource. Once the first call to CUDA is executed, the system will figure out which device it is using. If both cards per node is in use by a single application, please use 'cudaSetDevice'.

Debugging CUDA code

cuda-gdb can be used to debug CUDA codes. module load cuda will make it available to you. For more information on how to use the CUDA-GDB please visit http://developer.nvidia.com/cuda-gdb.

Detecting memory access errors

CUDA-MEMCHECK could be used for detecting the source and cause of memory access errors in your program. For more information on how to use CUDA-MEMCHECK please visit http://developer.nvidia.com/cuda-memcheck.

Setting the GPU compute mode on Ruby

The GPUs on Ruby can be set to different compute modes as listed here.

They can be set by adding the following to the GPU specification:

-l nodes=1:ppn=20:gpus=1:default
-l nodes=1:ppn=20:gpus=1:exclusive
-l nodes=1:ppn=20:gpus=1:exclusive_process

Note, the prohibited mode is not an option.

Batch Usage on Ruby

When you log into ruby.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. 

Interactive Batch Session

For an interactive batch session one can run the following command:

qsub -I -l nodes=1:ppn=20:gpus=1 -l walltime=00:20:00 

which requests one whole node with 20 cores ( -l nodes=1:ppn=20), for a walltime of 20 minutes ( -l walltime=00:20:00). You may adjust the numbers per your need.

Non-interactive Batch Job (Serial Run)

A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. Below is the example batch script (  job.txt) for a serial run:

#PBS -l walltime=01:00:00
#PBS -l nodes=1:ppn=20:gpus=1
#PBS -N compute
#PBS -j oe
module load cuda
cd $HOME/cuda
cp mycudaApp $TMPDIR
cd $TMPDIR
./mycudaApp

Further Reading

Online documentation is available at http://developer.nvidia.com/nvidia-gpu-computing-documentation

See Also

Supercomputer: 
Service: 
Technologies: 
Fields of Science: 

Caffe

Caffe is "a fast open framework for deep learning".

From their README:

Caffe is a deep learning framework made with expression, speed, and modularity in mind. It is developed by the Berkeley Vision and Learning Center (BVLC) and by community contributors. Yangqing Jia created the project during his PhD at UC Berkeley. Caffe is released under the BSD 2-Clause license.

Caffe also includes interfaces for both Python and Matlab.

Availability and Restrictions

Versions

The following versions of Caffe are available on OSC clusters:

version Ruby Owens
1.0.0-rc3 X  

 

The current versions of Caffe require the use of python/2.7.8, cuda/7.5.18, boost/1.56.0, opencv/3.1.0 on Ruby.

Feel free to contact OSC Help if you need other versions for your work.

Access 

Caffe is available to all OSC users without restriction.

Usage

Usage on Ruby

Setup on Ruby

To configure the Oakley or Ruby cluster for the use of Caffe, use the following commands:

module load caffe

Batch Usage on Oakley or Ruby

Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations for Ruby, and Scheduling Policies and Limits for more info.  In particular, Caffe should be run on a GPU-enabled compute node.

An Example of Using Caffe with MNIST Training Data

Below is an example batch script ( job.txt) for using Caffe, see this resource for a detailed explanation http://caffe.berkeleyvision.org/gathered/examples/mnist.html

#PBS -N Caffe
#PBS -l nodes=1:ppn=20:gpus=1:default
#PBS -l walltime=30:00
#PBS -j oe
#PBS -S /bin/bash 

#!/bin/bash
#PBS -l nodes=1:ppn=16:gpus=1:default
#PBS -q debug
#PBS -l walltime=10:00
#PBS -j oe

cd $PBS_O_WORKDIR
. /etc/profile.d/lmod.sh

# Load the modules for Caffe
ml caffe

# Migrate to job temp directory and copy folders over
cd $TMPDIR
cp -a $CAFFE_HOME/{examples,build,data} .

# Download, create, train
./data/mnist/get_mnist.sh
./examples/mnist/create_mnist.sh
./examples/mnist/train_lenet.sh

# Serialize log files
echo; echo 'LOG 1'
cat convert_mnist_data.bin.$(hostname)*
echo; echo 'LOG 2'
cat caffe.INFO
echo; echo 'LOG 3'
cat convert_mnist_data.bin.INFO
echo; echo 'LOG 4'
cat caffe.$(hostname).*

cp examples/mnist/lenet_iter_10000.* $PBS_O_WORKDIR

In order to run it via the batch system, submit the job.txt  file with the following command:

qsub job.txt

Further Reading

http://caffe.berkeleyvision.org/

Supercomputer: 
Service: 
Technologies: 
Fields of Science: 

Cairo

Cairo is a 2D graphics library with support for multiple output devices. It is designed to produce consistent output on all output media while taking advantage of display hardware acceleration when available.

Availability and Restrictions

Versions

The following versions of Cairo are available on OSC clusters:

VERSION OAKLEY
1.12.18 X(GI)*
1.14.2 X(GI)

 

*: Default Version
  • X(GI): available with gnu and intel

You can use  module avail cairo  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Cairo is available to all OSC users without restriction.

Usage

Usage on Oakley

Set-up on Oakley

To configure your environment for use of Cairo on Oakley, run the following command:  module load cairo. The default version will be loaded. To select a particular Cairo version, use  module load cairo/version . For example, use module load cairo/1.14.2  to load Cairo 1.14.2 on Oakley.

Building with Cairo

The Cairo library provides the following variables for use at build time:

Variable Use
$CAIRO_CFLAGS Use during your compilation step for programs
$CAIRO_LIBS Use during your linking step programs

For example, to build the code myprog.c with the Cairo library you would use:

icc -o myprog myprog.c $CAIRO_LIBS $CAIRO_CFLAGS

Further Reading

Supercomputer: 
Service: 
Fields of Science: 

Cambridge Structural Database

Introduction

The Cambridge Structural Database (CSD) contains complete structure information on hundreds of thousands of small molecule crystals. The Cambridge Crystallographic Data Centre (CCDC) is a suite of programs for CSD search and analysis. The Cambridge Crystallographic Data Centre Home Page has additional information.

Version

Several versions of the software package are available at OSC. Search for csd in this list:

module avail csd

There are also a very limited number of licenses for the web version WebCSD; contact OSC Help for availability.

Availability

CSD is available on cambridge.osc.edu (oakley01.osc.edu).

Usage

In order to access this software license, please contact OSC Help for further instruction.

To run conquest:

module load csd
cq

Documentation

General documentation is available at the CCDC Documentation page and in the local machine directories.

Supercomputer: 
Service: 

Clustal W

Clustal W is a general purpose multiple sequence alignment program for DNA or proteins.It produces biologically meaningful multiple sequence alignments of divergent sequences. It calculates the best match for the selected sequences, and lines them up so that the identities, similarities and differences can be seen.

Availability & Restrictions

Clustal W is available without restriction to all OSC users.

The following versions of Clustal W are available on OSC systems:

Version Glenn Oakley
1.8.3 X  
2.1   X

Usage

Set-up

Setup prior to use of Clustal W is dependent on the system you are using. On the Glenn system, first load the biosoftware module, then load the clustalw module:

module load biosoftw
module load clustalw

On the Oakley system, just load the clustalw module directly:

module load clustalw

 

Using Clustal W

Once the clustalw module has been loaded, the commands are available for your use. On the Glenn system, the command is

clustalw

On the Oakley system, the command is

clustalw2

The options can be listed interactively by typing clustalw -help or clustalw -check on the command-line.

                DATA (sequences)
-INFILE=file.ext                             :input sequences.
-PROFILE1=file.ext  and  -PROFILE2=file.ext  :profiles (old alignment).

                VERBS (do things)
-OPTIONS            :list the command line parameters
-HELP  or -CHECK    :outline the command line params.
-ALIGN              :do full multiple alignment 
-TREE               :calculate NJ tree.
-BOOTSTRAP(=n)      :bootstrap a NJ tree (n= number of bootstraps; def. = 1000).
-CONVERT            :output the input sequences in a different file format.

                PARAMETERS (set things)
***General settings:****
-INTERACTIVE :read command line, then enter normal interactive menus
-QUICKTREE   :use FAST algorithm for the alignment guide tree
-TYPE=       :PROTEIN or DNA sequences
-NEGATIVE    :protein alignment with negative values in matrix
-OUTFILE=    :sequence alignment file name
-OUTPUT=     :GCG, GDE, PHYLIP, PIR or NEXUS
-OUTORDER=   :INPUT or ALIGNED
-CASE        :LOWER or UPPER (for GDE output only)
-SEQNOS=     :OFF or ON (for Clustal output only)
-SEQNO_RANGE=:OFF or ON (NEW: for all output formats) 
-RANGE=m,n   :sequence range to write starting m to m+n. 

***Fast Pairwise Alignments:***
-KTUPLE=n    :word size
-TOPDIAGS=n  :number of best diags.
-WINDOW=n    :window around best diags.
-PAIRGAP=n   :gap penalty
-SCORE       :PERCENT or ABSOLUTE

***Slow Pairwise Alignments:***
-PWMATRIX=    :Protein weight matrix=BLOSUM, PAM, GONNET, ID or filename
-PWDNAMATRIX= :DNA weight matrix=IUB, CLUSTALW or filename
-PWGAPOPEN=f  :gap opening penalty        
-PWGAPEXT=f   :gap opening penalty

***Multiple Alignments:***
-NEWTREE=      :file for new guide tree
-USETREE=      :file for old guide tree
-MATRIX=       :Protein weight matrix=BLOSUM, PAM, GONNET, ID or filename
-DNAMATRIX=    :DNA weight matrix=IUB, CLUSTALW or filename
-GAPOPEN=f     :gap opening penalty        
-GAPEXT=f      :gap extension penalty
-ENDGAPS       :no end gap separation pen. 
-GAPDIST=n     :gap separation pen. range
-NOPGAP        :residue-specific gaps off  
-NOHGAP        :hydrophilic gaps off
-HGAPRESIDUES= :list hydrophilic res.    
-MAXDIV=n      :% ident. for delay
-TYPE=         :PROTEIN or DNA
-TRANSWEIGHT=f :transitions weighting

***Profile Alignments:***
-PROFILE      :Merge two alignments by profile alignment
-NEWTREE1=    :file for new guide tree for profile1
-NEWTREE2=    :file for new guide tree for profile2
-USETREE1=    :file for old guide tree for profile1
-USETREE2=    :file for old guide tree for profile2

***Sequence to Profile Alignments:***
-SEQUENCES   :Sequentially add profile2 sequences to profile1 alignment
-NEWTREE=    :file for new guide tree
-USETREE=    :file for old guide tree

***Structure Alignments:***
-NOSECSTR1     :do not use secondary structure-gap penalty mask for profile 1 
-NOSECSTR2     :do not use secondary structure-gap penalty mask for profile 2
-SECSTROUT=STRUCTURE or MASK or BOTH or NONE   :output in alignment file
-HELIXGAP=n    :gap penalty for helix core residues 
-STRANDGAP=n   :gap penalty for strand core residues
-LOOPGAP=n     :gap penalty for loop regions
-TERMINALGAP=n :gap penalty for structure termini
-HELIXENDIN=n  :number of residues inside helix to be treated as terminal
-HELIXENDOUT=n :number of residues outside helix to be treated as terminal
-STRANDENDIN=n :number of residues inside strand to be treated as terminal
-STRANDENDOUT=n:number of residues outside strand to be treated as terminal 

***Trees:***
-OUTPUTTREE=nj OR phylip OR dist OR nexus
-SEED=n        :seed number for bootstraps.
-KIMURA        :use Kimura's correction.   
-TOSSGAPS      :ignore positions with gaps.
-BOOTLABELS=node OR branch :position of bootstrap values in tree display

Batch Usage

Sample batch script for the Oakley system:

#PBS -N clustalw
#PBS -l walltime=1:00:00
#PBS -l nodes=1:ppn=1
#PBS -j oe

cd $PBS_O_WORKDIR
module load clustalw
clustalw2 -INFILE=myfile.seqs -GAPOPEN=2 -GAPEXT=4

Further Reading

See Also

Supercomputer: 
Service: 
Fields of Science: 

EMBOSS

EMBOSS is "The European Molecular Biology Open Software Suite". EMBOSS is a free Open Source software analysis package specially developed for the needs of the molecular biology (e.g. EMBnet) user community. The software automatically copes with data in a variety of formats and even allows transparent retrieval of sequence data from the web. Also, as extensive libraries are provided with the package, it is a platform to allow other scientists to develop and release software in true open source spirit. EMBOSS also integrates a range of currently available packages and tools for sequence analysis into a seamless whole.

Within EMBOSS you will find around hundreds of programs (applications) covering areas such as:

  • Sequence alignment,
  • Rapid database searching with sequence patterns,
  • Protein motif identification, including domain analysis,
  • Nucleotide sequence pattern analysis---for example to identify CpG islands or repeats,
  • Codon usage analysis for small genomes,
  • Rapid identification of sequence patterns in large scale sequence sets,
  • Presentation tools for publication

Availability & Restrictions

The EMBOSS software package is available without restriction for all academic OSC users.

The following versions of EMBOSS are available on OSC systems:

Version Glenn Oakley
5.0.0 X  
6.4.0   X

Usage

Set-up

Setup of EMBOSS depends on what system you are using. For the Glenn system, run the following commands:

module load biosoftw
module load emboss

For the Oakly system, run the following command:

module load emboss

Using EMBOSS

The EMBOSS programs are typically used to perform one or more tasks on a large number of sequences.  Once the emboss module is loaded, all emboss commands are available for your use.

Batch Usage

Coming soon.

Further Reading

See Also

Supercomputer: 
Service: 
Fields of Science: 

Elmer

Elmer is an open source multiphysical simulation software developed by CSC - IT (Finland).

From their Github README:

Elmer is a finite element software for numerical solution of partial
differential equations. Elmer is capable of handling any number of equations
and is therefore ideally suited for the simulation of multiphysical problems.
It includes models, for example, of structural mechanics, fluid dynamics, heat
transfer and electromagnetics. Users can also write their own equations that
can be dynamically linked with the main program.

Elmer consists of several parts. The most important ones are ElmerSolver, the
finite element solver, ElmerGUI, the graphical user interface, and ElmerGrid,
the mesh creation and manipulation tool. Also a visualization tool, ElmerPost,
is included in the package but it is no longer developed.  

Elmer software is licensed under GPL except for the ElmerSolver library which
is licensed under LGPL license. 

Elmer is mainly developed at CSC - IT Center for Science, Finland. However,
there have been numerous contributions from other organizations and developers
as well, and the project is open for new contributions.

Availability and Restrictions

Versions

The following versions of Elmer are available on OSC clusters:

version oakley Ruby
8.1 X X

 

The current versions of Elmer are compiled with GNU 4.8.4 (requirement > 4.8) and BLAS libraries from MKL/11.2.3.

Feel free to contact OSC Help if you need other versions for your work.

Access 

Elmer is available to all OSC users without restriction.

Usage

Usage on Oakley or Ruby

Setup on Oakley or Ruby

To configure the Oakley or Ruby cluster for the use of Elmer 8.1, use the following commands:

module load gnu/4.8.4
module load mkl/11.2.3
module load elmer

Batch Usage on Oakley or Ruby

Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations for OakleyQueues and Reservations for Ruby, and Scheduling Policies and Limits for more info.

Converting Mesh with ElmerGrid and Running ElmerSolver

Below is an example batch script ( job.txt) for using Elmer:

#PBS -N Elmer
#PBS -l nodes=1:ppn=1 
#PBS -l walltime=30:00
#PBS -j oe 
#PBS -S /bin/bash 

# Load the modules for Elmer
module load gnu/4.8.4
module load mkl/11.2.3
module load elmer
# Move to the problem directory
cd $PBS_O_WORKDIR
# Copy files to $TMPDIR and move there to execute the program
cp -r * $TMPDIR
cd $TMPDIR
# Convert the mesh - see Elmer webpage for more examples of 'ElmerGrid'
ElmerGrid 14 2 beam.msh -autoclean
# Run the solver 
ElmerSolver beam.sif
# Finally, copy files back to your home directory 
cp -r * $PBS_O_WORKDIR

In order to run it via the batch system, submit the job.txt  file with the following command:

qsub job.txt

Further Reading

https://www.csc.fi/web/elmer/

https://github.com/ElmerCSC/elmerfem

Supercomputer: 
Service: 
Fields of Science: 

FASTX-Toolkit

The FASTX-Toolkit is a collection of command line tools for Short-Reads FASTA/FASTQ files preprocessing.

Availability and Restrictions

The following versions of FASTX-Toolkit are available on OSC clusters:

Version Oakley Owens
0.0.14 X* X*
*: Current default version

You can use  module avail fastx to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

FASTX-Toolkit is available to all OSC users without restriction.

Usage on Oakley

Set-up

To configure your environment for use of FASTX-Toolkit, run the following command:   module load fastx. The default version will be loaded. To select a particular FASTX-Toolkit version, use   module load fastx/version. For example, use  module load fastx/0.0.14 to load FASTX-Toolkit 0.0.14.

Usage on Owens

Set-up

To configure your environment for use of FASTX-Toolkit, run the following command:   module load fastx. The default version will be loaded. To select a particular FASTX-Toolkit version, use   module load fastx/version. For example, use  module load fastx/0.0.14 to load FASTX-Toolkit 0.0.14.

Further Reading

Supercomputer: 
Service: 
Fields of Science: 

FFTW

FFTW is a C subroutine library for computing the Discrete Fourier Transform (DFT) in one or more dimensions, of arbitrary input size, and of both real and complex data. It is portable and performs well on a wide variety of platforms.

Availability and Restrictions

FFTW is available on Oakley, Ruby, and Owens Clusters. The versions currently available at OSC are:

Version OAKLEY RUBY Owens notes
2.1 X*      
2.1.5   X(GI)*    
3.1.2        
3.3 X     Default version on Oakley prior to 09/15/2015
3.3.1        
3.3.2        
3.3.3 X(I)      
3.3.4 X* X* X(GI)*  
  • X(G): gnu versions only,
  • X(I): intel version only
  • X(GI): available with both gnu and intel
  • X: available with gnu, intel, and pgi
* : Current Default Version (note: FFTW2 and FFTW3 are tracked separately in the module system)

You can use module spider fftw2  or module spider fftw3   to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

FFTW is available without restriction to all OSC Users

Usage on Oakley

Set-up

Initalizing the system for use of the FFTW library is dependent on the system you are using and the compiler you are using. A successful build of your program will depend on an understanding of what module fits your circumstances. To load a particular version, use  module load name  . For example, use  module load fftw3/3.3.3  to load FFTW3 version 3.3.3. You can use module spider fftw  to view available modules.

Building with FFTW

The following environment variables are setup when the FFTW library is loaded:

Variable Use
$FFTW3_CFLAGS Use during your compilation step for C programs.
$FFTW3_FFLAGS Use during your compilation step for Fortran programs.
$FFTW3_LIBS Use during your link step for the sequential version of the library.
$FFTW3_LIBS_OMP Use during your link step for the OpenMP version of the library.
$FFTW3_LIBS_MPI Use during your link step for the MPI version of the library.

below is a set of example commands used to build a file called my-fftw.c .

module load fftw3
icc $FFTW3_CFLAGS my-fftw.c -o my-fftw $FFTW3_LIBS 
ifort $FFTW3_FFLAGS more-fftw.f -o more-fftw $FFTW3_LIBS

Usage on Ruby

Set-up

Initalizing the system for use of the FFTW library is dependent on the system you are using and the compiler you are using.  A successful build of your program will depend on an understanding of what module fits your circumstances. To load a particular version, use  module load name  . For example, use  module load fftw3/3.3.4  to load FFTW3 version 3.3.4. You can use module spider fftw  to view available modules.

Building with FFTW

The following environment variables are setup when the FFTW library is loaded:

Variable Use
$FFTW3_CFLAGS Use during your compilation step for C programs.
$FFTW3_FFLAGS Use during your compilation step for Fortran programs.
$FFTW3_LIBS Use during your link step for the sequential version of the library.
$FFTW3_LIBS_OMP Use during your link step for the OpenMP version of the library.
$FFTW3_LIBS_MPI Use during your link step for the MPI version of the library.

below is a set of example commands used to build a file called my-fftw.c .

module load fftw3
icc $FFTW3_CFLAGS my-fftw.c -o my-fftw $FFTW3_LIBS 
ifort $FFTW3_FFLAGS more-fftw.f -o more-fftw $FFTW3_LIBS

Usage on Owens

Set-up

Initalizing the system for use of the FFTW library is dependent on the system you are using and the compiler you are using. A successful build of your program will depend on an understanding of what module fits your circumstances. To load a particular version, use  module load name  . For example, use  module load fftw3/3.3.4  to load FFTW3 version 3.3.4. You can use module spider fftw  to view available modules.

Building with FFTW

The following environment variables are setup when the FFTW library is loaded:

Variable Use
$FFTW3_CFLAGS Use during your compilation step for C programs.
$FFTW3_FFLAGS Use during your compilation step for Fortran programs.
$FFTW3_LIBS Use during your link step for the sequential version of the library.
$FFTW3_LIBS_OMP Use during your link step for the OpenMP version of the library.
$FFTW3_LIBS_MPI Use during your link step for the MPI version of the library.
$FFTW3_LIBS_THREADS Use during your link step for the "threads" version of the library.

below is a set of example commands used to build a file called my-fftw.c .

module load fftw3
icc $FFTW3_CFLAGS my-fftw.c -o my-fftw $FFTW3_LIBS 
ifort $FFTW3_FFLAGS more-fftw.f -o more-fftw $FFTW3_LIBS

Further Reading

See Also

Supercomputer: 
Service: 

Fitmodel

"'Fitmodel' estimates the parameters of various codon-based models of substitution, including those described in Guindon, Rodrigo, Dyer and Huelsenbeck (2004).  These models are especially useful as they accommodate site-specific switches between selection regimes without a priori knowledge of the positions in the tree where changes of selection regimes occurred.

The program will ask for two input files: a tree file and a sequence file.  The tree should be unrooted and in NEWICK format.  The sequences should be in PHYLIP interleaved or sequential format.  If you are planning to use codon-based models, the sequence length should be a multiple of 3.  The program provides four types of codon models: M1, M2, M2a, and M3 (see PAML manual).  Moreover, M2, M2a and M3 can be combined with 'switching' models (option 'M').  Two switching models are implemented: S1 and S2.  S1 constraints the rates of changes between dN/dS values to be uniform (e.g., the rates of changes between negative and positive selection is constrained to be the same as the rate of change between neutrality and positive selection) while S2 allows for differents rates of change between the different classes of dN/dS values.

If you are using a 'switching' model, 'fitmodel' will output file with the following names: your_sequence_file_trees_w1, your_sequence_file_trees_w2, your_sequence_file_trees_w3 and your_sequence_file_trees_wbest.  The w1, w2 and w3 files give the estimated tree with probabilities of w1, w2, and w3 (three maximum likelihood dN/dS ratio estimates) calculated on each edge of the tree and for each site.  Hence, the first tree in one of these files reports the probabilities calculated at the first site of the alignment.  Instead of probabilities, the wbest file allows you to identify which of the tree dN/dS is the most probable on any give edge, at any given site.  A branch with label 0.0 means that w1 is the most probable class, 0.5 indicates the w2 is the most probable and 1.0 means that w3 has the highest posterior probability." (README.txt)

Availability & Restrictions

Fitmodel is available to all OSC users without restriction.

The following versions of fitmodel are available on OSC systems:

VersionGlennOakley
0.5.3X 

Usage

Set-up

On the Glenn Cluster fitmodel is accessed by executing the following commands:

module load biosoftw
module load fitmodel

Using fitmodel

fitmodel will be added to the users PATH and can then be run with the following command:

fitmodel -treefile treefilename -seqfile seqfilename [options]

Options

-type nt or aa (default=nt)
-freq empirical or ml or uniform or F3X4 (defaults=empirical or F3X4)
-codon no or yes (defaults=no)
-model JC69, K80, F81, HKY85, F84, TN93, GTR, Dayhoff, JTT, MtREV, WAG, DCMut, M2 or M3 (default=HKY85)
-pinvar [0.0;1.0]
-optpinvar no or yes (default=no)
-kappa [0.01;100.0]
-optkappa no or yes (default=no)
-ncatg integer > 0
-alpha [0.01;100.0]
-optalpha no or yes (default=no)
-code 1,2,3,4,5,6,9,10,11,12,13,14,15,16,21,22,23 (see NCBI Taxonomy webpage) (default=yes)
-p1 [0.0;1.0]
-p2 [0.0;1.0]
-p3 [0.0;1.0]
-w1 [1E-7;1E+7]
-w2 [1E-7;1E+7]
-w3 [1E-7;1E+7]
-switches no or S1 or S2 (default=no)
-optpw yes or no (default=yes)
-multiple integer > 0
-interleaved yes or no (default=yes)
-optall yes or no (default=yes)

Batch Usage

Modified PAML's example brown.trees & brown.nuc files to be in NEWICK & PHYLIP formats respectively.

#PBS -N fitmodel_test
#PBS -l walltime=00:05:00
#PBS -l nodes=1:ppn=4
#PBS -j oe

module load biosoftw
module load fitmodel
cd $PBS_O_WORKDIR
echo "y" | fitmodel -treefile brown.newick -seqfilename brown.phylip

Further Reading

Supercomputer: 
Service: 

FreeType

FreeType is a freely available software library to render fonts. It is written in C, designed to be small, efficient, highly customizable, and portable while capable of producing high-quality output (glyph images) of most vector and bitmap font formats.

Availability and Restrictions

Versions

The following versions of FreeType are available on OSC clusters:

VERSION OAKLEY
2.5.5 X*
*: Default Version

You can use  module avail freetype  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

FreeType is available to all OSC users without restriction.

Usage

Usage on Oakley

Set-up on Oakley

To configure your environment for use of FreeType on Oakley, run the following command:  module load freetype . The default version will be loaded. To select a particular FreeType version, use  module load freetype/version . For example, use module load freetype/2.5.5  to load FreeType 2.5.5 on Oakley.

Further Reading

Supercomputer: 
Service: 
Fields of Science: 

GAMESS

Introduction

The General Atomic and Molecular Electronic Structure System (GAMESS) is a flexible ab initio electronic structure program. Its latest version can perform general valence bond, multiconfiguration self-consistent field, Möller-Plesset, coupled-cluster, and configuration interaction calculations. Geometry optimizations, vibrational frequencies, thermodynamic properties, and solution modeling are available. It performs well on open shell and excited state systems and can model relativistic effects. The GAMESS Home Page has additional information.

Availability

GAMESS is available on the Glenn and Oakley Clusters.

Version

Different versions of GAMESS are available on the Glenn and Oakley clusters.

VERSION GLENN OAKLEY
24 MAR 2007 (R3) X  
12 JAN 2009 (R3) X*  
1 MAY 2013 (R1)   X*

* - Default Version

Usage

Users are not required to complete a license agreement.

Location

The root of the GAMESS directory tree is /usr/local/gamess/

Execution

GAMESS usage is controlled via modules. Load one of the GAMESS modulefiles at the command line, in your shell initialization script, or in your batch scripts, for example:

module load gamess  

Examples

Further Reading

General documentation is available from the GAMESS Home page and in the local machine directories.

Supercomputer: 
Service: 

GATK

GATK is a software package for analysis of high-throughput sequencing data. The toolkit offers a wide variety of tools, with a primary focus on variant discovery and genotyping as well as strong emphasis on data quality assurance.

Availability and Restrictions

The following versions of GATK are available on OSC clusters:

Version Oakley Owens
3.5 X* X*
*: Current default version

You can use  module avail gatk to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access for Academic Users

GATK is available to academic OSC users with proper validation. In order to obtain validation, please contact OSC Help for further instruction. 

Usage on Oakley

Set-up

To configure your environment for use of GATK, run the following command:   module load gatk. The default version will be loaded. To select a particular GATK version, use   module load gatk/version. For example, use  module load gatk/3.5to load GATK 3.5.

Usage

This software is a Java executable .jar file; thus, it is not possible to add to the PATH environment variable.

From module load gatk , a new environment variable, GATK, will be set.

Thus, users can use the software by running the following command: java -jar $GATK {other options}

Usage on Owens

Set-up

To configure your environment for use of GATK, run the following command:   module load gatk. The default version will be loaded. To select a particular GATK version, use   module load gatk/version. For example, use module load gatk/3.5 to load GATK 3.5.

Usage

This software is a Java executable .jar file; thus, it is not possible to add to the PATH environment variable.

From module load gatk, a new environment variable, GATK, will be set.

Thus, users can use the software by running the following command: java -jar $GATK {other options}

Further Reading

Supercomputer: 
Service: 
Fields of Science: 

GLPK

GLPK (GNU Linear Programming Kit) is a set of open source LP (linear programming) and MIP (mixed integer problem) routines written in ANSI C, which can be called from within C programs. 

Availability & Restrictions

GLPK is available to all OSC users without restriction. 

The following versions are available on OSC systems:

versionglennoakley
4.48 X

Usage

Setup

To set up your environment for using GLPK on Oakley, run the following command:

module load glpk

Compiling and Linking

To compile your C code using GLPK API routines, use the environment variable $GLPK_CFLAGS provided by the module:

gcc $GLPK_CFLAGS -c my_prog.c

To link your code, use the variable $GLPK_LIBS:

gcc my_prog.o $GLPK_LIBS -o my_prog

glpsol

Additionally, the GLPK module contains a stand-alone LP/MIP solver, which can be used to process files written in the GNU MathProg modeling language.  The solver can be invoked using the following command syntax:

glpsol [options] [filename]

For a complete list of options, use the following command:

glpsol --help

Further Reading

GLPK Homepage

Supercomputer: 
Service: 

GMAP

GMAP: a genomic mapping and alignment program for mRNA and EST sequences.

Availability and Restrictions

The following versions of GMAP are available on OSC clusters:

Version Oakley Owens
2016-05-25 X*  
2016-06-09   X*
*: Current default version

You can use  module avail gmap   to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

GMAP is available to all OSC users without restriction.

Usage on Oakley

Set-up

To configure your environment for use of GMAP, run the following command:   module load gmap . The default version will be loaded. To select a particular GMAP version, use   module load gmap/version . For example, use  module load gmap/2016-05-25  to load GMAP 2016-05-25.

Usage on Owens

Set-up

To configure your environment for use of GMAP, run the following command:   module load gmap . The default version will be loaded. To select a particular GMAP version, use   module load gmap/version . For example, use module load gmap/2016-06-09  to load GMAP 2016-06-09.

Further Reading

Supercomputer: 
Service: 
Fields of Science: 

GNU Compilers

Fortran, C and C++ compilers produced by the GNU Project. 

Availability and Restrictions

GNU Compiler is available on Oakley, Ruby, and Owens Clusters. The versions currently available at OSC are:

Version Oakley Ruby Owens notes
4.4.7 X X   Default version on Oakley and Ruby prior to 09/15/2015
4.8.4 X* X*    
4.8.5     X*  
4.9.1 X X    
5.1.0 X      
5.2.0 X X    
6.1.0     X  
* : Current Default Version

You can use module avail gnu  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

The GNU compilers are available to all OSC users without restriction.

Usage

Usage on Oakley

Set-up

To configure your environment for use of the GNU compilers, run the following command (you may have to unload your selected compiler - if an error message appears, it will provide instructions):  module load gnu . The default version will be loaded. To select a particular GNU version, use module load gnu/version . For example, use module load gnu/4.8.4  to load GNU 4.8.4.

How to Compile

Once the module is loaded, follow the guides below for compile commands:

Language non-mpi mpi
Fortran 90 or 95 gfortran mpif90
Fortran 77 g77 mpif77
c gcc mpicc
c++ g++ mpicxx

Building Options

The GNU compilers recognize the following command line options :

Compiler Option Purpose
-fopenmp Enables compiler recognition of OpenMPdirectives (except g77 and mpi77)
-o FILENAME

Specifies the name of the object file

-O0 or no -O  option Disable optimization
-O1 or -O Ligh optimization
-O2 Heavy optimization
-O3 Most expensive optimization (Recommended)

There are numerous flags that can be used. For more information run man <compiler binary name> .

Batch Usage

When you log into oakley.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info.  Batch jobs run on the compute nodes of the system and not on the login node. It is desirable for big problems since more resources can be used.

Interactive Batch Session
For an interactive batch session, one can run the following command:
qsub -I -l nodes=1:ppn=12 -l walltime=1:00:00
which gives you 12 cores ( -l nodes=1:ppn=12 ) with 1 hour ( -l walltime=1:00:00 ). You may adjust the numbers per your need.
Non-interactive Batch Job (Serial Run)
A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. The following example batch script file will use the input file named hello.c  and the output file named hello_results
Below is the example batch script ( job.txt ) for a serial run:
#PBS -l walltime=1:00:00
#PBS -l nodes=1:ppn=12
#PBS -N jobname
#PBS -j oe
cd $PBS_O_WORKDIR
module load gnu
cp hello.c $TMPDIR
cd $TMPDIR
gcc -O3 hello.c -o hello
./hello > hello_results
cp hello_results $PBS_O_WORKDIR

In order to run it via the batch system, submit the job.txt file with the following command:

qsub job.txt
Non-interactive Batch Job (Parallel Run)
Below is the example batch script ( job.txt ) for a parallel run:
#PBS -l walltime=1:00:00
#PBS -l nodes=2:ppn=12
#PBS -N jobname
#PBS -j oe
cd $PBS_O_WORKDIR
module load gnu
mpicc -O3 hello.c -o hello
pbsdcp hello $TMPDIR
cd $TMPDIR
mpiexec hello > hello_results
pbsdcp -g hello_results $PBS_O_WORKDIR

Usage on Ruby

Set-up

To configure your environment for use of the GNU compilers, run the following command (you may have to unload your selected compiler - if an error message appears, it will provide instructions):   module load gnu .  The default version will be loaded. To select a particular GNU version, use  module load gnu/version  . For example, use module load gnu/4.9.1  to load GNU 4.9.1.

How to Compile

Once the module is loaded, follow the guides below for compile commands:

Language non-mpi mpi
Fortran 90 or 95 gfortran mpif90
Fortran 77 g77 mpif77
c gcc mpicc
c++ g++ mpicxx

Building Options

The GNU compilers recognize the following command line options :

Compiler Option Purpose
-fopenmp Enables compiler recognition of OpenMPdirectives (except g77 and mpi77)
-o FILENAME

Specifies the name of the object file

-O0 or no -O  option Disable optimization
-O1 or -O Ligh optimization
-O2 Heavy optimization
-O3 Most expensive optimization (Recommended)

There are numerous flags that can be used. For more information run man <compiler binary name> .

Batch Usage

When you log into ruby.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. 

Interactive Batch Session
For an interactive batch session, one can run the following command:
qsub -I -l nodes=1:ppn=20 -l walltime=1:00:00
which gives you 20 cores (  -l nodes=1:ppn=20 )  with 1 hour (  -l walltime=1:00:00 ).  You may adjust the numbers per your need.
Non-interactive Batch Job (Serial Run)
batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. The following example batch script file will use the input file named  hello.c  and the output file named  hello_results
Below is the example batch script (  job.txt  ) for a serial run:
#PBS -l walltime=1:00:00
#PBS -l nodes=1:ppn=20
#PBS -N jobname
#PBS -j oe
cd $PBS_O_WORKDIR
module load gnu
cp hello.c $TMPDIR
cd $TMPDIR
gcc -O3 hello.c -o hello
./hello > hello_results
cp hello_results $PBS_O_WORKDIR

In order to run it via the batch system, submit the  job.txt  file with the following command:

qsub job.txt
Non-interactive Batch Job (Parallel Run)
Below is the example batch script (  job.txt  ) for a parallel run:
#PBS -l walltime=1:00:00
#PBS -l nodes=2:ppn=20
#PBS -N jobname
#PBS -j oe
cd $PBS_O_WORKDIR
module load gnu
mpicc -O3 hello.c -o hello
cp hello $TMPDIR
cd $TMPDIR
mpiexec ./hello > hello_results
cp hello_results $PBS_O_WORKDIR

Usage on Owens

Set-up

To configure your environment for use of the GNU compilers, run the following command (you may have to unload your selected compiler - if an error message appears, it will provide instructions):   module load gnu .  The default version will be loaded. To select a particular GNU version, use  module load gnu/version  . For example, use module load gnu/4.8.5  to load GNU 4.8.5.

How to Compile

Once the module is loaded, follow the guides below for compile commands:

Language non-mpi mpi
Fortran 90 or 95 gfortran mpif90
Fortran 77 g77 mpif77
c gcc mpicc
c++ g++ mpicxx

Building Options

The GNU compilers recognize the following command line options :

Compiler Option Purpose
-fopenmp Enables compiler recognition of OpenMPdirectives (except g77 and mpi77)
-o FILENAME

Specifies the name of the object file

-O0 or no -O  option Disable optimization
-O1 or -O Ligh optimization
-O2 Heavy optimization
-O3 Most expensive optimization (Recommended)

There are numerous flags that can be used. For more information run man <compiler binary name> .

Batch Usage

When you log into owens.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. 

Interactive Batch Session
For an interactive batch session, one can run the following command:
qsub -I -l nodes=1:ppn=28 -l walltime=1:00:00
which gives you 28 cores (  -l nodes=1:ppn=28 )  with 1 hour (  -l walltime=1:00:00 ).  You may adjust the numbers per your need.
Non-interactive Batch Job (Serial Run)
batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. The following example batch script file will use the input file named  hello.c  and the output file named  hello_results
Below is the example batch script (  job.txt  ) for a serial run:
#PBS -l walltime=1:00:00
#PBS -l nodes=1:ppn=28
#PBS -N jobname
#PBS -j oe
cd $PBS_O_WORKDIR
module load gnu
cp hello.c $TMPDIR
cd $TMPDIR
gcc -O3 hello.c -o hello
./hello > hello_results
cp hello_results $PBS_O_WORKDIR

In order to run it via the batch system, submit the  job.txt  file with the following command:

qsub job.txt
Non-interactive Batch Job (Parallel Run)
Below is the example batch script (  job.txt  ) for a parallel run:
#PBS -l walltime=1:00:00
#PBS -l nodes=2:ppn=28
#PBS -N jobname
#PBS -j oe
cd $PBS_O_WORKDIR
module load gnu
mpicc -O3 hello.c -o hello
cp hello $TMPDIR
cd $TMPDIR
mpiexec ./hello > hello_results
cp hello_results $PBS_O_WORKDIR

Further Reading

See Also

Supercomputer: 
Service: 
Technologies: 
Fields of Science: 

GPU-Blast

GPU-BLAST is an accelerated version of the popular NCBI-BLAST (www.ncbi.nlm.nih.gov) that uses general-purpose graphics processing unit (GPU). In comparison to the sequential NCBI-BLAST, GPU-BLAST is nearly four times faster, while producing identical results.

Availability & Restrictions

GPU-BLAST is available without restriction to all OSC users.

The following versions of GPU-BLAST are available on OSC systems:

VERSIONNCBI-Blast VersionGLENNOAKLEY
1.02.2.24+ X
1.12.2.25+ X
1.12.2.26+ X

Usage

Set-up

To set up the environment for gpu-blast on Oakley cluster, do the following:

module load cuda/4.1.28
module load gpu-blast

Batch Usage

A sample batch script is below:

#PBS -l nodes=1:ppn=1:gpus=1
#PBS -l walltime=10:00
#PBS -N GPU-Blast
#PBS -S /bin/bash
#PBS -j oe

module load blast
module load cuda/4.1.28
set -x

cd $PBS_O_WORKDIR
mkdir $PBS_JOBID

cp 100.fasta $TMPDIR
cd $TMPDIR
/usr/bin/time blastn -db nt -query 100.fasta  -out test.out

cp * $PBS_O_WORKDIR/$PBS_JOBID

Further Reading

See Also

GROMACS

GROMACS is a versatile package of molecular dynamics simulation programs. It is primarily designed for biochemical molecules, but it has also been used on non-biological systems.  GROMACS generally scales well on OSC platforms. Starting with version 4.6 GROMACS includes GPU acceleration.

Availability and Restrictions

GROMACS is available on Oakley and Owens Clusters. The versions currently available at OSC are the following (S means serial single node executables, P means parallel multinode, and C means CUDA, i.e., GPU enabled):

Version Oakley Owens notes
4.5.5 SP   Default version on Oakley prior to 09/15/2015
4.6.3 SPC*    
5.1 SPC    
5.1.2   SP  
*: Current default version

You can use module avail gromacs  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

GROMACS is available to all OSC users without restriction.

Usage on Oakley

Set-up

To load the default version of GROMACS module/initalize your environement for use of GROMACS, use  module load gromacs . To select a particular software version, use   module load gromacs/version . For example, use  module load gromacs/5.1  to load GROMACS version 5.1; and use  module help gromacs/5.1  to view details, such as, compiler prerequisites, additional modules required for specific executables, etc.

Using GROMACS

The syntax of the GROMACS executable(s) changed in version 5. To execute a serial GROMACS versions 4 program interactively, simply run it on the command line, e.g.:
pdb2gmx

To execute a serial GROMACS versions 5 program interactively, simply run it on the command line, e.g.:

gmx pdb2gmx

Note that some serial executables are enabled for single node parallelism via OpenMP.  Parallel multinode GROMACS versions 4 programs should be run in a batch environment with mpiexec, e.g.:

mpiexec mdrun_mpi_d

Parallel multinode GROMACS versions 5 programs should be run in a batch environment with mpiexec, e.g.:

mpiexec gmx_mpi_d mdrun

Note that '_mpi' indicates a parallel executable and '_d' indicates a program built with double precision (_gpu enotes a GPU executable built with CUDA).  See the module help output for specific versions for more details on executable name conventions.

Batch Usage

When you log into oakley.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your analysis to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. 

Interactive Batch Session
For an interactive batch session on Oakley, one can run the following command:
qsub -I -l nodes=1:ppn=12 -l walltime=1:00:00
which gives you 12 cores ( -l nodes=1:ppn=12 ) with 1 hour ( -l walltime=1:00:00 ). You may adjust the numbers per your need.
Non-interactive Batch Job (Parallel Run)

batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. Sample batch scripts and Gaussian input files are available here:

~srb/workshops/compchem/gromacs/

This simple batch script demonstrates some important points:

# GROMACS Tutorial for Solvation Study of Spider Toxin Peptide
# see fwspider_tutor.pdf
#PBS -N fwsinvacuo.oakley
#PBS -l nodes=2:ppn=12
module load gromacs
# PBS_O_WORKDIR refers to the directory from which the job was submitted.
cd $PBS_O_WORKDIR
pbsdcp -p 1OMB.pdb em.mdp $TMPDIR
# Use TMPDIR for best performance.
cd $TMPDIR
pdb2gmx -ignh -ff gromos43a1 -f 1OMB.pdb -o fws.gro -p fws.top -water none
editconf -f fws.gro -d 0.7
editconf -f out.gro -o fws_ctr.gro -center 2.0715 1.6745 1.914
grompp -f em.mdp -c fws_ctr.gro -p fws.top -o fws_em.tpr
mpiexec mdrun_mpi -s fws_em.tpr -o fws_em.trr -c fws_ctr.gro -g em.log -e em.edr
cat em.log
cp -p * $PBS_O_WORKDIR/

Usage on Owens

Set-up

To load the default version of GROMACS module/initalize your environement for use of GROMACS, use module load gromacs . To select a particular software version, use   module load gromacs/version . For example, use  module load gromacs/5.1.2  to load GROMACS version 5.1.2; and use  module help gromacs/5.1.2  to view details, such as, compiler prerequisites, additional modules required for specific executables, etc.

Using GROMACS

To execute a serial GROMACS versions 5 program interactively, simply run it on the command line, e.g.:

gmx pdb2gmx

Parallel multinode GROMACS versions 5 programs should be run in a batch environment with mpiexec, e.g.:

mpiexec gmx_mpi_d mdrun

Note that '_mpi' indicates a parallel executable and '_d' indicates a program built with double precision (_gpu enotes a GPU executable built with CUDA).  See the module help output for specific versions for more details on executable name conventions.

Batch Usage

When you log into owens.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your analysis to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. 

Interactive Batch Session
For an interactive batch session on Oakley, one can run the following command:
qsub -I -l nodes=1:ppn=28 -l walltime=1:00:00
which gives you 28 cores ( -l nodes=1:ppn=28 ) with 1 hour ( -l walltime=1:00:00 ). You may adjust the numbers per your need.
Non-interactive Batch Job (Parallel Run)

batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. Sample batch scripts and Gaussian input files are available here:

~srb/workshops/compchem/gromacs/

This simple batch script demonstrates some important points:

# GROMACS Tutorial for Solvation Study of Spider Toxin Peptide
# see fwspider_tutor.pdf
#PBS -N fwsinvacuo.owens
#PBS -l nodes=2:ppn=28
module load gromacs
# PBS_O_WORKDIR refers to the directory from which the job was submitted.
cd $PBS_O_WORKDIR
pbsdcp -p 1OMB.pdb em.mdp $TMPDIR
# Use TMPDIR for best performance.
cd $TMPDIR
pdb2gmx -ignh -ff gromos43a1 -f 1OMB.pdb -o fws.gro -p fws.top -water none
editconf -f fws.gro -d 0.7
editconf -f out.gro -o fws_ctr.gro -center 2.0715 1.6745 1.914
grompp -f em.mdp -c fws_ctr.gro -p fws.top -o fws_em.tpr
mpiexec mdrun_mpi -s fws_em.tpr -o fws_em.trr -c fws_ctr.gro -g em.log -e em.edr
cat em.log
cp -p * $PBS_O_WORKDIR/

Further Reading

Supercomputer: 
Service: 

Gaussian

Gaussian is the most popular general purpose electronic structure program. Its latest version, g09, can perform density functional theory, Hartree-Fock, Möller-Plesset, coupled-cluster, and configuration interaction calculations among others. Geometry optimizations, vibrational frequencies, magnetic properties, and solution modeling are available. It performs well as black-box software on closed-shell ground state systems.

Availability and Restrictions

Gaussian is available on Ruby, Oakley, and Glenn Clusters. The versions currently available at OSC are:

Version Glenn Oakley Ruby notes
g03d01 X      
g03e01 X*      
g09a01 X      
g09b01 X X    
g09c01 X*

X

  Default version on Oakley prior to 09/15/2015
g09d01   X* X*  
g09e01   X X  
*: Current default version

You can use module avail gaussian  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access for Academic Users

Use of Gaussian for academic purposes requires validation. In order to obtain validation, please contact OSC Help for further instruction.

Access for Commerical Users

Contact OSC Help for getting access to Gaussian if you are a commerical user.

Usage

Usage on Glenn

Set-up on Glenn

To load the default version of Gaussian module/initalize your environement for use of Gaussian, use  module load gaussian   or module load g03   or module load g09 .  To select a particular software version, use   module load version . For example, use  module load g09a01  to load Gaussian version g09a01 on Glenn. 
Users are highly encouraged to switch to the g09 version as g03 is no longer supported by Gaussian and has significant limitations on Glenn.

Using Gaussian

To execute Gaussian, simply run the Gaussian binary with the input file on the command line:

g09 < input.com

When the input file is redirected as above ( < ), the output will be standard output; in this form the output can be seen via 'qpeek jobid' when the job is running in a batch queue.  Alternatively, Gaussian can be invoked without file redirection:

g09 input.com

in which case the output file will be named 'input.log'; in this form the output cannot be seen via 'qpeek jobid' when the job is running in a batch queue.

Batch Usage on Glenn

When you log into glenn.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your analysis to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. 

Interactive Batch Session
For an interactive batch session on Glenn, one can run the following command:
qsub -I -l nodes=1:ppn=8 -l walltime=1:00:00
which gives you 8 cores ( -l nodes=1:ppn=8 ) with 1 hour ( -l walltime=1:00:00 ). You may adjust the numbers per your need.
Non-interactive Batch Job (Serial Run)

batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. Sample batch scripts and Gaussian input files are available here:

/nfs/10/srb/workshops/compchem/gaussian/

This simple batch script demonstrates the important points:

#PBS -N GaussianJob
#PBS -l nodes=1:ppn=1
# PBS_O_WORKDIR refers to the directory from which the job was submitted.
cd $PBS_O_WORKDIR
cp input.com $TMPDIR
# Use TMPDIR for best performance.
cd $TMPDIR
module load g09
g09 < input.com
cp -p input.log *.chk $PBS_O_WORKDIR
Note: OSC does not have a functional distributed parallel version (LINDA) of Gaussian. Parallelism of Gaussian at OSC is only via shared memory. Consequently, do not request more than one node for Gaussian jobs on OSC's clusters.

Usage on Oakley

Set-up on Oakley

To load the default version of Gaussian module/initalize your environement for use of Gaussian, use  module load gaussian . To select a particular software version, use   module load gaussian/version . For example, use  module load gaussian/g09b01  to load Gaussian version g09b01 on Oakley. 

Using Gaussian

To execute Gaussian, simply run the Gaussian binary with the input file on the command line:

g09 < input.com

When the input file is redirected as above ( < ), the output will be standard output; in this form the output can be seen via 'qpeek jobid' when the job is running in a batch queue.  Alternatively, Gaussian can be invoked without file redirection:

g09 input.com

in which case the output file will be named 'input.log'; in this form the output cannot be seen via 'qpeek jobid' when the job is running in a batch queue.

Batch Usage on Oakley

When you log into oakley.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your analysis to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. 

Interactive Batch Session
For an interactive batch session on Oakley, one can run the following command:
qsub -I -l nodes=1:ppn=12 -l walltime=1:00:00
which gives you 12 cores ( -l nodes=1:ppn=12 ) with 1 hour ( -l walltime=1:00:00 ). You may adjust the numbers per your need.
Non-interactive Batch Job (Serial Run)

batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. Sample batch scripts and Gaussian input files are available here:

/nfs/10/srb/workshops/compchem/gaussian/

This simple batch script demonstrates the important points:

#PBS -N GaussianJob
#PBS -l nodes=1:ppn=1
# PBS_O_WORKDIR refers to the directory from which the job was submitted.
cd $PBS_O_WORKDIR
cp input.com $TMPDIR
# Use TMPDIR for best performance.
cd $TMPDIR
module load gaussian
g09 < input.com
cp -p input.log *.chk $PBS_O_WORKDIR
Note: OSC does not have a functional distributed parallel version (LINDA) of Gaussian. Parallelism of Gaussian at OSC is only via shared memory. Consequently, do not request more than one node for Gaussian jobs on OSC's clusters.

Usage on Ruby

Set-up on Ruby

To load the default version of Gaussian module/initalize your environement for use of Gaussian, use  module load gaussian . To select a particular software version, use   module load gaussian/version . For example, use  module load gaussian/g09d01  to load Gaussian version g09d01 on Ruby. 

Using Gaussian

To execute Gaussian, simply run the Gaussian binary with the input file on the command line:

g09 < input.com

When the input file is redirected as above ( < ), the output will be standard output; in this form the output can be seen via 'qpeek jobid' when the job is running in a batch queue.  Alternatively, Gaussian can be invoked without file redirection:

g09 input.com

in which case the output file will be named 'input.log'; in this form the output cannot be seen via 'qpeek jobid' when the job is running in a batch queue.

Batch Usage on Ruby

When you log into ruby.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your analysis to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. 

Interactive Batch Session
For an interactive batch session on Ruby, one can run the following command:
qsub -I -l nodes=1:ppn=20 -l walltime=1:00:00
which gives you 20 cores ( -l nodes=1:ppn=20 ) with 1 hour ( -l walltime=1:00:00 ). You may adjust the numbers per your need.
Non-interactive Batch Job (Serial Run)

batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. Sample batch scripts and Gaussian input files are available here:

/nfs/10/srb/workshops/compchem/gaussian/

This simple batch script demonstrates the important points:

#PBS -N GaussianJob
#PBS -l nodes=1:ppn=20
# PBS_O_WORKDIR refers to the directory from which the job was submitted.
cd $PBS_O_WORKDIR
cp input.com $TMPDIR
# Use TMPDIR for best performance.
cd $TMPDIR
module load gaussian
g09 < input.com
cp -p input.log *.chk $PBS_O_WORKDIR
Note: OSC does not have a functional distributed parallel version (LINDA) of Gaussian. Parallelism of Gaussian at OSC is only via shared memory. Consequently, do not request more than one node for Gaussian jobs on OSC's clusters.

Further Reading

 

Supercomputer: 
Service: 

Gnuplot

Introduction

Gnuplot is a portable command-line driven data and function plotting utility.  It was originally intended to allow scientists and students to visualize mathematical functions and data.  

Gnuplot supports many types of plots in two or three dimensions.  It can draw using points, lines, boxes, contours, vector fields surfaces and various associated text.  It also supports various specialized plot types.  

Gnuplot supports many different types of output:  interactive screen display (with mouse and hotkey functionality), pen plotters (like hpgl), printers (including postscript and many color devices), and file formats as vectorial pseudo-devices like LaTeX, metafont, pdf, svg, or bitmap png.  

Version

 

Version Oakley Owens Notes
4.2.4      
4.6.4 X    
4.6 patchlevel 2   System Install No Module Needed

Usage on Oakley

To start a Gnuplot session, load the module and launch using the following commands:

module load gnuplot

gnuplot

To access the Gnuplot help menu, type ? into the Gnuplot command line.  

Usage on Owens

Since Gnuplot is a System Install, no module is needed to use the application.

Further Reading

For more information, visit the Gnuplot Homepage.  

Supercomputer: 
Service: 

HDF5

HDF5 is a general purpose library and file format for storing scientific data. HDF5 can store two primary objects: datasets and groups. A dataset is essentially a multidimensional array of data elements, and a group is a structure for organizing objects in an HDF5 file. Using these two basic objects, one can create and store almost any kind of scientific data structure, such as images, arrays of vectors, and structured and unstructured grids.

Availability and Restrictions

HDF5 is available on Oakley, Ruby, and Owens Clusters. The versions currently available at OSC are:

Version Oakley Ruby Owens notes
1.6.4        
1.6.5        
1.8.7        
1.8.8 X      
1.8.12   X(GI)   Default version on Ruby prior to 09/14/2015
1.8.13 X(GI) X(GI)

 

Default version on Oakley prior to 09/14/2015
1.8.15 X* X(PI)*    
1.8.17     X(GI)*  
  • X(P): pgi only
  • X(PI): available with pgi and intel
  • X(GI): available with gnu and intel
  • X; available with pgi, gnu, and intel
* : Current Default Version

You can use module spider hdf5  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

HDF5 is available without restriction to all OSC users.

Usage on Oakley

Set-up

Initalizing the system for use of the HDF5 library is dependent on the system you are using and the compiler you are using. To load the default HDF5 library, run the following command: module load hdf5 . To load a particular version, use  module load hdf5/version . For example, use  module load hdf5/1.8.15  to load HDF5 version 1.8.15. You can use module spider hdf5  to view available modules.

Building With HDF5

The HDF5 library provides the following variables for use at build time:

Variable Use
$HDF5_C_INCLUDE Use during your compilation step for C programs
$HDF5_CPP_INCLUDE Use during your compilation step for C++ programs (serial version only)
$HDF5_F90_INCLUDE Use during your compilation step for FORTRAN programs
$HDF5_C_LIBS Use during your linking step programs
$HDF5_F90_LIBS Use during your linking step for FORTRAN programs

 

For example, to build the code myprog.c or myprog.f90 with the hdf5 library you would use:

icc -c $HDF5_C_INCLUDE myprog.c
icc -o myprog myprog.o $HDF5_C_LIBS
ifort -c $HDF5_F90_INCLUDE myprog.f90
ifort -o myprog myprog.o $HDF5_F90_LIBS

Batch Usage

When you log into oakley.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info.  Batch jobs run on the compute nodes of the system and not on the login node. It is desirable for big problems since more resources can be used.

Non-interactive Batch Job (Serial Run)
A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. Below is the example batch script that executes a program built with the HDF5 library:
#PBS -N AppNameJob
#PBS -l nodes=1:ppn=12
module load hdf5
cd $PBS_O_WORKDIR
cp foo.dat $TMPDIR
cd $TMPDIR
appname
cp foo_out.h5 $PBS_O_WORKDIR

Usage on Ruby

Set-up

Initalizing the system for use of the HDF5 library is dependent on the system you are using and the compiler you are using. To load the default HDF5 library, run the following command: module load hdf5 . To load a particular version, use  module load hdf5/version . For example, use  module load hdf5/1.8.15  to load HDF5 version 1.8.15. You can use module spider hdf5  to view available modules.

Building With HDF5

The HDF5 library provides the following variables for use at build time:

Variable Use
$HDF5_C_INCLUDE Use during your compilation step for C programs
$HDF5_CPP_INCLUDE Use during your compilation step for C++ programs (serial version only)
$HDF5_F90_INCLUDE Use during your compilation step for FORTRAN programs
$HDF5_C_LIBS Use during your linking step programs
$HDF5_F90_LIBS

Use during your linking step for FORTRAN programs

 

For example, to build the code myprog.c or myprog.f90 with the hdf5 library you would use:

icc -c $HDF5_C_INCLUDE myprog.c
icc -o myprog myprog.o $HDF5_C_LIBS
ifort -c $HDF5_F90_INCLUDE myprog.f90
ifort -o myprog myprog.o $HDF5_F90_LIBS

Batch Usage

When you log into ruby.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. 

Non-interactive Batch Job (Serial Run)
A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. Below is the example batch script that executes a program built with the HDF5 library:
#PBS -N AppNameJob
#PBS -l nodes=1:ppn=20
module load hdf5
cd $PBS_O_WORKDIR
cp foo.dat $TMPDIR
cd $TMPDIR
appname
cp foo_out.h5 $PBS_O_WORKDIR

Usage on Owens

Set-up

Initalizing the system for use of the HDF5 library is dependent on the system you are using and the compiler you are using. To load the default HDF5 library, run the following command: module load hdf5 . To load a particular version, use  module load hdf5/version . For example, use  module load hdf5/1.8.17  to load HDF5 version 1.8.17. You can use module spider hdf5  to view available modules.

Building With HDF5

The HDF5 library provides the following variables for use at build time:

Variable Use
$HDF5_C_INCLUDE Use during your compilation step for C programs
$HDF5_CPP_INCLUDE Use during your compilation step for C++ programs (serial version only)
$HDF5_F90_INCLUDE Use during your compilation step for FORTRAN programs
$HDF5_C_LIBS Use during your linking step programs
$HDF5_F90_LIBS

Use during your linking step for FORTRAN programs

 

For example, to build the code myprog.c or myprog.f90 with the hdf5 library you would use:

icc -c $HDF5_C_INCLUDE myprog.c
icc -o myprog myprog.o $HDF5_C_LIBS
ifort -c $HDF5_F90_INCLUDE myprog.f90
ifort -o myprog myprog.o $HDF5_F90_LIBS

Batch Usage

When you log into owens.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. 

Non-interactive Batch Job (Serial Run)
A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. Below is the example batch script that executes a program built with the HDF5 library:
#PBS -N AppNameJob
#PBS -l nodes=1:ppn=28
module load hdf5
cd $PBS_O_WORKDIR
cp foo.dat $TMPDIR
cd $TMPDIR
appname
cp foo_out.h5 $PBS_O_WORKDIR

Further Reading

Tag: 
Supercomputer: 
Service: 

HMMER

Profile hidden Markov models (profile HMMs) can be used to do sensitive database searching using statistical descriptions of a sequence family's consensus. HMMER uses profile HMMs, and can be useful in situations like:

  • if you are working with an evolutionarily diverse protein family, a BLAST search with any individual sequence may not find the rest of the sequences in the family.
  • the top hits in a BLAST search are hypothetical sequences from genome projects.
  • your protein consists of several domains which are of different types.

HMMER (pronounced 'hammer', as in a more precise mining tool than BLAST) was developed by Sean Eddy at Washington University in St. Louis.

HMMER is a very cpu-intensive program and is parallelized using threads, so that each instance of hmmsearch or the other search programs can use all the cpus available on a node. HMMER on OSC clusters are intended for those who need to run HMMER searches on large numbers of query sequences.

Availability & Restrictions

HMMER is available to all OSC users without restriction.

The following versions of HMMER are available on OSC systems:

Version Glenn Oakley
2.3.2 X  
3.0   X

Usage

Set-up

To use HMMER on Glenn, first run the following commands:

module load biosoftw
module load hmmer

To use HMMER on Oakley, first run the following command:

module load hmmer

Using HMMER

Once the hmmer module is loaded, the following commands will be available for your use:

Single sequence queries: new to HMMER3
phmmer Search a sequence against a sequence database. (BLASTP-like)
jackhmmer Iteratively search a sequence against a sequence database. (PSIBLAST-like)

Replacements for HMMER2’s functionality
hmmbuild Build a profile HMM from an input multiple alignment.
hmmsearch Search a profile HMM against a sequence database.
hmmscan Search a sequence against a profile HMM database.
hmmalign Make a multiple alignment of many sequences to a common profile HMM.

Other utilities
hmmconvert Convert profile formats to/from HMMER3 format.
hmmemit Generate (sample) sequences from a profile HMM.
hmmfetch Get a profile HMM by name or accession from an HMM database.
hmmpress Format an HMM database into a binary format for hmmscan.
hmmstat Show summary statistics for each profile in an HMM database.

If you need to know options for a command, type the command name followed by "-h", for example:

hmmalign -h 

Batch Usage

A sample batch job is below:

#PBS -N hmmer
#PBS -j oe
#PBS -l nodes=1:ppn=1
#PBS -S /bin/bash

cd $PBS_O_WORKDIR
hmmalign globins4.align globins45
hmmbuild globins4.hmm globins4.align
hmmsearch globins4.hmm /fdb/fastadb/nr.aa.fas > globins.out

Further Reading

See Also

Supercomputer: 
Service: 
Fields of Science: 

HOMER

HOMER (Hypergeometric Optimization of Motif EnRichment) is a suite of tools for Motif Discovery and ChIP-Seq analysis. It is a collection of command line programs for unix-style operating systems written in mostly perl and c++. Homer was primarily written as a de novo motif discovery algorithm that is well suited for finding 8-12 bp motifs in large scale genomics data.

Availability and Restrictions

The following versions of HOMER are available on OSC clusters:

Version Oakley owens
4.8 X* X*
*: Current default version

You can use  module avail homer  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

HOMER is available to all OSC users without restriction.

Usage on Oakley

Set-up

To configure your environment for use of HOMER, run the following command:   module load homer. The default version will be loaded. To select a particular HOMER version, use module load homer/version. For example, use  module load homer/4.8 to load HOMER 4.8.

Usage on Owens

Set-up

To configure your environment for use of HOMER, run the following command:   module load homer. The default version will be loaded. To select a particular HOMER version, use module load homer/version. For example, use module load homer/4.8 to load HOMER 4.8.

Further Reading

Supercomputer: 
Service: 
Fields of Science: 

Intel Compilers

The Intel compilers for both C/C++ and FORTRAN.

Availability and Restrictions

Intel Compiler is available on Glenn, Oakley and Ruby Clusters. The versions currently available at OSC are:

Version Glenn Oakley Ruby notes
9.1 X      
10.0 X      
10.0.023 X*      
11.1 X      
11.1.056 X      
12.1.4.319   X   Default version on Oakley prior to 09/15/2015
13.1.3.192   X    
14.0.0.080   X    
14.0.3 X   X  
15.0.0     X Default version on Ruby prior to 09/15/2015
15.0.1     X  
15.0.3   X* X*  
* : Current Default Version

You can use module avail intel  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

The Intel Compilers are available to all OSC users without restriction.

Usage

Usage on Glenn

Set-up on Glenn

After you ssh to Glenn, the default version of Intel compilers will be loaded for you automatically. To configure your environment for a particular Intel compilers version, use  module swap intel-compilers-10.0.023 intel-compilers-version . For example, use  module swap intel-compilers-10.0.023 intel-compilers-11.1  to load Intel  compilers 11.1 on Glenn.

Using the Intel Compilers

Once the intel compiler module has been loaded, the compilers are available for your use. The following table lists common compiler options available in all languages.

Compiler option Purpose
-c Compile only; do not link
-DMACRO[=value] Defines preprocessor macro MACRO with optional value (default value is 1)
-g Enables debugging; disables optimization
-I/directory/name Add /directory/name to the list of directories to be searched for #include files
-L/directory/name Adds /directory/name to the list of directories to be searched for library files
-lname Adds the library libname.a or libname.so to the list of libraries to be linked
-o outfile Names the resulting executable outfile instead of a.out
-UMACRO Removes definition of MACRO from preprocessor
-O0 Disable optimization
-O1 Light optimization
-O2 Heavy optimization (default)
-O3 Aggressive optimization; may change numerical results
-ipo Inline function expansion for calls to procedures defined in separate files
-funroll-loops Loop unrolling
-parallel Automatic parallelization
-openmp Enables translation of OpenMP directives

The following table lists some options specific to C/C++

-strict-ansi Enforces strict ANSI C/C++ compliance
-ansi Enforces loose ANSI C/C++ compliance

The following table lists some options specific to Fortran

-convert big_endian Use unformatted I/O compatible with Sun and SGI systems
-convert cray Use unformatted I/O compatible with Cray systems
-i8 Makes 8-byte INTEGERs the default
-module /dir/name Adds /dir/name to the list of directories searched for Fortran 90 modules
-r8 Makes 8-byte REALs the default
-fp-model strict Disables optimizations that can change the results of floating point calculations

See our compilation guide for suggestions on how to compile your software on our systems.

Batch Usage on Glenn

When you log into glenn.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info.  Batch jobs run on the compute nodes of the system and not on the login node. It is desirable for big problems since more resources can be used.

Interactive Batch Session
For an interactive batch session on Glenn, one can run the following command:
qsub -I -l nodes=1:ppn=8 -l walltime=1:00:00
which gives you 8 cores ( -l nodes=1:ppn=8  ) with 1 hour ( -l walltime=1:00:00 ). You may adjust the numbers per your need.
Non-interactive Batch Job (Serial Run)
A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. The following example batch script file will use the input file named hello.c  and the output file named hello_results .  Below is the example batch script ( job.txt ) for a serial run:
#PBS -l walltime=1:00:00
#PBS -l nodes=1:ppn=8
#PBS -N jobname
#PBS -j oe
cd $PBS_O_WORKDIR
module load intel
cp hello.c $TMPDIR
cd $TMPDIR
icc -O2 hello.c -o hello
./hello > hello_results
cp hello_results $PBS_O_WORKDIR

In order to run it via the batch system, submit the job.txt  file with the following command:

qsub job.txt
Non-interactive Batch Job (Parallel Run)
Below is the example batch script ( job.txt ) for a parallel run:
#PBS -l walltime=1:00:00
#PBS -l nodes=2:ppn=8
#PBS -N jobname
#PBS -j oe
cd $PBS_O_WORKDIR
module load intel
mpicc -O2 hello.c -o hello
pbsdcp hello $TMPDIR
cd $TMPDIR
mpiexec hello > hello_results
pbsdcp -g hello_results $PBS_O_WORKDIR

Usage on Oakley

Set-up on Oakley

After you ssh to Oakley, the default version of Intel compilers will be loaded for you automatically. To configure your environment for a particular Intel compilers version, use  module swap intel/12.1.4.39 intel/version . For example, use  module swap intel/12.1.4.39 intel/14.0.0.080  to load Intel  compilers 14.0.0.080 on Oakley.

Using the Intel Compilers

Once the intel compiler module has been loaded, the compilers are available for your use. The following table lists common compiler options available in all languages.

Compiler option Purpose
-c Compile only; do not link
-DMACRO[=value] Defines preprocessor macro MACRO with optional value (default value is 1)
-g Enables debugging; disables optimization
-I/directory/name Add /directory/name to the list of directories to be searched for #include files
-L/directory/name Adds /directory/name to the list of directories to be searched for library files
-lname Adds the library libname.a or libname.so to the list of libraries to be linked
-o outfile Names the resulting executable outfile instead of a.out
-UMACRO Removes definition of MACRO from preprocessor
-O0 Disable optimization
-O1 Light optimization
-O2 Heavy optimization (default)
-O3 Aggressive optimization; may change numerical results
-ipo Inline function expansion for calls to procedures defined in separate files
-funroll-loops Loop unrolling
-parallel Automatic parallelization
-openmp Enables translation of OpenMP directives

The following table lists some options specific to C/C++

-strict-ansi Enforces strict ANSI C/C++ compliance
-ansi Enforces loose ANSI C/C++ compliance

The following table lists some options specific to Fortran

-convert big_endian Use unformatted I/O compatible with Sun and SGI systems
-convert cray Use unformatted I/O compatible with Cray systems
-i8 Makes 8-byte INTEGERs the default
-module /dir/name Adds /dir/name to the list of directories searched for Fortran 90 modules
-r8 Makes 8-byte REALs the default
-fp-model strict Disables optimizations that can change the results of floating point calculations

Some parts of the C++11 standard are available by default in various versions of the Intel compilers; others require you to load an extra module. 

With the Intel 14 or 15 compilers:

module load cxx11

See our compilation guide for suggestions on how to compile your software on our systems.

Batch Usage on Oakley

When you log into oakley.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info.  Batch jobs run on the compute nodes of the system and not on the login node. It is desirable for big problems since more resources can be used.

Interactive Batch Session
For an interactive batch session on Oakley, one can run the following command:
qsub -I -l nodes=1:ppn=12 -l walltime=1:00:00
which gives you 12 cores ( -l nodes=1:ppn=12 ) with 1 hour ( -l walltime=1:00:00 ). You may adjust the numbers per your need.
Non-interactive Batch Job (Serial Run)
A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. The following example batch script file will use the input file named hello.c  and the output file named hello_results . Below is the example batch script ( job.txt ) for a serial run:
#PBS -l walltime=1:00:00
#PBS -l nodes=1:ppn=12
#PBS -N jobname
#PBS -j oe
cd $PBS_O_WORKDIR
module load intel
cp hello.c $TMPDIR
cd $TMPDIR
icc -O2 hello.c -o hello
./hello > hello_results
cp hello_results $PBS_O_WORKDIR

In order to run it via the batch system, submit the job.txt  file with the following command:

qsub job.txt
Non-interactive Batch Job (Parallel Run)
Below is the example batch script ( job.txt ) for a parallel run:
#PBS -l walltime=1:00:00
#PBS -l nodes=2:ppn=12
#PBS -N jobname
#PBS -j oe
cd $PBS_O_WORKDIR
module load intel
mpicc -O2 hello.c -o hello
pbsdcp hello $TMPDIR
cd $TMPDIR
mpiexec hello > hello_results
pbsdcp -g hello_results $PBS_O_WORKDIR

Usage on Ruby

Set-up on Ruby

After you ssh to Ruby, the default version of Intel compilers will be loaded for you automatically. To configure your environment for a particular Intel compilers version, use module swap intel/15.0.0 intel/version . For example, use  module swap intel/15.0.0 intel/15.0.1  to load Intel  compilers 15.0.1 on Ruby.

Using the Intel Compilers

Once the intel compiler module has been loaded, the compilers are available for your use. The following table lists common compiler options available in all languages.

Compiler option Purpose
-c Compile only; do not link
-DMACRO[=value] Defines preprocessor macro MACRO with optional value (default value is 1)
-g Enables debugging; disables optimization
-I/directory/name Add /directory/name to the list of directories to be searched for #include files
-L/directory/name Adds /directory/name to the list of directories to be searched for library files
-lname Adds the library libname.a or libname.so to the list of libraries to be linked
-o outfile Names the resulting executable outfile instead of a.out
-UMACRO Removes definition of MACRO from preprocessor
-O0 Disable optimization
-O1 Light optimization
-O2 Heavy optimization (default)
-O3 Aggressive optimization; may change numerical results
-ipo Inline function expansion for calls to procedures defined in separate files
-funroll-loops Loop unrolling
-parallel Automatic parallelization
-openmp Enables translation of OpenMP directives

The following table lists some options specific to C/C++

-strict-ansi Enforces strict ANSI C/C++ compliance
-ansi Enforces loose ANSI C/C++ compliance

The following table lists some options specific to Fortran

-convert big_endian Use unformatted I/O compatible with Sun and SGI systems
-convert cray Use unformatted I/O compatible with Cray systems
-i8 Makes 8-byte INTEGERs the default
-module /dir/name Adds /dir/name to the list of directories searched for Fortran 90 modules
-r8 Makes 8-byte REALs the default
-fp-model strict Disables optimizations that can change the results of floating point calculations

Some parts of the C++11 standard are available by default in various versions of the Intel compilers; others require you to load an extra module. 

With the Intel 14 or 15 compilers:

module load cxx11

See our compilation guide for suggestions on how to compile your software on our systems.

Batch Usage on Ruby

When you log into ruby.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. 

Interactive Batch Session
For an interactive batch session on Ruby, one can run the following command:
qsub -I -l nodes=1:ppn=20 -l walltime=1:00:00
which gives you 20 cores ( -l nodes=1:ppn=20 ) with 1 hour ( -l walltime=1:00:00 ). You may adjust the numbers per your need.
Non-interactive Batch Job (Serial Run)
batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. The following example batch script file will use the input file named  hello.c  and the output file named  hello_results . Below is the example batch script ( job.txt ) for a serial run:
#PBS -l walltime=1:00:00
#PBS -l nodes=1:ppn=20
#PBS -N jobname
#PBS -j oe
cd $PBS_O_WORKDIR
module load intel
cp hello.c $TMPDIR
cd $TMPDIR
icc -O2 hello.c -o hello
./hello > hello_results
cp hello_results $PBS_O_WORKDIR

In order to run it via the batch system, submit the job.txt  file with the following command:

qsub job.txt
Non-interactive Batch Job (Parallel Run)
Below is the example batch script (  job.txt   for a parallel run:
#PBS -l walltime=1:00:00
#PBS -l nodes=2:ppn=20
#PBS -N jobname
#PBS -j oe
cd $PBS_O_WORKDIR
module load intel
mpicc -O2 hello.c -o hello
cp hello $TMPDIR
cd $TMPDIR
mpiexec ./hello > hello_results
cp hello_results $PBS_O_WORKDIR​

Further Reading

See Also

Supercomputer: 
Service: 
Technologies: 
Fields of Science: 

Intel MPI

https://www.osc.edu/supercomputing/software/intel-compilers Intel's implementation of the Message Passing Interface (MPI) library.

Availability and Restrictions

Intel MPI is available on Glenn, Oakley and Ruby Clusters. This library may be used as an alternative to - but not in conjunction with - the MVAPICH2 MPI libraries. The versions currently available at OSC are:

Version Glenn Oakley ruby notes
3.2.2p-006 X      
4.0.0.028 X      
4.0.0pu-027 X      
4.0.3.008   X   Default version on Oakley prior to 09/15/2015
4.1.0.024   X    
4.1.1.036   X    
4.1.2.040   X    
4.1.3   X X  
4.1.3.049   X X  
5.0.1     X  
5.0.1.035     X  
5.0.3   X* X*  
* : Current Default Version

You can use module spider intelmpi  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

The Intel MPI areavailable to all OSC users without restriction.

Usage

Usage on Glenn

Set-up on Glenn

To configure your environment for a particular Intel MPI, use  module load name . For example, use  module load intel-mpi-3.2.2p-006  to load Intel MPI version 3.2.2p-006 on Glenn.

You can use module avail intel-mpi  to view available modules on Glenn.

Note: Since this module conflicts with MVAPICH installations, you should unload those modules first.

Using Intel MPI

Software compiled against this module will use the libraries at runtime.

Building With Intel MPI

On Glenn, we do not recommend building against these libraries. The modules are not configured to set up useful environment variables.

Batch Usage on Glenn

When you log into glenn.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info.  Batch jobs run on the compute nodes of the system and not on the login node. It is desirable for big problems since more resources can be used.

Non-interactive Batch Job (Parallel Run)
A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. The following example batch script file will run a program compiled against Intel MPI (called my-impi-application ) for five-hours on Glenn:
#PBS -N MyIntelMPIJob
#PBS -l nodes=4:ppn=8
#PBS -l walltime=5:00:00
module unload mpi
module load intel-mpi-3.2.2p-006
cd $PBS_O_WORKDIR
mpiexec my-impi-application

Usage on Oakley

Set-up on Oakley

To configure your environment for the default version of Intel MPI, use  module load intelmpi . To configure your environment for a specific version of Intel MPI, use  module load intelmpi/version . For example, use  module load intelmpi/4.1.3.049  to load Intel MPI version 4.1.3.049 on Oakley.

You can use module spider intelmpi  to view available modules on Oakley.

Note: This module conflicts with the default loaded MVAPICH installations, and Lmod will automatically replace with the correct one when you use module load intelmpi .

Using Intel MPI

Software compiled against this module will use the libraries at runtime.

Building With Intel MPI

On Oakley, we have defined several environment variables to make it easier to build and link with the Intel MPI libraries.

Variable Use
$MPI_CFLAGS Use during your compilation step for C programs.
$MPI_CXXFLAGS Use during your compilation step for C++ programs.
$MPI_FFLAGS Use during your compilation step for Fortran programs.
$MPI_F90FLAGS Use during your compilation step for Fortran 90 programs.
$MPI_LIBS Use when linking your program to Intel MPI.

In general, for any application already set up to use mpicc , (or similar), compilation should be fairly straightfoward. 

Batch Usage on Oakley

When you log into oakley.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info.  Batch jobs run on the compute nodes of the system and not on the login node. It is desirable for big problems since more resources can be used.

Non-interactive Batch Job (Parallel Run)
A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. The following example batch script file will run a program compiled against Intel MPI (called my-impi-application ) for five-hours on Oakley:
#PBS -N MyIntelMPIJob
#PBS -l nodes=4:ppn=12
#PBS -l walltime=5:00:00
module swap mvapich2 intelmpi
cd $PBS_O_WORKDIR
mpiexec my-impi-application

Usage on Ruby

Set-up on Ruby

To configure your environment for the default version of Intel MPI, use  module load intelmpi . To configure your environment for a specific version of Intel MPI, use  module load intelmpi/version . For example, use  module load intelmpi/4.1.3.049  to load Intel MPI version 4.1.3.049 on Ruby.

You can use module spider intelmpi  to view available modules on Ruby.

Note: This module conflicts with the default loaded MVAPICH installations, and Lmod will automatically replace with the correct one when you use module load intelmpi .

Using Intel MPI

Software compiled against this module will use the libraries at runtime.

Building With Intel MPI

On Ruby, we have defined several environment variables to make it easier to build and link with the Intel MPI libraries.

Variable Use
$MPI_CFLAGS Use during your compilation step for C programs.
$MPI_CXXFLAGS Use during your compilation step for C++ programs.
$MPI_FFLAGS Use during your compilation step for Fortran programs.
$MPI_F90FLAGS Use during your compilation step for Fortran 90 programs.
$MPI_LIBS Use when linking your program to Intel MPI.

In general, for any application already set up to use mpicc , (or similar), compilation should be fairly straightfoward. 

Batch Usage on Ruby

When you log into ruby.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. 

Non-interactive Batch Job (Parallel Run)
A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. The following example batch script file will run a program compiled against Intel MPI (called my-impi-application ) for five-hours on Ruby:
#PBS -N MyIntelMPIJob
#PBS -l nodes=4:ppn=20
#PBS -l walltime=5:00:00
module swap mvapich2 intelmpi
cd $PBS_O_WORKDIR
mpiexec my-impi-application

Further Reading

See Also

Supercomputer: 
Service: 
Technologies: 
Fields of Science: 

JasPer

Introduction

JasPer is an extensible open source utility designed for the manipulation, conversion, compression and decompression of digital images.  Currently supported image formats include bmp, jp2, jpc, jpg, pgx, mif and ras.    

Availability

Version 1.900.1 is available on the Oakley cluster.  

Usage

In order to use the JasPer library, you must load the JasPer module into your environment.  To load the JasPer module, type the following command:

module load jasper

Useful commands

In addition to the JasPer libraries, a few default applications are also provided when the JasPer module is loaded.     

To convert an image from one format to another, use the jasper command.   

jasper [options]

For a comprehensive list of options available for this command, type

jasper --help

To compare particular qualities of two images, use the imgcmp command.

imgcmp -f [file 1] -F [file 2] -m [metric]

For a full list of acceptable metric arguments, type

imgcmp --help

To view a sequence of one or more images, use the jiv command.  

jiv image

Alternatively, if you would like to view multiple images in sequence you can use the --wait option to specify the amount of time between the display of each image, creating a slideshow.  The command below will display each image for 5 seconds before displaying the next image.   

jiv --wait 5 [image1 image2 image3 ...]

For a full list of options, type

jiv --help

Further Reading

Additional information about JasPer can be found online at the JasPer Project Homepage.

Supercomputer: 
Service: 

Jmol

Jmol is a simple molecular visualization program. It can read many file formats and can write various formats. Animations of normal modes and simulations are implemented. Jmol is an OpenScience Java application.

Availability & Restrictions

Jmol is available to all OSC users without restirction.

The following versions of Jmol are available on OSC systems:

Version Glenn Oakley
12.0.8 X  

Jmol is best executed locally. Thus, we recommend downloading the latest version from http://jmol.sourceforge.net/download/. The installation process is trivial.

Usage

Set-up

To setup Glenn for use of Jmol, run the following command:

module load jmol

Using Jmol

To execute Jmol, simply run the jmol binary with an optional input file on the command line:

jmol

Batch Usage

Jmol should be executed interactively.

Further Reading

Supercomputer: 
Service: 

LAMMPS

The Large-scale Atomic/Molecular Massively Parallel Simulator (LAMMPS) is a classical molecular dynamics code designed for high-performance simulation of large atomistic systems.  LAMMPS generally scales well on OSC platforms, provides a variety of modeling techniques, and offers GPU accelerated computation.

Availability and Restrictions

LAMMPS is available on Oakley, Ruby, and Owens Clusters. The following versions are currently available at OSC (S means serial executables, P means parallel, and C means CUDA, i.e., GPU enabled):

Version Oakley Ruby Owens notes
Feb12 PC     Default version on Oakley prior to 09/15/2015
May12 PC      
Feb14 SPC      
5Sep14 P* P*    
7Dec15 PC PC    
14May16 PC   P  
*: Current default version

You can use module avail lammps to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

LAMMPS is available to all OSC users without restriction.

Usage on Oakley

Set-up

To load the default version of LAMMPS module and set up your environment, use  module load lammps . To select a particular software version, use   module load lammps/version . For example, use  module load lammps/12Feb12   to load LAMMPS version Feb12. 

Using LAMMPS

Once a module is loaded, LAMMPS can be run with the following command:
lammps < input.file

To see information on the packages and executables for a particular installation, run the module help command, for example:

module help lammps

Batch Usage

When you log into oakley.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info.  Batch jobs run on the compute nodes of the system and not on the login node. It is desirable for big problems since more resources can be used.

Interactive Batch Session

For an interactive batch session one can run the following command:

qsub -I -l nodes=1:ppn=1 -l walltime=00:20:00

which requests one core ( -l nodes=1:ppn=1 ), for a walltime of 20 minutes ( -l walltime=00:20:00 ). You may adjust the numbers per your need.

Non-interactive Batch Job (Parallel Run)

A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. Sample batch scripts and LAMMPS input files are available here:

~srb/workshops/compchem/lammps/

Below is a sample batch script. It asks for 24 processors and 10 hours of walltime. If the job goes beyond 10 hours, the job would be terminated.

#PBS -N chain  
#PBS -l nodes=2:ppn=12  
#PBS -l walltime=10:00:00  
#PBS -S /bin/bash  
#PBS -j oe  
module load lammps  
cd $PBS_O_WORKDIR  
pbsdcp chain.in $TMPDIR  
cd $TMPDIR  
lammps < chain.in  
pbsdcp -g '*' $PBS_O_WORKDIR

Usage on Ruby

Set-up

To load the default version of LAMMPS module and set up your environment, use  module load lammps . To select a particular software version, use   module load lammps/version . For example, use  module load lammps/5Sep14   to load LAMMPS version 5Sep14. 

Using LAMMPS

Once a module is loaded, LAMMPS can be run with the following command:
lammps < input.file

To see information on the packages and executables for a particular installation, run the module help command, for example:

module help lammps

Batch Usage

When you log into ruby.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. 

Interactive Batch Session

For an interactive batch session one can run the following command:

qsub -I -l nodes=1:ppn=20:gpus=1 -l walltime=00:20:00 

which requests one whole node with 20 cores ( -l nodes=1:ppn=20 ), for a walltime of 20 minutes ( -l walltime=00:20:00 ). You may adjust the numbers per your need.

Non-interactive Batch Job (Parallel Run)

A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. Sample batch scripts and LAMMPS input files are available here:

~srb/workshops/compchem/lammps/

Below is a sample batch script. It asks for 40 processors and 10 hours of walltime. If the job goes beyond 10 hours, the job would be terminated.

#PBS -N chain  
#PBS -l nodes=2:ppn=20  
#PBS -l walltime=10:00:00  
#PBS -S /bin/bash  
#PBS -j oe  
module load lammps  
cd $PBS_O_WORKDIR  
pbsdcp chain.in $TMPDIR  
cd $TMPDIR  
lammps < chain.in  
pbsdcp -g '*' $PBS_O_WORKDIR

Usage on Owens

Set-up

To load the default version of LAMMPS module and set up your environment, use  module load lammps . To select a particular software version, use   module load lammps/version . For example, use  module load lammps/14May16   to load LAMMPS version 14May16. 

Using LAMMPS

Once a module is loaded, LAMMPS can be run with the following command:
lammps < input.file

To see information on the packages and executables for a particular installation, run the module help command, for example:

module help lammps

Batch Usage

When you log into owens.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. 

Interactive Batch Session

For an interactive batch session one can run the following command:

qsub -I -l nodes=1:ppn=28:gpus=1 -l walltime=00:20:00 

which requests one whole node with 28 cores ( -l nodes=1:ppn=28 ), for a walltime of 20 minutes ( -l walltime=00:20:00 ). You may adjust the numbers per your need.

Non-interactive Batch Job (Parallel Run)

A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. Sample batch scripts and LAMMPS input files are available here:

~srb/workshops/compchem/lammps/

Below is a sample batch script. It asks for 56 processors and 10 hours of walltime. If the job goes beyond 10 hours, the job would be terminated.

#PBS -N chain  
#PBS -l nodes=2:ppn=28  
#PBS -l walltime=10:00:00  
#PBS -S /bin/bash  
#PBS -j oe  
module load lammps  
cd $PBS_O_WORKDIR  
pbsdcp chain.in $TMPDIR  
cd $TMPDIR  
lammps < chain.in  
pbsdcp -g '*' $PBS_O_WORKDIR

Further Reading

Supercomputer: 
Service: 

LS-DYNA

LS-DYNA is a general purpose finite element code for simulating complex structural problems, specializing in nonlinear, transient dynamic problems using explicit integration. LS-DYNA is one of the codes developed at Livermore Software Technology Corporation (LSTC).

Availability and Restrictions

LS-DYNA is available on Ruby, Oakley, and Glenn Clusters for both serial (smp solver for single node jobs) and parallel (mpp solver for multipe node jobs) versions. The versions currently available at OSC are:

Version Glenn Oakley Ruby notes
971-R4.2.1 smp  X      
mpp         
971-R5 smp  X      
mpp  X      
971-R5.1.1 smp   X   Default version on Oakley prior to 09/15/2015
mpp   X   Default version on Oakley prior to 09/15/2015
971-R7.0.0 smp X X X*  
mpp X X X*  

971-R7.1.1

smp    X*    
mpp   X*    
*: Current default version

Feel free to contact OSC Help if you need other versions for your work.

Access for Academic Users

OSC does not provide LS_DYNA license directly, however users with their own academic departmental license can use it on the OSC clusters.  Please contact OSC Help for further instruction.

Access for Commerical Users

Contact OSC Help for getting access to LS-DYNA if you are a commerical user.

Usage

Usage on Glenn

Set-up on Glenn

To view available modules installed on Glenn, use  module avail lsdyna  for smp solvers, and use  module avail mpp  for mpp solvers. In the module name, '_s' indicates single precision and '_d' indicates double precision. For example, mpp971_d_R5 is the mpp solver with double precision on Glenn. Use  module load name  to load LS-DYNA with a particular software version. For example, use  module load mpp971_d_R5  to load LS-DYNA mpp solver version 5.0 with double precision on Glenn.

Batch Usage on Glenn

When you log into glenn.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time, up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. Batch jobs run on the compute nodes of the system and not on the login node. It is desirable for big problems since more resources can be used.

Interactive Batch Session

For an interactive batch session one can run the following command:

qsub -I -l nodes=1:ppn=8 -l walltime=00:20:00 
which requests one whole node with 8 cores ( -l nodes=1:ppn=8 ), for a walltime of 20 minutes ( -l walltime=00:20:00 ). You may adjust the numbers per your need.
Non-interactive Batch Job (Serial Run)

A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. Please follow the steps below to use LS-DYNA via the batch system:

1) copy your input files ( explorer.k  in the example below) to your work directory at OSC

2) create a batch script, similar to the following file, saved as job.txt . It uses the smp solver for a serial job (nodes=1) on Glenn:

#PBS -N plate_test
#PBS -l walltime=5:00:00
#PBS -l nodes=1:ppn=8
#PBS -j oe
# The following lines set up the LSDYNA environment
module load lsdyna971_d_R5
#
# Move to the directory where the input files are located
#
cd $PBS_O_WORKDIR
#
# Run LSDYNA (number of cpus > 1)
#
lsdyna I=explorer.k NCPU=8

 3) submit the script to the batch queue with the command:  qsub job.txt

 When the job is finished, all the result files will be found in the directory where you submitted your job ($PBS_O_WORKDIR). Alternatively, you can submit your job from the temporary directory ($TMPDIR), which is faster to access for the system and might be beneficial for bigger jobs. Note that $TMPDIR is uniquely associated with the job submitted and will be cleared when the job ends. So you need to copy your results back to your work directory at the end of your script. 

Non-interactive Batch Job (Parallel Run)
Please follow the steps below to use LS-DYNA via the batch system:

1) copy your input files ( explorer.k  in the example below) to your work directory at OSC

2) create a batch script, similar to the following file, saved as job.txt ). It uses the mmp solver for a parallel job (nodes>1) on Glenn:

#PBS -N plate_test
#PBS -l walltime=5:00:00
#PBS -l nodes=2:ppn=8
#PBS -j oe
# The following lines set up the LSDYNA environment
module unload mpi
module load mpp971_d_R5
#
# Move to the directory where the input files are located
#
cd $PBS_O_WORKDIR
#
# Run LSDYNA (number of cpus > 1)
#
mpiexec mpp971 I=explorer.k NCPU=16

 3) submit the script to the batch queue with the command:  qsub job.txt

When the job is finished, all the result files will be found in the directory where you submitted your job ($PBS_O_WORKDIR). Alternatively, you can submit your job from the temporary directory ($TMPDIR), which is faster to access for the system and might be beneficial for bigger jobs. Note that $TMPDIR is uniquely associated with the job submitted and will be cleared when the job ends. So you need to copy your results back to your work directory at the end of your script. An example scrip should include the following lines:

...
cd $TMPDIR
cp $PBS_O_WORKDIR/explorer.k .
... #launch the solver and execute
pbsdcp -g '*' $PBS_O_WORKDIR

Usage on Oakley

Set-up on Oakley

To view available modules installed on Oakley, use  module avail ls-dyna  for smp solvers, and use  module spider mpp  for mpp solvers. In the module name, '_s' indicates single precision and '_d' indicates double precision. For example, mpp-dyna/971_d_R7.0.0 is the mpp solver with double precision on Oakley. Use  module load name  to load LS-DYNA with a particular software version. For example, use  module load mpp-dyna/971_d_R7.0.0   to load LS-DYNA mpp solver version 7.0.0 with double precision on Oakley.

Batch Usage on Oakley

When you log into oakley.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info.  Batch jobs run on the compute nodes of the system and not on the login node. It is desirable for big problems since more resources can be used.

Interactive Batch Session

For an interactive batch session one can run the following command:

qsub -I -l nodes=1:ppn=12 -l walltime=00:20:00 
which requests one whole node with 12 cores ( -l nodes=1:ppn=12 ), for a walltime of 20 minutes ( -l walltime=00:20:00 ). You may adjust the numbers per your need.
Non-interactive Batch Job (Serial Run)

A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. Please follow the steps below to use LS-DYNA via the batch system:

1) copy your input files ( explorer.k  in the example below) to your work directory at OSC

2) create a batch script, similar to the following file, saved as job.txt . It uses the smp solver for a serial job (nodes=1) on Oakley:

#PBS -N plate_test
#PBS -l walltime=5:00:00
#PBS -l nodes=1:ppn=12
#PBS -j oe
# The following lines set up the LSDYNA environment
module load ls-dyna/971_d_7.0.0
#
# Move to the directory where the input files are located
#
cd $PBS_O_WORKDIR
#
# Run LSDYNA (number of cpus > 1)
#
lsdyna I=explorer.k NCPU=12

 3) submit the script to the batch queue with the command:  qsub job.txt

 When the job is finished, all the result files will be found in the directory where you submitted your job ($PBS_O_WORKDIR). Alternatively, you can submit your job from the temporary directory ($TMPDIR), which is faster to access for the system and might be beneficial for bigger jobs. Note that $TMPDIR is uniquely associated with the job submitted and will be cleared when the job ends. So you need to copy your results back to your work directory at the end of your script. 

Non-interactive Batch Job (Parallel Run)
Please follow the steps below to use LS-DYNA via the batch system:

1) copy your input files ( explorer.k  in the example below) to your work directory at OSC

2) create a batch script, similar to the following file, saved as job.txt ). It uses the mmp solver for a parallel job (nodes>1) on Oakley:

#PBS -N plate_test
#PBS -l walltime=5:00:00
#PBS -l nodes=2:ppn=12
#PBS -j oe
# The following lines set up the LSDYNA environment
module swap mvapich2/1.7 intelmpi
module load mpp-dyna/971_d_R7.0.0
#
# Move to the directory where the input files are located
#
cd $PBS_O_WORKDIR
#
# Run LSDYNA (number of cpus > 1)
#
mpiexec mpp971 I=explorer.k NCPU=24

 3) submit the script to the batch queue with the command:  qsub job.txt

When the job is finished, all the result files will be found in the directory where you submitted your job ($PBS_O_WORKDIR). Alternatively, you can submit your job from the temporary directory ($TMPDIR), which is faster to access for the system and might be beneficial for bigger jobs. Note that $TMPDIR is uniquely associated with the job submitted and will be cleared when the job ends. So you need to copy your results back to your work directory at the end of your script. An example scrip should include the following lines:

...
cd $TMPDIR
cp $PBS_O_WORKDIR/explorer.k .
... #launch the solver and execute
pbsdcp -g '*' $PBS_O_WORKDIR

Usage on Ruby

Set-up on Ruby

To view available modules installed on Ruby, use  module avail ls-dyna  for smp solvers, and use  module spider mpp  for mpp solvers. In the module name, '_s' indicates single precision and '_d' indicates double precision. For example, mpp-dyna/971_d_R7.0.0 is the mpp solver with double precision on Ruby. Use  module load name  to load LS-DYNA with a particular software version. For example, use  module load mpp-dyna/971_d_R7.0.0   to load LS-DYNA mpp solver version 7.0.0 with double precision on Ruby.

Batch Usage on Ruby

When you log into ruby.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. 

Interactive Batch Session

For an interactive batch session one can run the following command:

qsub -I -l nodes=1:ppn=20 -l walltime=00:20:00 

which requests one whole node with 20 cores ( -l nodes=1:ppn=20 ), for a walltime of 20 minutes ( -l walltime=00:20:00 ). You may adjust the numbers per your need.

Non-interactive Batch Job (Serial Run)

A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. Please follow the steps below to use LS-DYNA via the batch system:

1) copy your input files ( explorer.k  in the example below) to your work directory at OSC

2) create a batch script, similar to the following file, saved as job.txt . It uses the smp solver for a serial job (nodes=1) on Ruby:

#PBS -N plate_test
#PBS -l walltime=5:00:00
#PBS -l nodes=1:ppn=20
#PBS -j oe
# The following lines set up the LSDYNA environment
module load ls-dyna/971_d_7.0.0
#
# Move to the directory where the input files are located
#
cd $PBS_O_WORKDIR
#
# Run LSDYNA (number of cpus > 1)
#
lsdyna I=explorer.k NCPU=20

 3) submit the script to the batch queue with the command:  qsub job.txt

 When the job is finished, all the result files will be found in the directory where you submitted your job ($PBS_O_WORKDIR). Alternatively, you can submit your job from the temporary directory ($TMPDIR), which is faster to access for the system and might be beneficial for bigger jobs. Note that $TMPDIR is uniquely associated with the job submitted and will be cleared when the job ends. So you need to copy your results back to your work directory at the end of your script. 

Non-interactive Batch Job (Parallel Run)

Please follow the steps below to use LS-DYNA via the batch system:

1) copy your input files ( explorer.k  in the example below) to your work directory at OSC

2) create a batch script, similar to the following file, saved as job.txt ). It uses the mmp solver for a parallel job (nodes>1) on Ruby:

#PBS -N plate_test
#PBS -l walltime=5:00:00
#PBS -l nodes=2:ppn=20
#PBS -j oe
# The following lines set up the LSDYNA environment
module swap mvapich2/1.7 intelmpi/5.0.1
module load mpp-dyna/971_d_R7.0.0
#
# Move to the directory where the input files are located
#
cd $PBS_O_WORKDIR
#
# Run LSDYNA (number of cpus > 1)
#
mpiexec mpp971 I=explorer.k NCPU=40

 3) submit the script to the batch queue with the command:  qsub job.txt

When the job is finished, all the result files will be found in the directory where you submitted your job ($PBS_O_WORKDIR). Alternatively, you can submit your job from the temporary directory ($TMPDIR), which is faster to access for the system and might be beneficial for bigger jobs. Note that $TMPDIR is uniquely associated with the job submitted and will be cleared when the job ends. So you need to copy your results back to your work directory at the end of your script. An example scrip should include the following lines:

...
cd $TMPDIR
cp $PBS_O_WORKDIR/explorer.k .
... #launch the solver and execute
pbsdcp -g '*' $PBS_O_WORKDIR

Further Reading

See Also

Supercomputer: 
Service: 

LS-PrePost

Introduction

LS-PrePost is an advanced pre and post-processor that is delivered free with LS-DYNA. The user interface is designed to be both efficient and intuitive. LS-PrePost runs on Windows, Linux, and Unix utilizing OpenGL graphics to achieve fast rendering and XY plotting. The latest builds can be downloaded from LSTC's FTP Site.

 

The prefered way of accessing LS-Prepost is through OnDemand's Glenn desktop application.  This gives you a preconfigured enviornment with GPU acceleration enabled.

Availability

LS-PrePost is available on both Glenn and Oakley clusters. A module was created for LS-Prepost 4.0 on Oakley, and can be loaded with 'module load lspp'. All other versions can be loaded with the corresponding dyna modules.

Version Glenn Oakley
v2.3 X  
v3.0   X
v3.2   X
v4.0   X

 

 

 

Usage

Running LS-PrePost on Oakley through OnDemand's Glenn Destkop

Below are instructions on running LS-PrePost on Oakley through Glenn's OnDemand desktop interface with GPU acceleration enabled.  To run LS-PrePost with a slower X-tunneling procedure, see the specified instructions below.

 

1) Log in to OnDemand with your HPC username and password.

2) Launch the Glenn Desktop from "Apps" menu. 

3) Open a Terminal window (Applications > Accessories > Terminal)

4) Type the following command to connect to oakley:

    ssh -X username@oakley.osc.edu       

          * Where "username" is your username.

5)  Once logged in to Oakley, submit an interactive job with the following command:

    qsub -X -I -l nodes=1:ppn=12:gpus=2:vis -l walltime=hh:mm:ss

          * pick a walltime that is close to the amount of time you will need to for working in the GUI application.

6) Once your job starts, make a note of the hostname for the compute node your job is running on.  You can find this information by typing the following command:

    hostname

7) Open another Terminal window, and type the following commands:

     module load VirtualGL
     vglconnect username@job-hostname

           * job-hostname is the information you found in step 6; your command might look something like this, for example:  

     vglconnect ahahn@n0656.ten.osc.edu

           You'll be asked a password to connect, which should be your HPC password.

8) Now, you should be connected to your running job's GPU node.  Run the following commands to launch LS-PrePost version 4.0:

     export LD_LIBRARY_PATH=/usr/local/MATLAB/R2013a/sys/opengl/lib/glnxa64
     /usr/local/lstc/ls-prepost/lsprepost4.0_centos6/lspp4

At startup LS-PrePost displays a graphical interface for model generation and results post-processing.

 

Running LS-PrePost on Oakley or Glenn with X-11 forwarding

The following procedure will result in a much slower and GUI interface, but may be useful if the above instructions are not working.  It can be done completely from the command line, with no need for logging into OnDemand.  You may need to edit your terminal settings to enabled x11 forwarding.

1) Login to Oakley or Glenn with X11 forwarding enabled

ssh -X username@oakley.osc.edu

or

ssh -X username@glenn.osc.edu

 

2) Submit a Interactive job

qsub -I -X -l nodes=1:ppn=12 -l walltime=hh:mm:ss

and wait for it to start. Use ppn=8 for Glenn.

 

3) Load the LS-Dyna module and start up LS-PrePost application

module load ls-dyna
lspp3

 

   For LS-Prepost 4.0 on Oakley, use the following commands instead:

module load lspp
lspp4

 

A x11 window should pop up.  If you get a error along the lines of:

Error: Unable to initialize gtk, is DISPLAY set properly?

 

Double check that you:

1) logged in with x11 forwarding enabled

2) have configured your x11 settings for your terminal

3) included the -X with the qsub command

4) have a x11 client running on your computer (Xming, Xorg, XQuarts, etc.).

Documentation

Documentation for LS-PrePost may be obtained at: http://www.lstc.com/lspp/

Supercomputer: 

User-Defined Material for LS-DYNA

This page describes how to specify user defined material to use within LS-DYNA.  The user-defined subroutines in LS-DYNA allow the program to be customized for particular applications.  In order to define user material, LS-DYNA must be recompiled.

Usage

The first step to running a simulation with user defined material is to build a new executable. The following is an example done with solver version mpp971_s_R7.1.1.

When you log into the Oakley system, load mpp971_s_R7.1.1 with the command:

module load mpp-dyna/R7.1.1

Next, copy the mpp971_s_R7.1.1 object files and Makefile to your current directory:

cp /usr/local/lstc/mpp-dyna/R7.1.1/usermat/* $PWD

Next, update the dyn21.f file with your user defined material model subroutine. Please see the LS-DYNA User's Manual (Keyword version) for details regarding the format and structure of this file.

Once your user defined model is setup correctly in dyn21.f, build the new mpp971 executable with the command:

make

To execute a multi processor (ppn > 1) run with your new executable, execute the following steps:

1) move your input file to a directory on Glenn (pipe.k in the example below)

2) copy your newly created mpp971 executable to this directory as well

3) create a batch script (lstc_umat.job) like the following:

#PBS -N LSDYNA_umat
#PBS -l walltime=1:00:00
#PBS -l nodes=2:ppn=8
#PBS -j oe
#PBS -S /bin/csh

# This is the template batch script for running a pre-compiled
# MPP 971 v7600 LS-DYNA.  
# Total number of processors is ( nodes x ppn )
#
# The following lines set up the LSDYNA environment
module load mpp-dyna/R7.1.1
#
# Move to the directory where the job was submitted from
# (i.e. PBS_O_WORKDIR = directory where you typed qsub)
#
cd $PBS_O_WORKDIR
#
# Run LSDYNA 
# NOTE: you have to put in your input file name
#
mpiexec mpp971 I=pipe.k NCPU=16

          4) Next, submit this job to the batch queue with the command:

       qsub lstc_umat.job

The output result files will be saved to the directory you ran the qsub command from (known as the $PBS_O_WORKDIR_

Documentation

On-line documentation is available on LSTC website.

See Also

 

 

Supercomputer: 
Service: 

MATLAB

MATLAB is a technical computing environment for high-performance numeric computation and visualization. MATLAB integrates numerical analysis, matrix computation, signal processing, and graphics in an easy-to-use environment where problems and solutions are expressed just as they are written mathematically--without traditional programming.

Availability and Restrictions

MATLAB is available on Oakley, Ruby, and Owens Clusters. The versions currently available at OSC are:

Version Oakley Ruby Owens notes
2013a X     Default version on Oakley prior to 09/15/2015
2013b X      
2014a X X   Default version on Ruby prior to 09/15/2015
2014b X* X*    
2015a X X    
2015b X X X*  
*: Current default version

You can use module avail matlab  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access to Ohio State University Students/Faculty

All Ohio State users must be added to the license server before using MATLAB.  Please contact OSC Help to be granted access.

Access to Non Ohio State University Students/Faculty

Academic users from outside of Ohio State University can use MATLAB at OSC, if they bring their own license. Please contact OSC Help to see if you are eligible. Currently, users from the following universities can use MATLAB upon request (please contact OSC Help).

  • University of Dayton
  • Miami University of Ohio
  • University of Cincinnati (Math department)

Another method to use MATLAB on OSC is to use the MATLAB Distributed Computing Server on Oakley via your local MATLAB client.  In order to use this feature, you should have your own MATLAB license, including the Parallel Computing Toolbox. See HOW-TO: Configure the MATLAB Parallel Computing Toolbox.

Usage on Oakley

Set-up

To load the default version of MATLAB module, use  module load matlab . For a list of all available MATLAB versions and the format expected, type: module spider matlab . To select a particular software version, use   module load matlab/version . For example, use  module load matlab/r2014b to load MATLAB version 2014b. 

Running MATLAB

The following command will start an interactive, command line version of MATLAB:

matlab -nodisplay 
If you are able to use X-11 forwarding and have enabled it in your SSH client software preferences, you can run MATLAB using the GUI by typing the command  matlab . For more information about the matlab command usage, type matlab –h  for a complete list of command line options.

The commands listed above will run MATLAB on the login node you are connected to. As the login node is a shared resource, running scripts that require significant computational resources will impact the usability of the cluster for others. As such, you should not use interactive MATLAB sessions on the login node for any significant computation. If your MATLAB script requires significant time, CPU power, or memory, you should run your code via the batch system.

Batch Usage

When you log into oakley.osc.edu you are actually logged into a Linux box referred to as the login node. To gain access to the multiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request multiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. Batch jobs run on the compute nodes of the system and not on the login node. It is desirable for big problems since more resources can be used.

Interactive Batch Session
For an interactive batch session using the command line version of MATLAB, one can run the following command:
qsub -I -X -l nodes=1:ppn=12 -l walltime=00:20:00 

which requests one whole node with 12 cores ( -l nodes=1:ppn=12 ), for a walltime of 20 minutes ( -l walltime=00:20:00 ). Here you can run MATLAB interactively by loading the MATLAB module and running MATLAB with the options of your choice as described above. The  –X flag enables X11 forwarding on the compute node, so you can use the MATLAB GUI if you choose.You may adjust the numbers per your need.

Non-interactive Batch Job (Serial Run)

In order to run MATLAB non-interactively via the batch system, you will require a batch submission script and a MATLAB M-file containing the script that will be run via the batch system. A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. Below is an example batch submission script and a simple M-file. The batch script runs the M-file via the batch system.Example batch script (  job.txt  ) for a serial run:

#PBS -N matlab_example
#PBS -l walltime=00:10:00
#PBS -l nodes=1:ppn=12
#PBS -j oe
module load matlab
matlab -nodisplay -nodesktop < hello.m
# end of example file

Example M-file, hello.m:

%Example M-file for Hello World
disp 'Hello World' 
exit   
% end of example file
In order to run hello.m via the batch system, submit the  job.txt file with the following command:
qsub job.txt 
This will run  hello.m via the batch system, and all output from the running of the script will be saved in the output.txt file.

Toolboxes

To view a complete list of the currently INSTALLED toolboxes, in the MATLAB command line type the command ver . OSC's current licenses support the following toolboxes (please contact oschelp@osc.edu for license-specific questions):

Bioinformatics_Toolbox
Communication_Toolbox
Control_Toolbox
Curve_Fitting_Toolbox
Data_Acq_Toolbox
Fuzzy_Toolbox
Image_Toolbox
Instr_Control_Toolbox
Neural_Network_Toolbox
Optimization_Toolbox
Distrib_Computing_Toolbox
PDE_Toolbox
RF_Toolbox
Robust_Toolbox
Signal_Toolbox
Statistics_Toolbox
Symbolic_Toolbox
Identification_Toolbox
Wavelet_Toolbox

Parallel Processing in MATLAB

MATLAB supports both implicit multiprocessing (multithreading) and explicit multiprocessing across multiple nodes.  

Multithreading

Multithreading allows some functions in MATLAB to distribute the work load between cores of the node that your job is running on. By default, all of the current versions of MATLAB available on the OSC clusters have multithreading enabled. 

The system will use a number of threads equal to the number of cores you request.  Therefore, if you request nodes=1:ppn=4, your job will only spawn four threads. 

Multithreading increases the speed of some linear algebra routines, but if you would like to disable multithreading you may request nodes=1:ppn=1 and include the option  -singleCompThread when running MATLAB. An example is given below:

#PBS -N disable_multithreading
#PBS -l walltime=00:10:00
#PBS -l nodes=1:ppn=1
#PBS -j oe
module load matlab
matlab -singleCompThread -nodisplay -nodesktop < hello.m
# end of example file
Parallel computing across multiple nodes

You can accomplish parallel processing using multiple nodes by using the Parallel Computing Toolbox in conjunction with the MATLAB Distributed Computing Server.  For more information about configuration and usage, see HOW-TO: Configure the MATLAB Parallel Computing Toolbox.  

Usage on Ruby

Set-up

To load the default version of MATLAB module, use  module load matlab . For a list of all available MATLAB versions and the format expected, type: module spider matlab . To select a particular software version, use   module load matlab/version . For example, use  module load matlab/r2014b to load MATLAB version 2014b. 

Running MATLAB

The following command will start an interactive, command line version of MATLAB:

matlab -nodisplay 
If you are able to use X-11 forwarding and have enabled it in your SSH client software preferences, you can run MATLAB using the GUI by typing the command  matlab . For more information about the matlab command usage, type matlab –h for a complete list of command line options.

The commands listed above will run MATLAB on the login node you are connected to. As the login node is a shared resource, running scripts that require significant computational resources will impact the usability of the cluster for others. As such, you should not use interactive MATLAB sessions on the login node for any significant computation. If your MATLAB script requires significant time, CPU power, or memory, you should run your code via the batch system.

Batch Usage

When you log into ruby.osc.edu you are actually logged into a Linux box referred to as the login node. To gain access to the multiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request multiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. 

Interactive Batch Session
For an interactive batch session using the command line version of MATLAB, one can run the following command:
qsub -I -X -l nodes=1:ppn=20 -l walltime=00:20:00 

which requests one whole node with 20 cores ( -l nodes=1:ppn=20 ), for a walltime of 20 minutes ( -l walltime=00:20:00 ). Here you can run MATLAB interactively by loading the MATLAB module and running MATLAB with the options of your choice as described above.  The  –X  flag enables X11 forwarding on the compute node, so you can use the MATLAB GUI if you choose.You may adjust the numbers per your need.

Toolboxes

To view a complete list of the currently INSTALLED toolboxes, in the MATLAB command line type the command ver . OSC's current licenses support the following toolboxes (please contact oschelp@osc.edu for license-specific questions):

Bioinformatics_Toolbox
Communication_Toolbox
Control_Toolbox
Curve_Fitting_Toolbox
Data_Acq_Toolbox
Fuzzy_Toolbox
Image_Toolbox
Instr_Control_Toolbox
Neural_Network_Toolbox
Optimization_Toolbox
Distrib_Computing_Toolbox
PDE_Toolbox
RF_Toolbox
Robust_Toolbox
Signal_Toolbox
Statistics_Toolbox
Symbolic_Toolbox
Identification_Toolbox
Wavelet_Toolbox

Parallel Processing in MATLAB

MATLAB supports both implicit multiprocessing (multithreading) and explicit multiprocessing across multiple nodes.  

Multithreading

Multithreading allows some functions in MATLAB to distribute the work load between cores of the node that your job is running on. By default, all of the current versions of MATLAB available on the OSC clusters have multithreading enabled. 

The system will run as many threads as there are cores on the nodes requested.  For the normal nodes, this mean matlab will start 20 threads to run on the 20 cores of the node.  This is due to the whole node scheduling policy of Ruby (see Batch Specifics).

Multithreading increases the speed of some linear algebra routines, but if you would like to disable multithreading you may include the option "   -singleCompThread " when running MATLAB. An example is given below:

#PBS -N disable_multithreading
#PBS -l walltime=00:10:00
#PBS -l nodes=1:ppn=20
#PBS -j oe
module load matlab
matlab -singleCompThread -nodisplay -nodesktop < hello.m
# end of example file
Parallel computing across multiple nodes

Parallel computing across multiple nodes is only available on Oakley.

Usage on Owens

Set-up

To load the default version of MATLAB module, use  module load matlab . For a list of all available MATLAB versions and the format expected, type: module spider matlab . To select a particular software version, use   module load matlab/version . For example, use  module load matlab/r2015b to load MATLAB version r2015b. 

Running MATLAB

The following command will start an interactive, command line version of MATLAB:

matlab -nodisplay 
If you are able to use X-11 forwarding and have enabled it in your SSH client software preferences, you can run MATLAB using the GUI by typing the command  matlab . For more information about the matlab command usage, type matlab –h for a complete list of command line options.

The commands listed above will run MATLAB on the login node you are connected to. As the login node is a shared resource, running scripts that require significant computational resources will impact the usability of the cluster for others. As such, you should not use interactive MATLAB sessions on the login node for any significant computation. If your MATLAB script requires significant time, CPU power, or memory, you should run your code via the batch system.

Batch Usage

When you log into owens.osc.edu you are actually logged into a Linux box referred to as the login node. To gain access to the multiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request multiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. 

Interactive Batch Session
For an interactive batch session using the command line version of MATLAB, one can run the following command:
qsub -I -X -l nodes=1:ppn=28 -l walltime=00:20:00 

which requests one whole node with 28 cores ( -l nodes=1:ppn=28), for a walltime of 20 minutes ( -l walltime=00:20:00 ). Here you can run MATLAB interactively by loading the MATLAB module and running MATLAB with the options of your choice as described above.  The  –X  flag enables X11 forwarding on the compute node, so you can use the MATLAB GUI if you choose.You may adjust the numbers per your need.

Toolboxes

To view a complete list of the currently INSTALLED toolboxes, in the MATLAB command line type the command ver . OSC's current licenses support the following toolboxes (please contact oschelp@osc.edu for license-specific questions):

Bioinformatics_Toolbox
Communication_Toolbox
Control_Toolbox
Curve_Fitting_Toolbox
Data_Acq_Toolbox
Fuzzy_Toolbox
Image_Toolbox
Instr_Control_Toolbox
Neural_Network_Toolbox
Optimization_Toolbox
Distrib_Computing_Toolbox
PDE_Toolbox
RF_Toolbox
Robust_Toolbox
Signal_Toolbox
Statistics_Toolbox
Symbolic_Toolbox
Identification_Toolbox
Wavelet_Toolbox

Parallel Processing in MATLAB

MATLAB supports both implicit multiprocessing (multithreading) and explicit multiprocessing across multiple nodes.  

Multithreading

Multithreading allows some functions in MATLAB to distribute the work load between cores of the node that your job is running on. By default, all of the current versions of MATLAB available on the OSC clusters have multithreading enabled. 

The system will run as many threads as there are cores on the nodes requested.  For the normal nodes, this mean matlab will start 28 threads to run on the 28 cores of the node.  This is due to the whole node scheduling policy of Owens (see Batch Specifics).

Multithreading increases the speed of some linear algebra routines, but if you would like to disable multithreading you may include the option "   -singleCompThread " when running MATLAB. An example is given below:

#PBS -N disable_multithreading
#PBS -l walltime=00:10:00
#PBS -l nodes=1:ppn=28
#PBS -j oe
module load matlab
matlab -singleCompThread -nodisplay -nodesktop < hello.m
# end of example file
Parallel computing across multiple nodes

Parallel computing across multiple nodes is only available on Oakley.

Further Reading

Official PDF documentation can be obtained from the MathWorks Website

 

Supercomputer: 
Service: 
Fields of Science: 

MINPACK

MINPACK is a library of Fortran routines for the solution of non-linear, multivariate minimization problems.

Availability & Restrictions

MINPACK is available to all OSC users without restrictions.

The following versions of MINPACK are available on OSC systems:

Version Glenn Oakley
n/a X  

Usage

Set-up

To use MINPACK on Glenn, first run the following command:

module load minpack

Building With MINPACK

The following environment variables are set when the minpack module is loaded:

Variable Use
$MINPACK Compiler and linker flags for building with MINPACK

To build a Fortran program named my_prog:

module load minpack
ifort -o my_prog my_prog.f $MINPACK

Further Reading

Supercomputer: 
Service: 

MIRA

MIRA - Sequence assembler and sequence mapping for whole genome shotgun and EST / RNASeq sequencing data. Can use Sanger, 454, Illumina and IonTorrent data. PacBio: CCS and error corrected data usable, uncorrected not yet.

Availability and Restrictions

The following versions of MIRA are available on OSC clusters:

Version Oakley Owens
4.0.2 X* X*
*: Current default version

You can use  module avail mira  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

MIRA is available to all OSC users without restriction.

Usage on Oakley

Set-up

To configure your environment for use of MIRA, run the following command:   module load mira. The default version will be loaded. To select a particular MIRA version, use   module load mira/version. For example, use  module load mira/4.0.2 to load MIRA 4.0.2.

Usage on Owens

Set-up

To configure your environment for use of MIRA, run the following command:   module load mira. The default version will be loaded. To select a particular MIRA version, use   module load mira/version. For example, use module load mira/4.0.2 to load MIRA 4.0.2.

Further Reading

Supercomputer: 
Service: 
Fields of Science: 

MKL - Intel Math Kernel Library

High-performance, multithreaded mathematics libraries for linear algebra, fast Fourier transforms, vector math, and more.

Availability and Restrictions

OSC supports single-process use of MKL for LAPACK and BLAS levels one through three. For multi-process applications, we also support the ScaLAPACK, FFTW2, and FFTW3 MKL wrappers. MKL modules are available for the Intel, GNU, and PGI compilers. MKL is available on Oakley, Ruby, and Owens Clusters. The versions currently available at OSC are:

Version Oakley Ruby owens notes
10.0.3        
10.3.0 X     Default version on Oakley prior to 09/15/2015
11.1.0 X      
11.2.0   X   Default version on Ruby prior to 09/15/2015
11.2.1   X    
11.2.3 X* X*    
11.3.2     X  
11.3.3     X*  
* : Current Default Version

You can use module spider mkl to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

All OSC users may use the MKL for academic purposes.

Usage on Oakley

Set-up

To load the default MKL, run the following command: module load mkl. To load a particular version, use  module load mkl/version . For example, use  module load mkl/11.2.3  to load MKL version 11.2.3. You can use module spider mkl  to view available modules.

This step is required for both building and running MKL applications.

Exception: The "mkl" module is usually not needed when using the Intel compilers; just use the "-mkl" flag on the compile and link steps.

Building With MKL

The MKL module will automatically configure your environment to locate the appropriate include files. When linking, however, you have a variety of options to choose from depending on your needs. These options come in the form of environment variables defined by the module. Which variable you include when linking will determine what particular configuration of MKL you get.

Environment Variable Function
$MKL_CFLAGS Include flags for C
$MKL_FFLAGS Include flags for Fortran
$MKL_LIBS Use multithreaded MKL with 32-bit integers. This is the standard configuration.
$MKL_LIBS_INT64 Use multithreaded MKL with 64-bit (“ILP64”) integers.
$MKL_LIBS_SEQ Use single-threaded (“ SEQ uential”) MKL with with 32-bit integers.
$MKL_LIBS_SEQ_INT64 Use single-threaded MKL with 64-bit integers.

Notes:

  • 64-bit integers are not supported by the FFTW2 wrappers. (See below.)
  • The Intel compilers are specially configured for Intel MKL. If you are using this toolchain, you can use the “ -mkl ” option when linking in place of $MKL_LIBS .
  • The default, multithreaded MKL libraries will automatically run in a single thread if they are called from an OpenMP parallel region. If you want to force single-threaded behavior throughout your program, choose one of the “ _SEQ ” variables from the list above.
Fortran 95 BLAS/LAPACK Wrappers

To compile Fortran 95 programs with modules, add the $MKL_F95FLAGS variable to your compilation step. If you need 64-bit integers, use the $MKL_F95_FLAGS_INT64 instead. When linking, you will also need to use $MKL_F95LIBS (or $MKL_F95LIBS_INT64 if using 64-bit integers). These variables will allow your programs to use the BLAS and LAPACK wrappers for MKL.

FFTW Wrappers

A number of “wrappers” are provided in the form of environment variables that allow you to use FFTW APIs with MKL. Variables ending in FLAGS should be included with your compilation step, while variables ending in LIBS should be included in your linking step.

Environment Variable Function
$MKL_FFTW_CFLAGS Compile variable for C programs.
$MKL_FFTW_FFLAGS Compile variable for Fortran programs.
$MKL_FFTW2_D_CLIBS Linking variable for double-precision FFTW2 C programs.
$MKL_FFTW2_D_FLIBS Linking variable for double-precision FFTW2 Fortran programs.
$MKL_FFTW2_S_CLIBS Linking variable for single-precision FFTW2 C programs.
$MKL_FFTW2_S_FLIBS Linking variable for single-precision FFTW2 Fortran programs.
$MKL_FFTW3_CLIBS Linking variable for FFTW3 C programs.
$MKL_FFTW3_FLIBS Linking variable for FFTW3 Fortran programs.

Notes:

  • FFTW3 is the default wrapper and should be picked up by your linker automatically. If you would prefer to use FFTW2, use the appropriate linking variable.
  • The FFTW2 wrappers do not support 64-bit (“ILP64”) integers.
Examples

The following examples use the Intel compilers, though they should work using the GNU and Portland Group compilers as well.

Compiling a Fortran77 program with MKL:

ifort -c my_prog.f
ifort -o my_prog my_prog.o $MKL_LIBS

Compiling a C program with MKL:

icc -c my_prog.c
icc -o my_prog my_prog.o $MKL_LIBS

Compiling a module-based Fortran90/95 program with MKL:

ifort -c my_prog.f95 $MKL_F95FLAGS
ifort -o my_prog my_prog.o $MKL_F95LIBS $MKL_LIBS

Compiling a C program with the double-precision FFTW2 wrapper:

icc -c $MKL_FFTW_CFLAGS -DMKL_DOUBLE my_prog.c
icc -o my_prog my_prog.o $MKL_FFTW2_D_CLIBS $MKL_LIBS

Compiling a Fortran77 program with the FFTW2 wrapper:

ifort -c $MKL_FFTW_FFLAGS my_prog.f
ifort -o my_prog my_prog.o $MKL_FFTW2_S_FLIBS $MKL_LIBS

Compiling a C program with the FFTW3 wrapper:

icc -c $MKL_FFTW_CFLAGS my_prog.c
icc -o my_prog my_prog.o $MKL_FFTW3_CLIBS $MKL_LIBS

Compiling a Fortran program with the FFTW3 wrapper:

ifort -c $MKL_FFTW_FFLAGS my_prog.f 
ifort -o my_prog my_prog.o $MKL_FFTW3_CLIBS $MKL_LIBS

Advanced Usage

If you are already familiar with building MKL applications with your chosen build tool (GCC, Intel, or PGI) and you do not wish to use the convenience variables discussed above, you may wish to use the $MKLROOT variable instead. This variable points to the installation directory for Intel MKL. All include files can be found in $MKLROOT/include , for example, and the libraries are in $MKLROOT/lib/intel64 .

Running MKL Programs

When running an MKL program, you need to be sure to take the following steps.

  1. Load the mkl  module:

    module load mkl
    
  2. If running with parallel MKL, set OMP_NUM_THREADS  to match the number of cores per node in your process. In the bash shell, you can accomplish this with:

    PPNPERNODE=$(expr $(cat $PBS_NODEFILE | wc -l) / $(uniq $PBS_NODEFILE | wc -l))
    export OMP_NUM_THREADS=$PPNPERNODE
    

Usage on Ruby

Set-up

To load the default MKL, run the following command: module load mkl . To load a particular version, use  module load mkl/version . For example, use  module load mkl/11.2.3  to load MKL version 11.2.3. You can use module spider mkl  to view available modules.

This step is required for both building and running MKL applications.

Exception: The "mkl" module is usually not needed when using the Intel compilers; just use the "-mkl" flag on the compile and link steps.

Usage on Owens

Set-up

To load the default MKL, run the following command: module load mkl . To load a particular version, use  module load mkl/version . For example, use  module load mkl/11.3.3  to load MKL version 11.3.3. You can use module spider mkl  to view available modules.

This step is required for both building and running MKL applications.

Exception: The "mkl" module is usually not needed when using the Intel compilers; just use the "-mkl" flag on the compile and link steps.

Further Reading

Tag: 
Supercomputer: 
Service: 

MPJ-Express

MPJ-Express is a Java library that provides message passing capabilities for parallel computing in Java applications.

MPJ-Express is available on Oakley.

Availability & Restrictions

MPJ-Express is available without restriction to all OSC users.

The following versions of MPJ-Express are available on OSC systems:

Version Oakley Ruby
0.38 X  
0.44 X  


Note: We have modified MPJ-Express slightly from the default distribution to work nicely with our infrastructure. Installing your own copy of a newer version will not work. Please contact OSC Help if you need a newer version of this library.

Usage

Set-up

To set up your environment for using the MPJ-Express libraries, you must load the appropriate module. On Oakley this is pretty straightforward:

module load mpj-express

Building With MPJ-Express

When you load the MPJ-Express module, we will modify your CLASSPATH to ensure the JAR files are located.

Batch Usage

Programs built with MPJ-Express can only be run in the batch environment at OSC. For information on writing batch scripts, see Batch Processing at OSC.

If you have loaded a Java module when compiling, be sure to load that same module in your batch script to avoid version mismatches.

Here is a basic "Hello World" program, which needs to be in a file called "HelloWorld.java", which we use in our example batch script:

 

import mpi.*;
public class HelloWorld
{
     public static void main(String[] args) 
                     throws Exception
     {
          MPI.Init(args);
          int me = MPI.COMM_WORLD.Rank();
          int size = MPI.COMM_WORLD.Size();
          System.out.println
            ("Hi from " + me + " of " + size + "\n");
          MPI.Finalize();
     }
}

 

Below we have included a sample batch script:

 

#PBS -N mpj-world
#PBS -l walltime=0:05:00
#PBS -l nodes=2:ppn=12
#PBS -S /bin/bash

# Set environment
module load mpj-express/0.44

# Change to the directory you submitted the job from
cd $PBS_O_WORKDIR

# Set up the directory for the daemon logs to go in
export MPJ_LOG="$PBS_O_WORKDIR/log"
mkdir $MPJ_LOG

# Re-compile our application - not strictly necessary,
# if the source is unchanged.
javac HelloWorld.java

# Start MPJ
mpjboot $PBS_NODEFILE

# Launch your parallel program
mpjrun.sh -np $PBS_NP -dev native HelloWorld

# Shutdown MPJ
mpjhalt $PBS_NODEFILE

 

Further Reading

See Also

Supercomputer: 
Service: 
Technologies: 

MVAPICH2

MPI is a standard library for performing parallel processing using a distributed-memory model. 

Availability & Restrictions

MPI is available without restriction to all OSC users.

The following versions of MVAPICH2 are available on OSC systems:

Version Oakley Ruby Owens
1.7 X    
1.8 X    
1.9 X X  
2.1 X* X*  
2.2     X*
NOTE: *Default version.

Usage

Set-up

To set up your environment for using the MPI libraries, you must load the appropriate module:

module load mvapich2

You will get the default version for the compiler you have loaded.

Note:Be sure to swap the intel compiler module for the gnu module if you're using gnu.

Building With MPI

To build a program that uses MPI, you should use the compiler wrappers provided on the system. They accept the same options as the underlying compiler. The commands are shown in the following table.

C mpicc
C++ mpicxx
FORTRAN 77 mpif77
Fortran 90 mpif90

For example, to build the code my_prog.c using the -O2 option, you would use:

mpicc -o my_prog -O2 my_prog.c

In rare cases you may be unable to use the wrappers. In that case you should use the environment variables set by the module.

Variable Use
$MPI_CFLAGS Use during your compilation step for C programs.
$MPI_CXXFLAGS Use during your compilation step for C++ programs.
$MPI_FFLAGS Use during your compilation step for Fortran 77 programs.
$MPI_F90FLAGS Use during your compilation step for Fortran 90 programs.
$MPI_LIBS Use when linking your program to the MPI libraries.

For example, to build the code my_prog.c without using the wrappers you would use:

mpicc -c $MPI_CFLAGS my_prog.c

mpicc -o my_prog my_prog.o $MPI_LIBS

Batch Usage

Programs built with MPI can only be run in the batch environment at OSC. For information on starting MPI programs using the mpiexec command, see Batch Processing at OSC.

Be sure to load the same compiler and mvapich modules at execution time as at build time.

Further Reading

See Also

Supercomputer: 
Service: 
Technologies: 
Fields of Science: 

Maven (Apache)

Apache Maven is a plugin-based build automation tool, similar in purpose to GNU Make or Apache Ant. It is most commonly used with Java projects, but also supports other languages such as .NET and C/C++.

Availability & Compatibility

Version Glenn Oakley
3.0.4   X

Restrictions

There are no restrictions for this software; any OSC user may make use of the Apache Maven.

Usage

Set-up

To use Maven, load the the "maven" module with the following command.

module load maven

To test that the install worked correctly, run "mvn --version". You should see output similar to that shown below:

 

$ mvn --version
Apache Maven 3.0.4 (r1232337; 2012-01-17 03:44:56-0500)
Maven home: /usr/local/maven/3.0.4
Java version: 1.6.0_29, vendor: Sun Microsystems Inc.
Java home: /usr/lib/jvm/java-1.6.0-sun-1.6.0.29.x86_64/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "2.6.32-131.17.1.el6.x86_64", arch: "amd64", family: "unix"

 

Basic Usage

Once the module is loaded, you can use Maven just as you would on your local machine. For example, the session below illustrates initializing a new project call "my-app":

 

$ mvn archetype:generate -DgroupId=com.mycompany.app -DartifactId=my-app -DarchetypeArtifactId=maven-archetype-quickstart -DinteractiveMode=false
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building Maven Stub Project (No POM) 1
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] >>> maven-archetype-plugin:2.2:generate (default-cli) @ standalone-pom >>>
[INFO] 
[INFO] <<< maven-archetype-plugin:2.2:generate (default-cli) @ standalone-pom <<<
[INFO] 
[INFO] --- maven-archetype-plugin:2.2:generate (default-cli) @ standalone-pom ---
[INFO] Generating project in Batch mode
[INFO] ----------------------------------------------------------------------------
[INFO] Using following parameters for creating project from Old (1.x) Archetype: maven-archetype-quickstart:1.0
[INFO] ----------------------------------------------------------------------------
[INFO] Parameter: groupId, Value: com.mycompany.app
[INFO] Parameter: packageName, Value: com.mycompany.app
[INFO] Parameter: package, Value: com.mycompany.app
[INFO] Parameter: artifactId, Value: my-app
[INFO] Parameter: basedir, Value: /nfs/12/jmccance
[INFO] Parameter: version, Value: 1.0-SNAPSHOT
[INFO] project created from Old (1.x) Archetype in dir: /nfs/12/jmccance/my-app
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4.429s
[INFO] Finished at: Fri Jun 08 14:30:57 EDT 2012
[INFO] Final Memory: 9M/361M
[INFO] ------------------------------------------------------------------------

Further Reading

See Also

Supercomputer: 
Service: 

MrBayes

MrBayes is a program for the Bayesian estimation of phylogeny. Bayesian inference of phylogeny is based upon a quantity called the posterior probability distribution of trees, which is the probability of a tree conditioned on the observations. The conditioning is accomplished using Bayes's theorem. The posterior probability distribution of trees is impossible to calculate analytically; instead, MrBayes uses a simulation technique called Markov chain Monte Carlo (or MCMC) to approximate the posterior probabilities of trees.

Availability & Restrictions

MrBayes is available without restriction to all OSC users.

The following versions of MrBayes are available:

Version Glenn Oakley
3.1.2 X  
3.2.1   X

Usage

Set-up

To load the MrBayes software on the Glenn system, use the following commands:

module load biosoftw
module load mrbayes

On the Oakley system, use the following command:

module load mrbayes

Using MrBayes

Once mrbayes module is loaded, the commands are available for your use. The command for serial version is

mb

Parallel Mrbayes can only be used via batch system. The command is

mb-parallel

Batch Usage

Below is a sample batch script for running serial mrbayes job. It asks for 1 processor and 30 minutes of walltime.

#PBS -l walltime=30:00
#PBS -l nodes=1:ppn=1
#PBS -N mb
#PBS -j oe
set -x
date
cd $PBS_O_WORKDIR
cp ./primates.nex $TMPDIR
cp ./primates.nxs $TMPDIR
cd $TMPDIR
module load mrbayes-3.1.2
/usr/bin/time mb ./primates.nxs > mb.log
cp * $PBS_O_WORKDIR
date

Below is a sample batch script for running a parallel job on Oakley. It asks for 2 nodes, 24 processors and 30 minutes of walltime.

#PBS -l walltime=30:00
#PBS -l nodes=2:ppn=12
#PBS -N mb-parallel
#PBS -j oe
set -x
date
cd $PBS_O_WORKDIR
pbsdcp ./primates.nex $TMPDIR
pbsdcp ./primates.nxs $TMPDIR
cd $TMPDIR
module load mrbayes
/usr/bin/time mpiexec mb-parallel ./primates.nxs > mb-paralllel.log
pbsdcp -g '*' $PBS_O_WORKDIR
date

Further Reading

See Also

  • Paup
Supercomputer: 
Service: 
Fields of Science: 

MuTect

MuTect is a method developed at the Broad Institute for the reliable and accurate identification of somatic point mutations in next generation sequencing data of cancer genomes.

Availability and Restrictions

The following versions of MuTect are available on OSC clusters:

Version Oakley owens
1.1.4 X* X*
*: Current default version

You can use  module avail mutect  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access for Academic Users

MuTect is available to academic OSC users with proper validation. In order to obtain validation, please contact OSC Help for further instruction. 

Usage on Oakley

Set-up

To configure your environment for use of MuTect, run the following command:   module load mutect . The default version will be loaded. To select a particular MuTect version, use   module load mutect/version . For example, use  module load mutect/1.1.4  to load MuTect 1.1.4.
 
NOTE: Java will also need to be loaded in order to use MuTect: module load java

Usage

This software is a Java executable .jar file; thus, it is not possible to add to the PATH environment variable.

From module load mutect , a new environment variable, MUTECT, will be set.

Thus, users can use the software by running the following command:  java -jar $MUTECT {other options}

Usage on Owens

Set-up

To configure your environment for use of MuTect, run the following command:   module load mutect . The default version will be loaded. To select a particular MuTect version, use   module load mutect/version . For example, use module load mutect/1.1.4  to load MuTect 1.1.4.
 
NOTE: Java 1.7.0 will also need to be loaded in order to use MuTect: module load java/1.7.0

Usage

This software is a Java executable .jar file; thus, it is not possible to add to the PATH environment variable.

From module load mutect , a new environment variable, MUTECT, will be set.

Thus, users can use the software by running the following command:  java -jar $MUTECT {other options}

Further Reading

Supercomputer: 
Service: 
Fields of Science: 

NAMD

NAMD is a parallel molecular dynamics code designed for high-performance simulation of large biomolecular systems. NAMD generally scales well on OSC platforms and offers a variety of modelling techniques. NAMD is file-compatible with AMBER, CHARMM, and X-PLOR.

Availability & Restrictions

NAMD is available without restriction to all OSC users.

The following versions of NAMD are available:

Version Glenn Oakley
2.6 X  
2.9           X
2.10  

X

2.11   X

Usage

Set-up

To load the NAMD software on the Glenn system, use the following command:

module load namd-2.6-mpi

To load the latest NAMD software on the Oakley system, use the following command:

module load namd/2.11

Using NAMD

NAMD is rarely executed interactively because preparation for simulations is typically performed with extraneous tools, such as, VMD.

Batch Usage

Sample batch scripts and input files are available here:

/nfs/10/srb/workshops/compchem/namd/

The simple batch script for Glenn below demonstrates some important points. It requests 16 processors and 2 hours of walltime. If the job goes beyond 2 hours, the job would be terminated.

#PBS -N apoa1
#PBS -l nodes=2:ppn=8
#PBS -l walltime=2:00:00
#PBS -S /bin/bash
#PBS -j oe

module load namd-2.6-mpi
cd $PBS_O_WORKDIR
pbsdcp -p apoa1.namd apoa1.pdb apoa1.psf *.xplor $TMPDIR
cd $TMPDIR
run_namd apoa1.namd
pbsdcp -pg '*' $PBS_O_WORKDIR

Further Reading

Supercomputer: 
Service: 

NCL/NCARG

NCL (The NCAR Command Language), is a free interpreted language designed specifically for scientific data processing and visualization. It is a product of the Computational & Information Systems Laboratory at the National Center for Atmospheric Research (NCAR) and sponsored by the National Science Foundation. NCL has robust file input and output: it can read and write netCDF-3, netCDF-4 classic, HDF4, binary, and ASCII data, and read HDF-EOS2, GRIB1, and GRIB2. The graphics are based on NCAR Graphics.

Availability and Restrictions

Versions

NCL/NCAR Graphics is available on Glenn and Oakley Cluster. The versions currently available at OSC are:

version glenn oakley ruby
5.0.0 X*     
6.0.0   X  
6.2.1    X  
6.3.0   X*  
* : Default Version

You can use module avail ncl  or  module avail ncarg to view available NCL/NCAR Graphics modules. Feel free to contact OSC Help if you need other versions for your work.

Access 

NCL/NCAR Graphics is available for use by all OSC users.

Usage

Usage on Glenn

Set-up on Glenn

To load the default version, use module load ncarg. To select a particular version, use   module load ncarg-version. For example, use module load ncarg-5.0.0 to load NCAR Graphics 5.0.0 on Glenn Cluster. 

Batch Usage on Glenn

Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations for Glenn and Scheduling Policies and Limits for more info. 
Interactive Batch Session
For an interactive batch session on Glenn, one can run the following command:
qsub -I -l nodes=1:ppn=8 -l walltime=1:00:00
which gives you 8 cores ( -l nodes=1:ppn=8 ) with 1 hour ( -l walltime=1:00:00 ). You may adjust the numbers per your need.
Non-interactive Batch Job (Serial Run)
batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. The following example batch script file will use the input file named agex06.f . 
Below is the example batch script ( job.txt) for a serial run:
#PBS -l walltime=1:00:00
#PBS -l nodes=1:ppn=8
#PBS -N jobname
#PBS -j oe
module load ncarg
cp agex06.f $TMPDIR
cd $TMPDIR
ncargex agex06
cp * $PBS_O_WORKDIR

In order to run it via the batch system, submit the job.txt  file with the following command:

qsub job.txt

Usage on Oakley

Set-up on Oakley

To load the default version, first load the cairo module, then use either  module load ncl or module load ncarg. To select a particular version, use   module load ncl/version or  module load ncarg/versionFor example, use  module load ncl/6.2.1 to load NCL version 6.2.1 on Oakley. For the default version of ncarg, use
module load cairo
module load ncarg

Batch Usage on Oakley

Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations for Oakley and Scheduling Policies and Limits for more info. 
Interactive Batch Session
For an interactive batch session on Oakley, one can run the following command:
qsub -I -l nodes=1:ppn=12 -l walltime=1:00:00
which gives you 12 cores ( -l nodes=1:ppn=12) with 1 hour ( -l walltime=1:00:00). You may adjust the numbers per your need.
Non-interactive Batch Job (Serial Run)
A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. The following example batch script file will use the input file named omega.ncl
Below is the example batch script job.txt for a serial run:
#PBS -l walltime=1:00:00
#PBS -l nodes=1:ppn=12
#PBS -N jobname
#PBS -j oe
module load ncl
cp omega.ncl $TMPDIR
cd $TMPDIR
ncl omega.ncl
cp * $PBS_O_WORKDIR

In order to run it via the batch system, submit the job.txt file with the following command:

qsub job.txt

Further Reading

Official documentation can be obtained from http://www.ncl.ucar.edu/overview.shtml 

Supercomputer: 
Service: 
Fields of Science: 

NWChem

NWChem aims to provide its users with computational chemistry tools that are scalable both in their ability to treat large scientific computational chemistry problems efficiently, and in their use of available parallel computing resources from high-performance parallel supercomputers to conventional workstation clusters.

Availability and Restrictions

The following versions of NWChem are available on OSC clusters:

Version Oakley
6.1 X*
*: Current default version

You can use  module avail nwchem   to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

NWChem is available to all OSC users without restriction.

Usage on Oakley

Set-up

To configure your environment for use of NWChem, run the following command:   module load nwchem  . The default version will be loaded. To select a particular NWChem version, use   module load nwchem/version  . For example, use  module load nwchem/6.1   to load NWChem 6.1.

Further Reading

Supercomputer: 
Service: 

Ncview

Ncview is a visual browser for netCDF format files. Typically you would use ncview to get a quick and easy, push-button look at your netCDF files. You can view simple movies of the data, view along various dimensions, take a look at the actual data values, change color maps, invert the data, etc.

Availability and Restrictions

Versions

The following versions of Ncview are available on OSC clusters:

VERSION Owens
2.1.7 X*
*: Default Version

You can use  module avail ncview  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Ncview is available to all OSC users without restriction.

Usage

Usage on Owens

Set-up

To configure your environment for use of Ncview, run the following command:  module load ncview  . The default version will be loaded. To select a particular Ncview version, use  module load ncview/version . For example, use module load ncview/2.1.7  to load Ncview 2.1.7.

Further Reading

Supercomputer: 
Service: 
Fields of Science: 

NetCDF

NetCDF (Network Common Data Form) is an interface for array-oriented data access and a library that provides an implementation of the interface. The netcdf library also defines a machine-independent format for representing scientific data. Together, the interface, library, and format support the creation, access, and sharing of scientific data.

Availability and Restrictions

NetCDF is available on Oakley, Ruby, and Owens Clusters. The versions currently available at OSC are:

Version Oakley Ruby owens notes
3.6.2        
4.1.3 X     Default version on Oakley prior to 09/15/2015
4.3.2   X    
4.3.3.1

X*

X* X*  
* : Current Default Version

You can use module spider netcdf  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

NetCDF is available without restriction to all OSC users.

Usage on Oakley

Set-up

Initalizing the system for use of the NetCDF is dependent on the system you are using and the compiler you are using. To load the default NetCDF, run the following command: module load netcdf . To use the parallel implementation of NetCDF, run the following command instead: module load pnetcdf . To load a particular version, use  module load netcdf/version . For example, use  module load netcdf/4.3.3.1  to load NetCDF version 4.3.3.1. You can use module spider netcdf  to view available modules.

Building With NetCDF

With the netcdf library loaded, the following environment variables will be available for use:

Variable Use
$NETCDF_CFLAGS Use during your compilation step for C or C++ programs.
$NETCDF_FFLAGS Use during your compilation step for Fortran programs.
$NETCDF_LIBS Use when linking your program to NetCDF.

 

Similarly, when the pnetcdf module is loaded, the following environment variables will be available:

VARIABLE USE
$PNETCDF_CFLAGS Use during your compilation step for C programs.
$PNETCDF_FFLAGS Use during your compilation step for Fortran programs.
$PNETCDF_LIBS Use when linking your program to NetCDF.
 

 

For example, to build the code myprog.c with the netcdf library you would use:

icc -c $NETCDF_CFLAGS myprog.c
icc -o myprog myprog.o $NETCDF_LIBS

Batch Usage

When you log into oakley.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info.  Batch jobs run on the compute nodes of the system and not on the login node. It is desirable for big problems since more resources can be used.

Non-interactive Batch Job (Serial Run)
A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. You must load the netcdf or pnetcdf module in your batch script before executing a program which is built with the netcdf library. Below is the example batch script that executes a program built with NetCDF:
#PBS -N AppNameJob
#PBS -l nodes=1:ppn=12
module load netcdf
cd $PBS_O_WORKDIR
cp foo.dat $TMPDIR
cd $TMPDIR
appname < foo.dat > foo.out
cp foo.out $PBS_O_WORKDIR

Usage on Ruby

Set-up

Initalizing the system for use of the NetCDF is dependent on the system you are using and the compiler you are using. To load the default NetCDF, run the following command: module load netcdf . To use the parallel implementation of NetCDF, run the following command instead: module load pnetcdf . To load a particular version, use  module load netcdf/version . For example, use  module load netcdf/4.3.3.1  to load NetCDF version 4.3.3.1. You can use module spider netcdf  to view available modules.

Building With NetCDF

With the netcdf library loaded, the following environment variables will be available for use:

Variable Use
$NETCDF_CFLAGS Use during your compilation step for C or C++ programs.
$NETCDF_FFLAGS Use during your compilation step for Fortran programs.
$NETCDF_LIBS Use when linking your program to NetCDF.

 

Similarly, when the pnetcdf module is loaded, the following environment variables will be available:

VARIABLE USE
$PNETCDF_CFLAGS Use during your compilation step for C programs.
$PNETCDF_FFLAGS Use during your compilation step for Fortran programs.
$PNETCDF_LIBS Use when linking your program to NetCDF.
 

 

For example, to build the code myprog.c with the netcdf library you would use:

icc -c $NETCDF_CFLAGS myprog.c
icc -o myprog myprog.o $NETCDF_LIBS

Batch Usage

When you log into ruby.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. 

Non-interactive Batch Job (Serial Run)
A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. You must load the netcdf or pnetcdf module in your batch script before executing a program which is built with the netcdf library. Below is the example batch script that executes a program built with NetCDF:
#PBS -N AppNameJob
#PBS -l nodes=1:ppn=20
module load netcdf
cd $PBS_O_WORKDIR
cp foo.dat $TMPDIR
cd $TMPDIR
appname < foo.dat > foo.out
cp foo.out $PBS_O_WORKDIR

Usage on Owens

Set-up

Initalizing the system for use of the NetCDF is dependent on the system you are using and the compiler you are using. To load the default NetCDF, run the following command: module load netcdf . To use the parallel implementation of NetCDF, run the following command instead: module load pnetcdf . To load a particular version, use  module load netcdf/version . For example, use  module load netcdf/4.3.3.1  to load NetCDF version 4.3.3.1. You can use module spider netcdf  to view available modules.

Building With NetCDF

With the netcdf library loaded, the following environment variables will be available for use:

Variable Use
$NETCDF_CFLAGS Use during your compilation step for C or C++ programs.
$NETCDF_FFLAGS Use during your compilation step for Fortran programs.
$NETCDF_LIBS Use when linking your program to NetCDF.

 

Similarly, when the pnetcdf module is loaded, the following environment variables will be available:

VARIABLE USE
$PNETCDF_CFLAGS Use during your compilation step for C programs.
$PNETCDF_FFLAGS Use during your compilation step for Fortran programs.
$PNETCDF_LIBS Use when linking your program to NetCDF.
 

 

For example, to build the code myprog.c with the netcdf library you would use:

icc -c $NETCDF_CFLAGS myprog.c
icc -o myprog myprog.o $NETCDF_LIBS

Batch Usage

When you log into owens.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. 

Non-interactive Batch Job (Serial Run)
A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. You must load the netcdf or pnetcdf module in your batch script before executing a program which is built with the netcdf library. Below is the example batch script that executes a program built with NetCDF:
#PBS -N AppNameJob
#PBS -l nodes=1:ppn=28
module load netcdf
cd $PBS_O_WORKDIR
cp foo.dat $TMPDIR
cd $TMPDIR
appname < foo.dat > foo.out
cp foo.out $PBS_O_WORKDIR

Further Reading

See Also

Tag: 
Supercomputer: 
Service: 

OSCusage

Introduction

OSCusage is command developed at OSC for use on OSC's systems.  It allows for a user to see information on their project's current RU balance, including which users and jobs incurred what charges.

Availability

Oakley ruby owens
X X X

 

Usage

OSCusage takes the following options and parameters.

usage: OSCusage [-qvn] -[r|h] [MM/DD/YYYY] [MM/DD/YYYY]
         or [-i]
Note: the year requires all 4 digits.
OSCusage can only provide information on a user's primary project, visible with the command id -gn.  If you are a part of other projects, visible under id -Gn, you will be unable to pull information on those projects.  Contact OSC Help for assistance in getting such information.

Today's Usage

Running OSCusage with no options or parameters specified will provide RU usage information per a user for the current day.

$ OSCusage

                     Usage Statistics for project PAS0435
                            for 5/3/2016 to 5/3/2016

                   PI: Dr. Judith Gardiner
                             RU Balance: -17.88635

        Username  Dates                            RUs  Status

          osu525    5/3/2016 to   5/3/2016     0.00000  ACTIVE
         *osu391    5/3/2016 to   5/3/2016     0.00000  ACTIVE
         osu6396    5/3/2016 to   5/3/2016     0.00000  ACTIVE
         osu6445    5/3/2016 to   5/3/2016     0.00000  PASSWORD
         osu8968    5/3/2016 to   5/3/2016     0.00000  ACTIVE
 ===============   PAS0435 TOTAL    0.00000

Users on Project

Specifying -i provides information on the accounts under the project.

$ OSCusage -i
                        Information for project PAS0435
                   PI: Dr. Judith Gardiner
                             RU Balance: -17.88635

            Username    Status  Full Name

             *osu391    ACTIVE  Dr. Judith Gardiner
             osu6396    ACTIVE  Dr. Judith Gardiner
             osu8968    ACTIVE  Koh Schooley

RU Usage in Timeframe

If you specify a timeframe you can get RU utilization information specifically for jobs that completed within that period.

$ OSCusage 01/01/2016 05/01/2016

                     Usage Statistics for project PAS0435
                          for 01/01/2016 to 05/01/2016

                   PI: Dr. Judith Gardiner
                             RU Balance: -17.88635

        Username  Dates                            RUs  Status

          osu525  01/01/2016 to 05/01/2016     0.00000  ACTIVE
         *osu391  01/01/2016 to 05/01/2016     0.00000  ACTIVE
         osu6396  01/01/2016 to 05/01/2016     0.00000  ACTIVE
         osu6445  01/01/2016 to 05/01/2016     0.00000  PASSWORD
         osu8968  01/01/2016 to 05/01/2016     0.15752  ACTIVE
 ===============   PAS0435 TOTAL    0.15752

Omit Users with No Utilization

Specify -q to omit information for accounts with no usage.

$ OSCusage -q 01/01/2016 05/01/2016

                     Usage Statistics for project PAS0435
                          for 01/01/2016 to 05/01/2016

                   PI: Dr. Judith Gardiner
                             RU Balance: -17.88635

        Username  Dates                            RUs  Status

         osu8968  01/01/2016 to 05/01/2016     0.15752  ACTIVE
 ===============   PAS0435 TOTAL    0.15752

Detailed Charges Breakdown

Specify -v to get detailed information on what job's incurred what charges.

$ OSCusage -v 01/01/2016 05/01/2016

                     Usage Statistics for project PAS0435
                          for 01/01/2016 to 05/01/2016

                   PI: Dr. Judith Gardiner
                             RU Balance: -17.88635

                        Start                 RUs     Charge           Status
        Username  Date  Date & Time          Used       Type     Queue           Job

 -------------            osu525 TOTAL    0.00000
 -------------           *osu391 TOTAL    0.00000
 -------------           osu6396 TOTAL    0.00000
 -------------           osu6445 TOTAL    0.00000
         osu8968 03/30  03/29 15:14:31    0.00277   CPU-Ruby    serial  E      STDIN
         osu8968 03/22  03/21 11:38:14    0.00005 CPU-Oakley    serial  E   login_en
         osu8968 03/22  03/21 11:48:25    0.00005 CPU-Oakley    serial  E   test_env
         osu8968 03/10  03/09 12:30:36    0.00066 CPU-Oakley    serial  E   test_pat
         osu8968 02/20  02/19 17:14:23    0.01266 CPU-Oakley  parallel  E   initial_
         osu8968 02/20  02/19 17:16:24    0.14133 CPU-Oakley  parallel  E   initial_
 -------------           osu8968 TOTAL    0.15752
 ===============   PAS0435 TOTAL    0.15752

View Usage in Hours

Specify -h to view the usage information in core hours, as opposed to RUs.

$ OSCusage -h 01/01/2016 05/01/2016

                     Usage Statistics for project PAS0435
                          for 01/01/2016 to 05/01/2016

                   PI: Dr. Judith Gardiner
                             RU Balance: -17.88635

                 Note: Listing is in HOURS, as opposed to RUs.

        Username  Dates                            HRS  Status

          osu525  01/01/2016 to 05/01/2016     0.00000  ACTIVE
         *osu391  01/01/2016 to 05/01/2016     0.00000  ACTIVE
         osu6396  01/01/2016 to 05/01/2016     0.00000  ACTIVE
         osu6445  01/01/2016 to 05/01/2016     0.00000  PASSWORD
         osu8968  01/01/2016 to 05/01/2016     1.57520  ACTIVE
 ===============   PAS0435 TOTAL    1.57520
Supercomputer: 
Service: 

Octave

Octave is a high-level language, primarily intended for numerical computations. It provides a convenient command line interface for solving linear and nonlinear problems numerically, and for performing other numerical experiments using a language that is mostly compatible with Matlab. It may also be used as a batch-oriented language.

Octave has extensive tools for solving common numerical linear algebra problems, finding the roots of nonlinear equations, integrating ordinary functions, manipulating polynomials, and integrating ordinary differential and differential-algebraic equations. It is easily extensible and customizable via user-defined functions written in Octave's own language, or using dynamically loaded modules written in C++, C, Fortran, or other languages.

Availability & Restrictions

Octave is available to all OSC users without restriction.

The following versions of Octave are available on OSC clusters:

Version Oakley Ruby Owens
3.6.1 X    
3.6.4 X    
4.0.0   X  
4.0.3     X

Usage

Set-up

To initialize either cluster for the use of Octave, run the following command:

module load octave

Using Octave

To run Octave, simply run the following command:

octave

Batch Usage

The following example batch script will an octave code file, mycode.o, via the batch processing system. The script requests one full node of cores on Oakley and 1 hour of walltime.

#PBS -N AppNameJob
#PBS -l nodes=1:ppn=12
#PBS -l walltime=01:00:00
#PBS -l software=appname

module load octave
cd $PBS_O_WORKDIR
cp mycode.o $TMPDIR
cd $TMPDIR

octave < mycode.o > data.out

cp data.out $PBS_O_WORKDIR

Further Reading

See Also

Supercomputer: 
Service: 

OpenACC

OpenACC is a standard for parallel programming on accelerators, such as Nvidia GPUs and Intel Phi. It consists primarily of a set of compiler directives for executing code on the accelerator, in C and Fortran. OpenACC is currently only supported by the PGI compilers installed on OSC systems.

Availability & Restrictions

OpenACC is available without restriction to all OSC users. It is supported by the PGI compilers.

Note: Some of the older compiler versions available on Glenn may not support OpenACC.

Usage

Set-up

OpenACC support is built into the compilers. There is no separate module to load.

Building With OpenACC

To build a program with OpenACC, use the compiler flag appropriate to your compiler. The correct libraries are included implicitly.

Compiler Family Flag
PGI -acc -ta=nvidia -Minfo=accel

Batch Usage

An OpenACC program will not run without an acelerator present. You need to ensure that your PBS resource request includes GPUs. For example, to run an OpenACC program on Oakley, your resource request should look something like this: #PBS -l nodes=1:ppn=12:gpus=2.

Further Reading

See Also

Supercomputer: 
Service: 
Fields of Science: 

OpenFOAM

OpenFOAM is a suite of computational fluid dynamics applications. It contains myriad solvers, both compressible and incompressible, as well as many utilities and libraries.

Availability and Restrictions

Versions

The following versions of OpenFOAM are available on OSC clusters:

version oakley Ruby Owens
2.2.2     X
2.3.0 X X  
2.4.0 X X  
3.0.0 X X  

 

Feel free to contact OSC Help if you need other versions for your work.

Access 

OpenFOAM is available to all OSC users without restriction.

Basic Structure for a OpenFOAM Case

The basic directory structure for a OpenFOAM case is shown as follows:

/home/yourusername/OpenFOAM_case
|-- 0
|-- U
|-- epsilon
|-- k
|-- p
`-- nut 
|-- constant
|-- RASProperties
|-- polyMesh
|   |-- blockMeshDict
|   `-- boundary
|-- transportProperties
`-- turbulenceProperties
|-- system
|-- controlDict
|-- fvSchemes
|-- fvSolution
`-- snappyHexMeshDict

IMPORTANT: To run in parallel, you need to also create the decomposeParDict file in the system directory. If you do not create this file, the decomposePar command will fail.

Usage

Usage on Oakley

Setup on Oakley

To configure the Oakley cluster for the use of OpenFOAM 1.7.x, use the following commands:

. /usr/local/OpenFOAM/OpenFOAM-1.7.x/etc/bashrc

To configure the Oakley cluster for the use of OpenFOAM 2.1.0, use the following commands:

. /usr/local/OpenFOAM/OpenFOAM-2.1.0/etc/bashrc

To configure the Oakley cluster for the use of OpenFOAM 2.3.0, use the following commands:

module load openfoam/2.3.0 # permissible with default Intel and MVAPICH2 compilers

To configure the Oakley cluster for the use of OpenFOAM 3.0.0, use the following commands:

module load openmpi/1.10 # currently only 3.0.0 is installed using OpenMPI libraries
module load openfoam/3.0.0

Batch Usage on Oakley

Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems.

On Oakley, refer to Queues and Reservations for Oakley and Scheduling Policies and Limits for more info. 

Interactive Batch Session

For an interactive batch session on Oakley, one can run the following command:

qsub -I -l nodes=1:ppn=12 -l walltime=1:00:00

which gives you 12 cores ( -l nodes=1:ppn=12) with 1 hour ( -l walltime=1:00:00). You may adjust the numbers per your need. 

Non-interactive Batch Job (Serial Run)

A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. Below is the example batch script ( job.txt) for a serial run, note that this would use the default installation of OpenFOAM 2.3.0 on Oakley:

#PBS -N serial_OpenFOAM 
#PBS -l nodes=1:ppn=1 
#PBS -l walltime=24:00:00 
#PBS -j oe 
#PBS -S /bin/bash 
# Initialize OpenFOAM on Oakley Cluster
# This only works if you are using default modules
module load openfoam
# Move to the case directory, where the 0, constant and system directories reside
cd $PBS_O_WORKDIR
# Copy files to $TMPDIR and move there to execute the program
cp * $TMPDIR
cd $TMPDIR
# Mesh the geometry
blockMesh
# Run the solver
icoFoam
# Finally, copy files back to your home directory
cp * $PBS_O_WORKDIR

In order to run it via the batch system, submit the job.txt   file with the following command:

qsub job.txt
Non-interactive Batch Job (Parallel Run)

Below is the example batch script ( job.txt) for a parallel run:

#PBS -N parallel_OpenFOAM
#PBS -l nodes=2:ppn=8
#PBS -l walltime=6:00:00
#PBS -j oe
#PBS -S /bin/bash 
# Initialize OpenFOAM on Oakley Cluster
# This only works if you are using default modules
module load openfoam
#Move to the case directory, where the 0, constant and system directories reside
cd $PBS_O_WORKDIR
#Mesh the geometry
blockMesh
#Decompose the mesh for parallel run
decomposePar
#Run the solver
mpiexec simpleFoam -parallel 
#Reconstruct the parallel results
reconstructPar

Usage on Ruby

Setup on Ruby

To configure the Ruby cluster for the use of OpenFOAM 2.3.0, use the following commands:
module load openfoam/2.3.0 # permissible with default Intel and MVAPICH2 compilers

To configure the Ruby cluster for the use of OpenFOAM 3.0.0, use the following commands:

module load openmpi/1.10 # currently only 3.0.0 is installed using OpenMPI libraries
module load openfoam/3.0.0

Batch Usage on Ruby

Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems.

On Oakley, refer to Queues and Reservations for Ruby and Scheduling Policies and Limits for more info. 

Interactive Batch Session

For an interactive batch session on Oakley, one can run the following command:

qsub -I -l nodes=1:ppn=20 -l walltime=1:00:00

which gives you 20 cores ( -l nodes=1:ppn=20 ) with 1 hour ( -l walltime=1:00:00 ). You may adjust the numbers per your need. 

Non-interactive Batch Job (Serial Run)

A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. Below is the example batch script ( job.txt ) for a serial run:

#PBS -N serial_OpenFOAM 
#PBS -l nodes=1:ppn=1 
#PBS -l walltime=24:00:00 
#PBS -j oe 
#PBS -S /bin/bash 
# Initialize OpenFOAM on Ruby Cluster
# This only works if you are using default modules
module load openfoam
# Move to the case directory, where the 0, constant and system directories reside
cd $PBS_O_WORKDIR
# Copy files to $TMPDIR and move there to execute the program
cp * $TMPDIR
cd $TMPDIR
# Mesh the geometry
blockMesh
# Run the solver
icoFoam
# Finally, copy files back to your home directory
cp * $PBS_O_WORKDIR

In order to run it via the batch system, submit the job.txt  file with the following command:

qsub job.txt
Non-interactive Batch Job (Parallel Run)

Below is the example batch script ( job.txt) for a parallel run:

#PBS -N parallel_OpenFOAM
#PBS -l nodes=2:ppn=8
#PBS -l walltime=6:00:00
#PBS -j oe
#PBS -S /bin/bash 
# Initialize OpenFOAM on Ruby Cluster
# This only works if you are using default modules
module load openfoam
# Move to the case directory, where the 0, constant and system directories reside
cd $PBS_O_WORKDIR
# Mesh the geometry
blockMesh
# Decompose the mesh for parallel run
decomposePar
# Run the solver
mpiexec simpleFoam -parallel 
# Reconstruct the parallel results
reconstructPar

Further Reading

The OpenFOAM home page

Supercomputer: 
Service: 
Fields of Science: 

OpenMP

OpenMP is a standard for parallel programming on shared-memory systems, including multicore systems. It consists primarily of a set of compiler directives for sharing work among multiple threads. OpenMP is supported by all the Fortran, C, and C++ compilers installed on OSC systems.

Availability & Restrictions

OpenMP is available without restriction to all OSC users. It is supported by the Intel, PGI, and gnu compilers.

Note: Some of the older compiler versions available on Glenn may not support OpenMP.

Usage

Set-up

OpenMP support is built into the compilers. There is no separate module to load.

Building With OpenMP

To build a program with OpenMP, use the compiler flag appropriate to your compiler. The correct libraries are included implicitly.

Compiler Family Flag
Intel -qopenmp or  -openmp
gnu -fopenmp
PGI -mp

Batch Usage

An OpenMP program by default will use a number of threads equal to the number of processor cores available. To use a different number of threads, set the environment variable OMP_NUM_THREADS.

Further Reading

See Also

Service: 
Fields of Science: 

OpenMPI

MPI is a standard library for performing parallel processing using a distributed-memory model. The Oakley, Ruby, and Owens clusters at OSC can use the OpenMPI implementation of the Message Passing Interface (MPI), which is based on MPICH.

Availability & Restrictions

OpenMPI is available without restriction to all OSC users.

Installations are available for the Intel, PGI, and GNU compilers.

The following versions of OpenMPI are available on OSC systems:

Version CompilerS Supported Oakley Ruby Owens
1.10       gnu/4.8.4
      intel/15.0.3
      pgi/15.4.0
X    
1.10       gnu/4.4.7
      gnu/4.8.4
      gnu/4.9.1
      intel/14.0.3
      intel/15.0.0
      intel/15.0.1
      intel/15.0.3
      pgi/13.5.0
      pgi/14.9.0
      pgi/15.4.0
  X  
1.10
2.0
      gnu/4.8.5
      intel/16.0.3
      pgi/16.5.0
    X
         
         
         
         
         

Usage

Setup on Oakley or Ruby

To set up your environment for using the MPI libraries, you must load the appropriate module. On any OSC system, this is performed by:

module load openmpi

You will get the default version for the compiler you have loaded. Note that only the default compilers are supported on Oakley, whereas there is more compiler cross-support for OpenMPI on Ruby.  On Owens, there is a choice of 1.10 and 2.0, but currently we recommend the default version which uses an optimizations specific to using Mellanox infiniband.

Building With MPI

To build a program that uses MPI, you should use the compiler wrappers provided on the system. They accept the same options as the underlying compiler. The commands are shown in the following table.

C mpicc
C++ mpicxx
FORTRAN 77 mpif77
Fortran 90 mpif90

For example, to build the code my_prog.c using the -O2 option, you would use:

mpicc -o my_prog -O2 my_prog.c

In rare cases, you may be unable to use the wrappers. In that case you should use the environment variables set by the module.

Variable Use
$MPI_CFLAGS Use during your compilation step for C programs.
$MPI_CXXFLAGS Use during your compilation step for C++ programs.
$MPI_FFLAGS Use during your compilation step for Fortran 77 programs.
$MPI_F90FLAGS Use during your compilation step for Fortran 90 programs.

Batch Usage

Programs built with MPI can only be run in the batch environment at OSC. For information on starting MPI programs using the mpiexec command, see Batch Processing at OSC.

Be sure to load the same compiler and OpenMPI modules at execution time as at build time.

Further Reading

See Also

Supercomputer: 
Service: 
Technologies: 
Fields of Science: 

PAML

"PAML (for Phylogentic Analysis by Maximum Likelihood) contains a few programs for model fitting and phylogenetic tree reconstruction using nucleotide or amino-acid sequence data." (doc/pamlDOC.pdf)

Availability & Restrictions

PAML is available to all OSC users without restriction.

The following versions of PAML are available on OSC systems:

Version Glenn Oakley
4.4d X  

Usage

Set-up

On the Glenn Cluster paml is accessed by executing the following commands:

module load biosoftw
module load paml

Using PAML

PAML is a collection of several programs that will be added to the users PATH: baseml, basemlg, chi2, codeml, ds, evolver, mcmctree, pamp, and yn00.  Each of the programs has separate, but typically similar usage and options.

Options

  • baseml / basemlg   Maximum likelihood analysis of nucleotide sequences using a faster discrete model / Implements the (continuous) gamma model of Yang (Intensive Computation)

Both baseml and basemlg require a baseml.ctl in the current directory with the following variables set: seqfile, outfile, treefile

The following are optional variable to set in baseml.ctl: noisy, verbose, runmode, model, Mgene, ndata, clock, fix_kappa, kappa, fix_alpha, alpha, Malpha, ncatG, fix_rho,nparK, nhomo, getSE, RateAncestor, Small_Diff, cleandata, icode, fix_blength, method

chi2   Calculates the x2 critical value and p value for conducting the likelihood ratio test
chi2 [p | INTEGER DOUBLE]
chi2                   prints x2 critical values at set significance levels until ‘q+ENTER’ is reached
chi2 p                 interactive set the degrees of freedom and x2 value
chi2 INTEGER DOUBLE    Computes the probability for INTEGER df and DOUBLE x2

 

  • codeml   Implements the codon substitution model of Goldman & Yang for DNA and amino acid sequences

codeml requires codeml.ctl to be located in the current directory with the following variables set: seqfile, outfile, treefile, aaRatefile

The following are optional variables to set in codeml.ctl: noisy, verbose, runmode, seqtype, CodonFreq, ndata, aaDist, model, NSsites, icode, Mgene, fix_kappa, kappa, fix_omega, omega, fix_alpha, alpha, Malpha, ncatG, getSE, RateAncestor, Small_Diff, cleandata, fix_blength, method

  • ds   Computes descriptive statistics from a baseml/basemlg analysis

ds filename.type

  • evolver   Simulates sequences under nucleotide, codon, and amino acid substitution models; generates random trees; and calculates the partition distances between trees

EVOLVER in paml version 4.4d, March 2011
Results for options 1-4 & 8 go into evolver.out
Options
      (1) Get random UNROOTED trees?
      (2) Get random ROOTED trees?
      (3) List all UNROOTED trees?
      (4) List all ROOTED trees?
      (5) Simulate nucleotide data sets (use MCbase.dat)?
      (6) Simulate codon data sets      (use MCcodon.dat)?
      (7) Simulate amino acid data sets (use MCaa.dat)?
      (8) Calculate identical bi-partitions between trees?
      (9) Calculate clade support values (read 2 treefiles)?
      (11) Label clades?
      (0) Quit?

evolver’s option 5 requires MCbase.dat.  evolver’s option 6 requires MCcodon.dat.  evolver’s option 7 requires MCaa.dat and dat/mtmam.dat.  evolver’s option 9 requires truetree rst1 (formed from stewart.trees & codeml's output rst1).  evolver’s option 11 requires name.tress with user input.

  • mcmctree   Implements the Bayesian MCMC algorithm of Yang and Rannala for estimating species divergence times

mcmctree requires mcmctree.ctl to be located in the current directory with the following variables set: seqfile, treefile, outfile, RootAge, usedata
The following are optional variables to set in mcmctree.ctl: seed, ndata, clock, model, alpha, ncatG, cleandata, BDparas, kappa_gamma, alpha_gamma, rgene_gamma, sigma2_gamma, finetune, print, burnin, sampfreq, nsample

  • pamp   Implements the parsimony-based analysis of Yang and Kumar

pamp requires pamp.ctl to be located in the current directory with the following variables set: seqfile, treefile, outfile
The following are optional variables to set in pamp.ctl: seqtype, ncatG, nhomo

  • yn00   Implements the method of Yang and Nielson for estimating synonymous and nonsynonymous substitution rates in pairwise comparisons of protein-coding DNA sequences

yn00 requires yn00.ctl to be located in the current directory with the following variables set: seqfile, outfile
The following are optional variables to set in yn00.ctl: verbose, icode, weighting, commonf3x4, ndata

Control Files

All .ctl files (baseml.ctl, codeml.ctl, mcmctree.ctl, pamp.ctl, and yn00.ctl) have comment line starting with '*'.

Batch Usage

#PBS -N paml_test
#PBS -l walltime=0:05:00
#PBS -l nodes=1:ppn=4

cd $PBS_O_WORKDIR
module load biosoftw
module load paml
export PAML_DIR=/usr/local/biosoftw/paml44
cp $PAML_DIR/*.* .
cp -r $PAML_DIR/dat .
cp -r $PAML_DIR/examples .
baseml
chi2 1 3.84
codeml
ds in.baseml
echo -e "1\n5\n5 5\n0\n2\n5\n5 5\n0\n3\n5\n4\n5\n5\n6\n7\n8\n" | evolver"
mcmctree
pamp
yn00

Further Reading

  • Four pdf documents are located in the following folder on Glenn:  /usr/local/biosoftw/paml44/doc/
  • An online discussion group for users is paml is located at the following website: http://www.rannala.org/phpBB2/
Supercomputer: 
Service: 

PAPI

PAPI provides the tool designer and application engineer with a consistent interface and methodology for use of the performance counter hardware found in most major microprocessors. PAPI enables software engineers to see, in near real time, the relation between software performance and processor events.

This software will be of interest only to HPC experts.

Availability and Restrictions

PAPI is available on Oakley, Ruby, and Owens Clusters. The versions currently available at OSC are:

Version Oakley Ruby Owens Notes
5.2.0     System Install* No Module Needed
5.4.1 X* X*    
* : Current Default Version

You can use module spider papi to view available modules for a given machine. For now, PAPI is available only with the Intel and gnu compilers. Feel free to contact OSC Help if you need other versions for your work.

Access

PAPI is available without restriction to all OSC users.

Usage on Oakley

Set-up

To load the default PAPI library, run the following command: module load papi . To load a particular version, use  module load papi/version . For example, use  module load papi/5.4.1  to load PAPI version 5.4.1. You can use module spider papi  to view available modules.

Building With PAPI

The PAPI library provides the following variables for use at build time:

Variable Use
$PAPI_CFLAGS Use during your compilation step for C/C++ programs
$PAPI_FFLAGS Use during your compilation step for FORTRAN programs 
$PAPI_LIB Use during your linking step programs

For example, to build the code myprog.c with the PAPI library you would use:

module load papi
gcc -c myprog.c $PAPI_CFLAGS
gcc -o myprog myprog.o $PAPI_LIB

Batch Usage

When you log into oakley.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info.  Batch jobs run on the compute nodes of the system and not on the login node. It is desirable for big problems since more resources can be used.

Usage on Ruby

Set-up

To load the default PAPI library, run the following command: module load papi . To load a particular version, use  module load papi/version . For example, use  module load papi/5.4.1  to load PAPI version 5.4.1. You can use module spider papi  to view available modules.

Building With PAPI

The PAPI library provides the following variables for use at build time:

Variable Use
$PAPI_CFLAGS Use during your compilation step for C/C++ programs
$PAPI_FFLAGS Use during your compilation step for FORTRAN programs 
$PAPI_LIB Use during your linking step programs

For example, to build the code myprog.c with the PAPI library you would use:

module load papi
gcc -c myprog.c $PAPI_CFLAGS
gcc -o myprog myprog.o $PAPI_LIB

Batch Usage

When you log into ruby.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. 

Usage on Owens

Set-up

Since PAPI is a System Install, no module is needed to run the application.

Building With PAPI

To build the code myprog.c with the PAPI library you would use:

gcc -c myprog.c -lpapi
gcc -o myprog myprog.o -lpapi

Batch Usage

When you log into owens.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. 

Further Reading

Supercomputer: 

PAUP

PAUP is a leading program for performing phylogenetic analysis for bioinformatics sequences. PAUP currently runs as a single processor program. No further enhancements are suggested.

Availability & Restrictions

PAUP is available to all OSC users without restriction.

The following versions of PAUP are available on OSC systems:

Version Glenn Oakley
4b10 X X

Usage

Set-up

To configure your environment for using PAUP, run the following command:

module load paup

Using PAUP

After loading the PAUP module, PAUP can be run with commands similar to the following:

paup nexus_file > nexus_file.out

Batch Usage

PAUP is best run via the batch processing system. The following example batch script file will use the input file nexus_file and the output file nexus_file.out

#PBS -l walltime=10:00:00
#PBS -l nodes=1:ppn=1
#PBS -N paup
#PBS -j oe

cd $PBS_O_WORKDIR
cp ./nexus_file $TMPDIR
cd $TMPDIR

module load paup
paup nexus_file > nexus_file.out
cp * $PBS_O_WORKDIR

Further Reading

  • The PAUP home page
  • PDF forms of the documentation are located at /usr/local/paup/paup4b10-opt-linux-a/Docs/ on Glenn and /usr/local/paup/4b10/Docs/ on Oakley
    • Cmd_ref_v2.pdf – command reference manual
    • Quick_start_v1.pdf – quick start guide for the command line version

See Also

  • Mrbayes
Supercomputer: 
Service: 
Fields of Science: 

PGI Compilers

Fortran, C, and C++ compilers from the Portland Group

Availability and Restrictions

PGI compiler is available on Oakley, Ruby, and Owens Clusters. The versions currently available at OSC are:

Version Oakley ruby Owens notes
12.10 X     Default version on Oakley prior to 09/15/2015
13.5.0   X    
14.9.0   X   Default version on Ruby prior to 09/15/2015
15.4.0 X* X*    
16.5.0     X*  
* : Current Default Version

You can use module avail pgi  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

The PGI Compilers are available to all OSC users without restriction.

Usage on Oakley

Set-up

To configure your environment for the default version of the PGI Compilers, use  module load pgi . To configure your environment for a particular PGI compiler version, use  module load pgi/version . For example, use  module load pgi/12.10  to load PGI compiler version 12.10.

Using the PGI Compilers

Once the module is loaded, compiling with the PGI compilers requires understanding which binary should be used for which type of code. Specifically, use the pgcc binary for C codes, the pgCC binary for C++ codes, the pgf77 for FORTRAN-77 codes and the pgf90 for FORTRAN-90 codes.

See our compilation guide for a more detailed breakdown of the compilers.

Building with the PGI Compilers

The PGI compilers recognize the following command line options (this list is not exhaustive, for more information run man <compiler binary name> ):

Compiler option Purpose
-c Compile into object code only; do not link
-DMACRO[=value] Defines preprocessor macro MACRO with optional value (default value is 1)
-g Enables debugging; disables optimization
-I/directory/name Add /directory/name to the list of directories to be searched for #include files
-L/directory/name Adds /directory/name to the list of directories to be searched for library files
-lname Adds the library libname.a or libname.so to the list of libraries to be linked
-o outfile Names the resulting executable outfile instead of a.out
-UMACRO Removes definition of MACRO from preprocessor
-O0 Disable optimization; default if -g is specified
-O1 Light optimization; default if -g is not specified
-O or -O2 Heavy optimization
-O3 Aggressive optimization; may change numerical results
-Mipa Inline function expansion for calls to procedures defined in separate files; implies -O2
-Munroll Loop unrolling; implies -O2
-Mconcur Automatic parallelization; implies -O2
-mp Enables translation of OpenMP directives
 

Usage on Ruby

Set-up

To configure your environment for the default version of the PGI Compilers, use  module load pgi . To configure your environment for a particular PGI compiler version, use  module load pgi/version . For example, use  module load pgi/14.9.0  to load PGI compiler version 14.9.

Using the PGI Compilers

Once the module is loaded, compiling with the PGI compilers requires understanding which binary should be used for which type of code. Specifically, use the pgcc binary for C codes, the pgCC binary for C++ codes, the pgf77 for FORTRAN-77 codes and the pgf90 for FORTRAN-90 codes.

See our compilation guide for a more detailed breakdown of the compilers.

Building with the PGI Compilers

The PGI compilers recognize the following command line options (this list is not exhaustive, for more information run man <compiler binary name> ):

Compiler option Purpose
-c Compile into object code only; do not link
-DMACRO[=value] Defines preprocessor macro MACRO with optional value (default value is 1)
-g Enables debugging; disables optimization
-I/directory/name Add /directory/name to the list of directories to be searched for #include files
-L/directory/name Adds /directory/name to the list of directories to be searched for library files
-lname Adds the library libname.a or libname.so to the list of libraries to be linked
-o outfile Names the resulting executable outfile instead of a.out
-UMACRO Removes definition of MACRO from preprocessor
-O0 Disable optimization; default if -g is specified
-O1 Light optimization; default if -g is not specified
-O or -O2 Heavy optimization
-O3 Aggressive optimization; may change numerical results
-Mipa Inline function expansion for calls to procedures defined in separate files; implies -O2
-Munroll Loop unrolling; implies -O2
-Mconcur Automatic parallelization; implies -O2
-mp Enables translation of OpenMP directives
 

Usage on Owens

Set-up

To configure your environment for the default version of the PGI Compilers, use  module load pgi . To configure your environment for a particular PGI compiler version, use  module load pgi/version . For example, use module load pgi/16.5.0  to load PGI compiler version 16.5.0.

Using the PGI Compilers

Once the module is loaded, compiling with the PGI compilers requires understanding which binary should be used for which type of code. Specifically, use the pgcc binary for C codes, the pgc++ binary for C++ codes, the pgf77 for FORTRAN-77 codes and the pgf90 for FORTRAN-90 codes.

See our compilation guide for a more detailed breakdown of the compilers.

Building with the PGI Compilers

The PGI compilers recognize the following command line options (this list is not exhaustive, for more information runman <compiler binary name> ):

COMPILER OPTION PURPOSE
-c Compile into object code only; do not link
-DMACRO[=value] Defines preprocessor macro MACRO with optional value (default value is 1)
-g Enables debugging; disables optimization
-I/directory/name Add /directory/name to the list of directories to be searched for #include files
-L/directory/name Adds /directory/name to the list of directories to be searched for library files
-lname Adds the library libname.a or libname.so to the list of libraries to be linked
-o outfile Names the resulting executable outfile instead of a.out
-UMACRO Removes definition of MACRO from preprocessor
-O0 Disable optimization; default if -g is specified
-O1 Light optimization; default if -g is not specified
-O or -O2 Heavy optimization
-O3 Aggressive optimization; may change numerical results
-Mipa Inline function expansion for calls to procedures defined in separate files; implies -O2
-Munroll Loop unrolling; implies -O2
-Mconcur Automatic parallelization; implies -O2
-mp Enables translation of OpenMP directives

Further Reading

See Also

Supercomputer: 
Service: 
Technologies: 
Fields of Science: 

PNG

PNG (Portable Network Graphics) is an extensible file format for the lossless, portable, well-compressed storage of raster images. PNG provides a patent-free replacement for GIF and can also replace many common uses of TIFF. Indexed-color, grayscale, and truecolor images are supported, plus an optional alpha channel. Sample depths range from 1 to 16 bits.

Availability and Restrictions

Versions

The following versions of PNG are available on OSC clusters:

VERSION OAKLEY
1.2.52 X
1.5.9 X*
*: Default Version

You can use  module avail png  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

PNG is available to all OSC users without restriction.

Usage

Usage on Oakley

Set-up on Oakley

To configure your environment for use of PNG on Oakley, run the following command:  module load png . The default version will be loaded. To select a particular PNG version, use  module load png/version . For example, use module load png/1.2.52 to load PNG 1.2.52 on Oakley.

Further Reading

Supercomputer: 
Service: 
Fields of Science: 

ParMETIS / METIS

ParMETIS (Parallel Graph Partitioning and Fill-reducing Matrix Ordering) is an MPI-based parallel library that implements a variety of algorithms for partitioning unstructured graphs, meshes, and for computing fill-reducing orderings of sparse matrices. ParMETIS extends the functionality provided by METIS and includes routines that are especially suited for parallel AMR computations and large scale numerical simulations. The algorithms implemented in ParMETIS are based on the parallel multilevel k-way graph-partitioning, adaptive repartitioning, and parallel multi-constrained partitioning schemes developed in Karypis lab.

METIS (Serial Graph Partitioning and Fill-reducing Matrix Ordering) is a set of serial programs for partitioning graphs, partitioning finite element meshes, and producing fill reducing orderings for sparse matrices. The algorithms implemented in METIS are based on the multilevel recursive-bisection, multilevel k-way, and multi-constraint partitioning schemes developed in Karypis lab.

Availability and Restrictions

ParMETIS is available on Oakley, Ruby, and Owens Clusters. The versions currently available at OSC are:

Version Oakley Ruby owens
4.0.3 X* X* X*
* : Current Default Version                                                        X: available for gnu and intel compilers

METIS is available on Oakley, Ruby, and Owens Clusters. The versions currently available at OSC are:

Version Oakley Ruby Owens
5.1.0 X* X* X*
* : Current Default Version                                                       X: available for gnu, intel, and pgi compilers

You can use module -r spider '.*metis.*'  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

ParMETIS / METIS is available without restriction to all OSC users.

Usage on Oakley

Set-up

To load ParMETIS, run the following command: module load parmetis . To use the serial implementation, METIS, run the following command instead: module load metis . You can use module spider metis   and module spider parmetis to view available modules. Use module spider metis/version and module spider parmetis/version to check what modules should be loaded before load ParMETIS / METIS.

Building With ParMETIS / METIS

With the ParMETIS library loaded, the following environment variables will be available for use:

Variable Use
$PARMETIS_CFLAGS Use during your compilation step for C or C++ programs.
$PARMETIS_LIBS

Use when linking your program to ParMETIS.

 

Similarly, when the METIS module is loaded, the following environment variables will be available:

VARIABLE USE
$METIS_CFLAGS Use during your compilation step for C programs.
$METIS_LIBS Use when linking your program to METIS.

 

For example, to build the code myprog.cc with the METIS library you would use:

g++ -o myprog myprog.cc $METIS_LIBS

Batch Usage

When you log into oakley.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info.  Batch jobs run on the compute nodes of the system and not on the login node. It is desirable for big problems since more resources can be used.

Non-interactive Batch Job (Serial Run)
A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. You must load the ParMETIS / METIS module in your batch script before executing a program which is built with the ParMETIS / METIS library. Below is the example batch script that executes a program built with ParMETIS:
#PBS -N myprogJob
#PBS -l nodes=1:ppn=12
module load gnu/4.8.4
module load parmetis
cd $PBS_O_WORKDIR
cp foo.dat $TMPDIR
cd $TMPDIR
myprog < foo.dat > foo.out
cp foo.out $PBS_O_WORKDIR

Usage on Ruby

Set-up

To load ParMETIS, run the following command: module load parmetis . To use the serial implementation, METIS, run the following command instead: module load metis . You can use module spider metis   and module spider parmetis to view available modules. Use module spider metis/version and module spider parmetis/version to check what modules should be loaded before load ParMETIS / METIS.

Building With ParMETIS / METIS

With the ParMETIS library loaded, the following environment variables will be available for use:

Variable Use
$PARMETIS_CFLAGS Use during your compilation step for C or C++ programs.
$PARMETIS_LIBS

Use when linking your program to ParMETIS.

 

Similarly, when the METIS module is loaded, the following environment variables will be available:

VARIABLE USE
$METIS_CFLAGS Use during your compilation step for C programs.
$METIS_LIBS Use when linking your program to METIS.

 

For example, to build the code myprog.cc with the METIS library you would use:

g++ -o myprog myprog.cc $METIS_LIBS

Batch Usage

When you log into ruby.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info.  Batch jobs run on the compute nodes of the system and not on the login node. It is desirable for big problems since more resources can be used.

Non-interactive Batch Job (Serial Run)
A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. You must load the ParMETIS / METIS module in your batch script before executing a program which is built with the ParMETIS / METIS library. Below is the example batch script that executes a program built with ParMETIS:
#PBS -N myprogJob
#PBS -l nodes=1:ppn=20
module load gnu/4.8.4
module load parmetis
cd $PBS_O_WORKDIR
cp foo.dat $TMPDIR
cd $TMPDIR
myprog < foo.dat > foo.out
cp foo.out $PBS_O_WORKDIR

Usage on Owens

Set-up

To load ParMETIS, run the following command: module load parmetis . To use the serial implementation, METIS, run the following command instead: module load metis . You can use module spider metis   and module spider parmetis to view available modules. Use module spider metis/version and module spider parmetis/version to check what modules should be loaded before load ParMETIS / METIS.

Building With ParMETIS / METIS

With the ParMETIS library loaded, the following environment variables will be available for use:

Variable Use
$PARMETIS_CFLAGS Use during your compilation step for C or C++ programs.
$PARMETIS_LIBS

Use when linking your program to ParMETIS.

 

Similarly, when the METIS module is loaded, the following environment variables will be available:

VARIABLE USE
$METIS_CFLAGS Use during your compilation step for C programs.
$METIS_LIBS Use when linking your program to METIS.

 

For example, to build the code myprog.cc with the METIS library you would use:

g++ -o myprog myprog.cc $METIS_LIBS

Batch Usage

When you log into owens.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. Batch jobs run on the compute nodes of the system and not on the login node. It is desirable for big problems since more resources can be used.

Non-interactive Batch Job (Serial Run)
A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. You must load the ParMETIS / METIS module in your batch script before executing a program which is built with the ParMETIS / METIS library. Below is the example batch script that executes a program built with ParMETIS:
#PBS -N myprogJob
#PBS -l nodes=1:ppn=28
module load gnu/4.8.5
module load parmetis
cd $PBS_O_WORKDIR
cp foo.dat $TMPDIR
cd $TMPDIR
myprog < foo.dat > foo.out
cp foo.out $PBS_O_WORKDIR

Further Reading

Tag: 
Supercomputer: 
Service: 

ParaView

ParaView is an open-source, multi-platform application designed to visualize data sets of size varying from small to very large. ParaView was developed to support distributed computational models for processing large data sets and to create an open, flexible user interface.

Availability and Restrictions

ParaView is available on Oakley, Ruby, and Owens Clusters. The versions currently available at OSC are:

VERSION Oakley Ruby Owens NOTEs
3.3.0        
3.8.0        
3.14.1 X*      
4.3.1   X*    
4.4.0 X   X*  
*: Current default version

You can use module avail paraview  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

ParaView is available for use by all OSC users.

Usage

Usage on Oakley

Set-up

To load the default version of ParaView module, use  module load paraview . To select a particular software version, use   module load paraview/version . For example, use  module load paraview/4.4.0  to load ParaView version 4.4.0. Following a successful loading of the ParaView module, you can access the ParaView program:
paraview

Usage on Ruby

Set-up

To load the default version of ParaView module, use  module load paraview . To select a particular software version, use   module load paraview/version . For example, use  module load paraview/4.3.1  to load ParaView version 4.3.1. Following a successful loading of the ParaView module, you can access the ParaView program:
paraview

Usage on Owens

Set-up

To load the default version of ParaView module, use  module load paraview . To select a particular software version, use   module load paraview/version . For example, use  module load paraview/4.4.0  to load ParaView version 4.4.0. Following a successful loading of the ParaView module, you can access the ParaView program:
paraview

Documentation

ParaView documentation is available on-line at http://www.paraview.org/paraview-guide/.

Supercomputer: 
Service: 
Fields of Science: 

Picard

Picard is a set of command line tools for manipulating high-throughput sequencing (HTS) data and formats such as SAM/BAM/CRAM and VCF.

Availability and Restrictions

The following versions of Picard are available on OSC clusters:

Version Oakley Owens
2.3.0 X* X*
*: Current default version

You can use  module avail picard  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Picard is available to all OSC users without restriction.

Usage on Oakley

Set-up

To configure your environment for use of Picard, run the following command:  module load picard . The default version will be loaded. To select a particular Picard version, use   module load picard/version . For example, use  module load picard/2.3.0.  to load Picard 2.3.0.

Usage

This software is a Java executable .jar file; thus, it is not possible to add to the PATH environment variable.

From module load picard , a new environment variable, PICARD, will be set.

Thus, users can use the software by running the following command:  java -jar $PICARD {other options}

Usage on Owens

Set-up

To configure your environment for use of Picard, run the following command:  module load picard . The default version will be loaded. To select a particular Picard version, use   module load picard/version . For example, use module load picard/2.3.0.  to load Picard 2.3.0.

Usage

This software is a Java executable .jar file; thus, it is not possible to add to the PATH environment variable.

From module load picard , a new environment variable, PICARD, will be set.

Thus, users can use the software by running the following command:  java -jar $PICARD {other options}

Further Reading

Supercomputer: 
Service: 
Fields of Science: 

Pixman

Pixman is a low-level software library for pixel manipulation, providing features such as image compositing and trapezoid rasterization. Important users of pixman are the cairo graphics library and the X server. Pixman is implemented as a library in the C programming language. It runs on many platforms, including Linux, BSD Derivatives, MacOS X, and Windows.

Availability and Restrictions

Versions

The following versions of Pixman are available on OSC clusters:

VERSION OAKLEY
0.32.6 X*
*: Default Version

You can use  module avail pixman  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Pixman is available to all OSC users without restriction.

Usage

Usage on Oakley

Set-up on Oakley

To configure your environment for use of Pixman on Oakley, run the following command:  module load pixman . The default version will be loaded. To select a particular Pixman version, use  module load pixman/version . For example, use module load pixman/0.32.6   to load Pixman 0.32.6 on Oakley.

Further Reading

See Also

Supercomputer: 
Service: 
Fields of Science: 

PnetCDF

PnetCDF is a library providing high-performance parallel I/O while still maintaining file-format compatibility with  Unidata's NetCDF, specifically the formats of CDF-1 and CDF-2. Although NetCDF supports parallel I/O starting from version 4, the files must be in HDF5 format. PnetCDF is currently the only choice for carrying out parallel I/O on files that are in classic formats (CDF-1 and 2). In addition, PnetCDF supports the CDF-5 file format, an extension of CDF-2, that supports more data types and allows users to define large dimensions, attributes, and variables (>2B elements).

Availability & Restrictions

PnetCDF is available without restriction to all OSC users.

The following versions of PnetCDF are available at OSC:

Version Ruby Owens
1.5.0 X*  
1.7.0   X*

Usage on Ruby

Set-up

To initalize the system prior to using PnetCDF, run the following comand:

module load pnetcdf

Building With PnetDCF

With the PnetCDF module loaded, the following environment variables will be available for use:

Variable Use
$PNETCDF_CFLAGS Use during your compilation step for C or C++ programs.
$PNETCDF_FFLAGS Use during your compilation step for Fortran programs.
$PNETCDF_LIBS

Use when linking your program to PnetCDF.

$PNETCDF Path to the PnetCDF installation directory

 

For example, to build the code myprog.c with the pnetcdf library you would use:

mpicc -c $PNETCDF_CFLAGS myprog.c
mpicc -o myprog myprog.o $PNETCDF_LIBS

Batch Usage

#PBS -N AppNameJob
#PBS -l nodes=1:ppn=20
cd $PBS_O_WORKDIR
mpiexec ./myprog

Usage on Owens

Set-up

To initalize the system prior to using PnetCDF, run the following comand:

module load pnetcdf

Building With PnetDCF

With the PnetCDF module loaded, the following environment variables will be available for use:

VARIABLE USE
$PNETCDF_CFLAGS Use during your compilation step for C or C++ programs.
$PNETCDF_FFLAGS Use during your compilation step for Fortran programs.
$PNETCDF_LIBS

Use when linking your program to PnetCDF.

$PNETCDF Path to the PnetCDF installation directory

 

For example, to build the code myprog.c with the pnetcdf library you would use:

mpicc -c $PNETCDF_CFLAGS myprog.c
mpicc -o myprog myprog.o $PNETCDF_LIBS

Batch Usage

#PBS -N AppNameJob
#PBS -l nodes=1:ppn=28
cd $PBS_O_WORKDIR
mpiexec ./myprog

Further Reading

Supercomputer: 
Service: 

Python

Python is a high-level, multi-paradigm programming language that is both easy to learn and useful in a wide variety of applications.  Python has a large standard library as well as a large number of third-party extensions, most of which are completely free and open source. 

Availability and Restrictions

Python is available on Oakley, Ruby, and Owens Clusters. The versions currently available at OSC are:

Version Oakley ruby owens notes
2.7     X The Latest Version of Python 2.7
2.7.1 X     Default version on Oakley prior to 09/15/2015
2.7.8 X(A)      
2.7.9   X(A)    
3.4.2 X(A)* X(A)*    
3.5     X* The Latest Version of Python 3.5
*: Current default version

NOTE:

You can use module avail python to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Python is available for use by all OSC users.

Usage on Oakley

Set-up

To load the default version of Python module, use  module load python. To select a particular software version, use   module load python/version. For example, use  module load python/2.7.1 to load Python version 2.7.1. After the module is loaded, you can run the interpreter by using the command python. To unload the Python 2.7.1 module, use the command module unload python/2.7.1

Installed Modules

We have installed a number of Python packages and tuned them for optimal performance on our systems. When Python module is loaded, executing  module help python will help you view the current list of packages available.

NOTE:
  • numpy and scipy have been compiled to use MKL for best performance
  • Due to architecture differences between our supercomputers, we recommend NOT installing your own packages in  ~/.local. Instead, you should install them in some other directory and set  $PYTHONPATH in your default environment. For more information about installing your own Python modules, please see our HOWTO.

Batch Usage

When you log into oakley.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info.  Batch jobs run on the compute nodes of the system and not on the login node. It is desirable for big problems since more resources can be used.

Interactive Batch Session

For an interactive batch session one can run the following command:

qsub -I -l nodes=1:ppn=12 -l walltime=00:20:00 
which requests one whole node with 12 cores ( -l nodes=1:ppn=12), for a walltime of 20 minutes ( -l walltime=00:20:00). You may adjust the numbers per your need.
Non-interactive Batch Job (Serial Run)

Python can technically be used in both interactive and non-interactive batch jobs, but you will most likely only want to run a Python script non-interactively. To do this, you will need to write a batch submission script and a Python script using any text editor you like in a working directory on the system of your choice. A batch script can be created and submitted for a serial or parallel run. Below is a sample batch submission script( job.txt) and a simple “Hello World” program ( hello.py ) written in Python.

Example batch submission script  job.txt:
#PBS –N hello
#PBS –l nodes=1:ppn=12
#PBS –l walltime=1:00:00
#PBS –j oe
module load python/2.7.1
python hello.py
#end of sample.job
Example Python script,  hello.py:
import sys
hello = [“Hello”, “World”]
for i in hello:
print i
sys.exit(0)
#end of hello.py

Usage on Ruby

Set-up

To load the default version of Python module, use  module load python. To select a particular software version, use   module load python/version. For example, use  module load python/3.4.2 to load Python version 3.4.2. After the module is loaded, you can run the interpreter by using the command python. To unload the Python 3.4.2 module, use the command module unload python/3.4.2

Installed Modules

We have installed a number of Python packages and tuned them for optimal performance on our systems.  When using the Anaconda distributions of python you can run conda list to view the installed packages.

NOTE:
  • Due to architecture differences between our supercomputers, we recommend NOT installing your own packages in  ~/.local. Instead, you should install them in some other directory and set  $PYTHONPATH  in your default environment. For more information about installing your own Python modules, please see our HOWTO.

Batch Usage

When you log into ruby.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. 

Interactive Batch Session

For an interactive batch session one can run the following command:

qsub -I -l nodes=1:ppn=20 -l walltime=00:20:00 
which requests one whole node with 20 cores ( -l nodes=1:ppn=20), for a walltime of 20 minutes ( -l walltime=00:20:00). You may adjust the numbers per your need.

Usage on Owens

Set-up

To load the default version of Python module, use  module load python . To select a particular software version, use   module load python/version. For example, use  module load python/3.5 to load the latest version of Python 3.5. After the module is loaded, you can run the interpreter by using the command python. To unload the latest version of Python 3.5 module, use the command module unload python/3.5

Installed Modules

We have installed a number of Python packages and tuned them for optimal performance on our systems.  When using the Anaconda distributions of python you can run conda list to view the installed packages.

NOTE:
  • Due to architecture differences between our supercomputers, we recommend NOT installing your own packages in  ~/.local. Instead, you should install them in some other directory and set  $PYTHONPATH in your default environment. For more information about installing your own Python modules, please see our HOWTO.

Batch Usage

When you log into owens.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. 

Interactive Batch Session

For an interactive batch session one can run the following command:

qsub -I -l nodes=1:ppn=28 -l walltime=00:20:00 
which requests one whole node with 28 cores ( -l nodes=1:ppn=28), for a walltime of 20 minutes (-l walltime=00:20:00). You may adjust the numbers per your need.

Further reading

Extensive documentation of the Python programming language and software downloads can be found at the Official Python Website.  

See Also

HOWTO: Install your own python modules

Supercomputer: 
Service: 
Fields of Science: 

Q-Chem

Q-Chem is a general purpose ab initio electronic structure program. Its latest version emphasizes Self-Consistent Field, especially Density Functional Theory, post Hartree-Fock, and innovative algorithms for fast performance and reduced scaling calculations. Geometry optimizations, vibrational frequencies, thermodynamic properties, and solution modeling are available. It performs reasonably well within its single reference paradigm on open shell and excited state systems. The Q-Chem Home Page has additional information.

Availability and Restrictions

Q-Chem is available on the Oakley Cluster. The versions currently available at OSC are

VERSION OAKLEY notes
4.0.1 X Default version on Oakley prior to 09/15/2015
4.2.1 X  
4.2.2 X  
4.3 X*  
4.3.1 X  
4.4 X  
*: Current default version

You can use module avail qchem  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Q-Chem is available for use by all OSC users.

Usage

Usage on Oakley

Set-up on Oakley

Q-Chem usage is controlled via modules. Load one of the Q-Chem modulefiles at the command line, in your shell initialization script, or in your batch scripts. To load the default version of Q-Chem module, use  module load qchem . To select a particular software version, use   module load qchem/version . For example, use  module load qchem/4.3   to load Q-Chem version 4.3 on Oakley. 

Examples

  • The root of the Q-Chem directory tree is /usr/local/qchem/ 
  • Example Q-Chem input files are in the samples subdirectory
  • Computational Chemistry Workshop materials are in /home/srb/workshops/compchem/qchem

Batch Usage on Oakley

When you log into oakley.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info.  Batch jobs run on the compute nodes of the system and not on the login node. It is desirable for big problems since more resources can be used.

Interactive Batch Session

For an interactive batch session one can run the following command:

qsub -I -l nodes=1:ppn=1 -l walltime=00:20:00

which requests one core ( -l nodes=1:ppn=1 ), for a walltime of 20 minutes ( -l walltime=00:20:00 ). You may adjust the numbers per your need.

Further Reading

General documentation is available from the Q-Chem Home page and in the doc subdirectories.

Supercomputer: 
Service: 

Quantum Espresso

Quantum ESPRESSO is a program package for ab-initio molecular dynamics (MD) simulations and electronic structure calculations.  It is based on density-functional theory, plane waves, and pseudopotentials.

Availability & Restrictions.

Quantum ESPRESSO is open source and available without restriction.  We recommend that Oakley be used.

The following versions are available on OSC systems:

Version Glenn Oakley
5.0.3   X*
5.2.1   X
*: Default Version

Usage

Set-up

You can configure your environment for the usage of Quantum ESPRESSO by running the following command:

module load espresso

Batch Usage

Sample batch scripts and input files are available here:

/nfs/10/srb/workshops/compchem/espresso/

Further Reading

See Also

Supercomputer: 
Service: 

R

R is a language and environment for statistical computing and graphics. It is similar to the S language and environment developed at Bell Laboratories (formerly AT&T, now Lucent Technologies). R provides a wide variety of statistical and graphical techniques, and is highly extensible.

Availability & Restrictions

R is available to all OSC users without restriction.

The following versions of R are available on OSC systems: 

Version Oakley Owens
2.14.1 X  
2.15.0 X  
2.15.2 X  
3.0.1 X  
3.1.3 X  
3.2.0 X*  
3.3.1 X X
NOTE: * means it is the default version.
NOTE: GNU/5.2.0 is required for R/3.3.1 on Oakley

Set-up

In order to configure your environment for the usage of R, run the following command:

module load R

Using R

Once your environment is configured, R can be started simply by entering the following command:

R

For a listing of command line options, run:

R --help

Batch Usage

Running R interactivly is not recommended and may violate OSC usage policy. In order to run R in batch, refrence the example batch script below. This script requests one full node on the Oakley cluster for 1 hour of walltime.

#PBS -N R_ExampleJob
#PBS -l nodes=1:ppn=12

module load R
cd $PBS_O_WORKDIR
cp in.dat $TMPDIR
cd $TMPDIR

R CMD BATCH test.R test.Rout

cp test.Rout $PBS_O_WORKDIR
NOTE: Owens users should use ppn=28 instead of ppn=12.

Further Reading

See Also

Supercomputer: 
Service: 
Fields of Science: 

RAxML

"RAxML is a fast implementation of maximum-likelihood (ML) phylogeny estimation that operates on both nucleotide and protein sequence alignments."
(http://www.embl.de/~seqanal/courses/commonCourseContent/usingRaxml.html)

Availability & Restrictions

RAxML is available to all OSC users without restriction.

The following versions of RAxML are available on OSC systems:

Version Glenn Oakley
7.0.4 X  
7.4.2 X X

Usage

Set-up

On the Glenn Cluster RAxML is accessed by executing the following commands:

module load biosoftw
module load RAxML

Using RAxML

raxmlHPC[-MPI|-PTHREADS] -s sequenceFileName -n outputFileName -m substitutionModel [-a weightFileName] [-b bootstrapRandomNumberSeed] [-c numberOfCategories][-d] [-e likelihoodEpsilon] [-E excludeFileName] [-f a|b|c|d|e|g|h|i|j|m|n|o|p|s|t|w|x] [-g groupingFileName] [-h] [-i initialRearrangementSetting] [-j] [-k] [-l sequenceSimilarityThreshold] [-L sequenceSimilarityThreshold] [-M] [-o outGroupName1[,outGroupName2[,...]]] [-p parsimonyRandomSeed] [-P proteinModel] [-q multipleModelFileName] [-r binaryConstraintTree] [-t userStartingTree] [-T numberOfThreads] [-u multiBootstrapSearches] [-v][-w workingDirectory] [-x rapidBootstrapRandomNumberSeed][-y][-z multipleTreesFile] [-#|-N numberOfRuns]

Options

-a    Specify a column weight file name to assign individual weights to each column of the alignment. Those weights must be integers separated by any type and number of whitespaces within a separate file, see file "example_weights" for an example.
-b    Specify an integer number (random seed) and turn on bootstrapping.  DEFAULT: OFF
-c    Specify number of distinct rate catgories for RAxML when modelOfEvolution is set to GTRCAT or GTRMIX.  Individual per-site rates are categorized into numberOfCategories rate categories to accelerate computations.  DEFAULT: 25
-d    start ML optimization from random starting tree.  DEFAULT: OFF
-e    set model optimization precision in log likelihood units for final optimization of tree topology under MIX/MIXI or GAMMA/GAMMAI.  DEFAULT: 0.1   for models not using proportion of invariant sites estimate 0.001 for models using proportion of invariant sites estimate
-E    specify an exclude file name, that contains a specification of alignment positions you wish to exclude.  Format is similar to Nexus, the file shall contain entries like "100-200 300-400", to exclude a single column write, e.g., "100-100", if you use a mixed model, an appropriately adapted model file will be written.
-f    select algorithm:
      "-f a": rapid Bootstrap analysis and search for best-scoring ML tree in one program run
      "-f b": draw bipartition information on a tree provided with "-t" based on multiple trees (e.g. form a bootstrap) in a file specifed by "-z"
      "-f c": check if the alignment can be properly read by RAxML
      "-f d": new rapid hill-climbing
      "-f e": optimize model+branch lengths for given input tree under GAMMA/GAMMAI only
      "-f g": compute per site log Likelihoods for one ore more trees passed via "-z" and write them to a file that can be read by CONSEL
      "-f h": compute log likelihood test (SH-test) between best tree passed via "-t" and a bunch of other trees passed via "-z"
      "-f i": perform a really thorough bootstrap, refinement of final BS tree under GAMMA and a more exhaustive algorithm
      "-f j": generate a bunch of bootstrapped alignment files from an original alignment file
      "-f m": Compare bipartitions between two bunches of trees passed via "-t" and "-z" respectively. This will return the Pearson correlation between all bipartitions found in the two tree files. A file called RAxML_bipartitionFrequencies.outpuFileName will be printed that contains the pair-wise bipartition frequencies of the two sets
      "-f n": Compute the log likelihood score of all trees contained in a tree file provided by "-z" under GAMMA or GAMMA+P-Invar
      "-f o": old and slower rapid hill-climbing
      "-f p": perform pure stepwise MP addition of new sequences to an incomplete starting tree
      "-f s": split up a multi-gene partitioned alignment into the respective subalignments
      "-f t": do randomized tree searches on one fixed starting tree
      "-f w": compute ELW test on a bunch of trees passed via "-z"
      "-f x": compute pair-wise ML distances, ML model parameters will be estimated on an MP starting tree or a user-defined tree passed via "-t", only allowed for GAMMA-based models of rate heterogeneity.  DEFAULT: new rapid hill climbing
-g    specify the file name of a multifurcating constraint tree this tree does not need to be comprehensive, i.e. must not contain all taxa
-h    Display this help message.
-i    Initial rearrangement setting for the subsequent application of topological changes phase.  DEFAULT: determined by program
-j    Specifies if checkpoints will be written by the program. If checkpoints (intermediate tree topologies) shall be written by the program specify "-j"  DEFAULT: OFF
-k    Specifies that bootstrapped trees should be printed with branch lengths.  The bootstraps will run a bit longer, because model parameters will be optimized at the end of each run. Use with CATMIX/PROTMIX or GAMMA/GAMMAI.  DEFAULT: OFF
-l    Specify a threshold for sequence similarity clustering. RAxML will then print out an alignment to a file called sequenceFileName.reducedBy.threshold that only contains sequences <= the specified thresold that must be between  0.0 and 1.0.  RAxML uses the QT-clustering algorithm to perform this task. In addition, a file called RAxML_reducedList.outputFileName will be written that contains clustering information.  DEFAULT: OFF
-L    Same functionality as "-l" above, but uses a less exhasutive and thus faster clustering algorithm.  This is intended for very large datasets with more than 20,000-30,000 sequences.  DEFAULT: OFF
-m    Model of Nucleotide or Amino Acid Substitution:
NUCLEOTIDES:
    "-m GTRCAT"                      : GTR + Optimization of substitution rates + Optimization of site-specific evolutionary rates which are categorized into numberOfCategories distinct rate categories for greater computational efficiency if you do a multiple analysis with "-#" or "-N" but without bootstrapping the program will use GTRMIX instead
    "-m GTRGAMMA"                    : GTR + Optimization of substitution rates + GAMMA model of rate heterogeneity (alpha parameter will be estimated)
    "-m GTRMIX"                      : Inference of the tree under GTRCAT and thereafter evaluation of the final tree topology under GTRGAMMA
    "-m GTRCAT_GAMMA"                : Inference of the tree with site-specific evolutionary rates.  However, here rates are categorized using the 4 discrete GAMMA rates. Evaluation of the final tree topology under GTRGAMMA
    "-m GTRGAMMAI"                   : Same as GTRGAMMA, but with estimate of proportion of invariable sites
    "-m GTRMIXI"                     : Same as GTRMIX, but with estimate of proportion of invariable sites
    "-m GTRCAT_GAMMAI"               : Same as GTRCAT_GAMMA, but with estimate of proportion of invariable sites
AMINO ACIDS:
    "-m PROTCATmatrixName[F]"        : specified AA matrix + Optimization of substitution rates + Optimization of site-specific evolutionary rates which are categorized into numberOfCategories distinct rate categories for greater computational efficiency if you do a multiple analysis with  "-#" or "-N" but without bootstrapping the program will use PROTMIX... instead
    "-m PROTGAMMAmatrixName[F]"      : specified AA matrix + Optimization of substitution rates + GAMMA model of rate heterogeneity (alpha parameter will be estimated)
    "-m PROTMIXmatrixName[F]"        : Inference of the tree under specified AA matrix + CAT and thereafter evaluation of the final tree topology under specified AA matrix + GAMMA
    "-m PROTCAT_GAMMAmatrixName[F]"  : Inference of the tree under specified AA matrix and site-specific evolutionary rates.  However, here rates are categorized using the 4 discrete GAMMA rates. Evaluation of the final tree topology under specified AA matrix + GAMMA
    "-m PROTGAMMAImatrixName[F]"     : Same as PROTGAMMAmatrixName[F], but with estimate of proportion of invariable sites
    "-m PROTMIXImatrixName[F]"       : Same as PROTMIXmatrixName[F], but with estimate of proportion of invariable sites
    "-m PROTCAT_GAMMAImatrixName[F]" : Same as PROTCAT_GAMMAmatrixName[F], but with estimate of proportion of invariable sites
Available AA substitution models: DAYHOFF, DCMUT, JTT, MTREV, WAG, RTREV, CPREV, VT, BLOSUM62, MTMAM, GTR
With the optional "F" appendix you can specify if you want to use empirical base frequencies.  Please not that for mixed models you can in addition specify the per-gene AA model in the mixed model file (see manual for details)
-M    Switch on estimation of individual per-partition branch lengths. Only has effect when used in combination with "-q". Branch lengths for individual partitions will be printed to separate files.  A weighted average of the branch lengths is computed by using the respective partition lengths.  DEFAULT: OFF
-n    Specifies the name of the output file.
-o    Specify the name of a single outgrpoup or a comma-separated list of outgroups, eg "-o Rat" or "-o Rat,Mouse", in case that multiple outgroups are not monophyletic the first name in the list will be selected as outgroup, don't leave spaces between taxon names!
-q    Specify the file name which contains the assignment of models to alignment partitions for multiple models of substitution. For the syntax of this file please consult the manual.
-p    Specify a random number seed for the parsimony inferences. This allows you to reproduce your results and will help me debug the program. This option HAS NO EFFECT in the parallel MPI version
-P    Specify the file name of a user-defined AA (Protein) substitution model. This file must contain 420 entries, the first 400 being the AA substitution rates (this must be a symmetric matrix) and the last 20 are the empirical base frequencies
-r    Specify the file name of a binary constraint tree.  This tree does not need to be comprehensive, i.e. must not contain all taxa
-s    Specify the name of the alignment data file in PHYLIP format
-t    Specify a user starting tree file name in Newick format
-T    PTHREADS VERSION ONLY! Specify the number of threads you want to run.  Make sure to set "-T" to at most the number of CPUs you have on your machine, otherwise, there will be a huge performance decrease!
-u    Specify the number of multiple BS searches per replicate to obtain better ML trees for each replicate.  DEFAULT: One ML search per BS replicate
-v    Display version information
-w    Name of the working directory where RAxML will write its output files.  DEFAULT: current directory
-x    Specify an integer number (random seed) and turn on rapid bootstrapping
-y    If you want to only compute a parsimony starting tree with RAxML specify "-y", the program will exit after computation of the starting tree.  DEFAULT: OFF
-z    Specify the file name of a file containing multiple trees e.g. from a bootstrap that shall be used to draw bipartition values onto a tree provided with "-t", It can also be used to compute per site log likelihoods in combination with "-f g" and to read a bunch of trees for a couple of other options ("-f h", "-f m", "-f n").
-#|-N Specify the number of alternative runs on distinct starting trees.  In combination with the "-b" option, this will invoke a multiple bootstrap analysis.  Note that "-N" has been added as an alternative since "-#" sometimes caused problems with certain MPI job submission systems, since "-#" is often used to start comments.  DEFAULT: 1 single analysis

Example

The file brown.philip is from the PAML example and modified to be in PHYLIP format.

#PBS -N RAxML_test
#PBS -l walltime=10:00
#PBS -l nodes=1:ppn=4

cd $PBS_O_WORKDIR
module load biosoftw
module load RAxML
raxmlHPC-PTHREAD -T 4 -m GTRCAT -n test -s brown.phylip

If you want to compute more than one tree add the following line before calling the program:

module load pvfs2

And substitute raxmlHPC-MPI for raxmlHPC-PTHREAD.

Further Reading

Supercomputer: 
Service: 
Fields of Science: 

RECON

“Proper identification of repetitive sequences is an essential step in genome analysis. The RECON package performs de novo identification and classification of repeat sequence families from genomic sequences. The underlying algorithm is based on extensions to the usual approach of single linkage clustering of local pairwise alignments between genomic sequences. Specifically, our extensions use multiple alignment information to define the boundaries of individual copies of the repeats and to distinguish homologous but distinct repeat element families. RECON should be useful for first-pass automatic classification of repeats in newly sequenced genomes.” (http://selab.janelia.org/recon.html)

Availability & Restrictions

RECON is available to all OSC users without restriction.

The following versions of RECON are available on OSC systems:

Version Glenn Oakley
1.0.5 X  

Usage

Set-up

On the Glenn Cluster, RECON is accessed by executing the following commands:

module load biosoftw
module load RECON

RECON is a collection of several programs that will be added to the users PATH: imagespread, eledef, eleredef, edgeredef, and famdef.  Also added to the users PATH is recon.pl, which executes each of the aforementioned programs.

Using RAxML

raxmlHPC[-MPI|-PTHREADS] -s sequenceFileName -n outputFileName -m substitutionModel [-a weightFileName] [-b bootstrapRandomNumberSeed] [-c numberOfCategories][-d] [-e likelihoodEpsilon] [-E excludeFileName] [-f a|b|c|d|e|g|h|i|j|m|n|o|p|s|t|w|x] [-g groupingFileName] [-h] [-i initialRearrangementSetting] [-j] [-k] [-l sequenceSimilarityThreshold] [-L sequenceSimilarityThreshold] [-M] [-o outGroupName1[,outGroupName2[,...]]] [-p parsimonyRandomSeed] [-P proteinModel] [-q multipleModelFileName] [-r binaryConstraintTree] [-t userStartingTree] [-T numberOfThreads] [-u multiBootstrapSearches] [-v][-w workingDirectory] [-x rapidBootstrapRandomNumberSeed][-y][-z multipleTreesFile] [-#|-N numberOfRuns]

Example

Not currently available.

Further Reading

Supercomputer: 
Service: 
Fields of Science: 

RNA-SeQC

RNA-SeQC is a java program which computes a series of quality control metrics for RNA-seq data. The input can be one or more BAM files. The output consists of HTML reports and tab delimited files of metrics data. This program can be valuable for comparing sequencing quality across different samples or experiments to evaluate different experimental parameters. It can also be run on individual samples as a means of quality control before continuing with downstream analysis.

Availability and Restrictions

The following versions of RNA-SeQC are available on OSC clusters:

Version Oakley Owens
1.1.8 X* X*
*: Current default version

You can use  module avail rna-seqc  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

RNA-SeQC is available to all OSC users without restriction.

Usage on Oakley

Set-up

To configure your environment for use of RNA-SeQC, run the following command:   module load rna-seqc . The default version will be loaded. To select a particular RNA-SeQC version, use   module load rna-seqc/version. For example, use  module load rna-seqc/1.1.8  to load RNA-SeQC 1.1.8.

Usage

This software is a Java executable .jar file; thus, it is not possible to add to the PATH environment variable.

From module load rna-seqc, a new environment variable, RNA_SEQC, will be set.

Thus, users can use the software by running the following command: java -jar $RNA_SEQC {other options}

Usage on Owens

Set-up

To configure your environment for use of RNA-SeQC, run the following command:   module load rna-seqc . The default version will be loaded. To select a particular RNA-SeQC version, use   module load rna-seqc/version. For example, use  module load rna-seqc/1.1.8  to load RNA-SeQC 1.1.8.

Usage

This software is a Java executable .jar file; thus, it is not possible to add to the PATH environment variable.

From module load rna-seqc, a new environment variable, RNA_SEQC, will be set.

Thus, users can use the software by running the following command: java -jar $RNA_SEQC {other options}

Further Reading

Supercomputer: 
Service: 
Fields of Science: 

RepeatMasker

"RepeatMasker is a program that screens DNA sequences for interspersed repeats and low complexity DNA sequences. The output of the program is a detailed annotation of the repeats that are present in the query sequence as well as a modified version of the query sequence in which all the annotated repeats have been masked (default: replaced by Ns). On average, almost 50% of a human genomic DNA sequence currently will be masked by the program." (http://www.repeatmasker.org/)

Availability & Restrictions

RepeatMasker is available to all OSC users without restriction.

The following versions of RepeatMasker are available on OSC systems:

Version Glenn Oakley
2.1 X  

Usage

Set-up

On the Glenn Cluster RepeatMasker is accessed by executing the following commands:

module load biosoftw
module load RepeatMasker

RepeatMasker will be added to the users PATH and can be run with the command:

RepeatMasker [-options] <seqfiles(s) in fasta format>

Options

-h(elp)
      Detailed help
      Default settings are for masking all type of repeats in a primate sequence.
-pa(rallel) [number]
      The number of processors to use in parallel (only works for batch files or sequences over 50 kb)
-s    Slow search; 0-5% more sensitive, 2-3 times slower than default
-q    Quick search; 5-10% less sensitive, 2-5 times faster than default
-qq   Rush job; about 10% less sensitive, 4->10 times faster than default (quick searches are fine under most circumstances) repeat options
-nolow /-low
      Does not mask low_complexity DNA or simple repeats
-noint /-int
      Only masks low complex/simple repeats (no interspersed repeats)
-norna
      Does not mask small RNA (pseudo) genes
-alu
      Only masks Alus (and 7SLRNA, SVA and LTR5)(only for primate DNA)
-div [number]
      Masks only those repeats < x percent diverged from consensus seq
-lib [filename]
      Allows use of a custom library (e.g. from another species)
-cutoff [number]
      Sets cutoff score for masking repeats when using -lib (default 225)
-species <query species>
      Specify the species or clade of the input sequence. The species name must be a valid NCBI Taxonomy Database species name and be contained in the RepeatMasker repeat database. Some examples are:
      -species human
      -species mouse
      -species rattus
      -species "ciona savignyi"
      -species arabidopsis
      Other commonly used species: mammal, carnivore, rodentia, rat, cow, pig, cat, dog, chicken, fugu, danio, "ciona intestinalis" drosophila, anopheles, elegans, diatoaea, artiodactyl, arabidopsis, rice, wheat, and maize

Contamination options

-is_only
      Only clips E coli insertion elements out of fasta and .qual files
-is_clip
      Clips IS elements before analysis (default: IS only reported)
-no_is
      Skips bacterial insertion element check
-rodspec
      Only checks for rodent specific repeats (no repeatmasker run)
-primspec
      Only checks for primate specific repeats (no repeatmasker run)

Running options

-gc [number]
      Use matrices calculated for 'number' percentage background GC level
-gccalc
      RepeatMasker calculates the GC content even for batch files/small seqs
-frag [number]
      Maximum sequence length masked without fragmenting (default 40000, 300000 for DeCypher)
-maxsize [nr]
      Maximum length for which IS- or repeat clipped sequences can be produced (default 4000000). Memory requirements go up with higher maxsize.
-nocut
      Skips the steps in which repeats are excised
-noisy
      Prints search engine progress report to screen (defaults to .stderr file)
-nopost
      Do not postprocess the results of the run ( i.e. call ProcessRepeats).
       NOTE: This options should only be used when ProcessRepeats will be run manually on the results.

Output options

-dir [directory name]
      Writes output to this directory (default is query file directory, "-dir ." will write to current directory).
-a(lignments)
      Writes alignments in .align output file; (not working with -wublast)
-inv
      Alignments are presented in the orientation of the repeat (with option -a)
-lcambig
      Outputs ambiguous DNA transposon fragments using a lower case name.  All other repeats are listed in upper case. Ambiguous fragments match multiple repeat elements and can only be called based on flanking repeat information.
-small
      Returns complete .masked sequence in lower case
-xsmall
      Returns repetitive regions in lowercase (rest capitals) rather than masked
-x    Returns repetitive regions masked with Xs rather than Ns
-poly
      Reports simple repeats that may be polymorphic (in file.poly)
-source
      Includes for each annotation the HSP "evidence". Currently this option is only available with the "-html" output format listed below.
-html
      Creates an additional output file in xhtml format.
-ace
      Creates an additional output file in ACeDB format
-gff
      Creates an additional Gene Feature Finding format output
-u    Creates an additional annotation file not processed by ProcessRepeats
-xm   Creates an additional output file in cross_match format (for parsing)
-fixed
      Creates an (old style) annotation file with fixed width columns
-no_id
      Leaves out final column with unique ID for each element (was default)
-e(xcln)
      Calculates repeat densities (in .tbl) excluding runs of >=20 N/Xs in the query

Example

#PBS -N RepeatMasker_test
#PBS -l walltime=4:00:00
#PBS -l nodes=1:ppn=4

module load biosoftw
module load RepeatMasker
cp /usr/local/biosoftw/bowtie-0.12.7/genomes/NC_008253.fna .
RepeatMasker –pa 4 NC_008253.fna

Errors

The following commands result in errors:  RepeatMasker -w, RepeatMasker -de, RepeatMasker -e.

Further Reading

Supercomputer: 
Service: 
Fields of Science: 

SAMtools

SAM format is a generic format for storing large nucleotide sequence alignments. SAMtools provide various utilities for manipulating alignments in the SAM format, including sorting, merging, indexing and generating alignments in a per-position format.

Availability and Restrictions

The following versions of SAMtools are available on OSC clusters:

Version Oakley Owens
1.3.1 X* X*
*: Current default version

You can use  module avail samtools  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

SAMtools is available to all OSC users without restriction.

Usage on Oakley

Set-up

To configure your environment for use of SAMtools, run the following command:   module load samtools . The default version will be loaded. To select a particular SAMtools version, use   module load samtools/version . For example, use  module load samtools/1.3.1  to load SAMtools 1.3.1.

Usage on Owens

Set-up

To configure your environment for use of SAMtools, run the following command:    module load samtools  . The default version will be loaded. To select a particular SAMtools version, use    module load samtools/version  . For example, use   module load samtools/1.3.1   to load SAMtools 1.3.1.

Further Reading

Supercomputer: 
Service: 
Fields of Science: 

SIESTA

SIESTA is both a method and its computer program implementation, to perform efficient electronic structure calculations and ab initio molecular dynamics simulations of molecules and solids. More information can be found from here.

Availability & Restrictions

SIESTA is available on the Oakley cluster. A serial and a parallel build were created in order to meet users' computational needs.

VERSION GLENN OAKLEY
3.2-pl-4   X

The SIESTA program is distributed freely to academics. Use of SIESTA for academic purposes requires validation. In order to obtain validation, please contact OSC Help for further insruction. Commercial users need to show proof of license in order to use SIESTA at OSC.

Batch Usage

When you log into oakley.osc.edu, you are actually logged into a linux box referred to as the login node. To gain access to the 4000+ processors in the computing environment, you must submit your SIESTA job to the batch system for execution.

Assume that you have a test case in your work directory (where you submit your job, represented by $PBS_O_WORKDIR), with the input file 32_h2o.fdf. A batch script can be created and submitted for a serial or parallel run. The following are the sample batch scripts for running serial and parallel SIESTA jobs.

Sample Batch Script for Serial Jobs

#PBS -l walltime=0:30:00
#PBS -l nodes=1:ppn=12
#PBS -N siesta
#PBS -j oe
#
cd $PBS_O_WORKDIR
#
# Set up the package environment
module load siesta
#
# Execute the serial solver (nodes=1, ppn<=12)
siesta <32_h2o.fdf> output
exit

Sample Batch Script for Parallel Jobs

#PBS -l walltime=0:30:00
#PBS -l nodes=2:ppn=12
#PBS -N siesta
#PBS -j oe
#
cd $PBS_O_WORKDIR
#
# Set up the package environment
module swap intel/12.1.4.319 intel/13.1.3.192
module load siesta_par
#
# Execute the parallel solver (nodes>1, ppn=12)
mpiexec -np 24 siesta <32_h2o.fdf> output
exit

Further Reading

Online documentation is available at: http://departments.icmab.es/leem/siesta/Documentation/index.html.

Citations

1. “Self-consistent order-N density-functional calculations for very large systems”, P. Ordejón, E. Artacho and J. M. Soler, Phys. Rev. B (Rapid Comm.) 53, R10441-10443 (1996).

2. “The SIESTA method for ab initio order-N materials simulation”, J. M. Soler, E. Artacho,J. D. Gale, A. García, J. Junquera, P. Ordejón, and D. Sánchez-Portal, J. Phys.: Condens. Matt. 14, 2745-2779 (2002).


 

Supercomputer: 
Service: 

SPRNG

Computational stochastic approaches (Monte Carlo methods) based on random sampling are becoming extremely important research tools not only in their "traditional" fields such as physics, chemistry or applied mathematics but also in social sciences and, recently, in various branches of industry. An indication of importance is, for example, the fact that Monte Carlo calculations consume about one half of the supercomputer cycles. One of the indispensable and important ingredients for reliable and statistically sound calculations is the source of pseudo random numbers. SPRNG provides a scalable package for parallel pseudo random number generation which will be easy to use on a variety of architectures, especially in large-scale parallel Monte Carlo applications.

SPRNG 1.0 provides the user the various SPRNG random number generators each in its own library. For most users this is acceptable, as one rarely uses more than one type of generator in a single program. However, if the user desires this added flexibility, SPRNG 2.0 provides it. In all other respects, SPRNG 1.0 and SPRNG 2.0 are identical.

Availability and Restrictions

SPRNG is available on Glenn and Oakley Clusters. The versions currently available at OSC are:

Version Glenn Oakley notes
1.0 X*    
2.0 X X*  
* : Current Default Version

You can use module avail sprng  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

SPRNG is available to all OSC users without restriction.

Usage

Usage on Glenn

Set-up on Glenn

Initalizing the system for use of SPRNG is dependent on the system you are using and the compiler you are using. To configure your environment for use of SPRNG 1.0 on Glenn, use the following command: module load sprng . To configure your environment for use of SPRNG 2.0 on Glenn, use the following command: module load sprng2 . You can use module avail sprng  to view available modules on Glenn.

Note: SPRNG is dependent on the compiler and mpi module and may not be supported for all combinations.

Building With SPRNG 1.0 and 2.0 on Glenn

Once SPRNG 1.0 is loaded, the following environment variables will be defined for use with compilation tools:

Variable Function
$SPRNG_INCLUDE Include path for SPRNG header files
$SPRNG_CMRG Library path and library for CMRG generator
$SPRNG_LCG Library path and library for LCG generator
$SPRNG_LCG64 Library path and library for 64-bit LCG generator
$SPRNG_LFG Library path and library for LFG generator
$SPRNG_MLFG Library path and library for MLFG generator

Alternativly, when SPRNG 2.0 is loaded, the following variables will be defined:

VARIABLE Function
$SPRNG_INCLUDE Include path for SPRNG 2.0 header files
$SPRNG_LIBS Library path and library for SPRNG 2.0 generators

Usage on Oakley

Set-up on Oakley

Initalizing the system for use of SPRNG is dependent on the system you are using and the compiler you are using. To configure your environment for use of SPRNG 2.0 on Oakley, use the following command: module load sprng/2.0 . You can use module avail sprng  to view available modules on Oakley.

Note: SPRNG is dependent on the compiler and mpi module and may not be supported for all combinations.

Building With SPRNG 2.0 on Oakley

When SPRNG 2.0 is loaded, the following variables will be defined:

VARIABLE Function
$SPRNG_INCLUDE Include path for SPRNG 2.0 header files
$SPRNG_LIBS Library path and library for SPRNG 2.0 generators

Further Reading

See Also

Supercomputer: 
Service: 
Fields of Science: 

SRA Toolkit

The Sequence Read Archive (SRA) stores raw sequence data from "next-generation" sequencing technologies including 454, IonTorrent, Illumina, SOLiD, Helicos and Complete Genomics. In addition to raw sequence data, SRA now stores alignment information in the form of read placements on a reference sequence. Use SRA Toolkit tools to directly operate on SRA runs.

Availability and Restrictions

The following versions of SRA Toolkit are available on OSC clusters:

Version Oakley Owens
2.6.3 X* X*
*: Current default version

You can use  module avail sratoolkit to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

SRA Toolkit is available to all OSC users without restriction.

Usage on Oakley

Set-up

To configure your environment for use of SRA Toolkit, run the following command:   module load sratoolkit. The default version will be loaded. To select a particular SRA Toolkit version, use   module load sratoolkit/version. For example, use  module load sratoolkit/2.6.3 to load SRA Toolkit 2.6.3.

Usage on Owens

Set-up

To configure your environment for use of SRA Toolkit, run the following command:   module load sratoolkit. The default version will be loaded. To select a particular SRA Toolkit version, use   module load sratoolkit/version. For example, use  module load sratoolkit/2.6.3 to load SRA Toolkit 2.6.3.

Further Reading

Supercomputer: 
Service: 
Fields of Science: 

STAR

STAR: Spliced Transcripts Alignment to a Reference.

Availability and Restrictions

The following versions of STAR are available on OSC clusters:

Version Oakley Owens
2.5.2a X* X*
*: Current default version

You can use  module avail star  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

STAR is available to all OSC users without restriction.

Usage on Oakley

Set-up

To configure your environment for use of STAR, run the following command:   module load star . The default version will be loaded. To select a particular STAR version, use   module load star/version . For example, use  module load star/2.5.2a  to load STAR 2.5.2a.

Usage on Owens

Set-up

To configure your environment for use of STAR, run the following command:   module load star . The default version will be loaded. To select a particular STAR version, use   module load star/version . For example, use module load star/2.5.2a  to load STAR 2.5.2a.

Further Reading

Supercomputer: 
Service: 
Fields of Science: 

STAR-CCM+

STAR-CCM+ provides the world’s most comprehensive engineering physics simulation inside a single integrated package. Much more than a CFD code, STAR‑CCM+ provides an engineering process for solving problems involving flow (of fluids and solids), heat transfer and stress. STAR‑CCM+ is unrivalled in its ability to tackle problems involving multi‑physics and complex geometries.  Support is provided by CD-adapco. CD-adapco usually releases new version of STAR-CCM+ every four months.

Availability and Restrictions

STAR-CCM+ is available on Oakley Cluster. The versions currently available at OSC are:

VERSION OAKLEY notes
7.06.012 X Default version on Oakley prior to 09/15/2015
7.06.012r8 X  
8.02.011 X  
8.04.009 X  
8.06.007 X  
9.04.011r8 X  
9.04.012 X  
10.04.009 X*  
10.06.010 X  
11.02.010 X  
*: Current default version

You can use module avail starccm  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access for Academic Users

Use of STAR-CCM+ for academic purposes requires validation. In order to obtain validation, please contact OSC Help for further instruction. 

Currently, OSC has a 50 seat license (ccmpsuite, which allows up to 50 concurrent users), with 1200 additional core licenses (hpcdomains) for academic users. 

Access for Commerical Users

Contact OSC Help for getting access to STAR-CCM+ if you are a commerical user.

Usage

Usage on Oakley

Set-up on Oakley

We recommend to run STAR-CCM+ on only the compute nodes. Thus, all STAR-CCM+ jobs should be submitted via the batch scheduling system, either as interactive or non-interactive batch jobs. To load the default version of STAR-CCM+ module on Oakley, use  module load starccm . To select a particular software version, use   module load starccm/version . For example, use  module load starccm/9.04.012  to load STAR-CCM+ version 9.04.012 on Oakley.

Batch Usage on Oakley

When you log into oakley.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your STAR-CCM+ analysis to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info.  Batch jobs run on the compute nodes of the system and not on the login node. It is desirable for big problems since more resources can be used. STAR-CCM+ can be run on OSC clusters in either interactive mode or in non-interactive batch mode.

Interactive Batch Session

Interactive mode is similar to running STAR-CCM+ on a desktop machine in that the graphical user interface (GUI) will be sent from OSC and displayed on the local machine. To run interactive STAR-CCM+, it is suggested to request necessary compute resources from the login node, with X11 forwarding. The intention is that users can run STAR-CCM+ interactively for the purpose of building their model, preparing input file (.sim file), and checking results. Once developed this input file can then be run in no-interactive batch mode. For example, the following line requests one node with 12 cores( -l nodes=1:ppn=12 ), for a walltime of one hour ( -l walltime=1:00:00 ), with one STAR-CCM+ base license token (    -l software=starccm+1   ) on Oakley:

qsub -I -X -l walltime=1:00:00 -l nodes=1:ppn=12 -l software=starccm+1

This job will queue until resources become available. Once the job is started, you're automatically logged in on the compute node; and you can launch STAR-CCM+ GUI with the following commands:

module load starccm
starccm+
Non-interactive Batch Job (Serial Run using 1 Base Token)

A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice.

Below is the example batch script (  job.txt ) for a serial run with an input file starccm.sim ) on Oakley:

#PBS -N star-ccm_test  
#PBS -l walltime=10:00:00  
#PBS -l nodes=1:ppn=1  
#PBS -l software=starccm+1
#PBS -j oe
#PBS -S /bin/bash
cd $TMPDIR  
cp $PBS_O_WORKDIR/starccm.sim .  
module load starccm  
starccm+ -batch starccm.sim >&output.txt  
cp output.txt $PBS_O_WORKDIR

To run this job on OSC batch system, the above script is to be submitted with the command:

qsub job.txt
Non-interactive Batch Job (Parallel Run using HPC Tokens)

To take advantage of the powerful compute resources at OSC, you may choose to run distributed STAR-CCM+ for large problems. Multiple nodes and cores can be requested to accelerate the solution time. The following shows an example script if you need 2 nodes with 12 cores per node on Oakley using the inputfile named  starccm.sim  :

#PBS -N starccm_test 
#PBS -l walltime=3:00:00 
#PBS -l nodes=2:ppn=12
#PBS -W x=GRES:starccm+1%starccmpar+24
#PBS -j oe
#PBS -S /bin/bash
cd $PBS_0_WORKDIR
cp starccm.sim $TMPDIR
cd $TMPDIR
module load starccm
starccm+ -np 24 -batch -machinefile $PBS_NODEFILE starccm.sim >&output.txt
cp output.txt $PBS_0_WORKDIR

In addition to requesting the STAR-CCM+ base license token (  -l software=starccm+1  ), you need to request copies of the starccmpar license, i.e., HPC tokens. However the scheduler cannot handle two "software" flags simultaneously, so the syntax changes. The new option is   -W x=GRES:starccm+1%starccmpar+[n]  , where [n] is equal to the number of cores.

Further Reading

See Also

Supercomputer: 
Service: 

STAR-Fusion

STAR-Fusion is a component of the Trinity Cancer Transcriptome Analysis Toolkit (CTAT). STAR-Fusion uses the STAR aligner to identify candidate fusion transcripts supported by Illumina reads. STAR-Fusion further processes the output generated by the STAR aligner to map junction reads and spanning reads to a reference annotation set.

Availability and Restrictions

The following versions of STAR-Fusion are available on OSC clusters:

Version Oakley Owens
0.7.0 X* X*
*: Current default version

You can use  module avail star-fusion  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

STAR-Fusion is available to all OSC users without restriction.

Usage on Oakley

Set-up

To configure your environment for use of STAR-Fusion, run the following command:   module load star-fusion  . The default version will be loaded. To select a particular STAR-Fusion version, use   module load star-fusion/version . For example, use  module load star-fusion/0.7.0  to load STAR-Fusion 0.7.0.

Usage on Owens

Set-up

To configure your environment for use of STAR-Fusion, run the following command:   module load star-fusion  . The default version will be loaded. To select a particular STAR-Fusion version, use  module load star-fusion/version . For example, use  module load star-fusion/0.7.0  to load STAR-Fusion 0.7.0.

Further Reading

Supercomputer: 
Service: 
Fields of Science: 

ScaLAPACK

ScaLAPACK is a library of high-performance linear algebra routines for clusters supporting MPI. It contains routines for solving systems of linear equations, least squares problems, and eigenvalue problems.

This page documents usage of the ScaLAPACK library installed by OSC from source. An optimized implementation of ScaLAPACK is included in MKL; see the software documentation page for Intel Math Kernel Library for usage information.

Availability and Restrictions

ScaLAPACK is available on Oakley and Ruby Clusters. The versions currently available at OSC are:

Version Oakley ruby owens notes
1.7        
2.0.1 X     Default version on Oakley prior to 09/15/2015
2.0.2 X* X* X*  
* : Current Default Version

You can use module spider scalapack  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

ScaLAPACK is available to all OSC users without restriction.  If you need high performance, we recommend using MKL instead of the standalone ScaLAPACK module.

Usage on Oakley

Set-up

Initalizing the system for use of the ScaLAPACK libraries is dependent on the system you are using and the compiler you are using. To use the ScaLAPACK libraries in your compilation, run the following command: module load scalapack . To load a particular version, use  module load scalapack/version . For example, use  module load scalapack/2.0.2  to load ScaLAPACK version 2.0.2. You can use module spider scalapack  to view available modules.

Building with ScaLAPACK

Once loaded, the ScaLAPACK libraries can be linked in with your compilation. To do this, use the following environment variables:

Variable Use
$SCALAPACK_LIBS Used to link ScaLAPACK into either Fortran or C

Usage on Ruby

Set-up

Initalizing the system for use of the ScaLAPACK libraries is dependent on the system you are using and the compiler you are using. To use the ScaLAPACK libraries in your compilation, run the following command: module load scalapack . To load a particular version, use  module load scalapack/version . For example, use  module load scalapack/2.0.2  to load ScaLAPACK version 2.0.2. You can use module spider scalapack  to view available modules.

Usage on Owens

Set-up

Initalizing the system for use of the ScaLAPACK libraries is dependent on the system you are using and the compiler you are using. To use the ScaLAPACK libraries in your compilation, run the following command: module load scalapack . To load a particular version, use  module load scalapack/version . For example, use  module load scalapack/2.0.2  to load ScaLAPACK version 2.0.2. You can use module spider scalapack  to view available modules.

Further Reading

 

Supercomputer: 
Service: 

Schrodinger

The Schrodinger molecular modeling software suite includes a number of popular programs focused on drug design and materials science but of general applicability, for example Glide, Jaguar, and MacroModel.  Maestro is the graphical user interface for the suite.  It allows the user to construct and graphically manipulate both simple and complex chemical structures, to apply molecular mechanics and dynamics techniques to evaluate the energies and geometries of molecules in vacuo or in solution, and to display and examine graphically the results of the modeling calculations.

Availability

The Schrodinger suite is available on Oakley.  (MacroModel is available on the Glenn Cluster.)  Several versions are currently available at OSC. To view the possibilities, search for schrodinger (macromodel) in the output of
module avail

Schrodinger is available to all academic users

To use Schrodinger you will have to be added to the license server first.  Please contact OSC Help to be added. Please note that if you are a non-OSU user, we need to send your name, contact email, and affiliation information to Schrodinger in order to grant access.

Usage on Oakley

To set up your environment for schrodinger load one of its modulefiles:
module load schrodinger/schrodinger15

 

Using schrodinger interactively requires an X11 connection. Typically one will launch the graphical user interface:

maestro

 

Here is an example batch script that uses schrodinger non-interactively via the batch system:

# Example glide single node batch script.
#PBS -N glidebatch
#PBS -j oe
#PBS -m ae
#PBS -M srb@osc.edu
#PBS -l walltime=0:10:00
#PBS -l nodes=1:ppn=12
#PBS -S /bin/sh
qstat -f $PBS_JOBID
export
module load schrodinger
cd $PBS_O_WORKDIR
pbsdcp -p '*' $TMPDIR
cd $TMPDIR
host=`cat $PBS_NODEFILE|head -1`
nproc=`cat $PBS_NODEFILE|wc -l`
glide -WAIT -HOST ${host}:${nproc} receptor_glide.in
ls -l
pbsdcp -p '*' $PBS_O_WORKDIR

 

The glide command passes control to the Schrodinger Job Control utility which processes the two options: The WAIT option forces the glide command to wait until all tasks of the command are completed. This is necessary for PBS batch jobs to run effectively. The HOST option specifies how tasks are distributed over processors.

 

 

Usage of Macromodel on Glenn

A batch process is required when running MacroModel, either interactively or non-interactively. For interactive use with an X11 interface, you must be prepared to take part in an X11 session. Additionally, if you will use 3D graphics, your workstation must support the OpenGL X11 extensions. For a quick test of your connection, run the command "glxgears." If successful, animated gears will appear in a window on your terminal.

For more information about client-server protocols and X11, read the OSC FAQ web page, http://www.osc.edu/supercomputing/faq.shtml#sshwindows

As A Module
MacroModel is installed under the module paradigm; therefore you should have the modulefile macromodel loaded before you try to use MacroModel. 
module load macromodel

The latest installed version can be accessed via
module load macromodel9.8

Interactive
To interactively use the maestro interface, bring up an interactive batch job. [Here is a sample session] :

userid@opt-login1>  qsub -V -I -l walltime=01:00:00 -l nodes=1:ppn=1

[The use of the -V flag declares that all environment variables in the  qsub  command's environment are to be exported to the batch job. This will include the necessary environment settings for the X11 protocols.]
qsub: waiting for job 330978.nfs4.osc.edu:17001 to start
qsub: job 330978.nfs4.osc.edu:17001 ready

tset: unknown terminal type unknown
Terminal type? xterm
Enter UNIX terminal type, Terminal type? [vt100] xterm
[opt0747]%  module load macromodel
[opt0747]%  maestro


Maestro Molecular Modeling Interface
Copyright 1999-2003 Schr\uffffdinger, L.L.C.
Portions of this program include material copyrighted (c)
by Absoft Corporation 1988-1998
**********************************************************************
Checking environment variables...

SCHRODINGER:  /usr/local/schrodinger
MAESTRO_EXEC: /usr/local/schrodinger/maestro-v60107/bin/Linux-x86
LD_LIBRARY_PATH: /usr/local/schrodinger/maestro-v60107/lib/Linux-x86:
/usr/local/globus-2.4.3/lib:/usr/local/mpich-1.2.6-
ib/lib/shared:/usr/local/infiniband/lib:/usr/local/pvfs2/lib:
/usr/local/globus/lib:/usr/local/pgi-5.1.3/linux86/5.1/lib:/usr/local/mkl-
7.0.17/mkl70/lib/32:/usr/local/intel-8.0-20040716/lib
MMSHARE_EXEC: /usr/local/schrodinger/mmshare-v12014/bin/Linux-x86
MMOD_EXEC: /usr/local/schrodinger/macromodel-v85014/bin/Linux-x86

FIFO_LOCATION Environment Variable Not Defined
Setting and Using Default
FIFO_LOCATION  = /tmp/28729
TEMP_PROJECT Environment Variable Not Defined
Setting TEMP_PROJECT to $TMPDIR/$USER
TEMP_PROJECT  = /tmp/pbstmp.330978/userid
mkdir -p /tmp/pbstmp.330978/userid

Environment variables successfully initialized.  Starting up Maestro...
[After a minute, the maestro interface, the gui for all of Schrodinger's applications, will appear on your display.]

Non-interactive
For non-interactive use, a batch script is needed. [An example batch file follows.]:

my_mm_batch.job:
#PBS -l walltime=1:00:00
#PBS -l nodes=1:ppn=1
#PBS -N my_mm
#PBS -S /bin/ksh
#PBS -j oe
module load macromodel
bmin -WAIT inputfile

The macromodel commands, as well as maestro, pass control to the Schrodinger Job Control utility. The WAIT flag disables returning to the prompt, or scheduler, until the job is completed. This is necessary for PBS batch jobs to run effectively. When running interactively and using the macromodel commands from the command line, the use of WAIT option is recommended. Otherwise, the batch job might be exited, thus killing any processes initiated, before the macormodel processes are completed.

Documentation on Glenn

PDF formatted documentation may be downloaded using 'scp':

  1. $SCHRODINGER/docs/general/*/*.pdf
  2. $SCHRODINGER/docs/macromodel/*/*.pdf
  3. $SCHRODINGER/docs/maestro/*/*.pdf

 

Further Reading

Supercomputer: 
Service: 

SnpEff

SnpEff is a variant annotation and effect prediction tool. It annotates and predicts the effects of variants on genes (such as amino acid changes).

Availability and Restrictions

The following versions of SnpEff are available on OSC clusters:

Version Oakley owens
4.2 X* X*
*: Current default version

You can use  module avail snpeff  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

SnpEff is available to all OSC users without restriction.

Usage on Oakley

Set-up

To configure your environment for use of SnpEff, run the following command:   module load snpeff . The default version will be loaded. To select a particular SnpEff version, use   module load snpeff/version . For example, use  module load snpeff/4.2  to load SnpEff 4.2.

Usage

This software consists of Java executable .jar files; thus, it is not possible to add to the PATH environment variable.

From  module load snpeff , new environment variables, SNPEFF and SNPSIFT, will be set.

Thus, users can use the software by running the following command:  java -jar $SNPEFF {other options} , or java -jar $SNPSIFT {other options}

Usage on Owens

Set-up

To configure your environment for use of SnpEff, run the following command:   module load snpeff . The default version will be loaded. To select a particular SnpEff version, use   module load snpeff/version . For example, use module load snpeff/4.2 to load SnpEff 4.2.

Usage

This software consists of Java executable .jar files; thus, it is not possible to add to the PATH environment variable.

From  module load snpeff , new environment variables, SNPEFF and SNPSIFT, will be set.

Thus, users can use the software by running the following command:  java -jar $SNPEFF {other options} , orjava -jar $SNPSIFT {other options}

Further Reading

Supercomputer: 
Service: 
Fields of Science: 

Stata

Stata is a complete, integrated statistical package that provides everything needed for data analysis, data management, and graphics. Release 13 32-processor SMP is currently available at OSC.

Availability & Restrictions

Only academic use is allowed. Please contact oschelp@osc.edu to get validated for using Stata.

The following versions of Stata are available on OSC systems:

Version Glenn Oakley
13  

X

 

Stata is no longer available on Glenn. We have upgraded to Stata 13, and migrated to Oakley.

Usage

Set-up

To configure your environment on Oakley for the usage of Stata, run the following command:

module load stata

Using Stata

Due to licensing restrictions, Stata may ONLY be used via the batch system on Oakley. See below for information on how this is done.

Batch Usage

OSC has a 2-user license. However, there is no enforcement mechanism built into Stata. In order for us to stay within the 2-user limit, we require you to run in the context of PBS and to include this option when starting your batch job (the PBS system will enforce the 2 user limit):

#PBS -l software=stata

Non-Interactive batch example

Use the script below as a template for your usage.

#PBS -l walltime=01:00:00
#PBS -l nodes=1:ppn=12
#PBS -l software=stata
#PBS -N stata
#PBS -j oe

module load stata
cd $PBS_O_WORKDIR
stata-mp -b do bigjob

Interactive batch example

The following illustrates how to run Stata via an interactive batsh session. Please note that this requires an X11 server on the client system.

oakley01: qsub -X -I -l nodes=1:ppn=12 -l software=stata 
qsub: waiting for job 5860.oak-batch.osc.edu to start
qsub: job 5860.oak-batch.osc.edu ready

n0533: module load stata
n0533: xstata-mp
n0533: exit

Stata can take full advantage of our large 32 core, 1TB RAM node. Please see the Oakley documentation for information about how to access this node.

Further Reading

See Also

Supercomputer: 
Service: 

Subread

The Subread package comprises a suite of software programs for processing next-gen sequencing read data like Subread, Subjunc, featureCounts, and exactSNP.

Availability and Restrictions

The following versions of Subread are available on OSC clusters:

Version Oakley owens
1.5.0-p2 X* X*
*: Current default version

You can use  module avail subread  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Subread is available to all OSC users without restriction.

Usage on Oakley

Set-up

To configure your environment for use of Subread, run the following command:   module load subread . The default version will be loaded. To select a particular Subread version, use  module load subread/version . For example, use  module load subread/1.5.0-p2   to load Subread 1.5.0-p2.

Usage on Owens

Set-up

To configure your environment for use of Subread, run the following command:   module load subread . The default version will be loaded. To select a particular Subread version, use  module load subread/version . For example, use  module load subread/1.5.0-p2to load Subread 1.5.0-p2.

Further Reading

Supercomputer: 
Service: 
Fields of Science: 

SuiteSparse

SuiteSparse is a suite of sparse matrix algorithms, including: UMFPACK(multifrontal LU factorization), CHOLMOD(supernodal Cholesky, with CUDA acceleration), SPQR(multifrontal QR) and many other packages.

Availability and Restrictions

OSC supports most packages in SuiteSparse, including UMFPACK, CHOLMOD, SPQR, KLU and BTF, Ordering Methods (AMD, CAMD, COLAMD, and CCOLAMD) and CSparse. SuiteSparse modules are available for the Intel, GNU, and Portland Group compilers. The following versions of SuiteSparse are available at OSC.

Version oakley
4.4.4 X*
* : Current Default Version

You can use module spider suitesparse  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Notes:

  • SuiteSparse library on our clusters is built without METIS, which might matter if CHOLMOD package is included in your program.

Access

SuiteSparse is available without restriction to all OSC users.

Usage

Usage on Oakley

Set-up on Oakley

To use SuitSparse, loading the newest compilers is necessary. Here, for example, gnu/4.8.4 is loaded. module load gnu/4.8.4 Before load SuiteSparse library, MKL is required to be loaded.  module load mkl Now with the following command, SuiteSparse library is ready to be used. module load suitesparse  You can use module spider suitesparse to view available modules on Oakley.

Building With SuiteSparse

With the SuiteSparse library loaded, the following environment variables will be available for use:

Variable Use
$SUITESPARSE_CFLAGS Include flags for C or C++ programs.
$SUITESPARSE_LIBS Use when linking your program to SuiteSparse library.

For example, to build the code my_prog.c with the SuiteSparse library you would use:

icc -c my_prog.c
icc -o my_prog my_prog.o $SUITESPARSE_LIBS

Batch Usage on Oakley

When you log into oakley.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info.  Batch jobs run on the compute nodes of the system and not on the login node. It is desirable for big problems since more resources can be used.

Non-interactive Batch Job (Serial Run)
A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice. You must load the SuiteSparse module in your batch script before executing a program which is built with the SuiteSparse library. Below is the example batch script that executes a program built with SuiteSparse:
#PBS -N MyProgJob
#PBS -l nodes=1:ppn=12
module load gnu/4.8.4
module load mkl
module load suitesparse
cd $PBS_O_WORKDIR
cp foo.dat $TMPDIR
cd $TMPDIR
my_prog < foo.dat > foo.out
cp foo.out $PBS_O_WORKDIR

Further Reading

See Also

Supercomputer: 
Service: 

SuperLU

SuperLU is a library for the direct solution of large, sparse, nonsymmetric systems of linear equations on high performance machines.  It comes in two different flavors:  SuperLU_MT (multithreaded) for shared memory parallel machines and SuperLU_DIST for distributed memory parallel machines.

Availability and Restrictions

SuperLU is available on Glenn and Oakley Clusters. It comes in two different flavors:  SuperLU_MT (multithreaded) for shared memory parallel machines and SuperLU_DIST for distributed memory parallel machines. The versions currently available at OSC are:

Library Version Glenn Oakley notes
SuperLU_MT 2.0   X Default version on Oakley prior to 09/15/2015
SuperLU_MT 2.3 X*    
SuperLU_MT 3.0   X*  
SuperLU_DIST 2.3 X*    
SuperLU_DIST 3.0   X Default version on Oakley prior to 09/15/2015
SuperLU_DIST 3.1   X  
SuperLU_DIST 4.1   X*  

NOTE:

  • SuperLU is available for all compilers on Oakley
  • SuperLU is only available for Intel compilers on Glenn
* : Current Default Version

You can use module spider superlu  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

SuperLU is available to all OSC users without restriction.

Usage

Usage on Glenn

Set-up on Glenn

To use the SuperLU libraries in your compilation on Glenn, you must first load the acml module then load the appropriate superlu module as follows:

module load acml-intel
module load superlu-MT-intel
or
module load acml-intel
module load superlu-DIST-intel
You can use module avail superlu  to view available modules on Glenn.

Building With SuperLU on Glenn

Once loaded on Glenn, the SuperLU libraries can be linked in with your compilation. To do this, use the following environment variables:

Variable Use
$SUPERLU_INC Include path
$SUPERLU_LIBS Link flags for either library, depending on loaded module

Usage on Oakley

Set-up on Oakley

Initalizing the system for use of the SuperLU librarie is dependent on the system you are using and the compiler you are using. To use the SuperLU libraries in your compilation on Oakley, run the following command: module load superlu_mt . or  module load superlu_dist . You can use module spider superlu  to view available modules on Oakley.

Building With SuperLU on Oakley

Once loaded on Oakley, the SuperLU libraries can be linked in with your compilation. To do this, use the following environment variables:

Variable Use
$SUPERLU_MT_CFLAGS Include path for multithreaded libraries
$SUPERLU_MT_LIBS Link flags for multithreaded libraries
$SUPERLU_DIST_CFLAGS Include path for distributed-memory libraries
$SUPERLU_DIST_LIBS Link flags for distributed-memory libraries

Further Reading

See Also

Supercomputer: 
Service: 

TotalView Debugger

TotalView is a symbolic debugger which supports threads, MPI, OpenMP, C/C++ and Fortran, plus mixed-language codes.  Advanced features include on-demand memory leak detection, heap allocation debugging and the Standard Template Library Viewer (STLView).  Other features like dive, a wide variety of breakpoints, the Message Queue Graph/Visualizer, powerful data analysis and control at the thread level give you the power you need to solve tough problems.  

Availability and Restrictions

TotalView is available on Oaklye, Ruby, and Owens Clusters. The versions currently available at OSC are:

Version Oakley Ruby Owens notes
8.9.2-1 X(CS)     Default version on Oakley prior to 09/15/2015
8.14.1-8 X* X    
2016.04.08     X(S)*  
*: Current default version

NOTE:

  • X(S): standard version
  • X(CS): Both standard and CUDA versions are available
  • X: CUDA debugging included in standard module

You can use module avail totalview to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access and Licensing

TotalView is available for use by all OSC users.  The installed license permits the use of 32 tokens (processes).  This means that the maximum parallel tasks for a TotalView session is 32.  If the token limit is exceeded an error with the text "Unable to acquire a license for X processes." will be presented in a message box.

Usage on Oakley

Building your code for debugging with TotalView

You should build your code with the -g option for symbolic debugging.  For example:

icc -g -c mycode.c

or

mpicc -g -c mympicode.c

Set-up

To load the default standard version of Totalview module, use  module load totalview . To select a particular software version, use   module load totalview/version . For example, use  module load totalview/8.9.2-1 to load Totalview version 8.9.2-1. 

Using Totalview

To debug a serial or multithreaded program with TotalView, load the module and launch using the following commands. The -a argument is used only if you want to pass arguments to your program on the command line.

module load totalview
totalview mycode -a prog_args

Example code to debug can be found in $TOTALVIEW_HOME/linux-x86-64/examples .  Refer to the file README.TXT in this directory for instructions for using these examples.

To debug an MPI program, use the following commands in an interactive job:

module load totalview
mpiexec -tv [other mpiexec arguments] mympiprog [program arguments]

Totalview will stop your program at the beginning to allow you to debug. For a parallel program, the initial breakpoint is in mpiexec. Press "Go" and you will be given the option of stopping when your job goes parallel. Normally it will stop inside MPI_Init. For a parallel job running on a single node, however, it can run past that point and stop later in your program. You may need to have your program sleep for 30 seconds at the beginning.

In C:

sleep(30);

In Fortran:

call sleep(30)

Usage on Ruby

Building your code for debugging with TotalView

You should build your code with the -g  option for symbolic debugging.  For example:

icc -g -c mycode.c

or

mpicc -g -c mympicode.c

To debug an MPI program, you must build with a debug version of mvapich2.  Load the module before building code, for example:

module load mvapich2/2.1-debug

Set-up

To load the default standard version of Totalview module, use  module load totalview . To select a particular software version, use   module load totalview/version . For example, use  module load totalview/8.14.1-8 to load Totalview version 8.14.1-8. 

Using Totalview

To debug a serial or multithreaded program with TotalView, load the module and launch using the following commands. The -a argument is used only if you want to pass arguments to your program on the command line.

module load totalview
totalview mycode -a prog_args

Example code to debug can be found in $TOTALVIEW_HOME/linux-x86-64/examples .  Refer to the file README.TXT in this directory for instructions for using these examples.

Use the following commands in an interactive job:

module load totalview
totalview -args mpiexec [mpiexec arguments] ./mympiprog [program arguments]

Totalview will stop your program at the beginning to allow you to debug. For a parallel program, the initial breakpoint is in mpiexec. Press "Go" and you will be given the option of stopping when your job goes parallel. Normally it will stop inside MPI_Init. For a parallel job running on a single node, however, it can run past that point and stop later in your program. You may need to have your program sleep for 30 seconds at the beginning.

In C:

sleep(30);

In Fortran:

call sleep(30)

Usage on Owens

Building your code for debugging with TotalView

You should build your code with the -g  option for symbolic debugging.  For example:

icc -g -c mycode.c

or

mpicc -g -c mympicode.c

To debug an MPI program, you must build with a debug version of mvapich2.  Load the module before building code, for example:

module load mvapich2/2.2

Set-up

To load the default standard version of Totalview module, use  module load totalview . To select a particular software version, use   module load totalview/version . For example, use  module load totalview/2016.04.08 to load Totalview version 2016.04.08. 

Using Totalview

To debug a serial or multithreaded program with TotalView, load the module and launch using the following commands. The -a argument is used only if you want to pass arguments to your program on the command line.

module load totalview
totalview mycode -a prog_args

Example code to debug can be found in $TOTALVIEW_HOME/linux-x86-64/examples .  Refer to the file README.TXTin this directory for instructions for using these examples.

Use the following commands in an interactive job:

module load totalview
totalview -args mpiexec [mpiexec arguments] ./mympiprog [program arguments]

Totalview will stop your program at the beginning to allow you to debug. For a parallel program, the initial breakpoint is in mpiexec. Press "Go" and you will be given the option of stopping when your job goes parallel. Normally it will stop inside MPI_Init. For a parallel job running on a single node, however, it can run past that point and stop later in your program. You may need to have your program sleep for 30 seconds at the beginning.

In C:

sleep(30);

In Fortran:

call sleep(30)

Further Reading

  • On Oakley and Ruby, TotalView documentation in pdf format can be found in  $TOTALVIEW_HOME/doc/pdf
  • Additional information about TotalView can be found at the Rogue Wave Software page.  
Supercomputer: 
Service: 
Fields of Science: 

TreeBeST

"TreeBeST is an original tree builder for constrained neighbour-joining and tree merge, an efficient tool capable of duplication/loss/ortholog inference, and a versatile program facili- tating many tree-building routines, such as tree rooting, alignment filtering and tree plot- ting. TreeBeST stands for ‘(gene) Tree Building guided by Species Tree’. It is previously known as NJTREE as the first piece of codes of this project aimed to build a eighbour-joining tree.

TreeBeST is the core engine of TreeFam (Tree Families Database) project initiated by Richard Durbin. The basic idea of this project is to build a full tree constrained by a manually verified seed tree. The tree builder must know how to utilize the prior knowledge provided by human experts. This demand disqualifies any existing softwares. Given this fact, we devised a new algorithm to control the joining step of traditional neighbour-joining. This is origin the constrained neighbour-joining.

When trees are built, they are only meaningful to biologists. Computers generate trees, but they do not understand them. To understand gene trees, a computer must be equipped with some biological knowledges, the species tree. It will teach a computer how to discriminate a speciation from a duplication event and how to find orthologs, provided a correct gene tree.

Unfortunately, gene trees are not always correct. Since the advent of UPGMA algorithm in 1958, we have tried to find a ideal model for nearly half a century. But we failed. Evolution is so complex a thing. A model best fits in one lineage might mean a disaster in another. A unified model is far from being discovered. TreeBeST aims at improving the accuracy of tree building, but it does not try to set up a new model in a traditional way. Instead, it integrates two existing models with the help of species tree, finding the subtree that best fits the models and merging them together to build a new tree incorporating the advantages of the both. This is the tree algorithm." (treebest.pdf)

Availability & Restrictions

TreeBeST is available to all OSC users without restriction.

The following versions of TreeBeST are available on OSC systems:

Version Glenn Oakley
1.9.2 X  

Usage

Set up

On the Glenn Cluster TreeBeST is accessed by executing the following commands:

module load biosoftw
module load treebest

TreeBeST will be added to the users PATH and can then be run with the command:

treebest <command> [options]

Below lists the commands and their summaries for the treebest program.

Command

nj           build neighbour-joining tree, SDI, rooting
best         build tree with the help of a species tree
phyml        build phyml tree
sdi          speciation vs. duplication inference
spec         print species tree
format       reformat a tree
filter       filter a multi-alignment
trans        translate coding nucleotide alignment
backtrans    translate aa alignment back to nt
leaf         get external nodes
mfa2aln      convert MFA to ALN format
ortho        ortholog/paralog inference
distmat      distance matrix
treedist     topological distance between two trees
pwalign      pairwise alignment
mmerge       merge a forest
export       export a tree to EPS format
subtree       extract the subtree
simulate     simulate a gene tree
sortleaf     sort leaf order
estlen       estimate branch length
trimpoor     trim out leaves that affect the quality of a tree
root         root a tree by minimizing height

Options

treebest nj [options] <input_file>
      -c FILE          constrained tree(s) in NH format [null]
      -m FILE          tree to be compared [null]
      -s FILE          species tree in NH format [default taxa tree]
      -l FILE          ingroup list file [null]
      -t TYPE          codon NT: ntmm, dn, ds, dm; AA: mm, jtt, kimura [mm]
                       ntmm          p-distance (codon alignment)
                       dn            non-synonymous distance
                       ds            synonymous distance
                       dm            dn-ds merge (tree merge)
                       mm            p-distance (amino acid alignment)
                       jtt           JTT model (maximum likelihood)
                       kimura        mm + Kimura's correction
      -T NUM           time limit in seconds [no limit]
      -b NUM           bootstrapping times [100]
      -F NUM           quality cut-off [15]
      -o STR           outgroup for tree cutting [Bilateria]
      -S               treat the first constrained tree as the original tree
      -C               use the leaves of constrained trees as ingroup
      -M               do not apply alignment mask
      -N               do not mask poorly aligned segments
      -g               collapse alternative splicing
      -R               do not apply leaf-reordering
      -p               the root node is a putative node
      -a               branch mode that is used by most tree-builder
      -A               the input alignment is stored in ALN format
      -W               wipe out root (SDI information will be lost!)
      -v               verbose output
      -h               help
treebest best [options] <CDS_alignment>        
            General Options:
            -P               skip PHYML
            -S               ignore the prob. of gene evolution (NOT recommended)
            -A               apply constraint to PHYML
            -C FILE          constraining tree [null]
            -f FILE          species tree [default]
            -r               discard species that do not appear at all
            Output Options:
            -D               output some debug information
            -q               suppress part of PHYML warnings
            -p STR           prefix of intermediate trees [null]
            -o FILE          output tree [null]
            Alignment Preprocessing Options:
            -s               only build tree for genes from sequenced species
            -g               collapse alternative splicing forms
            -N               do not mask low-scoring segments
            -F INT           quality cut-off [11]
            PHYML Related Options:
            -c INT           number of rate categories for PHYML-HKY [2]
            -k FLOAT|e       tv/ts ratio (kappa), 'e' for estimatinig [e]
            -a FLOAT|e       alpha parameter for Gamma distribution [1.0]
            -d FLOAT         duplication probability [0.15]
            -l FLOAT         probability of a loss following a speciation [0.10]
            -L FLOAT         probability of a loss following a duplication [0.20]
            -b FLOAT         prob. of the presence of an inconsistent branch [0.01]
treebest phyml <alignment> [<tree>]
            General Options:
            -t task          build | opt | loglk | dist [build]
            -n               the input is a nucleotide alignment
            -s               print out some statistics
            -N               do not mask low-scoring segments
            -g               collapse alternative splicing
            -b INT           number of bootstraps (slow) [0]
            -o FILE          write output to file [stdout]
            -F INT           quality cut-off [15]
            Model Related Options:
            -m model         nt: JC69 | K2P | F81 | HKY | F84 | TN93 | GTR [HKY]
                             aa: JTT | MtREV | Dayhoff | WAG [WAG]
            -c INT           number of relative substitution rate categories [1]
            -k FLOAT|e       transversion/transition ratio, 'e' for estimating [e]
            -a FLOAT|e       alpha parameter for Gamma distribution [1.0]
            -i FLOAT|e       proportion of invariable sites [0]
            Options for TreeFam Extensions:
            -S               use a species tree to guide tree building
            -f FILE          species tree [TreeFam species tree]
            -d FLOAT         duplication probability  [0.15]
            -l FLOAT         probability of a loss following a speciation [0.10]
            -L FLOAT         probability of a loss following a duplication [0.20]
            -C FILE          constraining tree [NULL]
            -p FLOAT         prob. of the presence of an inconsistent branch [0.01]
treebest sdi [-r|-H|-R|-m <tree0>|-l <spec_list>] <tree>
            Options:
            -r               reroot
            -c               use core species tree instead of the default one
            -H               reroot by minimizing tree height, instead of by minimizing the number of duplication events.
            -R               do not reorder the leaves.
            -s FILE          species tree [default taxa tree]
            -l FILE          cut a subtree that contains genes whose species exist in list [null]
            -m FILE          compare topology with FILE and re-order the leaves [null]
treebest spec
treebest format [-1] <tree>
treebest filter [options] <alignment> 
            Options:
            -n               nucleotide alignment
            -g               collapse alternative splicing
            -M               do not apply alignment mask
            -N               do not mask low-scoring segments
            -F NUM           quality cut-off [15]
treebest trans <nucl_alignment>
treebest backtrans [-t <thres>] <aa_aln> <nt_seq>
treebest leaf <nh_tree>
treebest mfa2aln [-n] <fasta_align>
treebest ortho <tree>
treebest distmat <dn|ds|dm|jtt|kimura|mm|dns> <alignment>
treebest treedist <tree1> <tree2>
treebest pwalign [options] <nt2nt|aa2aa|nt2aa|splice> <seq1> <seq2> 
            Options :
            -f               generate full alignment
            -a               do not apply matrix mean in local alignment
            -d               just calculate alignment boundaries
            -o NUM           gap open penalty
            -e NUM           gap extension penalty
            -n NUM           gap end penalty for nt2nt or aa2aa
            -s NUM           frame-shift penalty for aa2nt
            -g NUM           good splicing penalty
            -w NUM           band-width
            -b NUM           bad splicing penalty
            -m               output miscellaneous information
            -h               help
treebest mmerge [-r <forest>
            Options:
            -r               reroot
treebest export [options] <tree>
            Options:
            -x NUM           width [640]
            -y NUM           height [480]
            -m NUM           margin [20]
            -f NUM           font size [11]
            -b FNUM          box size [4.0]
            -w FNUM          font width [font_size/2]
            -s FILE          species tree
            -B               suppress bootstrap value
            -M               black/white mode
            -S               show species name
            -d               speciation/duplication inference
            -p               pseudo-length
treebest subtree <tree> <list>
treebest simulate [options] 
            Options:
            -d FNUM          duplication probability [0.05]
            -l FNUM          loss probability [0.01]
            -p FNUM          loss probability after duplication [0.25]
            -m FNUM          max height [0.25]
            -n               not show internal name
            -h               help
treebest sortleaf <tree1> [<tree2>]
treebest estlen <tree> <matrix> <tag>
treebest trimpoor <tree> [<threshold>=0>]
treebest root <tree> 

Examples

#PBS -N treebest_test
#PBS -l walltime=00:05:00
#PBS -l nodes=1:ppn=4

cd $PBS_O_WORKDIR
module load biosoftw
module load treebest
cp /usr/local/biosoftw/treebest-1.9.2/examples/ex1.nucl.* .

treebest nj ex1.nucl.mfa > ex1.nucl.1.nhx
cp ex1.nucl.nhx ex1.nucl.1.forest
cat ex1.nucl.1.nhx >> ex1.nucl.1.forest
treebest nj -m ex1.nucl.nhx ex1.nucl.mfa > ex1.nucl.2.nhx
treebest nj -v ex1.nucl.mfa
treebest best ex1.nucl.mfa -o ex1.nucl.3.nhx
treebest best -c 1 -a 0.9 -d 0.14 -l 0.09 -L 0.19 -b 0.009 -o ex1.nucl.4.nhx ex1.nucl.mfa
treebest phyml -o ex1.nucl.1.nh ex1.nucl.mfa
treebest phyml -o ex1.nucl.2.nh ex1.nucl.mfa ex1.nucl.nhx
treebest phyml -s -C ex1.nucl.nhx -o ex1.nucl.4.nh ex1.nucl.mfa
treebest phyml -b 2 -o ex1.nucl.5.nh ex1.nucl.mfa
treebest sdi ex1.nucl.nhx > ex1.nucl.5.nhx
treebest sdi -r ex1.nucl.nhx > ex1.nucl.6.nhx
treebest sdi -r ex1.nucl.nhx > ex1.nucl.7.nhx
treebest spec > all_species.nh
treebest format ex1.nucl.nhx
treebest filter -n -M -N ex1.nucl.mfa > ex1.nucl.1.mfa
treebest trans ex1.nucl.mfa > ex1.aa.mfa
treebest backtrans ex1.aa.mfa ex1.nucl.mfa > ex1.nucl.2.mfa
treebest leaf ex1.nucl.nhx > ex1.nucl.1.leaf
head ex1.nucl.1.leaf | tail -7 > ex1.nucl.1.sublist
treebest mfa2aln -n ex1.nucl.mfa > ex1.nucl.1.aln
treebest ortho ex1.nucl.nhx > ex1.nucl.1.ortho
treebest distmat dn ex1.nucl.mfa > ex1.nucl.1.matrix.dn
treebest distmat ds ex1.nucl.mfa > ex1.nucl.1.matrix.ds
treebest distmat dm ex1.nucl.mfa > ex1.nucl.1.matrix.dm
treebest treedist ex1.nucl.nhx ex1.nucl.1.nhx > ex1.nucl.1.dist
treebest mmerge -r ex1.nucl.1.forest > ex1.nucl.8.nhx
treebest export ex1.nucl.nhx > ex1.nucl.1.eps
treebest subtree ex1.nucl.nhx ex1.nucl.1.sublist > ex1.nucl.9.nhx
treebest simulate > ex1.nucl.6.nh
treebest simulate -d 0.04 -l 0.02 -p 0.5 -m 0.1 > ex1.nucl.7.nh
treebest sortleaf ex1.nucl.nhx > ex1.nucl.sorted.nhx
treebest sortleaf ex1.nucl.nhx ex1.nucl.1.nhx > ex1.nucl.sorted.2.nhx
treebest estlen ex1.nucl.nhx ex1.nucl.1.matrix.ds ds_method > ex1.nucl.1.estlen.ds.nhx
treebest trimpoor ex1.nucl.nhx > ex1.nucl.10.nhx
treebest root ex1.nucl.nhx > ex1.nucl.11.nhx

Further Reading

Supercomputer: 
Service: 
Fields of Science: 

Trimmomatic

Trimmomatic performs a variety of useful trimming tasks for illumina paired-end and single ended data.The selection of trimming steps and their associated parameters are supplied on the command line.

Availability and Restrictions

The following versions of Trimmomatic are available on OSC clusters:

Version Oakley Owens
0.36 X* X*
*: Current default version

You can use  module avail trimmomatic   to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Trimmomatic is available to all OSC users without restriction.

Usage on Oakley

Set-up

To configure your environment for use of Trimmomatic, run the following command:   module load trimmomati. The default version will be loaded. To select a particular Trimmomatic version, use   module load trimmomatic/version. For example, use  module load trimmomatic/0.36 to load Trimmomatic 0.36.

Usage

This software is a Java executable .jar file; thus, it is not possible to add to the PATH environment variable.

From module load trimmomatic, a new environment variable, TRIMMOMATIC, will be set.

Thus, users can use the software by running the following command: java -jar $TRIMMOMATIC {other options}

Usage on Owens

Set-up

To configure your environment for use of Trimmomatic, run the following command:   module load trimmomati. The default version will be loaded. To select a particular Trimmomatic version, use   module load trimmomatic/version. For example, use  module load trimmomatic/0.36 to load Trimmomatic 0.36.

Usage

This software is a Java executable .jar file; thus, it is not possible to add to the PATH environment variable.

From module load trimmomatic, a new environment variable, TRIMMOMATIC, will be set.

Thus, users can use the software by running the following command: java -jar $TRIMMOMATIC {other options}

Further Reading

Supercomputer: 
Service: 
Fields of Science: 

TurboVNC

TurboVNC is an implementation of VNC optimized for 3D graphics rendering.  Like other VNC software, TurboVNC can be used to create a virtual desktop on a remote machine, which can be useful for visualizing CPU-intensive graphics produced remotely.

Availability and Restrictions

TurboVNC is available on Oakley and Glenn Clusters. The versions currently available at OSC are:

VERSION GLENN OAKLEY notes
1.0 X    
1.1 X* [1] Default version on Oakley prior to 09/15/2015
1.2 X X*[2]  
*: Current default version

NOTE:

  • [1] -- TurboVNC 1.1's version of vncviewer does not work on Oakley.  Use the 1.2 module's version.
  • [2] -- TurboVNC 1.2's version of vncserver does not work on Oakley.  Use the 1.1 module's version.

You can use  module avail turbovnc  to view available modules for a given machine. Feel free to contact OSC Help  if you need other versions for your work.

Access

TurboVNC is available for use by all OSC users.

Usage

Usage on Glenn

Set-up on Glenn

To load the default version of TurboVNC module, use  module load turbovnc . To select a particular software version, use   module load turbovnc-version . For example, use  module load turbovnc-1.1  to load TurboVNC version 1.1 on Glenn. 

Please do not SSH directly to compute nodes and start VNC sessions! This will negatively impact other users (even if you have been assigned a node via the batch scheduler), and we will consider repeated occurances an abuse of the resources. If you need to use VNC on a compute node, please see our HOWTO for instructions.

Using TurboVNC

To start a VNC server on your current host, use the following command:

vncserver  

After starting the VNC server you should see output similar to the following:  

New 'X' desktop is hostname:display
Starting applications specified in /nfs/nn/yourusername/.vnc/xstartup.turbovnc
Log file is /nfs/nn/yourusername/.vnc/hotsname:display.log

Make a note of the hostname and display number ("hostname:display"), because you will need this information later in order to connect to the running VNC server.  

To establish a standard unencrypted connection to an already running VNC server, X11 forwarding must first be enabled in your SSH connection.  This can usually either be done by changing the preferences or settings in your SSH client software application, or by using the -X or -Y option on your ssh command.     

Once you are certain that X11 forwarding is enabled, create your VNC desktop using the vncviewer  command in a new shell.

vncviewer

You will be prompted by a dialogue box asking for the VNC server you wish to connect to.  Enter "hostname:display".  

You may then be prompted for your HPC password.  Once the password has been entered your VNC desktop should appear, where you should see all of your home directory contents. 

When you are finished with your work on the VNC desktop, you should make sure to close the desktop and kill the VNC server that was originally started.  The VNC server can be killed using the following command in the shell where the VNC server was originally started:

vncserver -kill :[display]

For a full explanation of each of the previous commands, type man vncserver  or man vncviewer  at the command line to view the online manual.

Usage on Oakley

Set-up on Oakley

To load the default version of TurboVNC module, use  module load turbovnc . To select a particular software version, use   module load turbovnc/version . For example, use  module load turbovnc/1.1  to load TurboVNC version 1.1 on Oakley. 

Please do not SSH directly to compute nodes and start VNC sessions! This will negatively impact other users (even if you have been assigned a node via the batch scheduler), and we will consider repeated occurances an abuse of the resources. If you need to use VNC on a compute node, please see our HOWTO for instructions.

Using TurboVNC

To start a VNC server on your current host, use the following command:

vncserver  

After starting the VNC server you should see output similar to the following:  

New 'X' desktop is hostname:display
Starting applications specified in /nfs/nn/yourusername/.vnc/xstartup.turbovnc
Log file is /nfs/nn/yourusername/.vnc/hotsname:display.log

Make a note of the hostname and display number ("hostname:display"), because you will need this information later in order to connect to the running VNC server.  

To establish a standard unencrypted connection to an already running VNC server, X11 forwarding must first be enabled in your SSH connection.  This can usually either be done by changing the preferences or settings in your SSH client software application, or by using the -X or -Y option on your ssh command.     

Once you are certain that X11 forwarding is enabled, create your VNC desktop using the vncviewer  command in a new shell.

vncviewer

You will be prompted by a dialogue box asking for the VNC server you wish to connect to.  Enter "hostname:display".  

You may then be prompted for your HPC password.  Once the password has been entered your VNC desktop should appear, where you should see all of your home directory contents. 

When you are finished with your work on the VNC desktop, you should make sure to close the desktop and kill the VNC server that was originally started.  The VNC server can be killed using the following command in the shell where the VNC server was originally started:

vncserver -kill :[display]

For a full explanation of each of the previous commands, type man vncserver  or man vncviewer  at the command line to view the online manual.

Further Reading

Additional information about TurboVNC can be found at the VirtualGL Project's documentation page.  

See Also

Supercomputer: 
Service: 
Fields of Science: 

Turbomole

TURBOMOLE is an ab initio computational chemistry program that implements various quantum chemistry algorithms. It is developed at the group of Prof. Reinhart Ahlrichs at the University of Karlsruhe.

Availability and Restrictions

Turbomole is available on both Oakley and Glenn Clusters. The versions currently available at OSC are:

VERSION GLENN OAKLEY notes
5.9.0 serial X    
parallel X  
5.9.1 serial X    
parallel    
5.10.0 serial X    
parallel X
6.0.0 serial X*    
parallel X*
6.2 serial X    
parallel  
6.3.1 serial X X Default version on Oakley prior to 09/15/2015
parallel   X Default version on Oakley prior to 09/15/2015
6.4 serial   X  
parallel   X  
6.5 serial   X*  
parallel   X*  
7.0.1 serial   X  
parallel   X  
*: Current default version

You can use module avail turbomole  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access for Academic Users

Use of Turbomole for academic purposes requires validation. In order to obtain validation, please contact OSC Help for further instruction. 

Usage

Usage on Glenn

Set-up on Glenn

To load the default version of Turbomole module on Glenn, use  module load turbomole  for serial programs and  module load turbomole-parallel  for parallel programs (the "turbomole-parallel" prefix is needed for loading Turbomole's parallel module).  To select a particular software version, use   module load turbomole-version for serial programs and  module load turbomole-parallel-version  for parallel programs. For example, use  module load turbomole-6.3.1   to load Turbomole version 6.3.1 for serial programs on Glenn. 

Using Turbomole on Glenn

To execute a turbomole program for serial programs:

module load turbomole
<turbomole command>

To execute a turbomole program for for parallel programs:

module load turbomole-parallel
<turbomole command>

Batch Usage on Glenn

When you log into glenn.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time, up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. Batch jobs run on the compute nodes of the system and not on the login node. It is desirable for big problems since more resources can be used.

Interactive Batch Session

For an interactive batch session one can run the following command:

qsub -I -l nodes=1:ppn=8 -l walltime=00:20:00 
which requests one whole node with 8 cores ( -l nodes=1:ppn=8 ), for a walltime of 20 minutes ( -l walltime=00:20:00 ). You may adjust the numbers per your need.

Usage on Oakley

Set-up on Oakley

To load the default version of Turbomole module on Oakley, use  module load turbomole  for both serial and parallel programs. To select a particular software version, use   module load turbomole/version . For example, use  module load turbomole/6.4   to load Turbomole version 6.4 for both serial and parallel programs on Oakley. 

Using Turbomole on Oakley

To execute a turbomole program:

module load turbomole
<turbomole command>

Batch Usage on Oakley

When you log into oakley.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info.  Batch jobs run on the compute nodes of the system and not on the login node. It is desirable for big problems since more resources can be used.

Interactive Batch Session

For an interactive batch session one can run the following command:

qsub -I -l nodes=1:ppn=12 -l walltime=00:20:00

which requests one core ( -l nodes=1:ppn=12 ), for a walltime of 20 minutes ( -l walltime=00:20:00 ). You may adjust the numbers per your need.

Further Reading

Supercomputer: 
Service: 

VASP

The Vienna Ab initio Simulation Package, VASP, is a program package for ab-initio quantum-mechanical molecular dynamics (MD) simulations and electronic structure calculations, from first principles.

Availability and Restrictions

Access

Due to licensing considerations, OSC does not provide general access to this software.

However, we are available to assist with the configuration of individual research-group installations on all our systems. See the The VASP FAQ page or contact OSC Help for information regrading licensing.

Usage

Using VASP

See the VASP documentation page for tutorial materials.

Further Reading

See Also

Service: 

VCFtools

VCFtools is a program package designed for working with VCF files, such as those generated by the 1000 Genomes Project. The aim of VCFtools is to provide easily accessible methods for working with complex genetic variation data in the form of VCF files.

Availability and Restrictions

The following versions of VCFtools are available on OSC clusters:

Version Oakley Owens
0.1.14 X* X*
*: Current default version

You can use  module avail vcftools to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

VCFtools is available to all OSC users without restriction.

Usage on Oakley

Set-up

To configure your environment for use of VCFtools, run the following command:   module load vcftools. The default version will be loaded. To select a particular VCFtools version, use   module load vcftools/version . For example, use  module load vcftools/0.1.14 to load VCFtools 0.1.14.

Usage on Owens

Set-up

To configure your environment for use of VCFtools, run the following command:   module load vcftools. The default version will be loaded. To select a particular VCFtools version, use   module load vcftools/version . For example, use  module load vcftools/0.1.14 to load VCFtools 0.1.14.

Further Reading

Supercomputer: 
Service: 
Fields of Science: 

VTK

Introduction

The Visualization ToolKit (VTK) is an open source, freely available software system for 3D computer graphics, image processing and visualization. VTK consists of a C++ class library and several interpreted interface layers including Tcl/Tk, Java and Python.

Version

 

Version Glenn Oakley
5.0.4 X  

Usage

You can use the following command to load the VTK module:

module load vtk

VTK example code can be found on the Glenn cluster in $VTK_DIR/Examples. Refer to the file README.txt in this directory for information about the content of the examples.

Further Reading

Links to additional documentation can be found online at http://www.vtk.org/doc/release/5.0/html/.

Supercomputer: 
Service: 
Fields of Science: 

VarScan

VarScan is a platform-independent software tool developed at the Genome Institute at Washington University to detect variants in NGS data.

Availability and Restrictions

The following versions of VarScan are available on OSC clusters:

Version Oakley Owens
2.4.1 X* X*
*: Current default version

You can use  module avail varscan to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

VarScan is available to all OSC users without restriction.

Usage on Oakley

Set-up

To configure your environment for use of VarScan, run the following command:   module load varscan. The default version will be loaded. To select a particular VarScan version, use   module load varscan/version . For example, use  module load varscan/2.4.1 to load VarScan 2.4.1.

Usage

This software is a Java executable .jar file; thus, it is not possible to add to the PATH environment variable.

From module load varscan , a new environment variable, VARSCAN, will be set.

Thus, users can use the software by running the following command:  java -jar $VARSCAN {other options}

Usage on Owens

Set-up

To configure your environment for use of VarScan, run the following command:   module load varscan. The default version will be loaded. To select a particular VarScan version, use   module load varscan/version . For example, use  module load varscan/2.4.1 to load VarScan 2.4.1.

Usage

This software is a Java executable .jar file; thus, it is not possible to add to the PATH environment variable.

From module load varscan , a new environment variable, VARSCAN, will be set.

Thus, users can use the software by running the following command:  java -jar $VARSCAN {other options}

Further Reading

Supercomputer: 
Service: 
Fields of Science: 

WARP3D

From WARP3D's webpage:

WARP3D is under continuing development as a research code for the solution of large-scale, 3-Dsolid models subjected to static and dynamic loads. The capabilities of the code focus on 
fatigue & fracture analyses primarily in metals. WARP3D runs on laptops-to-supercomputers and can analyze models with several million nodes and elements. 

Availability and Restrictions

Versions

The following versions of WARP3D are available on OSC clusters:

version oakley Owens
17.5.3 X  
17.7.1   X

 

Feel free to contact OSC Help if you need other versions for your work.

Access 

WARP3D is available to all OSC users without restriction.

Usage

Usage on Owens

Setup on Owens

To configure the Owens cluster for the use of WARP3D, use the following commands:

module load intelmpi
module load warp3d

Batch Usage on Owens

Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations for OakleyQueues and Reservations for Ruby, and Scheduling Policies and Limits for more info.

Running WARP3D

Below is an example batch script ( job.txt) for using WARP3D:

#PBS -N WARP3D
#PBS -l nodes=1:ppn=28
#PBS -l walltime=30:00
#PBS -j oe 
#PBS -S /bin/bash 

# Load the modules for WARP3D
module load intelmpi
module load warp3d
# Copy files to $TMPDIR and move there to execute the program
cp $WARP3D_HOME/example_problems_for_READMEs/mt_cohes_*.inp $TMPDIR
cd $TMPDIR
# Run the solver using 4 MPI tasks and 6 threads per MPI task 
$WARP3D_HOME/warp3d_script_linux_hybrid 4 6 < mt_cohes_4_cpu.inp
# Finally, copy files back to your home directory 
cp -r * $PBS_O_WORKDIR

In order to run it via the batch system, submit the job.txt  file with the following command:

qsub job.txt

Further Reading

http://www.warp3d.net

https://github.com/rhdodds/warp3d

Supercomputer: 
Service: 
Fields of Science: 

bam2fastq

bam2fastq is used to extract raw sequences (with qualities) from programs like SAMtools, Picard, and Bamtools.

Availability and Restrictions

The following versions of bam2fastq are available on OSC clusters:

Version Oakley owens
1.1.0 X* X*
*: Current default version

You can use  module avail bam2fastq  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

bam2fastq is available to all OSC users without restriction.

Usage on Oakley

Set-up

To configure your environment for use of bam2fastq, run the following command:   module load bam2fastq . The default version will be loaded. To select a particular bam2fastq version, use   module load bam2fastq/version . For example, use  module load bam2fastq/1.1.0  to load bam2fastq 1.1.0.

Usage on Owens

Set-up

To configure your environment for use of bam2fastq, run the following command:   module load bam2fastq . The default version will be loaded. To select a particular bam2fastq version, use   module load bam2fastq/version . For example, use  module load bam2fastq/1.1.0  to load bam2fastq 1.1.0.

Further Reading

Supercomputer: 
Service: 
Fields of Science: 

bcftools

bcftools is a set of utilities that manipulate variant calls in the Variant Call Format (VCF) and its binary counterpart BCF.

Availability and Restrictions

The following versions of bcftools are available on OSC clusters:

Version Oakley OWEns
1.3.1 X* X*
*: Current default version

You can use  module avail bcftools to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

bcftools is available to all OSC users without restriction.

Usage on Oakley

Set-up

To configure your environment for use of bcftools, run the following command:   module load bcftools. The default version will be loaded. To select a particular bcftools version, use   module load bcftools/version . For example, use  module load bcftools/1.3.1 to load bcftools 1.3.1.

Usage on Owens

Set-up

To configure your environment for use of bcftools, run the following command: module load bcftools. The default version will be loaded. To select a particular bcftools version, use   module load bcftools/version . For example, use  module load bcftools/1.3.1 to load bcftools 1.3.1.

Further Reading

Supercomputer: 
Service: 
Fields of Science: 

bedtools

Collectively, the bedtools utilities are a swiss-army knife of tools for a wide-range of genomics analysis tasks. The most widely-used tools enable genome arithmetic: that is, set theory on the genome. While each individual tool is designed to do a relatively simple task, quite sophisticated analyses can be conducted by combining multiple bedtools operations on the UNIX command line.

Availability and Restrictions

The following versions of bedtools are available on OSC clusters:

Version Oakley Owens
2.25.0 X* X*
*: Current default version

You can use  module avail bedtools  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

bedtools is available to all OSC users without restriction.

Usage on Oakley

Set-up

To configure your environment for use of bedtools, run the following command:   module load bedtools . The default version will be loaded. To select a particular bedtools version, use   module load bedtools/version  . For example, use  module load bedtools/2.25.0   to load bedtools 2.25.0.

Usage on Owens

Set-up

To configure your environment for use of bedtools, run the following command:   module load bedtools . The default version will be loaded. To select a particular bedtools version, use   module load bedtools/version  . For example, use  module load bedtools/2.25.0   to load bedtools 2.25.0.

Further Reading

Supercomputer: 
Service: 
Fields of Science: 

eXpress

eXpress is a streaming tool for quantifying the abundances of a set of target sequences from sampled subsequences.

Availability and Restrictions

The following versions of eXpress are available on OSC clusters:

Version Oakley Owens
1.5.1 X* X*
*: Current default version

You can use  module avail express   to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

eXpress is available to all OSC users without restriction.

Usage on Oakley

Set-up

To configure your environment for use of eXpress, run the following command:   module load express  . The default version will be loaded. To select a particular eXpress version, use   module load express/version  . For example, use  module load express/1.5.1    to load eXpress 1.5.1.

Usage on Owens

Set-up

To configure your environment for use of eXpress, run the following command:   module load express  . The default version will be loaded. To select a particular eXpress version, use   module load express/version  . For example, use  module load express/1.5.1    to load eXpress 1.5.1.

Further Reading

Supercomputer: 
Service: 
Fields of Science: 

miRDeep2

miRDeep2 is a completely overhauled tool which discovers microRNA genes by analyzing sequenced RNAs. The tool reports known and hundreds of novel microRNAs with high accuracy in seven species representing the major animal clades. The low consumption of time and memory combined with user-friendly interactive graphic output makes miRDeep2 accessible for straightforward application in current reasearch.

Availability and Restrictions

The following versions of miRDeep2 are available on OSC clusters:

Version Oakley Owens
2.0.0.8 X* X*
*: Current default version

You can use  module avail mirdeep2  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

miRDeep2 is available to all OSC users without restriction.

Usage on Oakley

Set-up

To configure your environment for use of miRDeep2, run the following command:   module load mirdeep2 . The default version will be loaded. To select a particular miRDeep2 version, use   module load mirdeep2/version . For example, use  module load mirdeep2/2.0.0.8   to load miRDeep2 2.0.0.8.

Usage on Owens

Set-up

To configure your environment for use of miRDeep2, run the following command:   module load mirdeep2 . The default version will be loaded. To select a particular miRDeep2 version, use   module load mirdeep2/version . For example, use  module load mirdeep2/2.0.0.8   to load miRDeep2 2.0.0.8.

Further Reading

Supercomputer: 
Service: 
Fields of Science: 

mpiBLAST

mpiBLAST is a freely available, open-source, parallel implementation of NCBI BLAST. mpiBLAST takes advantage of distributed computational resources, i.e., a cluster, through explicit MPI communication and thereby utilizes all available resources unlike standard NCBI BLAST which can only take advantage of shared-memory multi-processors (SMPs).

Availability & Restrictions

mpiBLAST is available without restriction to all OSC users.

The following version of mpiBLAST are available on OSC systems:

Version Glenn Oakley
1.6.0 X X

Usage

Set-up

To load the mpiBLAST software on the Glenn system, use the following commands:

module load biosoftw
module load mpiblast

On the Oakley system, use the following command:

module load mpiblast

Using mpiBLAST

Once mpiblast module is loaded, the commands are available for your use.

mpiblast 
mpiblast_cleanup 
mpiformatdb

Formatting a database

Before processing blast queries the sequence database must be formatted with mpiformatdb. The command line syntax looks like this:
mpiformatdb -N 16 -i nt -o T

The above command would format the nt database into 16 fragments. Note that currently mpiformatdb does not support multiple input files.

mpiformatdb places the formatted database fragments in the same directory as the FASTA database. To specify a different target location, use the "-n" option as what is available in the NCBI formatdb.

Querying the database

mpiblast command line syntax is nearly identical to NCBI's blastall program. Running a query on 18 nodes would look like:
mpiexec -n 18 mpiblast -p blastn -d nt -i blast_query.fas -o blast_results.txt

The above command would query the sequences in blast_query.fas against the nt database and write out results to the blast_results.txt file in the current working directory. By default, mpiBLAST reads configuration information from ~/.ncbirc. Furthermore, mpiBLAST needs at least 3 processes to perform a search: two processes dedicated for scheduling tasks and coordinating file output, while any additional processes actually perform search tasks.

Extra options to mpiblast

  • --partition-size=[integer]
    Enable hierarchical scheduling with multiple masters. The partition size equals the number of workers in a partition plus 1 (the master process). For example, a partition size of 17 creates partitions consisting of 16 workers and 1 master. An individual output file will be generated for each partition. By default, mpiBLAST uses one partition. This option is only available for version 1.6 or above.
  • --replica-group-size=[integer]
    Specify how database fragments are replicated within a partition. Suppose the total number of database fragments is F, the number of MPI processes in a partition is N, and the replica-group-size is G, then in total (N-1)/G database replicas will be distributed in the partition (the master process does not host any database fragments), and each worker process will host F/G fragments. In other words, a database replica will be distributed to every G MPI processes.
  • --query-segment-size=[integer]
    The default value is 5. Specify the number of query sequences that will be fetched from the supermaster to the master at a time. This parameter controls the granularity of load balancing between different partitions. This option is only available for version 1.6 or above.
  • --use-parallel-write
    Enable the high-performance parallel output solution. Note the current implementation of parallel-write does not require a parallel file system.
  • --use-virtual-frags
    Enable workers to cache database fragments in memory instead of local storage. This is recommended on diskless platforms where there is no local storage attaching to each processor. Default to be enabled on Blue Gene systems.
  • --predistribute-db
    Distribute database fragments to workers before the search begins. Especially useful in reducing data input time when multiple database replicas need to be distributed to workers.
  • --output-search-stats
    Enable output of the search statistics in the pairwise and XML output format. This could cause performance degradation on some diskless systems such as Blue Gene.
  • --removedb
    Removes the local copy of the database from each node before terminating execution.
  • --copy-via=[cp|rcp|scp|mpi|none]
    Sets the method of copying files that each worker will use. Default = "cp"
    • cp : use standard file system "cp" command. Additional option is --concurrent.
    • rcp : use rsh "rcp" command. Additonal option is --concurrent.
    • scp : use ssh "scp" command. Additional option is --concurrent.
    • mpi : use MPI_Send/MPI_Recv to copy files. Additional option is --mpi-size.
    • none : do not copy files, instead use shared storage as local storage.
  • --debug[=filename]
    Produces verbose debugging output for each node, optionally logs the output to a file.
  • --time-profile=[filename]
    Reports execution time profile.
  • --version
    Print the mpiBLAST version.

Please refer to the README file in the mpiBLAST package for performance tuning guide.

Removing a database

The --removedb command line option will cause mpiBLAST to do all work in a temporary directory that will get removed from each node's local storage directory upon successful termination. For example:
mpiexec -n 18 mpiblast -p blastx -d yeast.aa -i ech_10k.fas -o results.txt --removedb

The above command would perform a 18 node (16 worker) search of the yeast.aa database, writing the output to results.txt. Upon completion, worker nodes would delete the yeast.aa database fragments from their local storage.

Databases can also be removed without performing a search in the following manner:
mpiexec -n 18 mpiblast_cleanup

Batch Usage

Below is a sample batch script for running mpiBLAST job. It asks for 24 processors and 4 hours of walltime.

#PBS -l walltime=4:00:00
#PBS -l nodes=2:ppn=12
#PBS -N mpiBLAST
#PBS -j oe

# Set up the variable MPIBLAST_SHARED. This is where mpiblast looks for database.
# Leave the variable MPIBLAST_LOCAL unset. It will be default to $TMPDIR/mpiBLAST_local_db/.
export MPIBLAST_SHARED=/nfs/gpfs/PZS0530/yzhang/db/mpiblast

# Load the software environment for mpiblast.
module load mpiblast

# Copy input file over to $TMPDIR on the compute nodes.
cd $PBS_O_WORKDIR
pbsdcp query.fasta $TMPDIR

# Break the database into 20 pieces 
# Note: you need to work in the directory $TMPDIR for this step. 
# This step fails if you try it in your home directory or lustre file system.
cd $TMPDIR
cp $MPIBLAST_SHARED/nr ./
/usr/bin/time mpiformatdb -N 20 -i nr -o T -p T
cp nr.* $MPIBLAST_SHARED/

# Run mpiblast. 
cp /usr/local/mpiblast/1.6.0/.ncbirc ./
/usr/bin/time mpiexec -n 22 mpiblast -p blastp -d nr -i query.fasta -o blast_results.txt

# Copy output file back to the working directory. 
# The result only exists on the head node. So we are using "cp" command here.
mkdir $PBS_O_WORKDIR/$PBS_JOBID
cp blast_results.txt $PBS_O_WORKDIR/$PBS_JOBID

Further Reading

See Also

Supercomputer: 
Service: 
Fields of Science: 

parallel-command-processor

Introduction

There are many instances where it is necessary to run the same serial program many times with slightly different input. Parametric runs such as these either end up running in a sequential fashion in a single batch job, or a batch job is submitted for each parameter that is varied (or somewhere in between.) One alternative to this is to allocate a number of nodes/processors to running a large number of serial processes for some period of time. The command parallel-command-processor allows the execution of large number of independent serial processes in parallel. parallel-command-processor works as follows: In a parallel job with N processors allocated, the PCP manager process will read the first N-1 commands in the command stream and distribute them to the other N-1 processors. As processes complete, the PCP manager will read the next one in the stream and send it out to an idle processor core. Once the PCP manager runs out of commands to run, it will wait on the remaining running processes to complete before shutting itself down.

 

Availability

Parallel-Command-Processor is available for Oakley, Ruby, and Owens for all users.

 

Usage

Here is an interactive batch session that demonstrate the use of parallel-command-processor with a config file, pconf. pconf contains several lines of simple commands, one per line. The output of the commands were redirected to individual files.

-bash-3.2$ qsub -I -l nodes=2:ppn=4 
-bash-3.2$ cd $PBS_O_WORKDIR 
-bash-3.2$ cp pconf $TMPDIR
-bash-3.2$ cd $TMPDIR
-bash-3.2$ cat pconf ls / > 1 
ls $TMPDIR > 2 
ls $HOME > 3 
ls /usr/local/ > 4 
ls /tmp > 5 
ls /usr/src > 6 
ls /usr/local/src > 7
ls /usr/local/etc > 8 
hostname > 9 
uname -a > 10 
df > 11
-bash-3.2$ mpiexec parallel-command-processor pconf
-bash-3.2$ pwd
/tmp/pbstmp.1371894 
-bash-3.2$ all -j ls -l $TMPDIR 
854 total 16 
-rw------- 1 yzhang G-3040 1082 Feb 18 16:26 11
-rw------- 1 yzhang G-3040 1770 Feb 18 16:26 4 
-rw------- 1 yzhang G-3040 67 Feb 18 16:26 5
-rw------- 1 yzhang G-3040 32 Feb 18 16:26 6 
-rw------- 1 yzhang G-3040 0 Feb 18 16:26 7 
855 total 28
-rw------- 1 yzhang G-3040 199 Feb 18 16:26 1
-rw------- 1 yzhang G-3040 111 Feb 18 16:26 10
-rw------- 1 yzhang G-3040 12 Feb 18 16:26 2
-rw------- 1 yzhang G-3040 87 Feb 18 16:26 3 
-rw------- 1 yzhang G-3040 38 Feb 18 16:26 8
-rw------- 1 yzhang G-3040 20 Feb 18 16:26 9
-rw------- 1 yzhang G-3040 163 Feb 18 16:25 pconf 
-bash-3.2$ exit

As the command "all -j ls -l $TMPDIR" shows, the output files are distributed on the two nodes. In a batch file, pbsdcp can be used to distribute copy the files to $TMPDIR on different nodes and gather output files once execution has completed. This step is important due to the load that executing many processes in parallel can place on the user home directories.

Here is a slightly more complex example showing the usage of parallel-command-processor and pbsdcp:

#PBS -l nodes=13:ppn=4 
#PBS -l walltime=1:00:00 
#PBS -S /bin/bash 
#PBS -N blast-PCP 
#PBS -j oe 
date

module load biosoftw 
module load blast

set -x

cd $PBS_O_WORKDIR 
pbsdcp query/query.fsa.* $TMPDIR 
pbsdcp db/rice.* $TMPDIR 
cd $TMPDIR

for i in $(seq 1 49)

do 
      cmd="blastall -p blastn -d rice -i query.fsa.$i -o out.$i" 
      echo ${cmd} >> runblast 
done

mpiexec parallel-command-processor runblast

mkdir $PBS_O_WORKDIR/output 
pbsdcp -g out.* $PBS_O_WORKDIR/output

date

Documentation

The parallel-command-processor command is documented as a man page: man parallel-command-processor

 

Supercomputer: 
Service: 

pkg-config

pkg-config is a helper tool used when compiling applications and libraries. It helps you insert the correct compiler options on the command line. This metadata is stored in pkg-config files. These files have the suffix .pc and reside in specific locations known to the pkg-config tool.

Availability and Restrictions

Versions

The following versions of pkg-config are available on OSC clusters:

VERSION OAKLEY
0.28 X*
*: Default Version

You can use  module avail pkg-config  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

pkg-config is available to all OSC users without restriction.

Usage

Usage on Oakley

Set-up on Oakley

To configure your environment for use of pkg-config on Oakley, run the following command:  module load pkg-config . The default version will be loaded. To select a particular pkg-config version, use  module load pkg-config/version . For example, use module load pkg-config/0.28 to load pkg-config 0.28 on Oakley.

Further Reading

Supercomputer: 
Service: