OSC periodically updates The NCBI BLAST database. 

Versions

BLAST database is available on the Owens and Oakley clusters. The versions currently available at OSC are:

* indicates default version

The version indicates the date of download. You can usemodule spider blast-database​ to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

BLAST database is available without restriction to all OSC users.

Usage

BLAST database can be accessed with the following module:

module load blast-database/version

To list all the BLAST database available

module spider blast-database

BLAST database can be accessed by the environmental variable BLASTDB. For blast-database/2018-08, it is as follows

BLASTDB=/fs/project/pub_data/blast-database/2018-08

Further Reading

BLAST package: https://www.osc.edu/resources/available_software/software_list/blast

Microbial Genomes is a database of prokaryotic genome sequencing project data. 

Availability and Restrictions

Versions

Microbial Genomes is available on the Owens cluster. The versions currently available at OSC are:

The version indicates the date of the installation. You can use module spider microbial-database to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Microbial Genomes is available without restriction to all OSC users.

Usage

Usage on Owens

Further Reading

• Microbial Genomes homepage

ABAQUS is a finite element analysis program owned and supported by SIMULIA, the Dassault Systèmes brand for Realistic Simulation.

Availability and Restrictions

Versions

The available programs are ABAQUS/CAE, ABAQUS/Standard and ABAQUS/Explicit. The versions currently available at OSC are:

You can use  module spider abaqus to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access for Academic Users

OSC's ABAQUS license can only be used for educational, institutional, instructional, and/or research purposes. Only users who are faculty, research staff or students at the following institutions are permitted to utilized OSC's license:

• The Ohio State University
• University of Toledo
• University of Cincinnati
• Miami University of Ohio
• University of Akron

Users from additional degree granting academic institutions may request to be added to this list per a cost by contacting OSC Help.

The use of ABAQUS for academic purposes requires validation. In order to obtain validation, please contact OSC Help for further instruction. 

(link sends e-mail)

Access for Commerical Users

Contac OSC Help for getting access to SOFTWARE if you are a commerical user.

Publisher/Vendor/Repository and License Type

Dassault Systemes, Commercial

Usage

Token Usage

ABAQUS software usage is monitored though a token-based license manager. This means very time you run a ABAQUS job, tokens are checked out from our pool for your tasks usage. To ensure your job is only started when its required ABAQUS tokens are available it is important to include a software flag within your job script's PBS directives.  A minimum of 5 tokens are required per a job, so a 1 node, 1 processor ABAQUS job would need the following PBS software flag:  #PBS -l software=abaqus+5 . Jobs requiring more cores will need to request more tokens as calculated with the formula:  M = int(5 x N^0.422) , where N is the total number of cores.  For common requests, you can refer to the following table:

Usage on Ruby

qsub -I -l nodes=1:ppn=20 -l walltime=1:00:00

#PBS -N knee
#PBS -l walltime=1:00:00
#PBS -l nodes=1:ppn=20
#PBS -l software=abaqus+17
#PBS -j oe
#
# The following lines set up the ABAQUS environment
#
module load abaqus
#
# Move to the directory where the job was submitted
#
cd $PBS_O_WORKDIR
cp *.inp $TMPDIR
cd $TMPDIR
#
# Run ABAQUS
#
abaqus job=knee_bolster cpus=20 interactive
#
# Now, copy data (or move) back once the simulation has completed
#
cp * $PBS_O_WORKDIR

#PBS -l walltime=1:00:00
#PBS -l nodes=2:ppn=20
#PBS -N my_abaqus_job
#PBS -l software=abaqus+23
#PBS -j oe
#
# The following lines set up the ABAQUS environment
#
module load abaqus
#
# Move to the directory where the job was submitted
#
cd $PBS_O_WORKDIR
cp *.inp $TMPDIR/
cd $TMPDIR
#
# Run ABAQUS, note that in this case we have provided the names of the input files explicitly
#
abaqus job=test input=<my_input_file_name1>.inp cpus=40 interactive
#
# Now, move data back once the simulation has completed
#
mv * $PBS_O_WORKDIR

Usage on Owens

qsub -I -l nodes=1:ppn=28 -l walltime=1:00:00

#PBS -N knee
#PBS -l walltime=1:00:00
#PBS -l nodes=1:ppn=1
#PBS -l software=abaqus+5
#PBS -j oe
#
# The following lines set up the ABAQUS environment
#
module load abaqus
#
# Move to the directory where the job was submitted
#
cd $PBS_O_WORKDIR
cp *.inp $TMPDIR
cd $TMPDIR
#
# Run ABAQUS
#
abaqus job=knee_bolster interactive
#
# Now, copy data (or move) back once the simulation has completed
#
cp * $PBS_O_WORKDIR

#PBS -l walltime=1:00:00
#PBS -l nodes=2:ppn=28:pfsdir
#PBS -N my_abaqus_job
#PBS -l software=abaqus+27
#PBS -j oe
#
# The following lines set up the ABAQUS environment
#
module load abaqus
#
# Move to the directory where the job was submitted
#
cd $PBS_O_WORKDIR
#
# Cope input files to /fs/scratch and run Abaqus there
#
cp *.inp $PFSDIR
cd $PFSDIR
#
# Run ABAQUS, note that in this case we have provided the names of the input files explicitly
#
abaqus job=test input=<my_input_file_name1>.inp cpus=56 interactive
#
# Now, move data back once the simulation has completed
#
mv * $PBS_O_WORKDIR

Further Reading

The Assisted Model Building with Energy Refinement (AMBER) package contains many molecular simulation programs targeted at biomolecular systems. A wide variety of modelling techniques are available. It generally scales well on modest numbers of processors, and the GPU enabled CUDA programs are very efficient.

Availability and Restrictions

Versions

AMBER is available on Ruby, and Owens Clusters. The following versions are currently available at OSC (S means serial executables, P means parallel, C means CUDA, i.e., GPU enabled, and M means MIC, i.e., Xeon Phi enabled):

You can use module spider amber  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access for Academic Users

OSC's Amber is available to not-for-profit OSC users; simply contact OSC Help to request the appropriate form for access.

Access for Commerical Users

For-profit OSC users must obtain their own Amber license. 

Publisher/Vendor/Repository and License Type

University of California, San Francisco, Commercial

Usage

Usage on Ruby

tleap

mpiexec pmemd.MPI

pmemd.cuda

qsub -I -l nodes=1:ppn=20 -l walltime=1:00:00

~srb/workshops/compchem/amber/

# AMBER Example Batch Script for the Basic Tutorial in the Amber manual
#PBS -N 6pti
#PBS -l nodes=1:ppn=20
#PBS -l walltime=0:20:00
module load amber
# Use TMPDIR for best performance.
cd $TMPDIR
# PBS_O_WORKDIR refers to the directory from which the job was submitted.
cp -p $PBS_O_WORKDIR/6pti.prmtop .
cp -p $PBS_O_WORKDIR/6pti.prmcrd .
# Running minimization for BPTI
cat << eof > min.in
# 200 steps of minimization, generalized Born solvent model
&cntrl
maxcyc=200, imin=1, cut=12.0, igb=1, ntb=0, ntpr=10,
/
eof
sander -i min.in -o 6pti.min1.out -p 6pti.prmtop -c 6pti.prmcrd -r 6pti.min1.xyz
cp -p min.in 6pti.min1.out 6pti.min1.xyz $PBS_O_WORKDIR

Usage on Owens

tleap

mpiexec pmemd.MPI

qsub -I -l nodes=1:ppn=28 -l walltime=1:00:00

~srb/workshops/compchem/amber/

# AMBER Example Batch Script for the Basic Tutorial in the Amber manual
#PBS -N 6pti
#PBS -l nodes=1:ppn=28
#PBS -l walltime=0:20:00
module load amber
# Use TMPDIR for best performance.
cd $TMPDIR
# PBS_O_WORKDIR refers to the directory from which the job was submitted.
cp -p $PBS_O_WORKDIR/6pti.prmtop .
cp -p $PBS_O_WORKDIR/6pti.prmcrd .
# Running minimization for BPTI
cat << eof > min.in
# 200 steps of minimization, generalized Born solvent model
&cntrl
maxcyc=200, imin=1, cut=12.0, igb=1, ntb=0, ntpr=10,
/
eof
sander -i min.in -o 6pti.min1.out -p 6pti.prmtop -c 6pti.prmcrd -r 6pti.min1.xyz
cp -p min.in 6pti.min1.out 6pti.min1.xyz $PBS_O_WORKDIR

Usage on Pitzer

tleap

mpiexec pmemd.MPI

qsub -I -l nodes=1:ppn=28 -l walltime=1:00:00

~srb/workshops/compchem/amber/

# AMBER Example Batch Script for the Basic Tutorial in the Amber manual
#PBS -N 6pti
#PBS -l nodes=1:ppn=28
#PBS -l walltime=0:20:00
module load amber
# Use TMPDIR for best performance.
cd $TMPDIR
# PBS_O_WORKDIR refers to the directory from which the job was submitted.
cp -p $PBS_O_WORKDIR/6pti.prmtop .
cp -p $PBS_O_WORKDIR/6pti.prmcrd .
# Running minimization for BPTI
cat << eof > min.in
# 200 steps of minimization, generalized Born solvent model
&cntrl
maxcyc=200, imin=1, cut=12.0, igb=1, ntb=0, ntpr=10,
/
eof
sander -i min.in -o 6pti.min1.out -p 6pti.prmtop -c 6pti.prmcrd -r 6pti.min1.xyz
cp -p min.in 6pti.min1.out 6pti.min1.xyz $PBS_O_WORKDIR

Troubleshooting

In general, the scientific method should be applied to usage problems.  Users should check all inputs and examine all outputs for the first signs of trouble.  When one cannot find issues with ones inputs, it is often helpful to ask fellow humans, especially labmates, to review the inputs and outputs.  Reproducibility of molecular dynamics simulations is subject to many caveats.  See page 24 of the Amber16 manual for a discussion.

Further Reading

• Amber Tutorials
• Amber Home Page

ANSYS offers a comprehensive software suite that spans the entire range of physics, providing access to virtually any field of engineering simulation that a design process requires. Supports are provided by ANSYS, Inc. 

Availability and Restrictions

Versions

OSC has Academic Multiphysics Campus Solution license from Ansys. The license includes most of all the features that Ansys provides. See "Academic Multiphyscis Campus Solution Products" in this table for all available products at OSC.

Access for Academic Users

OSC has an "Academic Research " license for ANSYS. This allows for academic use of the software by Ohio faculty and students, with some restrictions. To view current ANSYS node restrictions, please see ANSYS's Terms of Use.

Use of ANSYS products at OSC for academic purposes requires validation. Please contact OSC Help for further instruction.

Access for Commerical Users

Contact OSC Help for getting access to ANSYS if you are a commercial user.

Publisher/Vendor/Repository and License Type

Ansys, Inc., Commercial

Usage

For more information on how to use each ANSYS product at OSC systems, refer to its documentation page provided at the end of this page.

Note

Due to the way our Fluent and ANSYS modules are configured, simultaneously loading multiple of either module will cause a cryptic error. The most common case of this happening is when multiple of a user's jobs are started at the same time and all load the module at once. In order for this error to manifest, the modules have to be loaded at precisely the same time; a rare occurrence, but a probable occurrence over the long term.

If you encounter this error you are not at fault. Please resubmit the failed job(s).

If you frequently submit large amounts of Fluent or ANSYS jobs, we recommend you stagger your job submit times to lower the chances of two jobs starting at the same time, and hence loading the module at the same time. Another solution is to establish job dependencies between jobs, so jobs will only start one after another. To do this, you would add the PBS directive:

#PBS -W after:jobid

To jobs you want to only start after another job has started. You would replace jobid with the job ID of the job to wait for. If you have additional questions, please contact OSC Help.

Further Reading

• ANSYS homepage

See Also

ARM HPC tools analyze how HPC software runs. It consists of three applications, ARM DDT, ARM Performance Reports and ARM MAP: 

• ARM DDT: graphical debugger for HPC applications.
• ARM MAP: HPC application profiler with easy-to-use GUI environment.
• ARM Performance Reports: simple tool to generate a single-page HTML or plain text report that presents overall performance characteristics of HPC applications.

Availability & Restrictions

Versions

The following versions of ARM HPC tools are available on OSC clusters:

You can use module spider arm to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

ARM DDT, MAP and Performance Reports are available to all OSC users.

Publisher/Vendor/Repository and License Type

ARM, Commercial

Usage

ARM DDT

ARM DDT is a debugger for HPC software that automatically alerts users of memory bugs and divergent behavior. For more features and benefits, visit ARM HPC tools and libraries - DDT.

For usage instructions and more iformation, read ARM DDT.

ARM MAP

ARM MAP produces a detailed profile of HPC software. Unlike ARM Performance Reports, you must have the source code to run ARM MAP because its analysis details the software line-by-line. For more features and benefits, visit ARM HPC tools and libraries - MAP. 

For usage instructions and more information, read ARM MAP.

ARM Performance Reports

ARM Performance Reports analyzes and documents information on CPU, MPI, I/O, and Memory performance characteristics of HPC software, even third party code, to aid understanding about the overall performance. Although it should not be used all the time, ARM Performance Reports is recommended to OSC users as a viable option to analyze how an HPC application runs. View an example report to navigate the format of a typical report. For more example reports, features and benefits, visit ARM HPC tools and libraries - Performance Reports.

For usage instructions and more information, read ARM Performance Reports.

Troubleshooting

Using ARM software with MVAPICH2

This note from ARM's Getting Started Guide applies to both perf-report and MAP:

Some MPIs, most notably MVAPICH, are not yet supported by ARM's Express Launch mode
(in which you can just put “perf-report” in front of an existing mpirun/mpiexec line). These can
still be measured using the Compatibility Launch mode.

Instead of this Express Launch command:

perf-report mpiexec <mpi args> <program> <program args> # BAD

Use the compatibility launch version instead:

perf-report -n <num procs> --mpiargs="<mpi args>" <program> <program args>

Further Reading

• ARM DDT homepage
• ARM MAP homepage
• ARM Performance Reports homepage

See Also

HyperWorks is a high-performance, comprehensive toolbox of CAE software for engineering design and simulation.

Availability & Restrictions

Versions

The following version of Altair Hyperworks can be found for the following environments:

You can use module spider hyperworks to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

HyperWorks is available to all academic clients. Please contact OSC Help to request the appropriate form for access.

Publisher/Vendor/Repository and License Type

Altair Engineering, Commercial (state-wide)

Usage

Using HyperWorks through OSC installation

To use HyperWorks on the OSC clusters, first ensure that X11 forwarding is enabled as the HyperWorks workbench is a graphical application. Then, load the hyperworks module:

module load hyperworks

The HyperWorks GUI can be launched then with the following command:

hw

State-wide access for HyperWorks

For information on downloading and installing a local copy through the state-wide license, follow the steps below. 

NOTE: To run Altair HyperWorks, your computer must have access to the internet. The software contacts the license server at OSC to check out a license when it starts and periodically during execution. The amount of data transferred is small, so network connections over modems are acceptable.

Usage of HyperWorks on a local machine using the statewide license will vary from installation to installation.

1. Go to http://www.altairhyperworks.com/ 

2. Click on "Login" in the upper right hand corner of the page. 

3. If you have already registered with the Altair web site, enter the e-mail address that you registered with and your password and skip to step #5.

4. If you have not registered yet, click the link that says " Sign up for Altair Connect". You will be prompted for some contact information and an e-mail address which will be your unique identifier.

   • IMPORTANT: The e-mail address you give must be from your academic institution. Under the statewide license agreement, registration from Ohio universities is allowed on the Altair web site. Trying to log in with a yahoo or hotmail e-mail account will not work. If you enter your university e-mail and the system will not register you, please contact OSChelp at oschelp@osc.edu.

5. Once you have logged in, click on "SUPPORT" and then "SOFTWARE DOWNLOADS"

6. In addition to downloading the software, download the "Installation Guide and Release Notes" for instructions on how to install the software.

   • IMPORTANT: If you have any questions or problems, please contact OSChelp at oschelp@osc.edu, rather than HyperWorks support. The software agreements outlines that problems should first be sent to OSC. If the OSC support line cannot answer or resolve the question, they have the ability to raise the problem to Altair support.

7. Please contact OSC Help for further instruction and license server information. In order to be added to the allowed list for the state-wide software access, we will need your IP address/range of machine that will be running this software. 

8. You need to set an environment variable (ALTAIR_LICENSE_PATH) on your local machine to point at our license server (7790@license6.osc.edu). See this link for instructions if necessary.

Further Reading

For more information about HyperWorks, see the following:

• Altair HyperWorks Documentation Page (Requires registration with Altair)
• Altair HyperWorks Academic Blog

See Also

• ABAQUS
• Altair HyperWorks statewide license

The BLAS (Basic Linear Algebra Subprograms) are routines that provide standard building blocks for performing basic vector and matrix operations.

Availability and Restrictions

Access

A highly optimized implementation of the BLAS is available on all OSC clusters as part of the Intel Math Kernel Library (MKL). We recommend that you use MKL rather than building the BLAS for yourself. MKL is available to all OSC users.

Usage

See OSC's MKL software page for usage information. Note that there is no library named libblas.a or libblas.so. The flag "-lblas" on your link line will not work. You should modify your makefile or build script to link to the MKL libraries instead.

Further Reading

• BLAS Website
• Intel MKL Website

he BLAST programs are widely used tools for searching DNA and protein databases for sequence similarity to identify homologs to a query sequence. While often referred to as just "BLAST", this can really be thought of as a set of programs: blastp, blastn, blastx, tblastn, and tblastx.

Availability & Restrictions

Versions

The following versions of BLAST are available on OSC systems: 

You can use module spider blast to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

If you need to use blastx, you will need to load one of the C++ implimenations modules of blast (any version with a "+").

Access

BLAST is available without restriction to all OSC users.

Publisher/Vendor/Repository and License Type

National Institutes of Health, Open source

Usage

Set-up

To load BLAST, type the following into the command line:

module load blast

Then create a resource file .ncbirc, and put it under your home directory.

Using BLAST

The five flavors of BLAST mentioned above perform the following tasks:

blastp: compares an amino acid query sequence against a protein sequence database

blastn: compares a nucleotide query sequence against a nucleotide sequence database

blastx: compares the six-frame conceptual translation products of a nucleotide query sequence (both strands) against a protein sequence database

tblastn: compares a protein query sequence against a nucleotide sequence database dynamically translated in all six reading frames (both strands).

tblastx: compares the six-frame translations of a nucleotide query sequence against the six-frame translations of a nucleotide sequence database. (Due to the nature of tblastx, gapped alignments are not available with this option)

NCBI BLAST Database

Information on the NCBI BLAST database can be found here. https://www.osc.edu/resources/available_software/scientific_database_list/blast_database 

We provide local access to nt and refseq_protein databases. You can access the database by loading desired blast-database modules. If you need other databases, please send a request email to OSC Help .

Batch Usage

A sample batch script is below:

#PBS -l nodes=1:ppn=1
#PBS -l walltime=10:00
#PBS -N Blast
#PBS -S /bin/bash
#PBS -j oe
module load blast
module load blast-database
set -x
cd $PBS_O_WORKDIR
mkdir $PBS_JOBID
cp 100.fasta $TMPDIR
cd $TMPDIR
/usr/bin/time blastn -db nt -query 100.fasta -out test.out
cp test.out $PBS_O_WORKDIR/$PBS_JOBID

Further Reading

• The BLAST Homepage
• Bioperl
• BLAT

BWA is a software package for mapping low-divergent sequences against a large reference genome, such as the human genome. It consists of three algorithms: BWA-backtrack, BWA-SW and BWA-MEM.

Availability and Restrictions

Versions

The following versions of BWA are available on OSC clusters:

You can use module spider bwa to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

0.7.17-r1198 is the latest version at the time of the installation on July 2019.

Access

BWA is available to all OSC users without restriction.

Publisher/Vendor/Repository and License Type

Li H. and Durbin R., Open source

Usage

Usage on Owens

Usage on Pitzer

Further Reading

• The BWA Official Webpage

BamTools provides both a programmer's API and an end-user's toolkit for handling BAM files.

Availability and Restrictions

Versions

The following versions of BamTools are available on OSC clusters:

You can use module spider bamtools to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

BamTools is available to all OSC users without restriction.

Publisher/Vendor/Repository and License Type

Derek Barnett, Erik Garrison, Gabor Marth, and Michael Stromberg/ Open Source

Usage

Usage on Owens

Usage on Pitzer

Further Reading

• The BamTools Official Webpage

Blender is the free and open source 3D creation suite. It supports the entirety of the 3D pipeline—modeling, rigging, animation, simulation, rendering, compositing and motion tracking, even video editing and game creation.

Availability and Restrictions

Versions

The following versions of Blender are available on OSC systems: 

You can use module spider blender to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Blender is available to all OSC users without restriction.

Publisher/Vendor/Repository and License Type

Blender Foundation, Open source

Usage

Set-up

On Owens-Desktop 'vis' or 'any' node type, run the following command:

module load blender

Using Blender

To run hardware-rendering version of blender, connect to OSC OnDemand and luanch a virtual desktop, either a Virtual Desktop Interface (VDI) or an Interactive HPC 'vis'-type desktop 'vis' type Desktop, and in desktop open a terminal and run blender with VirtualGL: 

module load virtualgl
vglrun blender

You can also run software-rendering version of blender on any type Desktop: 

blender-softwaregl

Further Reading

• Blender Demo Files
• Blender User Manual

Boost is a set of C++ libraries that provide helpful data structures and numerous support functions in a wide range of aspects of programming, such as, image processing, gpu programming, concurrent programming, along with many algorithms.  Boost is portable and performs well on a wide variety of platforms.

Availability & Restrictions

Versions

The following version of Boost are available on OSC systems:

You can use module spider boost to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Boost is available without restriction to all OSC Users.

Publisher/Vendor/Repository and License Type

Beman Dawes, David Abrahams, Rene Rivera/ Open source

Usage

Usage on Ruby

module load boost

cp /usr/local/src/boost/boost-1_56_0/test.osc/example2.cpp ~
cp /usr/local/src/boost/boost-1_56_0/test.osc/jayne.txt ~

module load boost
g++ $BOOST_CFLAGS example2.cpp -o boostTest $BOOST_LIBS -lboost_regex
./boostTest < jayne.txt

Usage on Owens

module load boost

cp /usr/local/src/boost/boost-1_56_0/test.osc/example2.cpp ~
cp /usr/local/src/boost/boost-1_56_0/test.osc/jayne.txt ~

g++ example2.cpp -o boostTest -lboost_regex
./boostTest < jayne.txt

Usage on Pitzer

module load boost

cp /usr/local/src/boost/boost-1_56_0/test.osc/example2.cpp ~
cp /usr/local/src/boost/boost-1_56_0/test.osc/jayne.txt ~

g++ example2.cpp -o boostTest -lboost_regex
./boostTest < jayne.txt

Further Reading

• The Boost Homepage

Bowtie1 is an ultrafast, memory-efficient short read aligner. It aligns short DNA sequences (reads) to the human genome at a rate of over 25 million 35-bp reads per hour. Bowtie indexes the genome with a Burrows-Wheeler index to keep its memory footprint small: typically about 2.2 GB for the human genome (2.9 GB for paired-end).

Availability and Restrictions

Versions

The following versions of Bowtie1 are available on OSC clusters:

You can use module spider bowtie1 to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Bowtie1 is available to all OSC users without restriction.

Publisher/Vendor/Repository and License Type

Ben Langmead et al., Open source (Artistic 2.0)

Usage

Usage on Owens

Usage on Pitzer

Further Reading

• The Bowtie1 Official Webpage

Bowtie2 is an ultrafast and memory-efficient tool for aligning sequencing reads to long reference sequences. It is particularly good at aligning reads of about 50 up to 100s or 1,000s of characters, and particularly good at aligning to relatively long (e.g. mammalian) genomes. Bowtie 2 indexes the genome with an FM Index to keep its memory footprint small: for the human genome, its memory footprint is typically around 3.2 GB. Bowtie 2 supports gapped, local, and paired-end alignment modes.

Please note that bowtie (and tophat) cannot run in parallel, that is, on multiple nodes.  Submitting multi-node jobs will only waste resources.  In addition you must explicitly include the '-p' option to use multiple threads on a single node.

Availability and Restrictions

Versions

The following versions of Bowtie2 are available on OSC clusters:

You can use module spider bowtie2 to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Bowtie2 is available to all OSC users without restriction.

Publisher/Vendor/Repository and License Type

Ben Langmead et al., Open source

Usage

Usage on Owens

Usage on Pitzer

Further Reading

• The Bowtie2 Official Webpage

CMake is a family of compilation tools that can be used to build, test and package software.

Availability and Restrictions

Versions

The current versions of CMake available at OSC are:

You can use  module spider cmake  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

CMake is available to all OSC users.

Publisher/Vendor/Repository and License Type

Aaron C. Meadows et al., Open source

Usage

Usage on Ruby

Usage on Owens

Usage on Pitzer

Further Reading

For more information, visit the CMake homepage.

COMSOL Multiphysics (formerly FEMLAB) is a finite element analysis and solver software package for various physics and engineering applications, especially coupled phenomena, or multiphysics. owned and supported by COMSOL, Inc.

Availability and Restrictions

Versions

COMSOL is available on the Owens and Ruby clusters. The versions currently available at OSC are:

You can use module spider comsol  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access for Academic Users

COMSOL is for academic use, available to all Ohio State University users. OSC does not provide COMSOL licenses for academic use to students and faculty outside of Ohio State University due to licensing restrictions. If you or your institution have a network COMSOL license server, you may be able to use it on OSC. For connections to your license server from OSC, please read this document. If you need further help, please contact OSC Help.

To use COMSOL you will have to be added to the license server.  Please contact OSC Help to be added.

Access for Commerical Users

Contact OSC Help for getting access to COMSOL if you are a commercial user. 

Publisher/Vendor/Repository and License Type

Comsol Inc., Commercial

Usage

Usage on Ruby

qsub -I -l nodes=1:ppn=20 -l walltime=1:00:00 -l software=comsolscript

#PBS -N COMSOL
#PBS -l walltime=1:00:00
#PBS -l nodes=1:ppn=1
#PBS -j oe
#PBS -l software=comsolscript
#
# The following lines set up the COMSOL environment
#
module load comsol
#
# Move to the directory where the job was submitted
#
cd $PBS_O_WORKDIR
cp *.m $TMPDIR
cd $TMPDIR
#
# Run COMSOL
#
comsol batch mycomsol
#
# Now, copy data (or move) back once the simulation has completed
#
cp * $PBS_O_WORKDIR

#PBS -l walltime=01:00:00
#PBS -l nodes=2:ppn=20
#PBS -N COMSOL
#PBS -j oe
#PBS -r n
#PBS -l software=comsolscript
cd ${PBS_O_WORKDIR}
module load comsol
echo "--- Copy Input Files to TMPDIR and Change Disk to TMPDIR"
cp input_cluster.mph $TMPDIR
cd $TMPDIR
np=20
echo "--- Running on ${np} processes (cores) on the following nodes:"
cat $PBS_NODEFILE | uniq > hostfile
echo "--- COMSOL run"
comsol -nn 2 -np ${np} batch -f hostfile -mpirsh rsh -inputfile input_cluster.mph -outputfile output_cluster.mph
echo "--- Copy files back"
cp output_cluster.mph output_cluster.mph.status ${PBS_O_WORKDIR}
echo "---Job finished at: 'date'"
echo "---------------------------------------------"

Usage on Owens

qsub -I -l nodes=1:ppn=28 -l walltime=1:00:00 -l software=comsolscript

#PBS -N COMSOL
#PBS -l walltime=1:00:00
#PBS -l nodes=1:ppn=1
#PBS -j oe
#PBS -l software=comsolscript
#
# The following lines set up the COMSOL environment
#
module load comsol
#
# Move to the directory where the job was submitted
#
cd $PBS_O_WORKDIR
cp *.m $TMPDIR
cd $TMPDIR
#
# Run COMSOL
#
comsol batch mycomsol
#
# Now, copy data (or move) back once the simulation has completed
#
cp * $PBS_O_WORKDIR

#PBS -l walltime=01:00:00
#PBS -l nodes=2:ppn=28
#PBS -N COMSOL
#PBS -j oe
#PBS -r n
#PBS -l software=comsolscript
cd ${PBS_O_WORKDIR}
module load comsol
echo "--- Copy Input Files to TMPDIR and Change Disk to TMPDIR"
cp input_cluster.mph $TMPDIR
cd $TMPDIR
np=28
echo "--- Running on ${np} processes (cores) on the following nodes:"
cat $PBS_NODEFILE | uniq > hostfile
echo "--- COMSOL run"
comsol -nn 2 -np ${np} batch -f hostfile -mpirsh ssh -inputfile input_cluster.mph -outputfile output_cluster.mph
echo "--- Copy files back"
cp output_cluster.mph output_cluster.mph.status ${PBS_O_WORKDIR}
echo "---Job finished at: 'date'"
echo "---------------------------------------------"

Further Reading

• The Official COMSOL Website

CP2K is a quantum chemistry and solid state physics software package that can perform atomistic simulations of solid state, liquid, molecular, periodic, material, crystal, and biological systems. CP2K provides a general framework for different modeling methods such as DFT using the mixed Gaussian and plane waves approaches GPW and GAPW. Supported theory levels include DFTB, LDA, GGA, MP2, RPA, semi-empirical methods and classical force fields. CP2K can do simulations of molecular dynamics, metadynamics, Monte Carlo, Ehrenfest dynamics, vibrational analysis, core level spectroscopy, energy minimization, and transition state optimization using NEB or dimer method.

Availability and Restrictions

Versions

ORCA is available on the OSC clusters. These are the versions currently available:

You can use module spider cp2k to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

CP2K is available to all OSC users.

Publisher/Vendor/Repository and License Type

CP2K, GNU General Public License

Usage

Usage on Pitzer

qsub -I -l nodes=1:ppn=1 -l walltime=00:20:00

#PBS -N cp2k_popt_test
#PBS -l walltime=0:10:0
#PBS -l nodes=2:ppn=40
cd $PBS_O_WORKDIR
module load gnu/4.8.5
module load mvapich2/2.3
module load cp2k/6.1
module list

cd $PBS_O_WORKDIR
pbsdcp -p job.inp $TMPDIR
cd $TMPDIR
mpiexec -np $PBS_NP cp2k.popt -o job.out.$PBS_JOBID -i job.inp
pbsdcp -g job.out $PBS_O_WORKDIR 

Further Reading

General documentation is available from the CP2K website.

CUDA™ (Compute Unified Device Architecture) is a parallel computing platform and programming model developed by Nvidia that enables dramatic increases in computing performance by harnessing the power of the graphics processing unit (GPU).

Availability and Restrictions

Versions

CUDA is available on the clusters supporting GPUs. The versions currently available at OSC are:

You can use module spider cuda to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

CUDA is available for use by all OSC users.

Publisher/Vendor/Repository and License Type

Nvidia, Freeware 

Usage

Usage on Ruby

-l nodes=1:ppn=20:gpus=1:default
-l nodes=1:ppn=20:gpus=1:exclusive_process

qsub -I -l nodes=1:ppn=20:gpus=1 -l walltime=00:20:00 

#PBS -l walltime=01:00:00
#PBS -l nodes=1:ppn=1:gpus=1
#PBS -N compute
#PBS -j oe
module load cuda
cd $HOME/cuda
cp mycudaApp $TMPDIR
cd $TMPDIR
./mycudaApp

Usage on Owens

-l nodes=1:ppn=28:gpus=1:default
-l nodes=1:ppn=28:gpus=1:exclusive_process

qsub -I -l nodes=1:ppn=28:gpus=1 -l walltime=00:20:00 

#PBS -l walltime=01:00:00
#PBS -l nodes=1:ppn=1:gpus=1
#PBS -N compute
#PBS -j oe
module load cuda
cd $HOME/cuda
cp mycudaApp $TMPDIR
cd $TMPDIR
./mycudaApp

Usage on Pitzer

-l nodes=1:ppn=40:gpus=2:default
-l nodes=1:ppn=40:gpus=2:exclusive_process

qsub -I -l nodes=1:ppn=40:gpus=2 -l walltime=00:20:00 

#PBS -l walltime=01:00:00
#PBS -l nodes=1:ppn=1:gpus=1
#PBS -N compute
#PBS -j oe
module load cuda
cd $HOME/cuda
cp mycudaApp $TMPDIR
cd $TMPDIR
./mycudaApp

Further Reading

Online documentation is available on the CUDA homepage.

Compiler support for the latest version of CUDA is available here.

Caffe is "a fast open framework for deep learning."

From their README:

Caffe is a deep learning framework made with expression, speed, and modularity in mind. It is developed by the Berkeley Vision and Learning Center (BVLC) and by community contributors. Yangqing Jia created  the project during his PhD at UC Berkeley. Caffe is released under the BSD 2-Clause license.

Caffe also includes interfaces for both Python and Matlab, which have been built but have not been tested.

Availability and Restrictions

Versions

The following versions of Caffe are available on OSC clusters:

You can use module spider caffe to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

The current versions of Caffe on Ruby require the use of python/2.7.8, cuda/7.5.18, boost/1.56.0, opencv/3.1.0 on Ruby for GPU calculations.

The current version of Caffe on Owens requires cuda/8.0.44 for GPU calculations.

Access 

Caffe is available to all OSC users without restriction.

Publisher/Vendor/Repository and License Type

Berkeley AI Research, Open source

Usage

Usage on Ruby or Owens

module load caffe

#PBS -N Caffe
#PBS -l nodes=1:ppn=20:gpus=1:default
#PBS -l walltime=30:00
#PBS -j oe
#PBS -S /bin/bash 
cd $PBS_O_WORKDIR
. /etc/profile.d/lmod.sh
# Load the modules for Caffe
ml caffe
# Migrate to job temp directory and copy folders over
cd $TMPDIR
cp -r $CAFFE_HOME/{examples,build,data} .
# Download, create, train
./data/mnist/get_mnist.sh
./examples/mnist/create_mnist.sh
./examples/mnist/train_lenet.sh
# Serialize log files
echo; echo 'LOG 1'
cat convert_mnist_data.bin.$(hostname)*
echo; echo 'LOG 2'
cat caffe.INFO
echo; echo 'LOG 3'
cat convert_mnist_data.bin.INFO
echo; echo 'LOG 4'
cat caffe.$(hostname).*
cp examples/mnist/lenet_iter_10000.* $PBS_O_WORKDIR

qsub job.txt

#!/bin/bash
#PBS -N Caffe
#PBS -l nodes=1:ppn=28:gpu
#PBS -l walltime=30:00
#PBS -j oe
#PBS -S /bin/bash 
cd $PBS_O_WORKDIR
. /etc/profile.d/lmod.sh
# Load the modules for Caffe
ml caffe
# Migrate to job temp directory and copy folders over
cd $TMPDIR
cp -r $CAFFE_HOME/{examples,data} .
# Download, create, train
./data/mnist/get_mnist.sh
./examples/mnist/create_mnist.sh
./examples/mnist/train_lenet.sh
# Serialize log files
echo; echo 'LOG 1'
cat convert_mnist_data.bin.$(hostname)*
echo; echo 'LOG 2'
cat caffe.INFO
echo; echo 'LOG 3'
cat convert_mnist_data.bin.INFO
echo; echo 'LOG 4'
cat caffe.$(hostname).*
cp examples/mnist/lenet_iter_10000.* $PBS_O_WORKDIR

qsub job.txt

Further Reading

• Caffe homepage

Cufflinks is a program that analyzes RNA -Seq samples. It assembles aligned RNA-Seq reads into a set of transcripts, then inspects the transcripts to estimate abundances and test for differential expression and regulation in the RNA-Seq reads.

Availability and Restrictions

Versions

Cufflinks is available on the Owens Cluster. The versions currently available at OSC are:

You can use module spider cufflinks to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Cufflinks is available without restriction to all OSC users.

Publisher/Vendor/Repository and License Type

Cole Trapnell et al., Open source

Usage

Usage on Owens

Further Reading

• Cufflinks homepage

Darshan is a lightweight "scalable HPC I/O characterization tool".  It is intended to profile I/O by emitting log files to a consistent log location for systems administrators, and also provides scripts to create summary PDFs to characterize I/O in MPI-based programs.

Availability and Restrictions

Versions

The following versions of Darshan are available on OSC clusters:

You can use module spider darshan to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access 

Darshan is available to all OSC users without restriction.

Publisher/Vendor/Repository and License Type

MCSD, Argonne National Laboratory, Open source

Usage

Usage on Owens

module load darshan

gnu/4.8.5  mvapich2/2.2
gnu/4.8.5  mvapich2/2.2rc1
gnu/4.8.5  openmpi/1.10
intel/16.0.3  intelmpi/5.1.3
intel/16.0.3  mvapich2/2.2
intel/16.0.3  mvapich2/2.2rc1
intel/16.0.3  openmpi/1.10

# basic call to darshan 
mpiexec.darshan [args] ./my_mpi_program
# to show evidence that Darshan is working and to see internal timing
mpiexec.darshan.timing [args] ./my_mpi_program

#!/bin/bash
#PBS -l nodes=1:ppn=28:pfsdir
#PBS -j oe
# one may need to perform 'shopt -s expand_aliases' inside the shell before calling this script to expand darshan aliases
shopt -s expand_aliases
ml r --quiet
module load darshan
ml
function run_darshan() {
COMPILER=mpicc
MPIPROCS=$1
PROGNAME=$2
BLOKSIZE=$3
LOCATION=$4
cp ${DARSHAN_EXAMPLE}/${PROGNAME}.c $LOCATION
cd $LOCATION
$COMPILER -DDEBUG -D_FILE_OFFSET_BITS=64 -D_LARGEFILE64_SOURCE -o $PROGNAME ${PROGNAME}.c
mpiexec_darshan() { eval mpiexec.darshan "$@"; }
mpiexec_darshan -n $MPIPROCS ./$PROGNAME -f test -l $BLOKSIZE
du -sh test # show the human-readable file size
rm test # we are not keeping the large file around
}
# run the parallel version of mpi-io to write to a file but use the local tmp directory
run_darshan $PBS_NP mkrandpfile 256 $TMPDIR
# run the parallel version of mpi-io to write to a file but use the parallel file system scratch location
run_darshan $PBS_NP mkrandpfile 256 $PFSDIR
# for each darshan log generate a PDF report
JOBID=$([[ $tmp =~ ([0-9]*)\..* ]]; echo "${BASH_REMATCH[1]}")
for log in $(ls $DARSHAN_LOG/$(date "+%Y/%-m/%-d")/${USER:0:8}_*_id${JOBID}*.darshan); do
darshan-job-summary.pl $log
done
cp *.pdf $PBS_O_WORKDIR

qsub mpiio_with_darshan.qsub

Usage on Pitzer

module load darshan

intel/18.0.3  mvapich2/2.3
intel/18.0.4  mvapich2/2.3

# basic call to darshan 
mpiexec.darshan [args] ./my_mpi_program
# to show evidence that Darshan is working and to see internal timing
mpiexec.darshan.timing [args] ./my_mpi_program

#!/bin/bash
#PBS -l nodes=1:ppn=28:pfsdir
#PBS -j oe
# one may need to perform 'shopt -s expand_aliases' inside the shell before calling this script to expand darshan aliases
shopt -s expand_aliases
ml r --quiet
module load darshan
ml
function run_darshan() {
COMPILER=mpicc
MPIPROCS=$1
PROGNAME=$2
BLOKSIZE=$3
LOCATION=$4
cp ${DARSHAN_EXAMPLE}/${PROGNAME}.c $LOCATION
cd $LOCATION
$COMPILER -DDEBUG -D_FILE_OFFSET_BITS=64 -D_LARGEFILE64_SOURCE -o $PROGNAME ${PROGNAME}.c
mpiexec_darshan() { eval mpiexec.darshan "$@"; }
mpiexec_darshan -n $MPIPROCS ./$PROGNAME -f test -l $BLOKSIZE
du -sh test # show the human-readable file size
rm test # we are not keeping the large file around
}
# run the parallel version of mpi-io to write to a file but use the local tmp directory
run_darshan $PBS_NP mkrandpfile 256 $TMPDIR
# run the parallel version of mpi-io to write to a file but use the parallel file system scratch location
run_darshan $PBS_NP mkrandpfile 256 $PFSDIR
# for each darshan log generate a PDF report
JOBID=$([[ $tmp =~ ([0-9]*)\..* ]]; echo "${BASH_REMATCH[1]}")
for log in $(ls $DARSHAN_LOG/$(date "+%Y/%-m/%-d")/${USER:0:8}_*_id${JOBID}*.darshan); do
darshan-job-summary.pl $log
done
cp *.pdf $PBS_O_WORKDIR

qsub mpiio_with_darshan.qsub

Further Reading

• Darshan homepage

Desmond is a software package that perform high-speed molecular dynamics simulations of biological systems on conventional commodity clusters, general-purpose supercomputers, and GPUs. The code uses novel parallel algorithms and numerical techniques to achieve high performance and accuracy on platforms containing a large number of processors, but may also be executed on a single computer. Desmond includes code optimized for machines with an NVIDIA GPU.

Availability and Restrictions

Versions

The Desmond package is available on Owens. The versions currently available at OSC are:

You can use  module spider desmond to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access for Academic Users 

Desmond is available to not-for-profit OSC users; simply contact OSC Help to request the appropriate form for access. 

Publisher/Vendor/Repository and License Type

D E Shaw Research, Non-Commercial

Usage

Usage on Owens

​​module load desmond/2018.2

maestro 

# Example Desmond single-node batch script.
#PBS -N multisim-batch
#PBS -j oe
#PBS -m ae
#PBS -l walltime=0:20:00
#PBS -l nodes=1:ppn=28
#PBS -S /bin/sh

qstat -f $PBS_JOBID
export
module load desmond
module list

cd $PBS_O_WORKDIR
pbsdcp -p 'desmond_md_job_butane.*' $TMPDIR

cd $TMPDIR
$SCHRODINGER/utilities/multisim -HOST localhost -maxjob 1 -cpu 24 -m desmond_md_job_butane.msj -c desmond_md_job_butane.cfg desmond_md_job_butane.cms -mode umbrella -ATTACHED -WAIT

ls -l
pbsdcp -p '*' $PBS_O_WORKDIR

Further Reading

• Desmond Home Page

Elmer is an open source multiphysical simulation software developed by CSC - IT (Finland).

From their Github README:

Elmer is a finite element software for numerical solution of partial
differential equations. Elmer is capable of handling any number of equations
and is therefore ideally suited for the simulation of multiphysical problems.
It includes models, for example, of structural mechanics, fluid dynamics, heat
transfer and electromagnetics. Users can also write their own equations that
can be dynamically linked with the main program.
Elmer consists of several parts. The most important ones are ElmerSolver, the
finite element solver, ElmerGUI, the graphical user interface, and ElmerGrid,
the mesh creation and manipulation tool. Also a visualization tool, ElmerPost,
is included in the package but it is no longer developed.  
Elmer software is licensed under GPL except for the ElmerSolver library which
is licensed under LGPL license. 
Elmer is mainly developed at CSC - IT Center for Science, Finland. However,
there have been numerous contributions from other organizations and developers
as well, and the project is open for new contributions.

Availability and Restrictions

Versions

The following versions of Elmer are available on OSC clusters:

You can use module spider elmer to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

The current versions of Elmer are compiled with GNU 4.8.4 (requirement > 4.8) and BLAS libraries from MKL/11.2.3.

Access 

Elmer is available to all OSC users without restriction.

Publisher/Vendor/Repository and License Type

CSC - IT (Finland), Open source

Usage

Usage on Ruby

module load gnu/4.8.4
module load mkl/11.2.3
module load elmer

#PBS -N Elmer
#PBS -l nodes=1:ppn=1 
#PBS -l walltime=30:00
#PBS -j oe 
#PBS -S /bin/bash 
# Load the modules for Elmer
module load gnu/4.8.4
module load mkl/11.2.3
module load elmer
# Move to the problem directory
cd $PBS_O_WORKDIR
# Copy files to $TMPDIR and move there to execute the program
cp -r * $TMPDIR
cd $TMPDIR
# Convert the mesh - see Elmer webpage for more examples of 'ElmerGrid'
ElmerGrid 14 2 beam.msh -autoclean
# Run the solver 
ElmerSolver beam.sif
# Finally, copy files back to your home directory 
cp -r * $PBS_O_WORKDIR

qsub job.txt

Further Reading

• Elmer homepage
• Elmer Github

The FASTX-Toolkit is a collection of command line tools for Short-Reads FASTA/FASTQ files preprocessing.

Availability and Restrictions

Verisons

The following versions of FASTX-Toolkit are available on OSC clusters:

You can use  module spider fastx to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

FASTX-Toolkit is available to all OSC users without restriction.

Publisher/Vendor/Repository and License Type

Assaf Gordon, Open source

Usage

Usage on Owens

Further Reading

• The FASTX-Toolkit Official Webpage

FFTW is a C subroutine library for computing the Discrete Fourier Transform (DFT) in one or more dimensions, of arbitrary input size, and of both real and complex data. It is portable and performs well on a wide variety of platforms.

Availability and Restrictions

Versions

FFTW is available on Ruby, and Owens Clusters. The versions currently available at OSC are:

You can use module spider fftw2  or module spider fftw3  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

FFTW is available without restriction to all OSC Users.

Publisher/Vendor/Repository and License Type

www.fftw.org, Open source

Usage

Usage on Ruby

module load fftw3
icc $FFTW3_CFLAGS my-fftw.c -o my-fftw $FFTW3_LIBS 
ifort $FFTW3_FFLAGS more-fftw.f -o more-fftw $FFTW3_LIBS

Usage on Owens

module load fftw3
icc $FFTW3_CFLAGS my-fftw.c -o my-fftw $FFTW3_LIBS 
ifort $FFTW3_FFLAGS more-fftw.f -o more-fftw $FFTW3_LIBS

Usage on Pitzer

module load fftw3
icc $FFTW3_CFLAGS my-fftw.c -o my-fftw $FFTW3_LIBS 
ifort $FFTW3_FFLAGS more-fftw.f -o more-fftw $FFTW3_LIBS

Further Reading

• The FFTW homepage

See Also

• The Intel MKL

FSL is a library of tools for analyzing FMRI, MRI and DTI brain imaging data.

Availability and Restrictions

Versions

The following versions of FSL are available on OSC clusters:

You can use module spider fsl to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

FSL is available to all academic OSC users with proper validation. In order to obtain validation, please contact OSC Help for further instruction.

Publisher/Vendor/Repository and License Type

Analysis Group, University of Oxford/ freeware

Usage

Usage on Owens

source $FSLDIR/etc/fslconf/fsl.sh
fsl

source $FSLDIR/etc/fslconf/fsl.csh
fsl 

Further Reading 

• FSL website

FastQC provides quality control checks of high throughput sequence data that identify areas of the data that may cause problems during further analysis.

Availability and Restrictions

Versions

FastQC is available on the Owens cluster. The versions currently available at OSC are:

You can use module spider fastqc to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

FastQC is available without restriction to all OSC users.

Publisher/Vendor/Repository and License Type

Babraham Bioinformatics, Open source

Usage

Usage on Owens

Usage on Pitzer

Further Reading

• FastQC homepage

FreeSurfer is a software package used to anaylze nueroimaging data.

Availability & Restrictions

Versions

The following versions of FreeSurfer are available on OSC clusters:

You can use module spider freesurfer to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

FreeSurfer is available to all academic OSC users with proper validation. In order to obtain validation, please contact OSC Help for further instruction. 

Publisher/Vendor/Repository and License Type

Athinoula A. Martinos Center, Open source

Usage

Usage on Owens

source $FREESURFER_HOME/SetUpFreeSurfer.sh

source $FREESURFER_HOME/SetUpFreeSurfer.csh

export SUBJECTS_DIR=<path to subject data>

setenv SUBJECTS_DIR=<path to subject data>

source $FREESURFER_HOME/bin/cuda5_setup.sh

source $FREESURFER_HOME/bin/cuda5_setup.csh

Further Reading 

• FreeSurfer website

The General Atomic and Molecular Electronic Structure System (GAMESS) is a flexible ab initio electronic structure program. Its latest version can perform general valence bond, multiconfiguration self-consistent field, Möller-Plesset, coupled-cluster, and configuration interaction calculations. Geometry optimizations, vibrational frequencies, thermodynamic properties, and solution modeling are available. It performs well on open shell and excited state systems and can model relativistic effects. The GAMESS Home Page has additional information.

Availability and Restrictions

Versions

The current versions of GAMESS available on the Oakley and Owens Clusters are:

You can use module spider gamess to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

GAMESS is available to all OSC users.

Publisher/Vendor/Repository and License Type

Gordon research group, Iowa State Univ./ Open source

Usage

Set-up

GAMESS usage is controlled via modules. Load one of the GAMESS modulefiles at the command line, in your shell initialization script, or in your batch scripts, for example:

module load gamess  

Examples

• Test input files are in the tests subdirectory
• Computational Chemistry Workshop materials are in ~srb/workshops/compchem/gamess

Further Reading

General documentation is available from the GAMESS Home page and in the local machine directories.

GATK is a software package for analysis of high-throughput sequencing data. The toolkit offers a wide variety of tools, with a primary focus on variant discovery and genotyping as well as strong emphasis on data quality assurance.

Availability and Restrictions

Versions

The following versions of GATK are available on OSC clusters:

You can use module spider gatk to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

If you use GATK with CBLAS, you may get errors. In this case, we recommend using the GATK container. In order to pull the container, use singularity pull docker://broadinstitute/gatk:4.1.2.0. You can read more about container in general from here. If you have any further questions, please contact OSC Help.

Access for Academic Users

GATK is available to academic OSC users with proper validation. In order to obtain validation, please contact OSC Help for further instruction. 

Publisher/Vendor/Repository and License Type

Broad Institute, Inc., Open source

Usage

Usage on Owens

Usage on Pitzer

Further Reading

• The GATK Official Webpage

GLPK (GNU Linear Programming Kit) is a set of open source LP (linear programming) and MIP (mixed integer problem) routines written in ANSI C, which can be called from within C programs. 

Availability and Restrictions

Versions

The following versions are available on OSC systems:

You can use module spider glpk to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

GLPK is available to all OSC users without restriction. 

Publisher/Vendor/Repository and License Type

GNU, Open source

Usage

Set-up

To set up your environment for using GLPK on Oakley, run the following command:

module load glpk

Compiling and Linking

To compile your C code using GLPK API routines, use the environment variable $GLPK_CFLAGS provided by the module:

gcc $GLPK_CFLAGS -c my_prog.c

To link your code, use the variable $GLPK_LIBS:

gcc my_prog.o $GLPK_LIBS -o my_prog

glpsol

Additionally, the GLPK module contains a stand-alone LP/MIP solver, which can be used to process files written in the GNU MathProg modeling language.  The solver can be invoked using the following command syntax:

glpsol [options] [filename]

For a complete list of options, use the following command:

glpsol --help

Further Reading

• GLPK Homepage

GMAP is a genomic mapping and alignment program for mRNA and EST sequences.

Availability and Restrictions

Versions

The following versions of GMAP are available on OSC clusters:

You can use module spider gmap to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

GMAP is available to all OSC users without restriction.

Publisher/Vendor/Repository and License Type

Genentech, Inc., Open source

Usage

Usage on Owens

Further Reading

• The GMAP Official Webpage

Fortran, C and C++ compilers produced by the GNU Project. 

Availability and Restrictions

Versions

GNU compilers are available on all our clusters. These are the versions currently available:

You can use module spider gnu to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

To find out what version of gcc you are using, type gcc --version.

Access

The GNU compilers are available to all OSC users without restriction.

Publisher/Vendor/Repository and License Type

https://www.gnu.org/software/gcc/, Open source

Usage

Usage on Ruby

qsub -I -l nodes=1:ppn=20 -l walltime=1:00:00

#PBS -l walltime=1:00:00
#PBS -l nodes=1:ppn=20
#PBS -N jobname
#PBS -j oe
cd $PBS_O_WORKDIR
module load gnu
cp hello.c $TMPDIR
cd $TMPDIR
gcc -O3 hello.c -o hello
./hello > hello_results
cp hello_results $PBS_O_WORKDIR

qsub job.txt

#PBS -l walltime=1:00:00
#PBS -l nodes=2:ppn=20
#PBS -N jobname
#PBS -j oe
cd $PBS_O_WORKDIR
module load gnu
mpicc -O3 hello.c -o hello
cp hello $TMPDIR
cd $TMPDIR
mpiexec ./hello > hello_results
cp hello_results $PBS_O_WORKDIR

Usage on Owens

qsub -I -l nodes=1:ppn=28 -l walltime=1:00:00

#PBS -l walltime=1:00:00
#PBS -l nodes=1:ppn=28
#PBS -N jobname
#PBS -j oe
cd $PBS_O_WORKDIR
module load gnu
cp hello.c $TMPDIR
cd $TMPDIR
gcc -O3 hello.c -o hello
./hello > hello_results
cp hello_results $PBS_O_WORKDIR

qsub job.txt

#PBS -l walltime=1:00:00
#PBS -l nodes=2:ppn=28
#PBS -N jobname
#PBS -j oe
cd $PBS_O_WORKDIR
module load gnu
mpicc -O3 hello.c -o hello
cp hello $TMPDIR
cd $TMPDIR
mpiexec ./hello > hello_results
cp hello_results $PBS_O_WORKDIR

Usage on Pitzer

Further Reading

• GCC, the GNU Compiler Collection

See Also

• Intel Compilers
• PGI Compilers
• Compilation Guide

GPU-BLAST is an accelerated version of the popular NCBI-BLAST (www.ncbi.nlm.nih.gov) that uses general-purpose graphics processing unit (GPU). In comparison to the sequential NCBI-BLAST, GPU-BLAST is nearly four times faster, while producing identical results.

Availability and Restrictions

Versions

The following versions of GPU-BLAST are available on OSC systems:

You can use module spider gpu-blast to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Note that GPU-BLAST is not available on the Owens system. GPU-BLAST requires the CUDA nvcc compiler, but the GPU-BLAST software is only compatible with CUDA versions up to 7.5. CUDA 8, installed on the Owens cluster, is incompatible with GPU-BLAST.

Access

GPU-BLAST is available without restriction to all OSC users.

Publisher/Vendor/Repository and License Type

Panagiotis D. Vouzis and Nikolaos V. Sahinidis/ freeware

Usage

Usage on Ruby

module load cuda/7.5.18
module load gpu-blast

#PBS -l nodes=1:ppn=1:gpus=1
#PBS -l walltime=10:00
#PBS -N GPU-Blast
#PBS -S /bin/bash
#PBS -j oe
module load gpu-blast
module load cuda/7.5.18
set -x
cd $PBS_O_WORKDIR
mkdir $PBS_JOBID
cp 100.fasta $TMPDIR
cd $TMPDIR
/usr/bin/time blastn -db nt -query 100.fasta -out test.out
cp * $PBS_O_WORKDIR/$PBS_JOBID

Further Reading

• The GPU-BLAST homepage

See Also

• BLAST
• BLAT

GROMACS is a versatile package of molecular dynamics simulation programs. It is primarily designed for biochemical molecules, but it has also been used on non-biological systems.  GROMACS generally scales well on OSC platforms. Starting with version 4.6 GROMACS includes GPU acceleration.

Availability and Restrictions

Versions

GROMACS is available on Pitzer and Owens Clusters. Both single and double precision executables are installed. The versions currently available at OSC are the following:

You can use module spider gromacs  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

GROMACS is available to all OSC users without restriction.

Publisher/Vendor/Repository and License Type

http://www.gromacs.org/ Open source

Usage 

Usage on Owens

gmx pdb2gmx

mpiexec gmx_mpi_d mdrun

qsub -I -l nodes=1:ppn=28 -l walltime=1:00:00

~srb/workshops/compchem/gromacs/

# GROMACS Tutorial for Solvation Study of Spider Toxin Peptide
# see fwspider_tutor.pdf
#PBS -N fwsinvacuo.owens
#PBS -l nodes=2:ppn=28
module load gromacs
# PBS_O_WORKDIR refers to the directory from which the job was submitted.
cd $PBS_O_WORKDIR
pbsdcp -p 1OMB.pdb em.mdp $TMPDIR
# Use TMPDIR for best performance.
cd $TMPDIR
pdb2gmx -ignh -ff gromos43a1 -f 1OMB.pdb -o fws.gro -p fws.top -water none
editconf -f fws.gro -d 0.7
editconf -f out.gro -o fws_ctr.gro -center 2.0715 1.6745 1.914
grompp -f em.mdp -c fws_ctr.gro -p fws.top -o fws_em.tpr
mpiexec gmx_mpi mdrun -s fws_em.tpr -o fws_em.trr -c fws_ctr.gro -g em.log -e em.edr
cat em.log
cp -p * $PBS_O_WORKDIR/

Usage on Pitzer

gmx pdb2gmx

mpiexec gmx_mpi_d mdrun

qsub -I -l nodes=1:ppn=40 -l walltime=1:00:00

~srb/workshops/compchem/gromacs/

# GROMACS Tutorial for Solvation Study of Spider Toxin Peptide
# see fwspider_tutor.pdf
#PBS -N fwsinvacuo.pitzer
#PBS -l nodes=2:ppn=40
module load gromacs
# PBS_O_WORKDIR refers to the directory from which the job was submitted.
cd $PBS_O_WORKDIR
pbsdcp -p 1OMB.pdb em.mdp $TMPDIR
# Use TMPDIR for best performance.
cd $TMPDIR
pdb2gmx -ignh -ff gromos43a1 -f 1OMB.pdb -o fws.gro -p fws.top -water none
editconf -f fws.gro -d 0.7
editconf -f out.gro -o fws_ctr.gro -center 2.0715 1.6745 1.914
grompp -f em.mdp -c fws_ctr.gro -p fws.top -o fws_em.tpr
mpiexec gmx_mpi mdrun -ntomp 1 -s fws_em.tpr -o fws_em.trr -c fws_ctr.gro -g em.log -e em.edr
cat em.log
cp -p * $PBS_O_WORKDIR/

Further Reading

• The GROMACS homepage

Gaussian is the most popular general purpose electronic structure program. Recent versions can perform density functional theory, Hartree-Fock, Möller-Plesset, coupled-cluster, and configuration interaction calculations among others. Geometry optimizations, vibrational frequencies, magnetic properties, and solution modeling are available. It performs well as black-box software on closed-shell ground state systems. 

Availability and Restrictions

Versions

Gaussian is available on Ruby, Pitzer, and Owen Clusters. These versions are currently available at OSC (S means single node serial/parallel and C means CUDA, i.e., GPU enabled):

You can use module spider gaussian to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access for Academic Users

Use of Gaussian for academic purposes requires validation. In order to obtain validation, please contact OSC Help for further instruction.

Access for Commerical Users

Contact OSC Help to obtain access to Gaussian if you are a commerical user.

Publisher/Vendor/Repository and License Type

Gaussian, commercial

Usage

Usage on Ruby

g16 < input.com

g16 input.com

qsub -I -l nodes=1:ppn=20 -l walltime=1:00:00

/users/appl/srb/workshops/compchem/gaussian/

#PBS -N GaussianJob
#PBS -l nodes=1:ppn=20
# PBS_O_WORKDIR refers to the directory from which the job was submitted.
cd $PBS_O_WORKDIR
cp input.com $TMPDIR
# Use TMPDIR for best performance.
cd $TMPDIR
module load gaussian
g16 < input.com
cp -p input.log *.chk $PBS_O_WORKDIR

Usage on Owens

g16 < input.com

g16 input.com

qsub -I -l nodes=1:ppn=28 -l walltime=1:00:00

/users/appl/srb/workshops/compchem/gaussian/

#PBS -N GaussianJob
#PBS -l nodes=1:ppn=28
# PBS_O_WORKDIR refers to the directory from which the job was submitted.
cd $PBS_O_WORKDIR
cp input.com $TMPDIR
# Use TMPDIR for best performance.
cd $TMPDIR
module load gaussian
g16 < input.com
cp -p input.log *.chk $PBS_O_WORKDIR

Usage on Pitzer

g16 < input.com

g16 input.com

qsub -I -l nodes=1:ppn=40 -l walltime=1:00:00

/users/appl/srb/workshops/compchem/gaussian/

#PBS -N GaussianJob
#PBS -l nodes=1:ppn=40
# PBS_O_WORKDIR refers to the directory from which the job was submitted.
cd $PBS_O_WORKDIR
cp input.com $TMPDIR
# Use TMPDIR for best performance.
cd $TMPDIR
module load gaussian
g16 < input.com
cp -p input.log *.chk $PBS_O_WORKDIR

Running Gaussian jobs with GPU

#PBS -S /bin/tcsh
#PBS -N methane 
#PBS -o methane.log
#PBS -l nodes=1:ppn=20:gpus=1:default
set echo
cd $TMPDIR
set INPUT=methane.com
# PBS_O_WORKDIR refers to the directory from which the job was submitted.
cp $PBS_O_WORKDIR/$INPUT .
module load gaussian/g16a03
g16 < ./$INPUT
ls -al
cp *.chk $PBS_O_WORKDIR

%nproc=20
%mem=8gb
%CPU=0-19
%GPUCPU=0=0
%chk=methane.chk
#b3lyp/6-31G(d) opt
methane B3LYP/6-31G(d) opt freq
0,1
C        0.000000        0.000000        0.000000
H        0.000000        0.000000        1.089000
H        1.026719        0.000000       -0.363000
H       -0.513360       -0.889165       -0.363000
H       -0.513360        0.889165       -0.363000

#PBS -S /bin/tcsh
#PBS -N methane 
#PBS -o methane.log
#PBS -l nodes=1:ppn=28:gpus=1:default
set echo
cd $TMPDIR
set INPUT=methane.com
# PBS_O_WORKDIR refers to the directory from which the job was submitted.
cp $PBS_O_WORKDIR/$INPUT .
module load gaussian/g16b01
g16 < ./$INPUT
ls -al
cp *.chk $PBS_O_WORKDIR

%nproc=28
%mem=8gb
%CPU=0-27
%GPUCPU=0=0
%chk=methane.chk
#b3lyp/6-31G(d) opt
methane B3LYP/6-31G(d) opt freq
0,1
C        0.000000        0.000000        0.000000
H        0.000000        0.000000        1.089000
H        1.026719        0.000000       -0.363000
H       -0.513360       -0.889165       -0.363000
H       -0.513360        0.889165       -0.363000

Further Reading

• The Gaussian homepage

Git is a version control system used for tracking file changes and facilitating collaborative work.

Availability and Restrictions

Versions

The following versions of Git are available on OSC clusters:

You can use module spider git to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Git is available to all OSC users without restriction.

Publisher/Vendor/Repository and License Type

Git, Open source

Usage

Usage on Owens

Usage on Ruby

Usage on Pitzer

Further Reading

• Git homepage

Gnuplot is a portable command-line driven data and function plotting utility.  It was originally intended to allow scientists and students to visualize mathematical functions and data.  

Gnuplot supports many types of plots in two or three dimensions.  It can draw using points, lines, boxes, contours, vector fields surfaces and various associated text.  It also supports various specialized plot types.  

Gnuplot supports many different types of output:  interactive screen display (with mouse and hotkey functionality), pen plotters (like hpgl), printers (including postscript and many color devices), and file formats as vectorial pseudo-devices like LaTeX, metafont, pdf, svg, or bitmap png.  

Availability and Restrictions

Versions

The current versions of Gnuplot available at OSC are:

You can use module spider gnuplot to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Gnuplot is available to all OSC users.

Publisher/Vendor/Repository and License Type

Thomas Williams, Colin Kelley/ Open source

Usage

Usage on Owens

module load gnuplot

gnuplot

Usage on Pitzer

module load gnuplot

gnuplot

Further Reading

For more information, visit the Gnuplot Homepage.  

HDF5 is a general purpose library and file format for storing scientific data. HDF5 can store two primary objects: datasets and groups. A dataset is essentially a multidimensional array of data elements, and a group is a structure for organizing objects in an HDF5 file. Using these two basic objects, one can create and store almost any kind of scientific data structure, such as images, arrays of vectors, and structured and unstructured grids.

Availability and Restrictions

Versions

HDF5 is available on Pitzer, Ruby, and Owens Clusters. The versions currently available at OSC are:

You can use module spider hdf5 to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

HDF5 is available without restriction to all OSC users.

Publisher/Vendor/Repository and License Type

The HDF Group, Open source (academic)

Usage

Usage on Ruby

icc -c $HDF5_C_INCLUDE myprog.c
icc -o myprog myprog.o $HDF5_C_LIBS
ifort -c $HDF5_F90_INCLUDE myprog.f90
ifort -o myprog myprog.o $HDF5_F90_LIBS

#PBS -N AppNameJob
#PBS -l nodes=1:ppn=20
module load hdf5
cd $PBS_O_WORKDIR
cp foo.dat $TMPDIR
cd $TMPDIR
appname
cp foo_out.h5 $PBS_O_WORKDIR

Usage on Owens

icc -c $HDF5_C_INCLUDE myprog.c
icc -o myprog myprog.o $HDF5_C_LIBS
ifort -c $HDF5_F90_INCLUDE myprog.f90
ifort -o myprog myprog.o $HDF5_F90_LIBS

#PBS -N AppNameJob
#PBS -l nodes=1:ppn=28
module load hdf5
cd $PBS_O_WORKDIR
cp foo.dat $TMPDIR
cd $TMPDIR
appname
cp foo_out.h5 $PBS_O_WORKDIR

Usage on Pitzer

icc -c $HDF5_C_INCLUDE myprog.c
icc -o myprog myprog.o $HDF5_C_LIBS
ifort -c $HDF5_F90_INCLUDE myprog.f90
ifort -o myprog myprog.o $HDF5_F90_LIBS

#PBS -N AppNameJob
#PBS -l nodes=1:ppn=28
module load hdf5
cd $PBS_O_WORKDIR
cp foo.dat $TMPDIR
cd $TMPDIR
appname
cp foo_out.h5 $PBS_O_WORKDIR

Further Reading

• The HDF5 home page
• HDF5 tutorials

• NetCDF

HISAT2 is a graph-based alignment program that maps DNA and RNA sequencing reads to a population of human genomes.

Availability and Restrictions

Versions

HISAT2 is available on the Owens Cluster. The versions currently available at OSC are:

You can use module spider hisat2 to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

HISAT2 is available without restriction to all OSC users.

Publisher/Vendor/Repository and License Type

https://ccb.jhu.edu/software/hisat2, Open source

Usage

Usage on Owens

Further Reading

• HISAT2 homepage

HOMER (Hypergeometric Optimization of Motif EnRichment) is a suite of tools for Motif Discovery and ChIP-Seq analysis. It is a collection of command line programs for unix-style operating systems written in mostly perl and c++. Homer was primarily written as a de novo motif discovery algorithm that is well suited for finding 8-12 bp motifs in large scale genomics data.

Availability and Restrictions

Versions

The following versions of HOMER are available on OSC clusters:

You can use  module spider homer to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

HOMER is available to all OSC users without restriction.

Publisher/Vendor/Repository and License Type

Christopher Benner, Open source

Usage

Usage on Owens

Further Reading

• The HOMER Official Webpage

HPC Toolkit is a collection of tools that measure a program's work, resource consumption, and inefficiency to analze performance.

Availability and Restrictions

Versions

The following versions of HPC Toolkitare available on OSC clusters:

You can use module spider hpctoolkit to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

HPC Toolkit is available to all OSC users without restriction.

Publisher/Vendor/Repository and License Type

Rice Univerity, Open source

Usage

Usage on Owens

Usage on Pitzer

Further Reading

• HPC Toolkit homepage

HTSlib is a C library used for reading and writing high-throughput sequencing data. HTSlib is the core library used by SAMtools. HTSlib also provides the bgzip, htsfile, and tabix utilities.

Availability and Restrictions

Versions

The versions of HTSlib currently available at OSC are:

You can use module spider htslib to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

HTSlib is available to all OSC users.

Publisher/Vendor/Repository and License Type

Genome Research Ltd., Open source

Usage

Usage on Owens

module load htslib/1.6

Further Reading

• HTSlib homepage

A hadoop cluster can be launched within the HPC environment, but managed by the PBS job scheduler using  Myhadoop framework developed by San Diego Supercomputer Center. (Please see http://www.sdsc.edu/~allans/MyHadoop.pdf)

Availability and Restrictions

Versions

The following versions of Hadoop are available on OSC systems: 

You can use module spider hadoop to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Hadoop is available to all OSC users without restriction.

Publisher/Vendor/Repository and License Type

Apache software foundation, Open source

Usage

Set-up

In order to configure your environment for the usage of Hadoop, run the following command:

module load hadoop

In order to access a particular version of Hadoop, run the following command

module load hadoop/3.0.0-alpha1

Using Hadoop

In order to run Hadoop in batch, reference the example batch script below. This script requests 6 node on the Owens cluster for 1 hour of walltime. 

#PBS -N hadoop-example

#PBS -l nodes=6:ppn=12

#PBS -l walltime=01:00:00

setenv WORK $PBS_O_WORKDIR

module load hadoop/3.0.0-alpha1

module load myhadoop/v0.40

setenv HADOOP_CONF_DIR $TMPDIR/mycluster-conf-$PBS_JOBID

cd $TMPDIR

myhadoop-configure.sh -c $HADOOP_CONF_DIR -s $TMPDIR

$HADOOP_HOME/sbin/start-dfs.sh

hadoop dfsadmin -report

hadoop  dfs -mkdir data

hadoop  dfs -put $HADOOP_HOME/README.txt  data/

hadoop  dfs -ls data

hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.0.0-alpha1.jar wordcount data/README.txt wordcount-out

hadoop  dfs -ls wordcount-out

hadoop  dfs  -copyToLocal -f  wordcount-out  $WORK

$HADOOP_HOME/sbin/stop-dfs.sh

myhadoop-cleanup.sh

Example Jobs

Please check /usr/local/src/hadoop/3.0.0-alpha1/test.osc folder for more examples of hadoop jobs

Further Reading

• Hadoop home page

See Also

• Spark

• "Horovod is a distributed training framework for TensorFlow, Keras, PyTorch, and MXNet. The goal of Horovod is to make distributed Deep Learning fast and easy to use. The primary motivation for this project is to make it easy to take a single-GPU TensorFlow program and successfully train it on many GPUs faster."

Quote from Horovod Github documentation. 

Installation

Please follow the link for general instruction on installing Horovod to use with GPU.

Step 1: Install NCCL 2

Please download NCCL 2 from https://developer.nvidia.com/nccl.

Add the library path to LD_LIBRARY_PATH environment variable

$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:Path_to_nccl/nccl-<version>/lib

Step 2: Install horovod python package

module load python/3.6-conda5.2

Create a local python environment for horovod installation with nccl and activate it

conda create -n horovod-withnccl python=3.6 anaconda
source activate horovod-withnccl

Install gpu version of tensorflow or pytorch

pip install https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-1.10.0-cp36-cp36m-linux_x86_64.whl

Load mavapich2 and cuda modules

module load mvapich2/2.3rc2-gpu 

module load cuda/9.1.85

Set the horovod environmental variables

export HOROVOD_CUDA_HOME=/usr/local/cuda/9.1.85

export HOROVOD_CUDA_INCLUDE=/usr/local/cuda/9.1.85/include/

export HOROVOD_CUDA_LIB=/usr/local/cuda/9.1.85/lib

install horovod python package

HOROVOD_NCCL_HOME=/path_to_nccl_home/ HOROVOD_GPU_ALLREDUCE=NCCL pip install --no-cache-dir horovod

Testing

Please see benchmark script here

#PBS -N TensorFlow
#PBS -l nodes=2:ppn=28:gpus=1:default
#PBS -l walltime=00:30:00
#PBS -j oe
#PBS -A projectID 
#PBS -S /bin/bash

module load python/3.6-conda5.2
module load cuda/9.1.85
source activate horovod-withnccl
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path_to_nccl_home/lib
module load cuda/9.1.85
ml mvapich2/2.3rc2-gpu
mpiexec -ppn 1 -binding none -env  NCCL_DEBUG=INFO python tf_cnn_benchmarks.py.py 

Feel free to contact OSC Help if you have any issues with installation.

Publisher/Vendor/Repository and License Type

https://eng.uber.com/horovod/, Open source

Further Reading

TensorFlow homepage

The Intel compilers for both C/C++ and FORTRAN.

Availability and Restrictions

Versions

The versions currently available at OSC are:

You can use module spider intel  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

The Intel Compilers are available to all OSC users without restriction.

Publisher/Vendor/Repository and License Type

Intel, Commercial (state-wide)

Usage

Usage on Ruby

module load cxx11

module load cxx14

qsub -I -l nodes=1:ppn=20 -l walltime=1:00:00

#PBS -l walltime=1:00:00
#PBS -l nodes=1:ppn=20
#PBS -N jobname
#PBS -j oe
cd $PBS_O_WORKDIR
module load intel
cp hello.c $TMPDIR
cd $TMPDIR
icc -O2 hello.c -o hello
./hello > hello_results
cp hello_results $PBS_O_WORKDIR

qsub job.txt

#PBS -l walltime=1:00:00
#PBS -l nodes=2:ppn=20
#PBS -N jobname
#PBS -j oe
cd $PBS_O_WORKDIR
module load intel
mpicc -O2 hello.c -o hello
cp hello $TMPDIR
cd $TMPDIR
mpiexec ./hello > hello_results
cp hello_results $PBS_O_WORKDIR​

Usage on Owens

module load cxx11

qsub -I -l nodes=1:ppn=28 -l walltime=1:00:00

#PBS -l walltime=1:00:00
#PBS -l nodes=1:ppn=28
#PBS -N jobname
#PBS -j oe
cd $PBS_O_WORKDIR
module load intel
cp hello.c $TMPDIR
cd $TMPDIR
icc -O2 hello.c -o hello
./hello > hello_results
cp hello_results $PBS_O_WORKDIR

qsub job.txt

#PBS -l walltime=1:00:00
#PBS -l nodes=2:ppn=28
#PBS -N jobname
#PBS -j oe
cd $PBS_O_WORKDIR
module load intel
mpicc -O2 hello.c -o hello
cp hello $TMPDIR
cd $TMPDIR
mpiexec ./hello > hello_results
cp hello_results $PBS_O_WORKDIR

Usage on Pitzer

qsub -I -l nodes=1:ppn=28 -l walltime=1:00:00

#PBS -l walltime=1:00:00
#PBS -l nodes=1:ppn=28
#PBS -N jobname
#PBS -j oe
cd $PBS_O_WORKDIR
module load intel
cp hello.c $TMPDIR
cd $TMPDIR
icc -O2 hello.c -o hello
./hello > hello_results
cp hello_results $PBS_O_WORKDIR

qsub job.txt

#PBS -l walltime=1:00:00
#PBS -l nodes=2:ppn=28
#PBS -N jobname
#PBS -j oe
cd $PBS_O_WORKDIR
module load intel
mpicc -O2 hello.c -o hello
cp hello $TMPDIR
cd $TMPDIR
mpiexec ./hello > hello_results
cp hello_results $PBS_O_WORKDIR

Further Reading

• The Intel Compiler Home Page

See Also

• Intel Math Kernel Library
• AMD Core Math Library
• Compilation Guide

Intel's implementation of the Message Passing Interface (MPI) library. See Intel Compilers for available compiler versions at OSC.

Availability and Restrictions

Versions

Intel MPI may be used as an alternative to - but not in conjunction with - the MVAPICH2 MPI libraries. The versions currently available at OSC are:

You can use module spider intelmpi to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Intel MPI is available to all OSC users without restriction.

Publisher/Vendor/Repository and License Type

Intel, Commercial

Usage

Usage on Ruby

#PBS -N MyIntelMPIJob
#PBS -l nodes=4:ppn=20
#PBS -l walltime=5:00:00
module swap mvapich2 intelmpi
cd $PBS_O_WORKDIR
mpiexec my-impi-application

Usage on Owens

#PBS -N MyIntelMPIJob
#PBS -l nodes=4:ppn=28
#PBS -l walltime=5:00:00
module swap mvapich2 intelmpi
cd $PBS_O_WORKDIR
mpiexec my-impi-application

Usage on Pitzer

#PBS -N MyIntelMPIJob
#PBS -l nodes=4:ppn=28
#PBS -l walltime=5:00:00
module swap mvapich2 intelmpi
cd $PBS_O_WORKDIR
mpiexec my-impi-application

Further Reading

• Intel MPI page at Intel.com
• Vendor documentation

See Also

• MVAPICH2
• OpenMPI
• Intel Compilers
• Intel Math Kernel Library

Java is a concurrent, class-based, object-oriented programming language.

Availability and Restrictions

Versions

The following versions of Java are available on OSC clusters:

You can use module spider java to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Java is available to all OSC users without restriction.

Publisher/Vendor/Repository and License Type

Oracle, Freeware

Usage

Usage on Owens

Usage on Ruby

Usage on Pitzer

Further Reading

• Java homepage

From julialang.org:

"Julia is a high-level, high-performance dynamic programming language for numerical computing. It provides a sophisticated compiler, distributed parallel execution, numerical accuracy, and an extensive mathematical function library. Julia’s Base library, largely written in Julia itself, also integrates mature, best-of-breed open source C and Fortran libraries for linear algebra, random number generation, signal processing, and string processing. In addition, the Julia developer community is contributing a number of external packages through Julia’s built-in package manager at a rapid pace. IJulia, a collaboration between the Jupyter and Julia communities, provides a powerful browser-based graphical notebook interface to Julia."

Availability and Restrictions

Versions

Julia is available on Owens Clusters. The versions currently available at OSC are:

You can use module spider julia to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Julia is available for use by all OSC users.

Publisher/Vendor/Repository and License Type

Jeff Bezanson et al., Open source

Usage 

Interactive Julia Notebooks

If you are using OnDemand, you can simply work with Jupyter and the selection of the Julia kernel to use interactive notebooks to work on an Owens compute node!

Navigate to ondemand.osc.edu and select a Jupyter notebook:

Kallisto is an RNA-seq quantification program. It quantifies abundances of transcripts from RNA-seq data and uses psedoalignment to determine the compatibility of reads with targets, without needing alignment.

Availability and Restrictions

Versions

Kallisto is available on the Owens Clusters. The versions currently available at OSC are:

You can use module spider kallisto to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Kallisto is available without restriction to all OSC users.

Publisher/Vendor/Repository and License Type

Nicolas Bray et al., Open source

Usage

Usage on Owens

Further Reading

• Kallisto homepage

The Large-scale Atomic/Molecular Massively Parallel Simulator (LAMMPS) is a classical molecular dynamics code designed for high-performance simulation of large atomistic systems.  LAMMPS generally scales well on OSC platforms, provides a variety of modeling techniques, and offers GPU accelerated computation.

Availability and Restrictions

Versions

LAMMPS is available on Ruby, and Owens Clusters. The following versions are currently available at OSC:

You can use module spider lammps  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

LAMMPS is available to all OSC users without restriction.

Publisher/Vendor/Repository and License Type

Sandia National Lab., Open source

Usage

Usage on Ruby

lammps < input.file

module help lammps

qsub -I -l nodes=1:ppn=20:gpus=1 -l walltime=00:20:00 

~srb/workshops/compchem/lammps/

#PBS -N chain  
#PBS -l nodes=2:ppn=20  
#PBS -l walltime=10:00:00  
#PBS -S /bin/bash  
#PBS -j oe  
module load lammps  
cd $PBS_O_WORKDIR  
pbsdcp chain.in $TMPDIR  
cd $TMPDIR  
lammps < chain.in  
pbsdcp -g '*' $PBS_O_WORKDIR

Usage on Owens

lammps < input.file

module help lammps

qsub -I -l nodes=1:ppn=28:gpus=1 -l walltime=00:20:00 

~srb/workshops/compchem/lammps/

#PBS -N chain  
#PBS -l nodes=2:ppn=28  
#PBS -l walltime=10:00:00  
#PBS -S /bin/bash  
#PBS -j oe  
module load lammps  
cd $PBS_O_WORKDIR  
pbsdcp chain.in $TMPDIR  
cd $TMPDIR  
lammps < chain.in  
pbsdcp -g '*' $PBS_O_WORKDIR

Usage on Pitzer

lammps < input.file

module help lammps

qsub -I -l nodes=1:ppn=28:gpus=1 -l walltime=00:20:00 

~srb/workshops/compchem/lammps/

#PBS -N chain  
#PBS -l nodes=2:ppn=28  
#PBS -l walltime=10:00:00  
#PBS -S /bin/bash  
#PBS -j oe  
module load lammps  
cd $PBS_O_WORKDIR  
pbsdcp chain.in $TMPDIR  
cd $TMPDIR  
lammps < chain.in  
pbsdcp -g '*' $PBS_O_WORKDIR

Further Reading

• The LAMMPS Homepage

LAPACK (Linear Algebra PACKage) provides routines for solving systems of simultaneous linear equations, least-squares solutions of linear systems of equations, eigenvalue problems, and singular value problems.

Availability and Restrictions

A highly optimized implementation of LAPACK is available on all OSC clusters as part of the Intel Math Kernel Library (MKL). We recommend that you use MKL rather than building LAPACK for yourself. MKL is available to all OSC users.

Publisher/Vendor/Repository and License Type

http://www.netlib.org/lapack/, Open source

Usage

See OSC's MKL software page for usage information. Note that there is no library named liblapack.a or liblapack.so. The flag "-llapack" on your link line will not work. You should modify your makefile or build script to link to the MKL libraries instead.

Further Reading

• LAPACK Website
• Intel MKL Website

LS-DYNA is a general purpose finite element code for simulating complex structural problems, specializing in nonlinear, transient dynamic problems using explicit integration. LS-DYNA is one of the codes developed at Livermore Software Technology Corporation (LSTC).

Availability and Restrictions

Versions

LS-DYNA is available on Owens and Oakley Clusters for both serial (smp solver for single node jobs) and parallel (mpp solver for multipe node jobs) versions. The versions currently available at OSC are:

You can use module spider ls-dyna to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access for Academic Users

OSC does not provide LS_DYNA license directly, however users with their own academic departmental license can use it on the OSC clusters.  Please contact OSC Help for further instruction.

Access for Commerical Users

Contact OSC Help for getting access to LS-DYNA if you are a commerical user.

Publisher/Vendor/Repository and License Type

Oasys, Commercial

Usage

Usage on Owens

qsub -I -l nodes=1:ppn=28 -l walltime=00:20:00 

#PBS -N plate_test
#PBS -l walltime=5:00:00
#PBS -l nodes=1:ppn=28
#PBS -j oe
# The following lines set up the LSDYNA environment
module load ls-dyna/971_d_9.0.1
#
# Move to the directory where the input files are located
#
cd $PBS_O_WORKDIR
#
# Run LSDYNA (number of cpus > 1)
#
lsdyna I=explorer.k NCPU=28

#PBS -N plate_test
#PBS -l walltime=5:00:00
#PBS -l nodes=2:ppn=28
#PBS -j oe
# The following lines set up the LSDYNA environment
module swap mvapich2/2.2 intelmpi
module load mpp-dyna/971_d_R9.0.1
#
# Move to the directory where the input files are located
#
cd $PBS_O_WORKDIR
#
# Run LSDYNA (number of cpus > 1)
#
mpiexec mpp971 I=explorer.k NCPU=56

...
cd $TMPDIR
cp $PBS_O_WORKDIR/explorer.k .
... #launch the solver and execute
pbsdcp -g '*' $PBS_O_WORKDIR

Further Reading

• The LSDYNA homepage
• The LSDYNA users manual (structural and keyword editions)
• The LSDYNA Theory Manual
• The LS-PREPOST Tutorial

See Also

MAGMA is a collection of next generation linear algebra (LA) GPU accelerated libraries designed and implemented by the team that developed LAPACK and ScaLAPACK. MAGMA is for heterogeneous GPU-based architectures, it supports interfaces to current LA packages and standards, e.g., LAPACK and BLAS, to allow computational scientists to effortlessly port any LA-relying software components. The main benefits of using MAGMA are that it can enable applications to fully exploit the power of current heterogeneous systems of multi/manycore CPUs and multi-GPUs, and deliver the fastest possible time to an accurate solution within given energy constraints.

Availability and Restrictions

Versions

MAGMA is available on Owens, and the following versions are currently available at OSC:

You can use module spider magma  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access for Academic Users

MAGMA is available without restriction to all OSC users. 

Publisher/Vendor/Repository and License Type

Computational Algebra Group, Univ. of Sydney, Open source

Usage

Usage on Owens

icc $MAGMA_CFLAGS example.c

ifort $MAGMA_F90FLAGS example.F90

qsub -I -l nodes=1:ppn=28 -l walltime=1:00:00

# MAGMA Example Batch Script for the Basic Tutorial in the MAGMA manual
#PBS -N 6pti
#PBS -l nodes=1:ppn=28
#PBS -l walltime=0:20:00
module load cuda
module load magma
# Use TMPDIR for best performance.
cd $TMPDIR
# PBS_O_WORKDIR refers to the directory from which the job was submitted.
cp $PBS_O_WORKDIR/example.c .
icc $MAGMA_CFLAGS example.c

Further Reading

• MAGMA Homepage

MATLAB is a technical computing environment for high-performance numeric computation and visualization. MATLAB integrates numerical analysis, matrix computation, signal processing, and graphics in an easy-to-use environment where problems and solutions are expressed just as they are written mathematically--without traditional programming.

Availability and Restrictions

Versions

MATLAB is available on Pitzer, Ruby, and Owens Clusters. The versions currently available at OSC are:

You can use module spider matlab to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access: Academic Users Only (non-commercial, non-government)

Any academic users at OSC can use Matlab. All users must be added to the license server before using MATLAB. Please contact OSC Help to be granted access.

Publisher/Vendor/Repository and License Type

MathWorks, Commercial (University site license)

Usage

Usage on Ruby

matlab -nodisplay 

qsub -I -X -l nodes=1:ppn=20 -l walltime=00:20:00 

#PBS -N disable_multithreading
#PBS -l walltime=00:10:00
#PBS -l nodes=1:ppn=20
#PBS -j oe
module load matlab
matlab -singleCompThread -nodisplay -nodesktop < hello.m
# end of example file

Usage on Owens