ABAQUS is a finite element analysis program owned and supported by SIMULIA, the Dassault Systèmes brand for Realistic Simulation.

Availability and Restrictions

Versions

The available programs are ABAQUS/CAE, ABAQUS/Standard and ABAQUS/Explicit. The versions currently available at OSC are:

You can use  module spider abaqus to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access for Academic Users

OSC's ABAQUS license can only be used for educational, institutional, instructional, and/or research purposes. Only users who are faculty, research staff, or students at the following institutions are permitted to utilized OSC's license:

• The Ohio State University
• University of Toledo
• University of Cincinnati
• University of Dayton
• University of Akron
• Miami University

Users from additional degree granting academic institutions may request to be added to this list per a cost by contacting OSC Help.

The use of ABAQUS for academic purposes requires validation. In order to obtain validation, please contact OSC Help for further instruction. 

(link sends e-mail)

Access for Commercial Users

Contact OSC Help for getting access to ABAQUS if you are a commercial user.

Publisher/Vendor/Repository and License Type

Dassault Systemes, Commercial

Usage

Token Usage

ABAQUS software usage is monitored though a token-based license manager. This means every time you run an ABAQUS job, tokens are checked out from our pool for your usage. To ensure your job is only started when its required ABAQUS tokens are available it is important to include a software flag within your job script's SBATCH directives.  A minimum of 5 tokens are required per job, so a 1 node, 1 processor ABAQUS job would need the following SBATCH software flag:  #SBATCH -L abaqus@osc:5 . Jobs requiring more cores will need to request more tokens as calculated with the formula:  M = int(5 x N^0.422) , where N is the total number of cores.  For common requests, you can refer to the following table:

Usage on Owens

sinteractive -A <project-account> -N 1 -n 28 -t 1:00:00 -L abaqus@osc:20

#!/bin/bash
#SBATCH --time=1:00:00
#SBATCH --nodes=1 --ntasks-per-node=1
#SBATCH -L abaqus@osc:5
#SBATCH --account=<project-account>
#
# The following lines set up the ABAQUS environment
#
module load abaqus
#
cp *.inp $TMPDIR
cd $TMPDIR
#
# Run ABAQUS
#
abaqus job=knee_bolster interactive
#
# Now, copy data (or move) back once the simulation has completed
#
cp * $SLURM_SUBMIT_DIR

#!/bin/bash 
#SBATCH --time=1:00:00 
#SBATCH --nodes=2 --ntasks-per-node=28 --gres=pfsdir
#SBATCH -L abaqus@osc:27
#SBATCH --account=<project-account>
#
# The following lines set up the ABAQUS environment
#
module load abaqus
#
# Cope input files to /fs/scratch and run Abaqus there
#
cp *.inp $PFSDIR
cd $PFSDIR
#
# Run ABAQUS, note that in this case we have provided the names of the input files explicitly
#
abaqus job=test input=<my_input_file_name1>.inp cpus=$SLURM_NTASKS interactive
#
# Now, move data back once the simulation has completed
#
mv * $SLURM_SUBMIT_DIR

Further Reading

AFNI (Analysis of Functional Neuro Images) is a leading software suite of C, Python, and R programs and shell scripst primarily developed for the analysis and display of multiple MRI modalities: anatomical, functional MRI (FMRI) and diffusion wieghted (DW) data. It is freely available (both as open source code and as precompiled binaries) for research purposes.

Availability and Restrictions

Versions

The following versions are available on OSC clusters:

You can use module spider afni to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

AFNI is available to all OSC users. If you have any questions, please contact OSC Help.

Publisher/Vendor/Repository and License Type

AFNI is distributed freely under the Gnu General Public License. Major portions of this software were written at the Medical College of Wisconsin, which owns the copyright to that code. For fuller details, see the file http://afni.nimh.nih.gov/pub/dist/src/README.copyright.

Usage

Usage on Pitzer

module load afni
singularity exec $AFNI_IMG suma

Usage on Owens

module load afni
singularity exec $AFNI_IMG suma

Further Reading

• Official AFNI Documentation
• Tutorials and Examples

The Assisted Model Building with Energy Refinement (AMBER) package, which includes AmberTools, contains many molecular simulation programs targeted at biomolecular systems. A wide variety of modelling techniques are available. It generally scales well on modest numbers of processors, and the GPU enabled CUDA programs are very efficient.

Availability and Restrictions

Versions

AMBER is available on the Owens, Pitzer, and Ascend clusters. The following versions are currently available at OSC (S means serial executables, P means parallel, and C means CUDA, i.e., GPU enabled):

You can use module spider amber to view available modules and use module spider amber/{version} to view installation details including applied Amber updates. Feel free to contact OSC Help if you need other versions or executables for your work.

Access for Academic Users

OSC's Amber is available to not-for-profit OSC users; simply contact OSC Help to request the appropriate form for access.

Access for Commercial Users

For-profit OSC users must obtain their own Amber license. 

Publisher/Vendor/Repository and License Type

University of California, San Francisco, Commercial

Usage

Usage on Owens

tleap

srun pmemd.MPI

sinteractive -A <project-account> -N 1 -n 28 -t 1:00:00

~srb/workshops/compchem/amber/

# AMBER Example Batch Script for the Basic Tutorial in the Amber manual
#!/bin/bash
#SBATCH --job-name 6pti
#SBATCH --nodes=1 --ntasks-per-node=28
#SBATCH --time=0:20:00
#SBATCH --account=<project-account>

module load amber
# Use TMPDIR for best performance.
cd $TMPDIR
# SLURM_SUBMIT_DIR refers to the directory from which the job was submitted.
cp -p $SLURM_SUBMIT_DIR/6pti.prmtop .
cp -p $SLURM_SUBMIT_DIR/6pti.prmcrd .
# Running minimization for BPTI
cat << eof > min.in
# 200 steps of minimization, generalized Born solvent model
&cntrl
maxcyc=200, imin=1, cut=12.0, igb=1, ntb=0, ntpr=10,
/
eof
sander -i min.in -o 6pti.min1.out -p 6pti.prmtop -c 6pti.prmcrd -r 6pti.min1.xyz
cp -p min.in 6pti.min1.out 6pti.min1.xyz $SLURM_SUBMIT_DIR

Usage on Pitzer

tleap

srun pmemd.MPI

sinteractive -A <project-account> -N 1 -n 48 -t 1:00:00

~srb/workshops/compchem/amber/

# AMBER Example Batch Script for the Basic Tutorial in the Amber manual 
#!/bin/bash
#SBATCH --job-name 6pti #
SBATCH --nodes=1 --ntasks-per-node=48 
SBATCH --time=0:20:00
#SBATCH --account=<project-account>

module load amber
# Use TMPDIR for best performance.
cd $TMPDIR
# SLURM_SUBMIT_DIR refers to the directory from which the job was submitted.
cp -p $SLURM_SUBMIT_DIR/6pti.prmtop .
cp -p $SLURM_SUBMIT_DIR/6pti.prmcrd .
# Running minimization for BPTI
cat << eof > min.in
# 200 steps of minimization, generalized Born solvent model
&cntrl
maxcyc=200, imin=1, cut=12.0, igb=1, ntb=0, ntpr=10,
/
eof
sander -i min.in -o 6pti.min1.out -p 6pti.prmtop -c 6pti.prmcrd -r 6pti.min1.xyz
cp -p min.in 6pti.min1.out 6pti.min1.xyz $SLURM_SUBMIT_DIR

Troubleshooting

In general, the scientific method should be applied to usage problems.  Users should check all inputs and examine all outputs for the first signs of trouble.  When one cannot find issues with ones inputs, it is often helpful to ask fellow humans, especially labmates, to review the inputs and outputs.  Reproducibility of molecular dynamics simulations is subject to many caveats.  See page 24 of the Amber18 manual for a discussion.

Further Reading

• Amber Tutorials
• Amber Home Page

ANSYS offers a comprehensive software suite that spans the entire range of physics, providing access to virtually any field of engineering simulation that a design process requires. Supports are provided by ANSYS, Inc. 

Availability and Restrictions

Versions

OSC has Academic Multiphysics Campus Solution license from Ansys. The license includes most of all the features that Ansys provides. See "Academic Multiphysics Campus Solution Products" in this table for all available products at OSC.

Access for Academic Users

OSC has an "Academic Research " license for ANSYS. This allows for academic use of the software by Ohio faculty and students, with some restrictions. To view current ANSYS node restrictions, please see ANSYS's Terms of Use.

Use of ANSYS products at OSC for academic purposes requires validation. Please contact OSC Help for further instruction.

Access for Commercial Users

Contact OSC Help for getting access to ANSYS if you are a commercial user.

Publisher/Vendor/Repository and License Type

Ansys, Inc., Commercial

Usage

For more information on how to use each ANSYS product at OSC systems, refer to its documentation page provided at the end of this page.

Known Issues

Simultaneously loading multiple of Fluent and ANSYS module cryptic error

Due to the way our Fluent and ANSYS modules are configured, simultaneously loading multiple of either module will cause a cryptic error. The most common case of this happening is when multiple of a user's jobs are started at the same time and all load the module at once. In order for this error to manifest, the modules have to be loaded at precisely the same time; a rare occurrence, but a probable occurrence over the long term.

If you encounter this error you are not at fault. Please resubmit the failed job(s).

If you frequently submit large amounts of Fluent or ANSYS jobs, we recommend you stagger your job submit times to lower the chances of two jobs starting at the same time, and hence loading the module at the same time. Another solution is to establish job dependencies between jobs, so jobs will only start one after another. To do this, you would add the SLURM directive:

#SBATCH --dependency=after:jobid

To jobs you want to only start after another job has started. You would replace jobid with the job ID of the job to wait for. If you have additional questions, please contact OSC Help.

Ansys DesignModeler with hardware acceleration

Further Reading

• ANSYS homepage

See Also

ARM HPC tools analyze how HPC software runs. It consists of three applications, ARM DDT, ARM Performance Reports and ARM MAP: 

• ARM DDT: graphical debugger for HPC applications.
• ARM MAP: HPC application profiler with easy-to-use GUI environment.
• ARM Performance Reports: simple tool to generate a single-page HTML or plain text report that presents overall performance characteristics of HPC applications.

Availability & Restrictions

Versions

The following versions of ARM HPC tools are available on OSC clusters:

You can use module spider arm to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

ARM DDT, MAP and Performance Reports are available to all OSC users.

Publisher/Vendor/Repository and License Type

ARM, Commercial

Usage

ARM DDT

ARM DDT is a debugger for HPC software that automatically alerts users of memory bugs and divergent behavior. For more features and benefits, visit ARM HPC tools and libraries - DDT.

For usage instructions and more iformation, read ARM DDT.

ARM MAP

ARM MAP produces a detailed profile of HPC software. Unlike ARM Performance Reports, you must have the source code to run ARM MAP because its analysis details the software line-by-line. For more features and benefits, visit ARM HPC tools and libraries - MAP. 

For usage instructions and more information, read ARM MAP.

ARM Performance Reports

ARM Performance Reports analyzes and documents information on CPU, MPI, I/O, and Memory performance characteristics of HPC software, even third party code, to aid understanding about the overall performance. Although it should not be used all the time, ARM Performance Reports is recommended to OSC users as a viable option to analyze how an HPC application runs. View an example report to navigate the format of a typical report. For more example reports, features and benefits, visit ARM HPC tools and libraries - Performance Reports.

For usage instructions and more information, read ARM Performance Reports.

Troubleshooting

Using ARM software with MVAPICH2

This note from ARM's Getting Started Guide applies to both perf-report and MAP:

Some MPIs, most notably MVAPICH, are not yet supported by ARM's Express Launch mode
(in which you can just put “perf-report” in front of an existing mpirun/mpiexec line). These can
still be measured using the Compatibility Launch mode.

Instead of this Express Launch command:

perf-report mpiexec <mpi args> <program> <program args> # BAD

Use the compatibility launch version instead:

perf-report -n <num procs> --mpiargs="<mpi args>" <program> <program args>

Further Reading

• ARM DDT homepage
• ARM MAP homepage
• ARM Performance Reports homepage

See Also

AlphaFold is a software package that provides an implementation of the inference pipeline of AlphaFold v2.0. This is a completely new model that was entered in CASP14 and pusblished in Nature.

Availability and Restrictions

Versions

You can use module spider alphafold to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

AlphaFold is available for all OSC users

Publisher/Vendor/Repository and License Type

Copyright 2021 DeepMind Technologies Limited

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at https://www.apache.org/licenses/LICENSE-2.0. Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See hte License for the specific language governing permissions and limitations under the License. See the License for specific langauge governing permissions and limitations under the License.

The AlphaFold parameters are made available for non-commercial use only, under the terms of the Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) license. You can find details at: https://creativecommons.org/licenses/by-nc/4.0/legalcode.

Usage

Usage on Pitzer

#!/bin/bash
#SBATCH --ntasks=8
#SBATCH --gpus-per-node=1
#SBATCH --gpu_cmode=shared


module reset
module load alphafold/2.1.2

run_alphafold.sh --use_gpu_relax=True --db_preset=reduced_dbs --fasta_paths=rcs_pdb_6VR4.fasta --max_template_date=2020-05-14 --output_dir=$(pwd -P)/output

run_alphafold.sh --helpshort

export TF_FORCE_UNIFIED_MEMORY=1
run_alphafold.sh ...

module reset
module load alphafold/2.2.2

export ALPHAFOLD_HOME=$HOME/alphafold
run_alphafold.sh --db_preset=reduced_dbs --fasta_paths=rcs_pdb_6VR4.fasta --max_template_date=2020-05-14 --output_dir=$(pwd -P)/output

#!/bin/bash
#SBATCH --ntasks=8
#SBATCH --gpus-per-node=2

module reset
module load alphafold/2.0.0

run_alphafold.sh --preset=reduced_dbs --fasta_paths=rcs_pdb_6VR4.fasta --max_template_date=2020-05-14 --output_dir=$(pwd -P)/output

--preset=recued_dbs, --preset=full_dbs, or --preset=casp14

Further Reading

Online documentation is available on the AlphaFold homepage.

Notes on AlphaFold output.

Notes on citing AlphaFold.

HyperWorks is a high-performance, comprehensive toolbox of CAE software for engineering design and simulation.

Availability & Restrictions

Versions

The following version of Altair Hyperworks can be found for the following environments:

You can use module spider hyperworks to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

HyperWorks is available to all academic clients. Please contact OSC Help to request the appropriate form for access.

Publisher/Vendor/Repository and License Type

Altair Engineering, Commercial (state-wide)

Usage

Using HyperWorks through OSC installation

To use HyperWorks on the OSC clusters, first ensure that X11 forwarding is enabled as the HyperWorks workbench is a graphical application. Then, load the hyperworks module:

module load hyperworks

The HyperWorks GUI can be launched then with the following command:

hw

The Hypermesh GUI can be launched then with the following command:

hm

State-wide access for HyperWorks

For information on downloading and installing a local copy through the state-wide license, follow the steps below. The versions of HyperWorks available statewide differ from the versions available at OSC on the Owens cluster. To check for the available statewide versions, complete steps 1 through 5 below.

NOTE: To run Altair HyperWorks, your computer must have access to the internet. The software contacts the license server at OSC to check out a license when it starts and periodically during execution. The amount of data transferred is small, so network connections over modems are acceptable.

Usage of HyperWorks on a local machine using the statewide license will vary from installation to installation.

1. Go to https://altairone.com/home

2. If you have already registered with the Altair website, click on "Sign In" in the upper right hand corner of the page, enter the e-mail address that you registered with and your password and skip to step #4. Otherwise click the "Sign Up" button instead and continue with step #3.

3. You will be prompted for some contact information and an e-mail address which will be your unique identifier.

   • IMPORTANT: The e-mail address you give must be from your academic institution. Under the statewide license agreement, registration from Ohio universities is allowed on the Altair web site. Trying to log in with a yahoo or hotmail e-mail account will not work. If you enter your university e-mail and the system will not register you, please contact OSChelp at oschelp@osc.edu.

4. Once you have logged in, go back to the home page and click on the button labeled "Altair Marketplace", where you can then press the button "Browse the Marketplace" which takes you to the Marketplace page.

5. From here, you can search for the app you would like to use, in this case you're looking for the one listed as "HyperWorks" which you can search for in the search bar at the upper left corner of the Marketplace page.

6. To download, you just need to press the "Download" button that appears in the side window that pops up after selecting the HyperWorks application from the marketplace page. From there you need to select the version you'd like and the target operating system for which it will run on. Then press the button that looks like an arrow point down at a "U" (aka Download symbol). In addition to downloading the software, download the "Installation Guide and Release Notes" for instructions on how to install the software.

   • NOTE: If you are a student and you click on the HyperWorks application in the marketplace, after creating an account and logging in, but see a "Try Now" button instead of a "Download" button then you may have not been added to the university account correctly (A known issue). To remedy this, please email support@altair.com with your name plus email, and ask the support team to update the account permissions so you can download the software.

   • IMPORTANT: If you have any questions or problems, please contact OSChelp at oschelp@osc.edu, rather than HyperWorks support. The software agreements outlines that problems should first be sent to OSC. If the OSC support line cannot answer or resolve the question, they have the ability to raise the problem to Altair support. If you have any general questions, or are looking for answers to frequently asked questions, you can check the Community Forums page for possible answers or help. But if you have problems, make sure to extend them to OSC first as stated above.

7. Please contact OSC Help for further instruction and license server information. In order to be added to the allowed list for the state-wide software access, we will need your IP address/range of machine that will be running this software. 

8. You need to set an environment variable (ALTAIR_LICENSE_PATH) on your local machine to point at our license server (7790@license6.osc.edu). See this link for instructions if necessary.

Further Reading

For more information about HyperWorks, see the following:

• Altair HyperWorks Documentation Page (Requires registration with Altair)
• Altair HyperWorks Academic Blog

See Also

• ABAQUS
• Altair HyperWorks statewide license

Apptainer/Singularity is a container system designed for use on High Performance Computing (HPC) systems. It allows users to run both Docker and Singularity containers.

From the Docker website: "A container image is a lightweight, stand-alone, executable package of a piece of software that includes everything needed to run it: code, runtime, system tools, system libraries, settings."

Availability and Restrictions

Versions

Apptainer/Singularity is available on all OSC clusters. Only one version is available at any given time. To find out the current version:

singularity version

Check the release page for the changelog: https://github.com/apptainer/apptainer/releases

Access

Apptainer/Singularity is available to all OSC users.

Publisher/Vendor/Repository and License Type

Apptainer project, established as Apptainer a Series of LF Projects LLC; 3-clause BSD License

Usage

Set-up

No setup is required. You can use Apptainer/Singularity directly on all clusters.

Using Singularity

See HOWTO: Use Docker and Singularity Containers at OSC for information about using Apptainer/Singularity on all OSC clusters, including some site-specific caveats.  

Example:  Run a container from the Singularity hub

[owens-login01]$ singularity run shub://singularityhub/hello-world
INFO:    Downloading library image
Tacotacotaco

Known Issues

Reached your pull rate limit

ERROR: toomanyrequests: Too Many Requests.

You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limits.

Failed to pull a large Docker image

[owens-login01]$ singularity pull docker://qimme2/core
FATAL: Unable to pull docker://qiime2/core While running mksquashfs: signal: killed

[owens-login01]$ sinteractive -n 1 -A PAS1234 
bash-4.2$ export APPTAINER_CACHEDIR=$TMPDIR
bash-4.2$ export APPTAINER_TMPDIR=$TMPDIR 
bash-4.2$ singularity pull docker://qiime2/core:2019.1
INFO:    Converting OCI blobs to SIF format
INFO:    Starting build...
...
INFO:    Creating SIF file...
bash-4.2$ exit

Failed to run a container directly or pull an image from Singularity or Docker hub

[owens-login01]$ singularity run shub://vsoch/hello-world
Progress |===================================| 100.0%
FATAL: container creation failed: mount error: can't mount image /proc/self/fd/13: failed to find loop device: could not attach image file too loop device: No loop devices available

singulariy cache clean

#/bin/bash
#SBATCH --job-name="singularity_test"
#SBSTCH --ntasks=1

export APPTAINER_CACHEDIR=$TMPDIR
singularity run shub://vsoch/hello-world

Workshop

• 2019-05-14 OSC Workshop: Containers for Research Computing. Please find the link in the workshop page for the tutorial material and slides. 

Further Reading

• Apptainer documents

The BLAS (Basic Linear Algebra Subprograms) are routines that provide standard building blocks for performing basic vector and matrix operations.

Availability and Restrictions

Access

A highly optimized implementation of the BLAS is available on all OSC clusters as part of the Intel Math Kernel Library (MKL). We recommend that you use MKL rather than building the BLAS for yourself. MKL is available to all OSC users.

Usage

See OSC's MKL software page for usage information. Note that there is no library named libblas.a or libblas.so. The flag "-lblas" on your link line will not work. You should modify your makefile or build script to link to the MKL libraries instead.

Further Reading

• BLAS Website
• Intel MKL Website

The BLAST programs are widely used tools for searching DNA and protein databases for sequence similarity to identify homologs to a query sequence. While often referred to as just "BLAST", this can really be thought of as a set of programs: blastp, blastn, blastx, tblastn, and tblastx.

Availability & Restrictions

Versions

The following versions of BLAST are available on OSC systems: 

You can use module spider blast to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

If you need to use blastx, you will need to load one of the C++ implimenations modules of blast (any version with a "+").

Access

BLAST is available to all OSC users. If you have any questions, please contact OSC Help.

Publisher/Vendor/Repository and License Type

National Institutes of Health, Open source

Usage

Set-up

To load BLAST, type the following into the command line:

module load blast

Then create a resource file .ncbirc, and put it under your home directory.

Using BLAST

The five flavors of BLAST mentioned above perform the following tasks:

• blastp: compares an amino acid query sequence against a protein sequence database

• blastn: compares a nucleotide query sequence against a nucleotide sequence database

• blastx: compares the six-frame conceptual translation products of a nucleotide query sequence (both strands) against a protein sequence database

• tblastn: compares a protein query sequence against a nucleotide sequence database dynamically translated in all six reading frames (both strands).

• tblastx: compares the six-frame translations of a nucleotide query sequence against the six-frame translations of a nucleotide sequence database. (Due to the nature of tblastx, gapped alignments are not available with this option)

NCBI BLAST Database

Information on the NCBI BLAST database can be found here. https://www.osc.edu/resources/available_software/scientific_database_list/blast_database 

We provide local access to nt and refseq_protein databases. You can access the database by loading desired blast-database modules. If you need other databases, please send a request email to OSC Help .

Batch Usage

A sample batch script on Owens and Pitzer is below:

#!/bin/bash
## --ntasks-per-node can be increased upto 48 on Pitzer
#SBATCH --nodes=1 --ntasks-per-node=28 
#SBATCH --time=00:10:00
#SBATCH --job-name Blast
#SBATCH --account=<project-account>

module load blast
module load blast-database/2018-08

cp 100.fasta $TMPDIR
cd $TMPDIR

tblastn -db nt -query 100.fasta -num_threads 16 -out 100_tblastn.out

cp 100_tblastn.out $SLURM_SUBMIT_DIR

Further Reading

• The BLAST Homepage
• Bioperl
• BLAT

BWA is a software package for mapping low-divergent sequences against a large reference genome, such as the human genome. It consists of three algorithms: BWA-backtrack, BWA-SW and BWA-MEM.

Availability and Restrictions

Versions

The following versions of BWA are available on OSC clusters:

You can use module spider bwa to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

BWA is available to all OSC users. If you have any questions, please contact OSC Help.

Publisher/Vendor/Repository and License Type

Li H. and Durbin R., Open source

Usage

Usage on Owens

Usage on Pitzer

Further Reading

• The BWA Official Webpage

BamTools provides both a programmer's API and an end-user's toolkit for handling BAM files.

Availability and Restrictions

Versions

The following versions of BamTools are available on OSC clusters:

You can use module spider bamtools to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

BamTools is available to all OSC users. If you have any questions, please contact OSC Help.

Publisher/Vendor/Repository and License Type

Derek Barnett, Erik Garrison, Gabor Marth, and Michael Stromberg/ Open Source

Usage

Usage on Owens

Usage on Pitzer

Further Reading

• The BamTools Official Webpage

Bismark is a program to map bisulfite treated sequencing reads to a genome of interest and perform methylation calls in a single step.

Availability and Restrictions

Versions

The following versions of bedtools are available on OSC clusters:

You can use module spider bismark to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Bismark is available to all OSC users. If you have any questions, please contact OSC Help.

Publisher/Vendor/Repository and License Type

Babraham Bioinformatics, GNU GPL v3

Usage

Usage on Owens

Usage on Pitzer

Further Reading

• The Bismark Official Webpage

Blender is the free and open source 3D creation suite. It supports the entirety of the 3D pipeline—modeling, rigging, animation, simulation, rendering, compositing and motion tracking, even video editing and game creation.

Availability and Restrictions

Versions

The following versions of Blender are available on OSC systems: 

You can use module spider blender to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Blender is available to all OSC users. If you have any questions, please contact OSC Help.

Publisher/Vendor/Repository and License Type

Blender Foundation, Open source

Usage

Set-up for Blender 3.6.3

module load blender/3.6.3

Using Blender 3.6.3

To run software-accelerated Blender, run either of the following equivalent commands:

apptainer exec $BLENDER_IMG blender

apptainer exec $BLENDER_IMG blender-softwaregl

Set-up for Blender 2.X

On Pitzer or Owens-Desktop 'vis' or 'any' node type, run the following command:

module load blender

Using Blender 2.X

To run hardware-rendering version of blender, connect to OSC OnDemand and luanch a virtual desktop, either a Lightweight Desktop or an Interactive HPC 'vis' type Desktop, and in desktop open a terminal and run blender with VirtualGL: 

module load virtualgl
vglrun blender

You can also run software-rendering version of blender on any type Desktop: 

blender-softwaregl

Further Reading

• Blender Demo Files
• Blender User Manual

Boost is a set of C++ libraries that provide helpful data structures and numerous support functions in a wide range of aspects of programming, such as, image processing, gpu programming, concurrent programming, along with many algorithms.  Boost is portable and performs well on a wide variety of platforms.

Availability & Restrictions

Versions

The following version of Boost are available on OSC systems:

You can use module spider boost to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Boost is available to all OSC users. If you have any questions, please contact OSC Help.

Publisher/Vendor/Repository and License Type

Beman Dawes, David Abrahams, Rene Rivera/ Open source

Usage

Usage on Owens

module load boost

cp /usr/local/src/boost/boost-1_56_0/test.osc/example2.cpp ~
cp /usr/local/src/boost/boost-1_56_0/test.osc/jayne.txt ~

g++ example2.cpp -o boostTest -lboost_regex
./boostTest < jayne.txt

Usage on Pitzer

module load boost

cp /usr/local/src/boost/boost-1_56_0/test.osc/example2.cpp ~
cp /usr/local/src/boost/boost-1_56_0/test.osc/jayne.txt ~

g++ example2.cpp -o boostTest -lboost_regex
./boostTest < jayne.txt

Usage on Ascend

module load boost

cp /usr/local/src/boost/boost-1_56_0/test.osc/example2.cpp ~
cp /usr/local/src/boost/boost-1_56_0/test.osc/jayne.txt ~

g++ example2.cpp -o boostTest -lboost_regex
./boostTest < jayne.txt

Further Reading

• The Boost Homepage

Bowtie1 is an ultrafast, memory-efficient short read aligner. It aligns short DNA sequences (reads) to the human genome at a rate of over 25 million 35-bp reads per hour. Bowtie indexes the genome with a Burrows-Wheeler index to keep its memory footprint small: typically about 2.2 GB for the human genome (2.9 GB for paired-end).

Availability and Restrictions

Versions

The following versions of Bowtie1 are available on OSC clusters:

You can use module spider bowtie1 to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Bowtie1 is available to all OSC users. If you have any questions, please contact OSC Help.

Publisher/Vendor/Repository and License Type

Ben Langmead et al., Open source (Artistic 2.0)

Usage

Usage on Owens

Usage on Pitzer

Further Reading

• The Bowtie1 Official Webpage

Bowtie2 is an ultrafast and memory-efficient tool for aligning sequencing reads to long reference sequences. It is particularly good at aligning reads of about 50 up to 100s or 1,000s of characters, and particularly good at aligning to relatively long (e.g. mammalian) genomes. Bowtie 2 indexes the genome with an FM Index to keep its memory footprint small: for the human genome, its memory footprint is typically around 3.2 GB. Bowtie 2 supports gapped, local, and paired-end alignment modes.

Availability and Restrictions

Versions

The following versions of Bowtie2 are available on OSC clusters:

You can use module spider bowtie2 to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Bowtie2 is available to all OSC users. If you have any questions, please contact OSC Help.

Publisher/Vendor/Repository and License Type

Ben Langmead et al., Open source

Usage

Usage on Owens

Usage on Pitzer

Further Reading

• The Bowtie2 Official Webpage

CIAO (also known as Chandra Interactive Analysis of Observations) is a X-Ray telescope analysis software package for astronomical observation. CIAO focuses on the Chandra X-ray observatory. It contains a toolset used to analyze fits files and is commonly used in conjuction with DS9 and Sherpa, and focuses on data flexibility. 

Availability and Restrictions

Versions

CIAO is available on Pitzer and Owens Clusters. The versions currently available at OSC are:

You can use module spider ciao to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Publisher/Vendor/Repository and License Type

Harvard & Smithsonian (Public)

Usage

Usage on Owens

ciaorun 

Usage on Pitzer

ciaorun 

Further Reading

Official documentation can be obtained from the CIAO's Website. 

References

1. DS9

CMake is a family of compilation tools that can be used to build, test and package software.

Availability and Restrictions

Versions

The current versions of CMake available at OSC are:

You can use  module spider cmake  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

CMake is available to all OSC users.

Publisher/Vendor/Repository and License Type

Aaron C. Meadows et al., Open source

Usage

Usage on Owens

Usage on Pitzer

Further Reading

For more information, visit the CMake homepage.

COMSOL Multiphysics (formerly FEMLAB) is a finite element analysis and solver software package for various physics and engineering applications, especially coupled phenomena, or multiphysics. owned and supported by COMSOL, Inc.

Availability and Restrictions

Versions

COMSOL is available on the Owens clusters. The versions currently available at OSC are:

You can use module spider comsol  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access for Academic Users

COMSOL is for academic use, available only to the Ohio State University users. OSC does not provide COMSOL licenses for academic use to students and faculty outside of the Ohio State University due to licensing restrictions. If you or your institution have a network COMSOL license server, you may be able to use it on OSC. For connections to your license server from OSC, please read this document. If you need further help, please contact OSC Help.

To use COMSOL you will have to be added to the license server.  Please contact OSC Help to be added.

Access for Commercial Users

Contact OSC Help for getting access to COMSOL if you are a commercial user. 

Publisher/Vendor/Repository and License Type

Comsol Inc., Commercial

Usage

Usage on Owens

sinteractive -A <project-account> -N 1 -n 28 -t 1:00:00 -L comsolscript@osc:1

#!/bin/bash
#SBATCH --time=1:00:00
#SBATCH --nodes=1 --ntasks-per-node=1
#SBATCH -L comsolscript@osc:1
#SBATCH --account=<project-account>
#
# The following lines set up the COMSOL environment
#
module load comsol
#
# Use TMPDIR for best performance
cp -p mycomsol.m $TMPDIR
cd $TMPDIR
#
# Run COMSOL
#
comsol batch mycomsol
#
# Now, copy data (or move) back once the simulation has completed
#
cp -p * $SLURM_SUBMIT_DIR

#!/bin/bash
#SBATCH --time=1:00:00
#SBATCH --nodes=2 --ntasks-per-node=4 --cpus-per-task=7
#SBATCH -L comsolscript@osc:1
#SBATCH --account=<project-account>

module load comsol
echo "--- Copy Input Files to TMPDIR and Change Disk to TMPDIR"
cp input_cluster.mph $TMPDIR
cd $TMPDIR

echo "--- COMSOL run"
comsol batch -mpibootstrap slurm -inputfile input_cluster.mph -outputfile output_cluster.mph
echo "--- Copy files back"
cp output_cluster.mph output_cluster.mph.status ${SLURM_SUBMIT_DIR}
echo "---Job finished at: 'date'"
echo "---------------------------------------------"

#!/bin/bash
#SBATCH --time=1:00:00
#SBATCH --nodes=2 --ntasks-per-node=28
#SBATCH -L comsolscript@osc:1
#SBATCH --account=<project-account>

module load comsol
echo "--- Copy Input Files to TMPDIR and Change Disk to TMPDIR"
cp input_cluster.mph $TMPDIR
cd $TMPDIR

echo "--- COMSOL run"
comsol -nn 2 batch -mpirsh ssh -inputfile input_cluster.mph -outputfile output_cluster.mph
echo "--- Copy files back"
cp output_cluster.mph output_cluster.mph.status ${SLURM_SUBMIT_DIR}
echo "---Job finished at: 'date'"
echo "---------------------------------------------"

AC/DC Module
Battery Design Module
CAD Import Module
CFD Module
Chemical Reaction Engineering Module
Heat Transfer Module
LiveLink for MATLAB
MEMS Module
Microfluidics Module
Particle Tracing Module
RF Module
Semiconductor Module
Structural Mechanics Module
Subsurface Flow Module

Further Reading

• The Official COMSOL Website

CP2K is a quantum chemistry and solid state physics software package that can perform atomistic simulations of solid state, liquid, molecular, periodic, material, crystal, and biological systems. CP2K provides a general framework for different modeling methods such as DFT using the mixed Gaussian and plane waves approaches GPW and GAPW. Supported theory levels include DFTB, LDA, GGA, MP2, RPA, semi-empirical methods and classical force fields. CP2K can do simulations of molecular dynamics, metadynamics, Monte Carlo, Ehrenfest dynamics, vibrational analysis, core level spectroscopy, energy minimization, and transition state optimization using NEB or dimer method.

Availability and Restrictions

Versions

CP2K is available on the OSC clusters. These are the versions currently available:

You can use module spider cp2k to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

CP2K is available to all OSC users.

Publisher/Vendor/Repository and License Type

CP2K, GNU General Public License

Usage

Usage

sinteractive -A <project-account> -n 1 -t 00:20:00

#!/bin/bash 
#SBATCH --nodes=2
#SBATCH --time=1:00:0
#SBATCH --account=<project-account>
#SBATCH --gres=pfsdir

module load  gnu/8.4.0 intelmpi/2019.7
module load cp2k/7.1
module list


cp job.inp $PFSDIR/job.inp
cd $PFSDIR
srun cp2k.popt -i job.inp -o job.out.$SLURM_JOB_ID 
cp job.out.$SLURM_JOB_ID $SLURM_SUBMIT_DIR/job.out.$SLURM_JOB_ID

Known Issues

CP2K 6.1 Floating-point exception on Pitzer Cascade Lakes (48-core) node:

Program received signal SIGFPE: Floating-point exception - erroneous arithmetic operation.

Backtrace for this error:

Thid could be a bug in libxsmm 1.9.0 which is released on Mar 15, 2018 (Cascade Lake is launched in 2019). The bug has been fixed in cp2k/7.1.

Further Reading

General documentation is available from the CP2K website.
Scratch Storage documentation is available from the Storage Guide

CUDA™ (Compute Unified Device Architecture) is a parallel computing platform and programming model developed by Nvidia that enables dramatic increases in computing performance by harnessing the power of the graphics processing unit (GPU).

Availability and Restrictions

Versions

CUDA is available on the clusters supporting GPUs. The versions currently available at OSC are:

You can use module spider cuda to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

CUDA is available for use by all OSC users.

Publisher/Vendor/Repository and License Type

Nvidia, Freeware 

Usage

Usage on Owens

#error -- unsupported GNU version! gcc versions later than X are not supported!

--nodes=1 --ntasks-per-node=28 --gpus-per-node=1 --gpu_cmode=exclusive

sinteractive -A <project-account> -N 1 -n 28 -g 1 -t 00:20:00 

#!/bin/bash
#SBATCH -- time=01:00:00
#SBATCH --nodes=1 --ntasks-per-node=1:gpus=1
#SBATCH --job-name compute
#SBATCH --account=<project-account>

module load cuda
cd $HOME/cuda
cp mycudaApp $TMPDIR
cd $TMPDIR
./mycudaApp

Usage on Pitzer

#error -- unsupported GNU version! gcc versions later than X are not supported!

--nodes=1 --ntasks-per-node=40 --gpus-per-node=1 --gpu_cmode=exclusive

sinteractive -A <project-account> -N 1 -n 40 -g 2 -t 00:20:00

#!/bin/bash
#SBATCH --time=01:00:00
#SBATCH --nodes=1 --ntasks-per-node=1 --gpus-per-node=1
#SBATCH --job-name Compute
#SBATCH --account=<project-account>

module load cuda
cd $HOME/cuda
cp mycudaApp $TMPDIR
cd $TMPDIR
./mycudaApp

GNU Compiler Support for NVCC 

Further Reading

Online documentation is available on the CUDA homepage.

Compiler support for the latest version of CUDA is available here.

CUDA optimization techniques.

Caffe is "a fast open framework for deep learning."

From their README:

Caffe is a deep learning framework made with expression, speed, and modularity in mind. It is developed by the Berkeley Vision and Learning Center (BVLC) and by community contributors. Yangqing Jia created  the project during his PhD at UC Berkeley. Caffe is released under the BSD 2-Clause license.

Caffe also includes interfaces for both Python and Matlab, which have been built but have not been tested.

Availability and Restrictions

Versions

The following versions of Caffe are available on OSC clusters:

You can use module spider caffe to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

The current version of Caffe on Owens requires cuda/8.0.44 for GPU calculations.

Access 

Caffe is available to all OSC users. If you have any questions, please contact OSC Help.

Publisher/Vendor/Repository and License Type

Berkeley AI Research, Open source

Usage

Usage on Owens

module load caffe

#!/bin/bash
#SBATCH --job-name=Caffe
#SBATCH --nodes=1 --ntask-per-node=28:gpu
#SBATCH --time=30:00
#SBATCH --account <project-account>

. /etc/profile.d/lmod.sh
# Load the modules for Caffe
ml caffe
# Migrate to job temp directory and copy folders over
cd $TMPDIR
cp -r $CAFFE_HOME/{examples,data} .
# Download, create, train
./data/mnist/get_mnist.sh
./examples/mnist/create_mnist.sh
./examples/mnist/train_lenet.sh
# Serialize log files
echo; echo 'LOG 1'
cat convert_mnist_data.bin.$(hostname)*
echo; echo 'LOG 2'
cat caffe.INFO
echo; echo 'LOG 3'
cat convert_mnist_data.bin.INFO
echo; echo 'LOG 4'
cat caffe.$(hostname).*
cp examples/mnist/lenet_iter_10000.* $SLURM_SUBMIT_DIR

sbatch job.txt

Further Reading

• Caffe homepage

Cell Ranger is a cell analysis library for generate feature-barcode matrices, perform Analysis for RNA samples. Cell Ranger works in pipelines for it's RNA  sequencing analysis which allows it to: process raw sequencing output, read alignment, generate gene-cell matrices, and can perform downstream analyses such as clustering and gene expression analysis.

Availability and Restrictions

Versions

Cell Ranger is available on the Pitzer Cluster. The versions currently available at OSC are:

You can use module spider cellranger to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Cell Ranger is available to only academic OSC users. Please review the license agreement and 10x Privacy Policy before use. If you have any questions, please contact OSC Help.

Publisher/Vendor/Repository and License Type

The 10x Genomics group, Closed source (academic)

Usage

Usage on Pitzer

Further Reading

• Cell Ranger Home Page

• Cell Ranger Tutorials

Clara Parabricks is a powerful toolkit designed for genomic analysis. It is primarily designed for GPU computation.

Availability and Restrictions

Versions

Clara Parabricks is available on Pitzer and Owens Clusters. The versions currently available at OSC are the following:

You can use module spider clara-parabricks  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Clara-Parabricks is available to all OSC users. If you have any questions, please contact OSC Help.

Publisher/Vendor/Repository and License Type

Open source

Usage 

Usage on Owens

sinteractive -A <project-account> -N 1 -n 28 -t 1:00:00

Usage on Pitzer

sinteractive -A <project-account> -N 1 -n 28 -t 1:00:00

Further Reading

• The Nvidia Clara Parabricks homepage

Clustal W is a multiple sequence alignment program written in C++.

Availability and Restrictions

Versions

The following versions of bedtools are available on OSC clusters:

You can use module spider clustalw to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Clustal W is available to all OSC users. If you have any questions, please contact OSC Help.

Publisher/Vendor/Repository and License Type

GNU Lesser GPL.

Usage

Usage on Owens

Usage on Pitzer

Further Reading

• Clustal W Official Webpage

Connectome is an open-source visualization and discovery tool used to explore data generated by the Human Connectome Project. The distribution includes wb_view, a GUI-based visualization platform, and wb_command, a command-line program for performing a variety of algorithmic tasks using volume, surface, and grayordinate data.

Availability and Restrictions

Versions

The following versions are available on OSC clusters:

You can use module spider connectome to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Connectome is available to all OSC users. If you have any questions, please contact OSC Help.

Publisher/Vendor/Repository and License Type

Connectome Workbench is made available via the GNU General Public License, version 2. The full license is available from here.

Usage

Usage on Pitzer

module load virutalgl/2.6.5
module load connectome
viglrun wb_view

Further Reading

• Connectome homepage
• The Connectome Coordination Facility homepage

Connectome Workbench is an open source, freely available visualization and analysis tool for neuroimaging data, especially data generated by the Human Connectome Project.

Availability and Restrictions

Versions

Connectome Workbench is available on Owens and Pitzer clusters. These are the versions currently available:

You can use module spider connectome-workbench to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Connectome Workbench is available to all OSC users.

Publisher/Vendor/Repository and License Type

Washington University School of Medicine, GPL

Usage

Set-up

To configure your environment for use of the workbench, run the following command:  module load connectome-workbench virtualgl. The default version will be loaded; the virtualgl module is required as well on some platforms. To select a particular version, use  module load connectome-workbench/version. For example, use  module load connectome-workbench/1.3.2 to load Connectome Workbench 1.3.2.

Further Reading

General documentation is available from the Connectome Workbench hompage.

Cufflinks is a program that analyzes RNA -Seq samples. It assembles aligned RNA-Seq reads into a set of transcripts, then inspects the transcripts to estimate abundances and test for differential expression and regulation in the RNA-Seq reads.

Availability and Restrictions

Versions

Cufflinks is available on the Owens Cluster. The versions currently available at OSC are:

You can use module spider cufflinks to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Cufflinks is available to all OSC users. If you have any questions, please contact OSC Help.

Publisher/Vendor/Repository and License Type

Cole Trapnell et al., Open source

Usage

Usage on Owens

Further Reading

• Cufflinks homepage

SAOImageDS9 is a astronomical imaging and data visualization application. DS9 provides support for FITS images, binary tables, multiple frame buffers, region manipulation, and colormaps display options

Availability and Restrictions

The following versions of DS9 are available on OSC clusters:

You can use  module spider ds9to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

DS9 is available to all OSC users. If you have any questions, please contact OSC Help.

Publisher/Vendor/Repository and License Type

Jessica Mink, Smithsonian Astrophysical Observatory/ Open source

Usage

Usage on Owens

Usage on Pitzer

Further Reading

• The DS9 Official Webpage

DSI Studio is a tractography software tool that maps brain connections and correlates findings with neuropsychological disorders. It is a collective implementation of several diffusion MRI methods, including diffusion tensor imaging (DTI), generalized q-sampling imaging (GQI), q-space diffeomorphic reconstruction (QSDR), diffusion MRI connectometry, and generalized deterministic fiber tracking.

Availability and Restrictions

The following versions of DSI Studio are available on OSC clusters:

You can use  module spider dsi-studio to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

DSI Studio is available to all OSC users. If you have any questions, please contact OSC Help.

Publisher/Vendor/Repository and License Type

DSI Studio is free and licensing information for both academic and non-academic licenses is available at the DSI Studio homepage.

Please refer to the citation page about how to acknowledge DSI Studio.

Usage

Usage on Pitzer

module load dsi-studio
singularity exec $DSI_IMG dsi_studio

Further Reading

• DSI Studio Home
• Documentation and Tutorials

Darshan is a lightweight "scalable HPC I/O characterization tool".  It is intended to profile I/O by emitting log files to a consistent log location for systems administrators, and also provides scripts to create summary PDFs to characterize I/O in MPI-based programs.

Availability and Restrictions

Versions

The following versions of Darshan are available on OSC clusters:

You can use module spider darshan to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access 

Darshan is available to all OSC users. If you have any questions, please contact OSC Help.

Publisher/Vendor/Repository and License Type

MCSD, Argonne National Laboratory, Open source

Usage

Usage on Owens & Pitzer

$ module spider darshan/3.2.1

------------------------------------------------------------------------------------------------
  darshan: darshan/3.2.1
------------------------------------------------------------------------------------------------

    You will need to load all module(s) on any one of the lines below before the "darshan/3.2.1" module is available to load.

      intel/19.0.3  intelmpi/2019.7
      intel/19.0.3  mvapich2/2.3.1
      intel/19.0.3  mvapich2/2.3.2
      intel/19.0.3  mvapich2/2.3.3
      intel/19.0.3  mvapich2/2.3.4
      intel/19.0.3  mvapich2/2.3.5
      intel/19.0.5  intelmpi/2019.3
      intel/19.0.5  intelmpi/2019.7
      intel/19.0.5  mvapich2/2.3.1
      intel/19.0.5  mvapich2/2.3.2
      intel/19.0.5  mvapich2/2.3.3
      intel/19.0.5  mvapich2/2.3.4
      intel/19.0.5  mvapich2/2.3.5

$ module load intel/19.0.5 mvapich2/2.3.5
$ module load darshan/3.2.1

module load darshan

# basic call to darshan
export MV2_USE_SHARED_MEM=0
export LD_PRELOAD=$OSC_DARSHAN_DIR/lib/libdarshan.so
srun [args] ./my_mpi_program

# to show evidence that Darshan is working and to see internal timing
export DARSHAN_INTERNAL_TIMING=yes
srun [args] ./my_mpi_program

#!/bin/bash
#SBATCH --job-name="darshan_mpi_pfsdir_test"
#SBATCH --ntasks=4
#SBATCH --ntasks-per-node=2
#SBATCH --output=rfm_darshan_mpi_pfsdir_test.out
#SBATCH --time=0:10:0
#SBATCH -p parallel
#SBATCH --gres=pfsdir:ess

# Setup Darshan
module load intel
module load mvapich2
module load darshan
export DARSHAN_LOGFILE=${LMOD_SYSTEM_NAME}_${SLURM_JOB_ID/.*/}_${SLURM_JOB_NAME}.log
export DARSHAN_INTERNAL_TIMING=yes
export MV2_USE_SHARED_MEM=0
export LD_PRELOAD=$OSC_DARSHAN_DIR/lib/libdarshan.so

# Prepare the scratch files and run the cases
cp ~support/share/reframe/source/darshan/io-sample.c .
mpicc -o io-sample io-sample.c -lm
for x in 0 1 2 3; do  dd if=/dev/zero of=$PFSDIR/read_only.$x bs=2097152000 count=1; done
shopt -s expand_aliases
srun ./io-sample -p $PFSDIR -b 524288000 -v

# Generat report
darshan-job-summary.pl --summary $DARSHAN_LOGFILE

sbatch darshan_mpi_pfsdir_test.sh

Further Reading

• Darshan homepage

Desmond is a software package that perform high-speed molecular dynamics simulations of biological systems on conventional commodity clusters, general-purpose supercomputers, and GPUs. The code uses novel parallel algorithms and numerical techniques to achieve high performance and accuracy on platforms containing a large number of processors, but may also be executed on a single computer. Desmond includes code optimized for machines with an NVIDIA GPU.

Availability and Restrictions

Versions

The Desmond package is available on Owens. The versions currently available at OSC are:

You can use  module spider desmond to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access for Academic Users 

Desmond is available to academic OSC users. Please review the license agreement carefully before use. If you have any questions, please contact OSC Help.

Publisher/Vendor/Repository and License Type

D. E. Shaw Research, Non-Commercial

Usage

Usage on Owens

​​module load desmond/2018.2

#!/bin/bash
#SBATCH --job-name multisim-batch
#SBATCH --time=0:20:00
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=24
#SBATCH --account=<account>

# Example Desmond single-node batch script. 

sstat -j $SLURM_JOB_ID
export
module load desmond/2018.2
module list

sbcast -p desmondbutane.msj $TMPDIR/desmondbutane.msj
sbcast -p desmondbutane.cfg $TMPDIR/desmondbutane.cfg
sbcast -p desmondbutane.cms $TMPDIR/desmondbutane.cms

cd $TMPDIR
$SCHRODINGER/utilities/multisim -HOST localhost -maxjob 1 -cpu 24 -m desmondbutane.msj -c desmondbutane.cfg desmondbutane.cms -mode umbrella -ATTACHED -WAIT
ls -l
cd $SLURM_SUBMIT_DIR
sgather -r $TMPDIR $SLURM_SUBMIT_DIR

module load desmond
maestro

#!/bin/bash
#SBATCH --time=0:20:00
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=8
#SBATCH --gpus-per-node=1
#SBATCH --account=<account>

module reset
module load desmond/2019.1

# Desmond job script starts here

"${SCHRODINGER}/utilities/multisim" -JOBNAME desmond_min_job_1 -HOST localhost -maxjob 1 -cpu 1 -m desmond_min_job_1.msj -c desmond_min_job_1.cfg -description Minimization desmond_min_job_1.cms -mode umbrella -set stage[1].set_family.md.jlaunch_opt=["-gpu"] -o desmond_min_job_1-out.cms -ATTACHED

"${SCHRODINGER}/utilities/multisim" -JOBNAME desmond_min_job_1 -HOST localhost -maxjob 1 -cpu 8 -m desmond_min_job_1.msj -c desmond_min_job_1.cfg -description Minimization desmond_min_job_1.cms -mode umbrella  -o desmond_min_job_1-out.cms -ATTACHED

#!/bin/bash
#SBATCH --time=0:20:00
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=24
#SBATCH --account=<account>

module reset
module load desmond/2019.1

# Desmond job script starts here

Further Reading

• Desmond Home Page

The FASTX-Toolkit is a collection of command line tools for Short-Reads FASTA/FASTQ files preprocessing.

Availability and Restrictions

Verisons

The following versions of FASTX-Toolkit are available on OSC clusters:

You can use  module spider fastx to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

FASTX-Toolkit is available to all OSC users. If you have any questions, please contact OSC Help.

Publisher/Vendor/Repository and License Type

Assaf Gordon, Open source

Usage

Usage on Owens

Further Reading

• The FASTX-Toolkit Official Webpage

FFTW is a C subroutine library for computing the Discrete Fourier Transform (DFT) in one or more dimensions, of arbitrary input size, and of both real and complex data. It is portable and performs well on a wide variety of platforms.

Availability and Restrictions

Versions

FFTW is available on Ruby, and Owens Clusters. The versions currently available at OSC are:

You can use module spider fftw2  or module spider fftw3  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

FFTW is available to all OSC users. If you have any questions, please contact OSC Help.

Publisher/Vendor/Repository and License Type

www.fftw.org, Open source

Usage

Usage on Owens

module load fftw3
icc $FFTW3_CFLAGS my-fftw.c -o my-fftw $FFTW3_LIBS 
ifort $FFTW3_FFLAGS more-fftw.f -o more-fftw $FFTW3_LIBS

Usage on Pitzer

module load fftw3
icc $FFTW3_CFLAGS my-fftw.c -o my-fftw $FFTW3_LIBS 
ifort $FFTW3_FFLAGS more-fftw.f -o more-fftw $FFTW3_LIBS

Usage on Ascend

module load fftw3
icc $FFTW3_CFLAGS my-fftw.c -o my-fftw $FFTW3_LIBS 
ifort $FFTW3_FFLAGS more-fftw.f -o more-fftw $FFTW3_LIBS

Further Reading

• The FFTW homepage

See Also

• The Intel MKL

FSL is a library of tools for analyzing FMRI, MRI and DTI brain imaging data.

Availability and Restrictions

Versions

The following versions of FSL are available on OSC clusters:

You can use module spider fsl to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

FSL is available to academic OSC users. Please review the license agreement carefully before use. If you have any questions, please contact OSC Help.

Publisher/Vendor/Repository and License Type

Analysis Group, University of Oxford/ freeware

Usage

Usage on Owens and Pitzer

source $FSLDIR/etc/fslconf/fsl.sh
fsl

source $FSLDIR/etc/fslconf/fsl.csh
fsl 

module load fsl/6.0.4
source $FSLDIR/etc/fslconf/fsl.sh
asl_gui --matplotlib

Further Reading 

• FSL website

FastQC provides quality control checks of high throughput sequence data that identify areas of the data that may cause problems during further analysis.

Availability and Restrictions

Versions

FastQC is available on the Owens cluster. The versions currently available at OSC are:

You can use module spider fastqc to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

FastQC is available to all OSC users. If you have any questions, please contact OSC Help.

Publisher/Vendor/Repository and License Type

Babraham Bioinformatics, Open source

Usage

Usage on Owens

Usage on Pitzer

Further Reading

• FastQC homepage

FreeSurfer is a software package used to anaylze nueroimaging data.

Availability & Restrictions

Versions

The following versions of FreeSurfer are available on OSC clusters:

You can use module spider freesurfer to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

FreeSurfer is available to academic OSC users. Please review the license agreement carefully before use. If you have any questions, please contact OSC Help.

Publisher/Vendor/Repository and License Type

Athinoula A. Martinos Center, Open source

Usage

Usage on Owens and Pitzer

source $FREESURFER_HOME/SetUpFreeSurfer.sh

source $FREESURFER_HOME/SetUpFreeSurfer.csh

export SUBJECTS_DIR=<path to subject data>

setenv SUBJECTS_DIR=<path to subject data>

source $FREESURFER_HOME/bin/cuda5_setup.sh

source $FREESURFER_HOME/bin/cuda5_setup.csh

Further Reading 

• FreeSurfer website

The General Atomic and Molecular Electronic Structure System (GAMESS) is a flexible ab initio electronic structure program. Its latest version can perform general valence bond, multiconfiguration self-consistent field, Möller-Plesset, coupled-cluster, and configuration interaction calculations. Geometry optimizations, vibrational frequencies, thermodynamic properties, and solution modeling are available. It performs well on open shell and excited state systems and can model relativistic effects. The GAMESS Home Page has additional information.

Availability and Restrictions

Versions

The current versions of GAMESS available on the Oakley and Owens Clusters are:

You can use module spider gamess to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

GAMESS is available to all OSC users. Please review the license agreement carefully before use. If you have any questions, please contact OSC Help.

Publisher/Vendor/Repository and License Type

Gordon research group, Iowa State Univ./ Proprietary freeware

Usage

Set-up

GAMESS usage is controlled via modules. Load one of the GAMESS modulefiles at the command line, in your shell initialization script, or in your batch scripts, for example:

module load gamess  

Examples

• Test input files are in the tests subdirectory
• Computational Chemistry Workshop materials are in ~srb/workshops/compchem/gamess

Further Reading

General documentation is available from the GAMESS Home page and in the local machine directories.

GATK is a software package for analysis of high-throughput sequencing data. The toolkit offers a wide variety of tools, with a primary focus on variant discovery and genotyping as well as strong emphasis on data quality assurance.

Availability and Restrictions

Versions

The following versions of GATK are available on OSC clusters:

You can use module spider gatk to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access for Academic Users

GATK4 is available to all OSC users under BSD 3-clause License.

GATK3 is available to academic OSC users. Please review the license agreement carefully before use. If you have any questions, please contact OSC Help.

Publisher/Vendor/Repository and License Type

Broad Institute, Inc., BSD 3-clause License (GATK4 only)

Usage

Usage on Owens

Usage on Pitzer

Known Issues

CBLAS undefined symbol error

INFO: successfully loaded /tmp/jniloader1239007313705592313netlib-native_system-linux-x86_64.so
java: symbol lookup error: /tmp/jniloader1239007313705592313netlib-native_system-linux-x86_64.so: undefined symbol: cblas_dspr
java: symbol lookup error: /tmp/jniloader1239007313705592313netlib-native_system-linux-x86_64.so: undefined symbol: cblas_dspr

$ module load lapack/3.8.0
$ module load gatk/4.1.2.0
$ LD_LIBRARY_PATH=$OSC_LAPACK_DIR/lib64 gatk AnyTool toolArgs

$ qsub -I -l nodes=1:ppn=1
$ cd $TMPDIR
$ export SINGULARITY_CACHEDIR=$TMPDIR
$ SINGULARITY_TMPDIR=$TMPDIR 
$ singularity pull docker://broadinstitute/gatk:4.1.2.0
$ cp gatk_4.1.2.0.sif ~/

$ singularity exec ~/gatk_4.1.2.0.sif gatk AnyTool ToolArgs

Further Reading

• The GATK Official Webpage

GLPK (GNU Linear Programming Kit) is a set of open source LP (linear programming) and MIP (mixed integer problem) routines written in ANSI C, which can be called from within C programs. 

Availability and Restrictions

Versions

The following versions are available on OSC systems:

You can use module spider glpk to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

GLPK is available to all OSC users. If you have any questions, please contact OSC Help.

Publisher/Vendor/Repository and License Type

GNU, Open source

Usage

Set-up

To set up your environment for using GLPK on Oakley, run the following command:

module load glpk

Compiling and Linking

To compile your C code using GLPK API routines, use the environment variable $GLPK_CFLAGS provided by the module:

gcc $GLPK_CFLAGS -c my_prog.c

To link your code, use the variable $GLPK_LIBS:

gcc my_prog.o $GLPK_LIBS -o my_prog

glpsol

Additionally, the GLPK module contains a stand-alone LP/MIP solver, which can be used to process files written in the GNU MathProg modeling language.  The solver can be invoked using the following command syntax:

glpsol [options] [filename]

For a complete list of options, use the following command:

glpsol --help

Further Reading

• GLPK Homepage

GMAP is a genomic mapping and alignment program for mRNA and EST sequences.

Availability and Restrictions

Versions

The following versions of GMAP are available on OSC clusters:

You can use module spider gmap to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

GMAP is available to all OSC users. If you have any questions, please contact OSC Help.

Publisher/Vendor/Repository and License Type

Genentech, Inc., Open source

Usage

Usage on Owens

Further Reading

• The GMAP Official Webpage

Fortran, C and C++ compilers produced by the GNU Project. 

Availability and Restrictions

Versions

GNU compilers are available on all our clusters. These are the versions currently available:

You can use module spider gnu to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

To find out what version of gcc you are using, type gcc --version.

Access

The GNU compilers are available to all OSC users. If you have any questions, please contact OSC Help.

Publisher/Vendor/Repository and License Type

https://www.gnu.org/software/gcc/, Open source

Usage

Usage on Owens

sinteractive -A <project-account> -N 1 -n 28 -l -t 1:00:00

#!/bin/bash
#SBATCH --time=1:00:00
#SBATCH --nodes=1 --ntasks-per-node=28
#SBATCH --job-name jobname
#SBATCH --account=<project-account>

module load gnu
cp hello.c $TMPDIR
cd $TMPDIR
gcc -O3 hello.c -o hello
./hello > hello_results
cp hello_results $SLURM_SUBMIT_DIR

sbatch job.txt

#!/bin/bash
#SBATCH --time=1:00:00
#SBATCH --nodes=2 --ntasks-per-node=28 
#SBATCH --job-name jobname
#SBATCH --account=<project-account>

module load gnu
mpicc -O3 hello.c -o hello
cp hello $TMPDIR
cd $TMPDIR
mpiexec ./hello > hello_results
cp hello_results $SLURM_SUBMIT_DIR

Usage on Pitzer

Known Issues

Type mismatch error

Error: Type mismatch between actual argument at (1) and actual argument at (2) (REAL(4)/REAL(8))

configure: error: The Fortran compiler gfortran will not compile files that call
the same routine with arguments of different types.

Multiple definition error

/.libs/libmca_mtl_psm.a(mtl_psm_component.o): multiple definition of `mca_mtl_psm_component'

Further Reading

• GCC, the GNU Compiler Collection

See Also

• Intel Compilers
• PGI Compilers
• Compilation Guide

GROMACS is a versatile package of molecular dynamics simulation programs. It is primarily designed for biochemical molecules, but it has also been used on non-biological systems.  GROMACS generally scales well on OSC platforms. Starting with version 4.6 GROMACS includes GPU acceleration.

Availability and Restrictions

Versions

GROMACS is available on Pitzer and Owens Clusters. Both single and double precision executables are installed. The versions currently available at OSC are the following:

You can use module spider gromacs  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

GROMACS is available to all OSC users. If you have any questions, please contact OSC Help.

Publisher/Vendor/Repository and License Type

http://www.gromacs.org/ Open source

Usage 

Usage on Owens

gmx pdb2gmx

srun gmx_mpi_d mdrun

sinteractive -A <project-account> -N 1 -n 28 -t 1:00:00

~srb/workshops/compchem/gromacs/

#!/bin/bash
# GROMACS Tutorial for Solvation Study of Spider Toxin Peptide
# see fwspider_tutor.pdf
#SBATCH --job-name fwsinvacuo.owens
#SBATCH --nodes=2 --ntasks-per-node=28
#SBATCH --account=PZS0711
# turn off verbosity for noisy module commands
set +vx
module purge
module load intel/18.0.3
module load mvapich2/2.3
module load gromacs/2018.2
module list
set -vx

cd $SLURM_SUBMIT_DIR
echo $SLURM_SUBMIT_DIR
sbcast -p 1OMB.pdb $TMPDIR/1OMB.pdb
sbcast -p em.mdp $TMPDIR/em.mdp

cd $TMPDIR
mpiexec -ppn 1 gmx pdb2gmx -ignh -ff gromos43a1 -f 1OMB.pdb -o fws.gro -p fws.top -water none
mpiexec -ppn 1 gmx editconf -f fws.gro -d 0.7

mpiexec -ppn 1 gmx editconf -f out.gro -o fws_ctr.gro -center 2.0715 1.6745 1.914
mpiexec -ppn 1 gmx grompp -f em.mdp -c fws_ctr.gro -p fws.top -o fws_em.tpr -maxwarn 1
mpiexec -ppn 1 ls -l 
mpiexec gmx_mpi mdrun -s fws_em.tpr -o fws_em.trr -c fws_ctr.gro -g em.log -e em.edr

cp -p * $SLURM_SUBMIT_DIR/

Usage on Pitzer

gmx pdb2gmx

srun gmx_mpi_d mdrun

sinteractive -A <project-account> -N 1 -n 40 -t 1:00:00

~srb/workshops/compchem/gromacs/

#!/bin/bash
# GROMACS Tutorial for Solvation Study of Spider Toxin Peptide
# see fwspider_tutor.pdf
#SBATCH --job-name fwsinvacuo.owens
#SBATCH --nodes=2 --ntasks-per-node=28
#SBATCH --account=PZS0711
# turn off verbosity for noisy module commands
set +vx
module purge
module load intel/18.0.3
module load mvapich2/2.3
module load gromacs/2018.2
module list
set -vx

cd $SLURM_SUBMIT_DIR
echo $SLURM_SUBMIT_DIR
sbcast -p 1OMB.pdb $TMPDIR/1OMB.pdb
sbcast -p em.mdp $TMPDIR/em.mdp

cd $TMPDIR
mpiexec -ppn 1 gmx pdb2gmx -ignh -ff gromos43a1 -f 1OMB.pdb -o fws.gro -p fws.top -water none
mpiexec -ppn 1 gmx editconf -f fws.gro -d 0.7

mpiexec -ppn 1 gmx editconf -f out.gro -o fws_ctr.gro -center 2.0715 1.6745 1.914
mpiexec -ppn 1 gmx grompp -f em.mdp -c fws_ctr.gro -p fws.top -o fws_em.tpr -maxwarn 1
mpiexec -ppn 1 ls -l 
mpiexec gmx_mpi mdrun -s fws_em.tpr -o fws_em.trr -c fws_ctr.gro -g em.log -e em.edr

cp -p * $SLURM_SUBMIT_DIR/

Further Reading

• The GROMACS homepage

GSL is a library of mathematical methods for C and C++ languages.

Availability and Restrictions

Versions

GSL is available on all clusters. The versions currently available at OSC are:

You can use module spider gslto view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

GSL is available to all OSC users. If you have any questions, please contact OSC Help.

Publisher/Vendor/Repository and License Type

GNU opensource

Usage

Usage on Owens

Usage on Pitzer

Usage on Ascend

Further Reading

• Gnu GSL software page

Gaussian is a very popular general purpose electronic structure program. Recent versions can perform density functional theory, Hartree-Fock, Möller-Plesset, coupled-cluster, and configuration interaction calculations among others. Geometry optimizations, vibrational frequencies, magnetic properties, and solution modeling are available. It performs well as black-box software on closed-shell ground state systems. 

Availability and Restrictions

Versions

Gaussian is available on the Pitzer and Owens Clusters. These versions are currently available at OSC (S means single node serial/parallel and C means CUDA, i.e., GPU enabled):

You can use module spider gaussian to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access for Academic Users

Use of Gaussian for academic purposes requires validation. In order to obtain validation, please contact OSC Help for further instruction.

Publisher/Vendor/Repository and License Type

Gaussian, commercial

Usage

Usage on Owens

g16 < input.com

g16 input.com

sinteractive -A <project-account> -N 1 -n 28 -t 1:00:00

/users/appl/srb/workshops/compchem/gaussian/

#!/bin/bash
#SBATCH --job-name=GaussianJob
#SBATCH --nodes=1 --ntasks-per-node=28
#SBATCH --time=1:00:00
#SBATCH --account=<project-account>

cp input.com $TMPDIR
# Use TMPDIR for best performance.
cd $TMPDIR
module load gaussian
g16 input.com
cp -p input.log *.chk $SLURM_SUBMIT_DIR

Usage on Pitzer

g16 < input.com

g16 input.com

sinteractive -A <project-account> -N 1 -n 40 -t 1:00:00

/users/appl/srb/workshops/compchem/gaussian/

#!/bin/bash
#SBATCH --job-name=GaussianJob
#SBATCH --nodes=1 --ntasks-per-node=40
#SBATCH --time=1:00:00
#SBATCH --account=<project-account>

cp input.com $TMPDIR
# Use TMPDIR for best performance.
cd $TMPDIR
module load gaussian
g16 input.com
cp -p input.log *.chk $SLURM_SUBMIT_DIR

Running Gaussian jobs with GPU

#!/bin/bash 
#SBATCH --job-name=GaussianJob 
#SBATCH --nodes=1 --ntasks-per-node=40
#SBATCH --gpus-per-node=1
#SBATCH --time=1:00:00
#SBATCH --account=<project-account>

set echo
cd $TMPDIR
set INPUT=methane.com
# SLURM_SUBMIT_DIR refers to the directory from which the job was submitted.
cp $SLURM_SUBMIT_DIR/$INPUT .
module load gaussian/g16b01
g16 < ./$INPUT
ls -al
cp -p *.chk $SLURM_SUBMIT_DIR

%nproc=28
%mem=8gb
%CPU=0-27
%GPUCPU=0=0
%chk=methane.chk
#b3lyp/6-31G(d) opt
methane B3LYP/6-31G(d) opt freq
0,1
C        0.000000        0.000000        0.000000
H        0.000000        0.000000        1.089000
H        1.026719        0.000000       -0.363000
H       -0.513360       -0.889165       -0.363000
H       -0.513360        0.889165       -0.363000

#!/bin/tcsh
#SBATCH --job-name=methane
#SBATCH --output=methane.log
#SBATCH --nodes=1 --ntasks-per-node=48
#SBATCH --gpus-per-node=1
#SBATCH --time=1:00:00
#SBATCH --account=<project-account>

set echo
cd $TMPDIR
set INPUT=methane.com
# SLURM_SUBMIT_DIR refers to the directory from which the job was submitted.
cp $SLURM_SUBMIT_DIR/$INPUT .
module load gaussian/g16b01
g16 < ./$INPUT
ls -al
cp -p *.chk $SLURM_SUBMIT_DIR

%nproc=48
%mem=8gb
%CPU=0-47
%GPUCPU=0=0
%chk=methane.chk
#b3lyp/6-31G(d) opt
methane B3LYP/6-31G(d) opt freq
0,1
C        0.000000        0.000000        0.000000
H        0.000000        0.000000        1.089000
H        1.026719        0.000000       -0.363000
H       -0.513360       -0.889165       -0.363000
H       -0.513360        0.889165       -0.363000

Known Issues

Out of Memory Problems for Large TMPDIR Jobs

For some Gaussian jobs, the operating system will start swapping and may trigger the out of memory (OOM) killer because of memory consumption by the local filesystem (TMPDIR) cache.  For these jobs %mem may not be critical, i.e., these jobs may not be big memory jobs per se; it is the disk usage that causes the OOM; known examples of this case are large ONIOM calculations.

While an investigation is ongoing, a simple workaround is to avoid putting the Gaussian internal files on TMPDIR.  The most obvious alternative to TMPDIR is PFSDIR, in which case the commands are

...
#SBATCH --gres=pfsdir
...
module load gaussian
export GAUSS_SCRDIR=$PFSDIR
...

Other workarounds exist; contact oschelp@osc.edu for details.

g16b01 G4 Problem

See the known issue and note that g16c01 is the current default module version.

Further Reading

• The Gaussian homepage

Git is a version control system used for tracking file changes and facilitating collaborative work.

Availability and Restrictions

Versions

The following versions of Git are available on OSC clusters:

You can use module spider git to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Git is available to all OSC users. If you have any questions, please contact OSC Help.

Publisher/Vendor/Repository and License Type

Git, Open source

Usage

Usage on Owens

Usage on Pitzer

Further Reading

• Git homepage

Gnuplot is a portable command-line driven data and function plotting utility.  It was originally intended to allow scientists and students to visualize mathematical functions and data.  

Gnuplot supports many types of plots in two or three dimensions.  It can draw using points, lines, boxes, contours, vector fields surfaces and various associated text.  It also supports various specialized plot types.  

Gnuplot supports many different types of output:  interactive screen display (with mouse and hotkey functionality), pen plotters (like hpgl), printers (including postscript and many color devices), and file formats as vectorial pseudo-devices like LaTeX, metafont, pdf, svg, or bitmap png.  

Availability and Restrictions

Versions

The current versions of Gnuplot available at OSC are:

You can use module spider gnuplot to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Gnuplot is available to all OSC users.

Publisher/Vendor/Repository and License Type

Thomas Williams, Colin Kelley/ Open source

Usage

Usage on Owens

module load gnuplot

gnuplot

Usage on Pitzer

module load gnuplot

gnuplot

Further Reading

For more information, visit the Gnuplot Homepage.  

Gurobi is a mathematical optimization solver that supports a variety of programming and modeling languages.

Availability and Restrictions

Versions

The following versions of bedtools are available on OSC clusters:

You can use module spider gurobi to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Gurobi is available to academic OSC users with proper validation. In order to obtain validation, please contact OSC Help for further instruction.

Publisher/Vendor/Repository and License Type

Gurobi Optimization, LLC/ Free academic floating license

Usage

Usage on Owens

Further Reading

• The Gurobi Official Webpage

HDF5 is a general purpose library and file format for storing scientific data. HDF5 can store two primary objects: datasets and groups. A dataset is essentially a multidimensional array of data elements, and a group is a structure for organizing objects in an HDF5 file. Using these two basic objects, one can create and store almost any kind of scientific data structure, such as images, arrays of vectors, and structured and unstructured grids.

Availability and Restrictions

Versions

HDF5 is available on the Pitzer and Owens Clusters. The versions currently available at OSC are:

You can use module spider hdf5 to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

HDF5 is available to all OSC users. If you have any questions, please contact OSC Help.

Publisher/Vendor/Repository and License Type

The HDF Group, Open source (academic)

API Compatibility issue on hdf5/1.12

hdf5/1.12 may not compatible with applications created with earlier hdf5 versions. In order to work around, users may use a compatibility macro mapping:

• To compile an application built with a version of HDF5 that includes deprecated symbols (the default), specify: -DH5_USE_110_API (autotools) or –DH5_USE_110_API:BOOL=ON (CMake)

However, users will not be able to take advantage of some of the new features in 1.12 if using these compatibility mappings. For more detail, please see release note.

Usage

Usage on Owens

icc -c $HDF5_C_INCLUDE myprog.c
icc -o myprog myprog.o $HDF5_C_LIBS
ifort -c $HDF5_F90_INCLUDE myprog.f90
ifort -o myprog myprog.o $HDF5_F90_LIBS

#!/bin/bash
#SBATCH --job-name=AppNameJob
#SBATCH --nodes=1 --ntasks-per-node=28
#SBATCH --account <project-account>

module load hdf5
cp foo.dat $TMPDIR
cd $TMPDIR
appname
cp foo_out.h5 $SLURM_SUBMIT_DIR

Usage on Pitzer

icc -c $HDF5_C_INCLUDE myprog.c
icc -o myprog myprog.o $HDF5_C_LIBS
ifort -c $HDF5_F90_INCLUDE myprog.f90
ifort -o myprog myprog.o $HDF5_F90_LIBS

#!/bin/bash
#SBATCH --job-name=AppNameJob 
#SBATCH --nodes=1 --ntasks-per-node=48
#SBATCH --account <project-account>

module load hdf5
cp foo.dat $TMPDIR
cd $TMPDIR
appname
cp foo_out.h5 $SLURM_SUBMIT_DIR

Further Reading

• The HDF5 home page
• HDF5 tutorials

• NetCDF

HISAT2 is a graph-based alignment program that maps DNA and RNA sequencing reads to a population of human genomes.

Availability and Restrictions

Versions

HISAT2 is available on the Owens Cluster. The versions currently available at OSC are:

You can use module spider hisat2 to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

HISAT2 is available to all OSC users. If you have any questions, please contact OSC Help.

Publisher/Vendor/Repository and License Type

https://ccb.jhu.edu/software/hisat2, Open source

Usage

Usage on Owens and Pitzer

Further Reading

• HISAT2 homepage

HMMER is used for searching sequence databases for sequence homologs, and for making sequence alignments. It implements methods using probabilistic models called profile hidden Markov models (profile HMMS). HMMER is designed to detect remote homologs as sensitively as possible, relying on the strength of its underlying probability models.

Availability and Restrictions

Versions

HMMER is available on the OSC clusters. These are the versions currently available:

You can use module spider hmmer to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

HMMER is available to all OSC users.

Publisher/Vendor/Repository and License Type

Copyright (C) 2020 Howard Hughes Medical Institute.

HMMER and its documentation are feely distributed under the 3-Clause BSD open source license. For a copy of the license, see opensource.org/licenses/BSD-3-Clause.

Usage

Usage on Owens

Usage on Pitzer

Further Reading

• HMMER homepage
• Official User Guide

HOMER (Hypergeometric Optimization of Motif EnRichment) is a suite of tools for Motif Discovery and ChIP-Seq analysis. It is a collection of command line programs for unix-style operating systems written in mostly perl and c++. Homer was primarily written as a de novo motif discovery algorithm that is well suited for finding 8-12 bp motifs in large scale genomics data.

Availability and Restrictions

Versions

The following versions of HOMER are available on OSC clusters:

You can use  module spider homer to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

HOMER is available to all OSC users. If you have any questions, please contact OSC Help.

Publisher/Vendor/Repository and License Type

Christopher Benner, Open source

Usage

HOMER data

We maintain the HOMER data in a central location, which can be accessed and shared by all versions of HOMER. Current availabe data are listed below:

You can access the data via the environment variable $HOMER_DATA after loading the homer module. If you need other data, please contact OSC Help.

Usage on Owens

annotatePeaks.pl input.bed $HOMER_DATA/genomes/mm10 > output.txt

#!/bin/bash
#SBATCH --job-name homer_data_test 
#SBATCH --time=1:00:00
#SBATCH --nodes=1 --ntasks-per-node=1
#SBATCH --account=<project-account>

cp output_test.fastq $TMPDIR

module load homer/4.10

cd $TMPDIR
homerTools trim -3 GTCTTT -mis 1 -minMatchLength 4 -min 15 output_test.fastq

sgather -pr $TMPDIR ${SLURM_SUBMIT_DIR}/sgather

Further Reading

• The HOMER Official Webpage

HPC Toolkit is a collection of tools that measure a program's work, resource consumption, and inefficiency to analze performance.

Availability and Restrictions

Versions

The following versions of HPC Toolkitare available on OSC clusters:

You can use module spider hpctoolkit to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

HPC Toolkit is available to all OSC users. If you have any questions, please contact OSC Help.

Publisher/Vendor/Repository and License Type

Rice Univerity, Open source

Usage

Usage on Owens

Usage on Pitzer

Further Reading

• HPC Toolkit homepage

HTSlib is a C library used for reading and writing high-throughput sequencing data. HTSlib is the core library used by SAMtools. HTSlib also provides the bgzip, htsfile, and tabix utilities.

Availability and Restrictions

Versions

The versions of HTSlib currently available at OSC are:

You can use module spider htslib to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

HTSlib is available to all OSC users.

Publisher/Vendor/Repository and License Type

Genome Research Ltd., Open source

Usage

Usage on Owens and Pitzer

Further Reading

• HTSlib homepage

A hadoop cluster can be launched within the HPC environment, but managed by the PBS/slurm job scheduler using  Myhadoop framework developed by San Diego Supercomputer Center. (Please see https://www.grid.tuc.gr/fileadmin/users_data/grid/documents/hadoop/Krish...)

Availability and Restrictions

Versions

The following versions of Hadoop are available on OSC systems: 

You can use module spider hadoop to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Hadoop is available to all OSC users. If you have any questions, please contact OSC Help.

Publisher/Vendor/Repository and License Type

Apache software foundation, Open source

Usage

Set-up

In order to configure your environment for the usage of Hadoop, run the following command:

module load hadoop

In order to access a particular version of Hadoop, run the following command

module load hadoop/3.0.0-alpha1

Using Hadoop

In order to run Hadoop in batch, reference the example batch script below. This script requests 6 node on the Owens cluster for 1 hour of walltime. 

#!/bin/bash
#SBATCH --job-name hadoop-example
#SBATCH --nodes=6 --ntasks-per-node=28
#SBATCH --time=01:00:00
#SBATCH --account <account>

export WORK=$SLURM_SUBMIT_DIR
module load hadoop/3.0.0-alpha1
module load myhadoop/v0.40
export HADOOP_CONF_DIR=$TMPDIR/mycluster-conf-$SLURM_JOBID

cd $TMPDIR

myhadoop-configure.sh -c $HADOOP_CONF_DIR -s $TMPDIR
$HADOOP_HOME/sbin/start-dfs.sh
hadoop dfsadmin -report
hadoop  dfs -mkdir data
hadoop  dfs -put $HADOOP_HOME/README.txt  data/
hadoop  dfs -ls data
hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.0.0-alpha1.jar wordcount data/README.txt wordcount-out
hadoop  dfs -ls wordcount-out
hadoop  dfs  -copyToLocal -f  wordcount-out  $WORK
$HADOOP_HOME/sbin/stop-dfs.sh
myhadoop-cleanup.sh

Example Jobs

Please check /usr/local/src/hadoop/3.0.0-alpha1/test.osc folder for more examples of hadoop jobs

Further Reading

• Hadoop home page

See Also

• Spark

"Horovod is a distributed training framework for TensorFlow, Keras, PyTorch, and MXNet. The goal of Horovod is to make distributed Deep Learning fast and easy to use. The primary motivation for this project is to make it easy to take a single-GPU TensorFlow program and successfully train it on many GPUs faster."

Quote from Horovod Github documentation. 

Installation

Please follow the link for general instructions on installing Horovod for use with GPUs. The commands below assume a Bourne type shell; if you are using a C type shell then the "source activate" command may not work; in general, you can load all the modules, define any environment variables, and then type "bash" and execute the other commands.

Step 1: Install NCCL 2

Please download NCCL 2 from https://developer.nvidia.com/nccl (select OS agnostic local installer; Download NCCL 2.7.8, for CUDA 10.2, July 24,2020 was used in the latest test of this recipe).

Add the nccl library path to LD_LIBRARY_PATH environment variable

$ export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:Path_to_nccl/nccl-<version>/lib

Step 2: Install horovod python package

module load python/3.6-conda5.2

Create a local python environment for a horovod installation with nccl and activate it

conda create -n horovod-withnccl python=3.6 anaconda
source activate horovod-withnccl

Install a GPU version of tensorflow or pytorch

pip install https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-1.10.0-cp36-cp36m-linux_x86_64.whl

Load mvapich2 and cuda modules

module load gnu/7.3.0  mvapich2-gdr/2.3.4 

module load cuda/10.2.89

Install the horovod python package

HOROVOD_NCCL_HOME=/path_to_nccl_home/ HOROVOD_GPU_ALLREDUCE=NCCL pip install --no-cache-dir horovod

Testing

Please get the benchmark script from here.

#!/bin/bash 
#SBATCH --job-name R_ExampleJob 
#SBATCH --nodes=2 --ntasks-per-node=48 
#SBATCH --time=01:00:00 
#SBATCH --account <account>

module load python/3.6-conda5.2 
module load cuda/10.2.89 
module load gnu/7.3.0 
module load mvapich2-gdr/2.3.4 
source activate horovod-withnccl export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/path_to_nccl_home/lib mpiexec -ppn 1 -binding none -env NCCL_DEBUG=INFO python tf_cnn_benchmarks.py

Feel free to contact OSC Help if you have any issues with installation.

Publisher/Vendor/Repository and License Type

https://eng.uber.com/horovod/, Open source

Further Reading

TensorFlow homepage

The Intel compilers for both C/C++ and FORTRAN.

Availability and Restrictions

Versions

The versions currently available at OSC are:

You can use module spider intel  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

The Intel Compilers are available to all OSC users. If you have any questions, please contact OSC Help.

Publisher/Vendor/Repository and License Type

Intel, Commercial (state-wide)

Usage

Usage on Owens

module load cxx14

    error: more than one instance of overloaded function "std::to_string" matches the argument list:
              detected during:
                instantiation of "..."

    error: class "std::vector<std::pair<short, short>, std::allocator<std::pair <short, short>>>" has no member "..."
              detected during:
                instantiation of "..."

sinteractive -A <project-account> -N 1 -n 28 -t 1:00:00

#!/bin/bash
#SBATCH --time=1:00:00 
#SBATCH --nodes=1 --ntasks-per-node=28 
#SBATCH --job-name jobname 
#SBATCH --account=<project-account>

module load intel 
cp hello.c $TMPDIR 
cd $TMPDIR 
icc -O2 hello.c -o hello 
./hello > hello_results 
cp hello_results $SLURM_SUBMIT_DIR

sbatch job.txt

#!/bin/bash
#SBATCH --time=1:00:00
#SBATCH--nodes=2 --ntasks-per-node=40
#SBATCH --job-name name
#SBATCH --account=<project-account>

module load intel
mpicc -O2 hello.c -o hello
cp hello $TMPDIR
cd $TMPDIR
mpiexec ./hello > hello_results
cp hello_results $SLURM_SUBMIT_DIR

Usage on Pitzer

    error: more than one instance of overloaded function "std::to_string" matches the argument list:
              detected during:
                instantiation of "..."

    error: class "std::vector<std::pair<short, short>, std::allocator<std::pair <short, short>>>" has no member "..."
              detected during:
                instantiation of "..."

sinteractive -A <project-account> -N 1 -n 40 -t 1:00:00

#!/bin/bash
#SBATCH --time=1:00:00
#SBATCH --nodes=1 --ntasks-per-node=40
#SBATCH --job-name hello
#SBATCH --account=<project-account>

module load intel
cp hello.c $TMPDIR
cd $TMPDIR
icc -O2 hello.c -o hello
./hello > hello_results
cp hello_results $SLURM_SUBMIT_DIR

sbatch job.txt

#!/bin/bash
#SBATCH --time=1:00:00
#SBATCH --nodes=2 --ntasks-per-node=40
#SBATCH --job-name name
#SBATCH --account=<project-account>

module load intel
module laod intelmpi
mpicc -O2 hello.c -o hello
cp hello $TMPDIR
cd $TMPDIR
sun ./hello > hello_results
cp hello_results $SLURM_SUBMIT_DIR

Further Reading

• The Intel Compiler Home Page

See Also

• Intel Math Kernel Library
• AMD Core Math Library
• Compilation Guide

Intel's implementation of the Message Passing Interface (MPI) library. See Intel Compilers for available compiler versions at OSC.

Availability and Restrictions

Versions

Intel MPI may be used as an alternative to - but not in conjunction with - the MVAPICH2 MPI libraries. The versions currently available at OSC are:

You can use module spider intelmpi to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Intel MPI is available to all OSC users. If you have any questions, please contact OSC Help.

Publisher/Vendor/Repository and License Type

Intel, Commercial

Usage

Usage on Owens

#!/bin/bash
#SBATCH --job-name MyIntelMPIJob
#SBATCH --nodes=4 --ntasks-per-node=28
#SBATCH --time=5:00:00
#SBATCH --account=<project-account>

module load intelmpi
srun my-impi-application

Usage on Pitzer

#!/bin/bash
#SBATCH --job-name MyIntelMPIJob
#SBATCH --nodes=2 --ntasks-per-node=48
#SBATCH --time=5:00:00
#SBATCH --account=<project-account>

module load intelmpi
srun my-impi-application

Usage on Ascend

#!/bin/bash
#SBATCH --job-name MyIntelMPIJob
#SBATCH --nodes=2 --ntasks-per-node=48
#SBATCH --time=5:00:00
#SBATCH --account=<project-account>

module load intelmpi
srun my-impi-application

Known Issues

A partial-node MPI job failed to start using mpiexec

[mpiexec@o0439.ten.osc.edu] wait_proxies_to_terminate (../../../../../src/pm/i_hydra/mpiexec/intel/i_mpiexec.c:532): downstream from host o0439 was killed by signal 11 (Segmentation fault)
[mpiexec@o0439.ten.osc.edu] main (../../../../../src/pm/i_hydra/mpiexec/mpiexec.c:2114): assert (exitcodes != NULL) failed

/var/spool/torque/mom_priv/jobs/11510761.owens-batch.ten.osc.edu.SC: line 30: 11728 Segmentation fault  

/var/spool/slurmd/job00884/slurm_script: line 24:  3180 Segmentation fault      (core dumped)

#SBATCH --nodes=1
#SBATCH --ntasks-per-node=N

Using mpiexec/mpirun with Slurm

unset I_MPI_PMI_LIBRARY I_MPI_HYDRA_BOOTSTRAP
export I_MPI_JOB_RESPECT_PROCESS_PLACEMENT=0   # the option -ppn only works if you set this before

MPI-IO issues on home directories

Further Reading

• Intel MPI page at Intel.com
• Vendor documentation

See Also

• MVAPICH2
• OpenMPI
• Intel Compilers
• Intel Math Kernel Library

Java is a concurrent, class-based, object-oriented programming language.