15 July 2016, 5:00PM update: some additional issues we are facing

Python

Python is a high-level, multi-paradigm programming language that is both easy to learn and useful in a wide variety of applications.  Python has a large standard library as well as a large number of third-party extensions, most of which are completely free and open source. 

Availability and Restrictions

Python is available on Oakley, Ruby, and Owens Clusters. The versions currently available at OSC are:

Version Oakley ruby owens notes
2.7     X The Latest Version of Python 2.7
2.7.1 X     Default version on Oakley prior to 09/15/2015
2.7.8 X(A)      
2.7.9   X(A)    
3.4.2 X(A)* X(A)*    
3.5     X* The Latest Version of Python 3.5
*: Current default version

NOTE:

You can use module avail python to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access

Python is available for use by all OSC users.

Usage on Oakley

Set-up

To load the default version of Python module, use  module load python. To select a particular software version, use   module load python/version. For example, use  module load python/2.7.1 to load Python version 2.7.1. After the module is loaded, you can run the interpreter by using the command python. To unload the Python 2.7.1 module, use the command module unload python/2.7.1

Installed Modules

We have installed a number of Python packages and tuned them for optimal performance on our systems. When Python module is loaded, executing  module help python will help you view the current list of packages available.

NOTE:
  • numpy and scipy have been compiled to use MKL for best performance
  • Due to architecture differences between our supercomputers, we recommend NOT installing your own packages in  ~/.local. Instead, you should install them in some other directory and set  $PYTHONPATH in your default environment. For more information about installing your own Python modules, please see our HOWTO.

Batch Usage

When you log into oakley.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info.  Batch jobs run on the compute nodes of the system and not on the login node. It is desirable for big problems since more resources can be used.

Interactive Batch Session

For an interactive batch session one can run the following command:

qsub -I -l nodes=1:ppn=12 -l walltime=00:20:00 
which requests one whole node with 12 cores ( -l nodes=1:ppn=12), for a walltime of 20 minutes ( -l walltime=00:20:00). You may adjust the numbers per your need.
Non-interactive Batch Job (Serial Run)

Python can technically be used in both interactive and non-interactive batch jobs, but you will most likely only want to run a Python script non-interactively. To do this, you will need to write a batch submission script and a Python script using any text editor you like in a working directory on the system of your choice. A batch script can be created and submitted for a serial or parallel run. Below is a sample batch submission script( job.txt) and a simple “Hello World” program ( hello.py ) written in Python.

Example batch submission script  job.txt:
#PBS –N hello
#PBS –l nodes=1:ppn=12
#PBS –l walltime=1:00:00
#PBS –j oe
module load python/2.7.1
python hello.py
#end of sample.job
Example Python script,  hello.py:
import sys
hello = [“Hello”, “World”]
for i in hello:
print i
sys.exit(0)
#end of hello.py

Usage on Ruby

Set-up

To load the default version of Python module, use  module load python. To select a particular software version, use   module load python/version. For example, use  module load python/3.4.2 to load Python version 3.4.2. After the module is loaded, you can run the interpreter by using the command python. To unload the Python 3.4.2 module, use the command module unload python/3.4.2

Installed Modules

We have installed a number of Python packages and tuned them for optimal performance on our systems.  When using the Anaconda distributions of python you can run conda list to view the installed packages.

NOTE:
  • Due to architecture differences between our supercomputers, we recommend NOT installing your own packages in  ~/.local. Instead, you should install them in some other directory and set  $PYTHONPATH  in your default environment. For more information about installing your own Python modules, please see our HOWTO.

Batch Usage

When you log into ruby.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. 

Interactive Batch Session

For an interactive batch session one can run the following command:

qsub -I -l nodes=1:ppn=20 -l walltime=00:20:00 
which requests one whole node with 20 cores ( -l nodes=1:ppn=20), for a walltime of 20 minutes ( -l walltime=00:20:00). You may adjust the numbers per your need.

Usage on Owens

Set-up

To load the default version of Python module, use  module load python . To select a particular software version, use   module load python/version. For example, use  module load python/3.5 to load the latest version of Python 3.5. After the module is loaded, you can run the interpreter by using the command python. To unload the latest version of Python 3.5 module, use the command module unload python/3.5

Installed Modules

We have installed a number of Python packages and tuned them for optimal performance on our systems.  When using the Anaconda distributions of python you can run conda list to view the installed packages.

NOTE:
  • Due to architecture differences between our supercomputers, we recommend NOT installing your own packages in  ~/.local. Instead, you should install them in some other directory and set  $PYTHONPATH in your default environment. For more information about installing your own Python modules, please see our HOWTO.

Batch Usage

When you log into owens.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your job to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. 

Interactive Batch Session

For an interactive batch session one can run the following command:

qsub -I -l nodes=1:ppn=28 -l walltime=00:20:00 
which requests one whole node with 28 cores ( -l nodes=1:ppn=28), for a walltime of 20 minutes (-l walltime=00:20:00). You may adjust the numbers per your need.

Further reading

Extensive documentation of the Python programming language and software downloads can be found at the Official Python Website.  

See Also

HOWTO: Install your own python modules

Supercomputer: 
Service: 
Fields of Science: