Update Tue Feb 10th 11:30am -- This issue is resolved.  

There is a bug in the changes we made to a part of our batch software during the downtime. The bug is affecting some users when they submit jobs to our system.

STAR-CCM+

STAR-CCM+ provides the world’s most comprehensive engineering physics simulation inside a single integrated package. Much more than a CFD code, STAR‑CCM+ provides an engineering process for solving problems involving flow (of fluids and solids), heat transfer and stress. STAR‑CCM+ is unrivalled in its ability to tackle problems involving multi‑physics and complex geometries.  Support is provided by CD-adapco. CD-adapco usually releases new version of STAR-CCM+ every four months.

Availability and Restrictions

STAR-CCM+ is available on Oakley and Glenn Clusters. The versions currently available at OSC are:

VERSION GLENN OAKLEY notes
6.04.016 X*    
6.06.017 X    
7.02.011 X    
7.04.011 X    
7.06.012   X Default version on Oakley prior to 09/15/2015
7.06.012r8   X  
8.02.011   X  
8.04.009 X X  
8.06.007   X  
9.04.011r8   X  
9.04.012   X  
10.04.009   X*  
10.06.010   X  
*: Current default version

You can use module avail starccm  to view available modules for a given machine. Feel free to contact OSC Help if you need other versions for your work.

Access for Academic Users

Use of STAR-CCM+ for academic purposes requires validation. In order to obtain validation, please contact OSC Help for further instruction. 

Currently, OSC has a 4 seat license (ccmpsuite, which allows up to 4 concurrent users), with 56 additional core licenses (hpcdomains) for academic users. 

Access for Commerical Users

Contact OSC Help for getting access to STAR-CCM+ if you are a commerical user.

Usage

Usage on Glenn

Set-up on Glenn

We recommend to run STAR-CCM+ on only the compute nodes. Thus, all STAR-CCM+ jobs should be submitted via the batch scheduling system, either as interactive or non-interactive batch jobs.  To load the default version of STAR-CCM+ module on Glenn, use  module load starccm . To select a particular software version, use   module load starccm-version . For example, use  module load starccm-8.04.009  to load STAR-CCM+ version 8.04.009 on Glenn. 

Batch Usage on Glenn

When you log into glenn.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your STAR-CCM+ analysis to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time, up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info. Batch jobs run on the compute nodes of the system and not on the login node. It is desirable for big problems since more resources can be used. STAR-CCM+ can be run on OSC clusters in either interactive mode or in non-interactive batch mode.

Interactive Batch Session

Interactive mode is similar to running STAR-CCM+ on a desktop machine in that the graphical user interface (GUI) will be sent from OSC and displayed on the local machine. To run interactive STAR-CCM+, it is suggested to request necessary compute resources from the login node, with X11 forwarding. The intention is that users can run STAR-CCM+ interactively for the purpose of building their model, preparing input file (.sim file), and checking results. Once developed this input file can then be run in no-interactive batch mode. For example, the following line requests one node with 8 cores ( -l nodes=1:ppn=8  ), for a walltime of one hour ( -l walltime=1:00:00 ), with one STAR-CCM+ base license token (   -l software=starccm+1  ) on Glenn:

qsub -I -X -l walltime=1:00:00 -l nodes=1:ppn=8 -l software=starccm+1

This job will queue until resources become available. Once the job is started, you're automatically logged in on the compute node; and you can launch STAR-CCM+ GUI with the following commands:

module load starccm
starccm+
Non-interactive Batch Job (Serial Run using 1 Base Token)

A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice.

Below is the example batch script (  job.txt ) for a serial run with an input file starccm.sim ) on Glenn:

#PBS -N star-ccm_test  
#PBS -l walltime=10:00:00  
#PBS -l nodes=1:ppn=1  
#PBS -l software=starccm+1
#PBS -j oe
#PBS -S /bin/bash
cd $TMPDIR  
cp $PBS_O_WORKDIR/starccm.sim .  
module load starccm  
starccm+ -batch starccm.sim >&output.txt  
cp output.txt $PBS_O_WORKDIR

To run this job on OSC batch system, the above script is to be submitted with the command:

qsub job.txt
Non-interactive Batch Job (Parallel Run using HPC tokens)

To take advantage of the powerful compute resources at OSC, you may choose to run distributed STAR-CCM+ for large problems. Multiple nodes and cores can be requested to accelerate the solution time. The following shows an example script if you need 2 nodes with 8 cores per node on Glenn using the inputfile named  starccm.sim  :

#PBS -N starccm_test 
#PBS -l walltime=3:00:00 
#PBS -l nodes=2:ppn=8
#PBS -W x=GRES:starccm+1%starccmpar+16
#PBS -j oe
#PBS -S /bin/bash
cd $PBS_0_WORKDIR
cp starccm.sim $TMPDIR
cd $TMPDIR
module load starccm
starccm+ -np 16 -batch -machinefile $PBS_NODEFILE starccm.sim >&output.txt
cp output.txt $PBS_0_WORKDIR

In addition to requesting the STAR-CCM+ base license token (  -l software=starccm+1  ), you need to request copies of the starccmpar license, i.e., HPC tokens. However the scheduler cannot handle two "software" flags simultaneously, so the syntax changes. The new option is   -W x=GRES:starccm+1%starccmpar+[n]  , where [n] is equal to the number of cores.

Usage on Oakley

Set-up on Oakley

We recommend to run STAR-CCM+ on only the compute nodes. Thus, all STAR-CCM+ jobs should be submitted via the batch scheduling system, either as interactive or non-interactive batch jobs. To load the default version of STAR-CCM+ module on Oakley, use  module load starccm . To select a particular software version, use   module load starccm/version . For example, use  module load starccm/9.04.012  to load STAR-CCM+ version 9.04.012 on Oakley.

Batch Usage on Oakley

When you log into oakley.osc.edu you are actually logged into a linux box referred to as the login node. To gain access to the mutiple processors in the computing environment, you must submit your STAR-CCM+ analysis to the batch system for execution. Batch jobs can request mutiple nodes/cores and compute time up to the limits of the OSC systems. Refer to Queues and Reservations and Batch Limit Rules for more info.  Batch jobs run on the compute nodes of the system and not on the login node. It is desirable for big problems since more resources can be used. STAR-CCM+ can be run on OSC clusters in either interactive mode or in non-interactive batch mode.

Interactive Batch Session

Interactive mode is similar to running STAR-CCM+ on a desktop machine in that the graphical user interface (GUI) will be sent from OSC and displayed on the local machine. To run interactive STAR-CCM+, it is suggested to request necessary compute resources from the login node, with X11 forwarding. The intention is that users can run STAR-CCM+ interactively for the purpose of building their model, preparing input file (.sim file), and checking results. Once developed this input file can then be run in no-interactive batch mode. For example, the following line requests one node with 12 cores( -l nodes=1:ppn=12 ), for a walltime of one hour ( -l walltime=1:00:00 ), with one STAR-CCM+ base license token (    -l software=starccm+1   ) on Oakley:

qsub -I -X -l walltime=1:00:00 -l nodes=1:ppn=12 -l software=starccm+1

This job will queue until resources become available. Once the job is started, you're automatically logged in on the compute node; and you can launch STAR-CCM+ GUI with the following commands:

module load starccm
starccm+
Non-interactive Batch Job (Serial Run using 1 Base Token)

A batch script can be created and submitted for a serial or parallel run. You can create the batch script using any text editor you like in a working directory on the system of your choice.

Below is the example batch script (  job.txt ) for a serial run with an input file starccm.sim ) on Oakley:

#PBS -N star-ccm_test  
#PBS -l walltime=10:00:00  
#PBS -l nodes=1:ppn=1  
#PBS -l software=starccm+1
#PBS -j oe
#PBS -S /bin/bash
cd $TMPDIR  
cp $PBS_O_WORKDIR/starccm.sim .  
module load starccm  
starccm+ -batch starccm.sim >&output.txt  
cp output.txt $PBS_O_WORKDIR

To run this job on OSC batch system, the above script is to be submitted with the command:

qsub job.txt
Non-interactive Batch Job (Parallel Run using HPC Tokens)

To take advantage of the powerful compute resources at OSC, you may choose to run distributed STAR-CCM+ for large problems. Multiple nodes and cores can be requested to accelerate the solution time. The following shows an example script if you need 2 nodes with 12 cores per node on Oakley using the inputfile named  starccm.sim  :

#PBS -N starccm_test 
#PBS -l walltime=3:00:00 
#PBS -l nodes=2:ppn=12
#PBS -W x=GRES:starccm+1%starccmpar+24
#PBS -j oe
#PBS -S /bin/bash
cd $PBS_0_WORKDIR
cp starccm.sim $TMPDIR
cd $TMPDIR
module load starccm
starccm+ -np 24 -batch -machinefile $PBS_NODEFILE starccm.sim >&output.txt
cp output.txt $PBS_0_WORKDIR

In addition to requesting the STAR-CCM+ base license token (  -l software=starccm+1  ), you need to request copies of the starccmpar license, i.e., HPC tokens. However the scheduler cannot handle two "software" flags simultaneously, so the syntax changes. The new option is   -W x=GRES:starccm+1%starccmpar+[n]  , where [n] is equal to the number of cores.

Further Reading

See Also

Supercomputer: 
Service: