Batch Limit Rules

Memory Limit:

It is strongly suggested to consider the memory use to the available per-core memory when users request OSC resources for their jobs.

Regular Dense Compute Node

On Owens, it equates to 4571MB/core or 124GB/node for the regular dense compute node. 

If your job requests less than a full node ( ppn< 28 ), it may be scheduled on a node with other running jobs. In this case, your job is entitled to a memory allocation proportional to the number of cores requested (4571MB/core).  For example, without any memory request ( mem=XX ), a job that requests  nodes=1:ppn=1  will be assigned one core and should use no more than 4571MB of RAM, a job that requests  nodes=1:ppn=3  will be assigned 3 cores and should use no more than 13713MB (~13GB) of RAM, and a job that requests  nodes=1:ppn=28  will be assigned the whole node (28 cores) with 124GB of RAM.  

Here is some information when you include memory request (mem=XX ) in your job. A job that requests  nodes=1:ppn=1,mem=12GB  will be assigned one core and have access to 12GB of RAM, and charged for 3 cores worth of Resource Units (RU).  However, a job that requests  nodes=1:ppn=5,mem=12GB  will be assigned 5 cores but have access to only 12GB of RAM, and charged for 5 cores worth of Resource Units (RU).  See Charging for memory use for more details

A multi-node job ( nodes>1 ) will be assigned the entire nodes with 124 GB/node and charged for the entire nodes regardless of ppn request. For example, a job that requests  nodes=10:ppn=1 will be charged for 10 whole nodes (28 cores/node*10 nodes, which is 280 cores worth of RU).  

Huge Memory Node

Beginning on Tuesday, March 10th, 2020, users are able to run jobs using less than a full huge memory node. It is strongly suggested to read the following instructions carefully before requesting a huge memory node on Owens. 

On Owens, it has 1,536GB/node (or 32 GB/core) for a huge memory node.

Please always specify a memory limit in your job request if your job requests less than a full node ( ppn< 48 ) ; otherwise, we will allocate 4571MB/core only for your job. For example, use nodes=1:ppn=30, mem=960GB  will be assigned 30 cores and 960 GB of RAM of a huge memory node. However, without any memory request ( mem=XX ), a job that requests  nodes=1:ppn=30  will be assigned 30 cores of a huge memory node and should use no more than 4571MB*30 of RAM (~133 GB of RAM); while a job that requests  nodes=1:ppn=1, mem=960GB  will be assigned 1 core of a huge memory node and should use no more than 960 GB of RAM. 

A job that requests huge-memory node ( nodes=1:ppn=48  ) will be allocated the entire huge-memory node with 1.5 TB of RAM and charged for the whole node (48 cores worth of RU).

See the table below for some comparisons:

Job request How many cores will be allocated How much memory will be allocated How the job will be charged
nodes=1:ppn=30 30 4571MB*30 (~133 GB) 30 cores worth of RU
nodes=1:ppn=1, mem=960GB 1 960 GB

30 cores worth of RU

(960GB/32GB)

nodes=1:ppn=30, mem=960GB 30 960 GB 30 cores worth of RU
nodes=1:ppn=48 48 (whole node) 1.536 GB (whole memory) 48 cores worth of RU

To manage and monitor your memory usage, please refer to Out-of-Memory (OOM) or Excessive Memory Usage.

Walltime Limit

Here are the queues available on Owens:

NAME

MAX WALLTIME

MAX JOB SIZE

NOTES

Serial

 168 hours

1 node

 

longserial 336 hours 1 node
  • Restricted access (contact OSC Help if you need access)

Parallel

96 hours

8 nodes

Jobs are scheduled to run within a single IB leaf switch

Largeparallel

96 hours

81 nodes

Jobs are scheduled across multiple switches

Hugemem

168 hours

1 node

16 nodes in this class
Parallel hugemem 96 hours 16 nodes
  • Restricted access (contact OSC Help if you need access)
  • Use "-q parhugemem" to access it

Debug

1 hour

2 nodes

  • 6 nodes in this class
  • Use "-q debug" to request it 

GPU Jobs

There is only one GPU per GPU node on Owens.

For serial jobs, we will allow node sharing on GPU nodes so a job may request any number of cores (up to 28)

(nodes=1:ppn=XX:gpus=1)

For parallel jobs (n>1), we will not allow node sharing.

Job/Core Limits

  Max Running Job Limit Soft Max Core/Processor Limit Hard Max Core/Processor Limt
Individual User 384 3080 3080
Project/Group 576 3080 4620

The soft and hard max limits above apply depending on different system resource availability. If resources are scarce, then the soft max limit is used to increase the fairness of allocating resources. Otherwise, if there are idle resources, then the hard max limit is used to increase system utilization.

An individual user can have up to the max concurrently running jobs and/or up to the max processors/cores in use.

However, among all the users in a particular group/project, they can have up to the max concurrently running jobs and/or up to the max processors/cores in use.

A user may have no more than 1000 jobs submitted to both the parallel and serial job queue separately.
Supercomputer: 
Service: