It is strongly suggested to consider the available per-core memory when users request OSC resources for their jobs.
On Owens, it equates to 4315MB/core or 118GB/node for the regular dense compute node.
If your job requests less than a full node ( ppn< 28 ), it may be scheduled on a node with other running jobs. In this case, your job is entitled to a memory allocation proportional to the number of cores requested (4315MB/core). For example, without any memory request ( mem=XX ), a job that requests nodes=1:ppn=1 will be assigned one core and should use no more than 4315MB of RAM, a job that requests nodes=1:ppn=3 will be assigned 3 cores and should use no more than 3*4315MB of RAM, and a job that requests nodes=1:ppn=28 will be assigned the whole node (28 cores) with 118GB of RAM.
Here is some information when you include memory request (mem=XX ) in your job. A job that requests nodes=1:ppn=1,mem=12GB will be assigned one core and have access to 12GB of RAM, and charged for 3 cores worth of usage. However, a job that requests nodes=1:ppn=5,mem=12GB will be assigned 5 cores but have access to only 12GB of RAM, and charged for 5 cores worth of usage.
A multi-node job ( nodes>1 ) will be assigned the entire nodes with 118 GB/node and charged for the entire nodes regardless of ppn request. For example, a job that requests nodes=10:ppn=1 will be charged for 10 whole nodes (28 cores/node*10 nodes, which is 280 cores worth of usage).
On Owens, it has 1,493GB/node with 48 cores for a huge memory node.
Please always specify a memory limit in your job request if your job requests less than a full node ( ppn< 48 ) ; otherwise, we will allocate 4315MB/core only for your job. For example, use nodes=1:ppn=30, mem=960GB will be assigned 30 cores and 960 GB of RAM of a huge memory node. However, without any memory request ( mem=XX ), a job that requests nodes=1:ppn=30 will be assigned 30 cores of a huge memory node and should use no more than 4315MB*30 of RAM; while a job that requests nodes=1:ppn=1, mem=960GB will be assigned 1 core of a huge memory node and should use no more than 960 GB of RAM.
A job that requests huge-memory node ( nodes=1:ppn=48 ) will be allocated the entire huge-memory node with 1493GB of RAM and charged for the whole node (48 cores worth of usage).
To manage and monitor your memory usage, please refer to Out-of-Memory (OOM) or Excessive Memory Usage.
Here are the queues available on Owens:
|
NAME |
MAX WALLTIME |
MAX JOB SIZE |
NOTES |
|---|---|---|---|
|
Serial |
168 hours |
1 node |
|
| longserial | 336 hours | 1 node |
|
|
Parallel |
96 hours |
81 nodes |
|
|
Hugemem |
168 hours |
1 node |
16 nodes in this class |
| Parallel hugemem | 96 hours | 16 nodes |
|
|
Debug |
1 hour |
2 nodes |
|
There is only one GPU per GPU node on Owens.
For serial jobs, we allow node sharing on GPU nodes so a job may request any number of cores (up to 28)
(nodes=1:ppn=XX:gpus=1)
For parallel jobs (n>1), we do not allow node sharing.
| Max Running Job Limit | Soft Max Core/Processor Limit | Hard Max Core/Processor Limt | |
|---|---|---|---|
| Individual User | 384 | 3080 | 3080 |
| Project/Group | 576 | 3080 | 4620 |
The soft and hard max limits above apply depending on different system resource availability. If resources are scarce, then the soft max limit is used to increase the fairness of allocating resources. Otherwise, if there are idle resources, then the hard max limit is used to increase system utilization.
An individual user can have up to the max concurrently running jobs and/or up to the max processors/cores in use.
However, among all the users in a particular group/project, they can have up to the max concurrently running jobs and/or up to the max processors/cores in use.