There are some commands that OSC has created custom versions of to be more useful to OSC users.
OSCfinger is a command developed at OSC for use on OSC's systems and is similar to the standard finger command. It allows various account information to be viewed.
| CARDINAL | PITZER | ASCEND |
|---|---|---|
|
X |
X |
X |
OSCfinger takes the following options and parameters.
$ OSCfinger -h
usage: OSCfinger.py [-h] [-e] [-g] USER
positional arguments:
USER
optional arguments:
-h, --help show this help message and exit
-e Extend search to include gecos/full name (user) or
category/institution (group)
-g, --group Query group instead of users
Query user:
OSCfinger foobar
Query by first or last name:
OSCfinger -e Foo
OSCfinger -e Bar
Query group:
OSCfinger -g PZS0001
Query group by category or insitituion:
OSCfinger -e -g OSC
The OSCfinger command can be used to view account information given a username.
$ OSCfinger jsmith Login: xxx Name: John Smith Directory: xxx Shell: /bin/bash E-mail: xxx Primary Group: PPP1234 Groups:
The OSCfinger command can also reveal details about a project using the -g flag.
$ OSCfinger -g PPP1234 Group: PPP1234 GID: 1234 Status: ACTIVE Type: Academic Principal Investigator: xxx Admins: NA Members: xxx Category: NA Institution: OHIO SUPERCOMPUTER CENTER Description: xxx ---
If the username is not known, a lookup can be initiated using the -e flag.
This example is shown using the lookup for a first and last name.
$ OSCfinger -e "John Smith" Login: jsmith Name: John Smith Directory: xxx Shell: /bin/bash E-mail: NA Primary Group: PPP1234 Groups: xxx Password Changed: Jul 04 1776 15:47 (calculated) Password Expires: Aug 21 1778 12:05 AM Login Disabled: FALSE Password Expired: FALSE ---
One can also lookup users with only the last name:
$ OSCfinger -e smith Login: jsmith Name: John Smith Directory: xxx Shell: /bin/bash E-mail: NA Primary Group: PPP1234 Groups: --- Login: asmith Name: Anne Smith Directory: xxx Shell: /bin/bash E-mail: xxx Primary Group: xxx Groups: ---
Only the first name can also be used, but many accounts are likely to be returned.
$ OSCfinger -e John Login: jsmith Name: John Smith Directory: xxx Shell: /bin/bash E-mail: xxx Primary Group: PPP1234 Groups: --- Login: xxx Name: John XXX Directory: xxx Shell: /bin/bash E-mail: xxx Primary Group: xxx Groups: --- Login: xxx Name: John XXX Directory: xxx Shell: /bin/ksh E-mail: xxx Primary Group: xxx Groups: --- ...(more accounts below)...
OSCgetent is a command developed at OSC for use on OSC's systems and is similar to the standard getent command. It lets one view group information.
| CARDINAL | ASCEND | PITZER |
|---|---|---|
|
X |
X |
X |
OSCgetent takes the following options and parameters.
$ OSCgetent -h
usage: OSCgetent.py [-h] {group} [name [name ...]]
positional arguments:
{group}
name
optional arguments:
-h, --help show this help message and exit
Query group:
OSCgetent.py group PZS0708
Query multiple groups:
OSCgetent.py group PZS0708 PZS0709
The OSCgetent command can be used to view group(s) members:
$ OSCgetent group PZS0712 PZS0712:*:5513:amarcum,amarcumtest,amarcumtest2,guilfoos,hhamblin,kcahill,xwang
$ OSCgetent group PZS0712 PZS0708 PZS0708:*:5509:djohnson,ewahl,kearley,kyriacou,linli,soottikkal,tdockendorf,troy PZS0712:*:5513:amarcum,amarcumtest,amarcumtest2,guilfoos,hhamblin,kcahill,xwang
OSCprojects is a command developed at OSC for use on OSC's systems and is used to view your logged in accounts project information.
| CARDINAL | ASCEND | PITZER |
|---|---|---|
|
X |
X |
X |
OSCprojects does not take any arguments or options:
$ OSCprojects OSC projects for user amarcumtest2: Project Status Members ------- ------ ------- PZS0712 ACTIVE amarcumtest2,amarcumtest,guilfoos,amarcum,xwang PZS0726 ACTIVE amarcumtest2,xwangtest,amarcum
This command returns the current users projects, whether those projects are active/restricted and the current members of the projects.
OSCusage is a command developed at OSC for use on OSC's systems. It allows a user to see information on their project's usage, including different users and their jobs.
| Pitzer | Cardinal | Ascend |
|---|---|---|
| X | X | X |
OSCusage takes the following options and parameters.
$ OSCusage --help
usage: OSCusage.py [-h] [-u USER]
[-s {opt,pitzer,glenn,bale,oak,oakley,owens,ruby}] [-A]
[-P PROJECT] [-q] [-H] [-r] [-n] [-v]
[start_date] [end_date]
positional arguments:
start_date start date (default: 2021-03-16)
end_date end date (default: 2021-03-17)
optional arguments:
-h, --help show this help message and exit
-u USER, --user USER username to run as. Be sure to include -P or -A.
(default: amarcum)
-s {opt,pitzer,glenn,bale,oak,oakley,owens,ruby}, --system {opt,pitzer,glenn,bale,oak,oakley,owens,ruby}
-A Show all
-P PROJECT, --project PROJECT
project to query (default: PZS0712)
-q show user data
-H show hours
-r show raw
-n show job ID
-v do not summarize
-J, --json Print data as JSON
-C, --current-unbilled show current unbilled usage
-p {month,quarter,annual}, --period {month,quarter,annual} Period used when showing unbilled usage (default: month)
-N JOB_NAME, --job-name JOB_NAME
Filter jobs by job name, supports substring match and
regex (does not apply to JSON output)
Usage Examples:
Specify start time:
OSCusage 2018-01-24
Specify start and end time:
OSCusage 2018-01-24 2018-01-25
View current unbilled usage:
OSCusage -C -p month
Running OSCusage with no options or parameters specified will provide the usage information in Dollars for the current day.
$ OSCusage
---------------- ------------------------------------
Usage Statistics for project PZS0712
Time 2021-03-16 to 2021-03-17
PI guilfoos@osc.edu
Remaining Budget -1.15
---------------- ------------------------------------
User Jobs Dollars Status
------------ ------ --------- --------
amarcum 0 0.0 ACTIVE
amarcumtest 0 0.0 ACTIVE
amarcumtest2 0 0.0 ACTIVE
guilfoos 0 0.0 ACTIVE
hhamblin 0 0.0 ACTIVE
kcahill 0 0.0 ACTIVE
wouma 0 0.0 ACTIVE
xwang 12 0.0 ACTIVE
-- -- --
TOTAL 12 0.0
If you specify a timeframe you can get utilization information specifically for jobs that completed within that period.
$ OSCusage 2020-01-01 2020-07-01 -H
---------------- ------------------------------------
Usage Statistics for project PZS0712
Time 2020-01-01 to 2020-07-01
PI Brian Guilfoos <guilfoos@osc.edu>
Remaining Budget -1.15
---------------- ------------------------------------
User Jobs core-hours Status
------------ ------ ------------ ----------
amarcum 86 260.3887 ACTIVE
amarcumtest 0 0.0 ACTIVE
amarcumtest2 0 0.0 RESTRICTED
guilfoos 9 29.187 ACTIVE
hhamblin 1 1.01 ACTIVE
kcahill 7 40.5812 ACTIVE
wouma 63 841.2503 ACTIVE
xwang 253 8148.2638 ACTIVE
-- -- --
TOTAL 419 9320.681
Specify -q to show only the current user's usage. This stacks with -u to specify which user you want to see.
$ OSCusage -u xwang -q 2020-01-01 2020-07-01 -H
---- -------------------------------
Usage Statistics for user xwang
Time 2020-01-01 to 2020-07-01
---- -------------------------------
User Jobs core-hours Status
------ ------ ------------ --------
xwang 253 8148.2638 -
-- -- --
TOTAL 253 8148.2638
By default, the tool shows your default (first) project. You can use -P to specify which charge code to report on.
$ OSCusage -P PZS0200 -H
---------------- ------------------------------------
Usage Statistics for project PZS0200
Time 2020-09-13 to 2020-09-14
PI David Hudak <dhudak@osc.edu>
Remaining Budget 0
---------------- ------------------------------------
User Jobs core-hours Status
---------- ------ ------------ ----------
adraghi 0 0.0 ARCHIVED
airani 0 0.0 ARCHIVED
alingg 0 0.0 ARCHIVED
You can show all of your charge codes/projects at once, by using -A .
By default, all charges are shown in the output. However, you can filter to show a particular system with -s .
$ OSCusage -s pitzer -H
---------------- ------------------------------------
Usage Statistics for project PZS0712
Time 2021-03-16 to 2021-03-17
PI guilfoos@osc.edu
Remaining Budget -1.15
---------------- ------------------------------------
User Jobs core-hours Status
------------ ------ ------------ --------
amarcum 0 0.0 ACTIVE
amarcumtest 0 0.0 ACTIVE
amarcumtest2 0 0.0 ACTIVE
guilfoos 0 0.0 ACTIVE
hhamblin 0 0.0 ACTIVE
kcahill 0 0.0 ACTIVE
wouma 0 0.0 ACTIVE
xwang 0 0.0 ACTIVE
-- -- --
TOTAL 0 0.0
The report can show usage dollars. You can elect to get usage in core-hours using -H or raw seconds using -r
$ OSCusage 2020-01-01 2020-07-01 -r
---------------- ------------------------------------
Usage Statistics for project PZS0712
Time 2020-01-01 to 2020-07-01
PI Brian Guilfoos <guilfoos@osc.edu>
Remaining Budget -1.15
---------------- ------------------------------------
User Jobs raw_used Status
------------ ------ ---------- ----------
amarcum 86 937397.0 ACTIVE
amarcumtest 0 0.0 ACTIVE
amarcumtest2 0 0.0 RESTRICTED
guilfoos 9 105073.0 ACTIVE
hhamblin 1 3636.0 ACTIVE
kcahill 7 146092.0 ACTIVE
wouma 63 3028500.0 ACTIVE
xwang 253 29333749.0 ACTIVE
-- -- --
TOTAL 419 33554447.0
Detailed Charges Breakdown
Specify -v to get detailed information jobs.
You can add the -n option to the -v option to add the job ID to the report output. OSCHelp will need the job ID to answer any questions about a particular job record.
Please contact OSC Help with questions.
gpu-seff is a command developed at OSC for use on OSC's systems and is similar providing GPU resource data, similar to the CPU resource data reported by the seff command.
| CARDINAL | PITZER | ASCEND |
|---|---|---|
|
X |
X |
X |
gpu-seff takes the following options and parameters.
$ gpu-seff -h
usage: gpu-seff [-h] [-M {pitzer,ascend,cardinal}] [-d] [-t TIMEBUFFER] [-v] [-j] [-mu] [-me] [-gu] [-ge] [-ji] [-gi] jobid
positional arguments:
jobid Jobid
optional arguments:
-h, --help show this help message and exit
-M {pitzer,ascend,cardinal}, -c {pitzer,ascend,cardinal}, --cluster {pitzer,ascend,cardinal}
Cluster
-d, --debug Debug
-t TIMEBUFFER, --timebuffer TIMEBUFFER
Time buffer (seconds, default 60)
-v, --verbose Detailed per-gpu report
-j, --json Output in json format
optional output options:
By default these options are enabled. If any flags are passed in, only those flags will be used.
-mu, --memory-util Include memory utilization data
-me, --memory-efficiency
Include memory efficiency data
-gu, --gpu-util Include GPU utilization data
-ge, --gpu-efficiency
Include GPU utilization data
-ji, --job-info Include basic information about the job
-gi, --gpu-info GPU-related information about the job
By default, the gpu-seff command will give an overview of the job resource usage, including total memory utilization, average memory efficiency, total utilization, and total efficiency across all allocated GPUs.
$ gpu-seff 100 Job ID: 100 Cluster: cardinal User/Group: xxx/PPP1234 State: COMPLETED Nodes: 1 Job Wall-clock time: 00:03:40 GPUs per node: 2 Total GPUs: 2 GPU Memory Utilized: 77.17 GB GPU Memory Efficiency: 41.49% of 186.01 GB GPU Utilization: 00:01:14 GPU Efficiency: 17.03% of 00:07:20 gpu-walltime
xxx@ascend-login01:~$ gpu-seff -M cardinal 100 Job ID: 100 Cluster: cardinal User/Group: xxx/PPP1234 State: COMPLETED Nodes: 1 Job Wall-clock time: 00:03:40 GPUs per node: 2 Total GPUs: 2 GPU Memory Utilized: 77.17 GB GPU Memory Efficiency: 41.49% of 186.01 GB GPU Utilization: 00:01:14 GPU Efficiency: 17.03% of 00:07:20 gpu-walltime
To get per-GPU statistics, rather than summary statistics across all gpus, pass the verbose flag
$ gpu-seff 100 --verbose Job ID: 100 Cluster: cardinal User/Group: xxx/PPP1234 State: COMPLETED Nodes: 1 Job Wall-clock time: 00:03:40 GPUs per node: 2 Total GPUs: 2 GPU Memory Utilized: Host c0818 GPU #0: 32.08 GB Host c0818 GPU #1: 45.09 GB GPU Memory Efficiency: Host c0818 GPU #0: 34.50% of 93.00 GB Host c0818 GPU #1: 48.48% of 93.00 GB GPU Utilization: Host c0818 GPU #0: 00:00:35 Host c0818 GPU #1: 00:00:39 GPU Efficiency: Host c0818 GPU #0: 15.96% of 00:03:40 gpu-walltime Host c0818 GPU #1: 18.11% of 00:03:40 gpu-walltime
To display the resourece information in an easily parsible json format, pass the --json flag.
$ gpu-seff 100 --json
{
"jobid": "100",
"cluster": "cardinal",
"user": "xxx",
"group": "PP1234",
"nodes": 1,
"walltime": "00:03:40",
"gpu_per_node": 2.0,
"total_gpus": 2,
"gputime": 440,
"mem_util": "77.17 GB",
"mem_eff": 41.48684832257049,
"gpu_util": 74,
"gpu_eff": 17.034495
}
By default, gpu-seff will display all of memory utilization, memory efficiency, gpu utilization and gpu efficiency, basic job information, and GPU resource details. If any of the optional output options are specified, then only those specified will be shown.
To only display, GPU details and exclude basic job information:
$ gpu-seff 100 -gi -mu -me -gu -ge GPUs per node: 2 Total GPUs: 2 GPU Memory Utilized: 77.17 GB GPU Memory Efficiency: 41.49% of 186.01 GB GPU Utilization: 00:01:14 GPU Efficiency: 17.03% of 00:07:20 gpu-walltime
osc-seff is a command developed at OSC for use on OSC's systems and provides a the CPU resource data of the seff command with the GPU resource data of gpu-seff.
| CARDINAL | PITZER | ASCEND |
|---|---|---|
|
X |
X |
X |
osc-seff takes the SLURM job ID has as it's only input.
$ osc-seff -h Usage: /opt/gpu-metrics/osc-seff <jobid>
By default, the gpu-seff command will give an overview of the job resource usage, including total memory utilization, average memory efficiency, total utilization, and total efficiency across all allocated GPUs.
$ osc-seff 100 Job ID: 100 Cluster: cardinal User/Group: xxx/PPP1234 State: COMPLETED (exit code 0) Nodes: 1 Cores per node: 32 CPU Utilized: 00:04:06 CPU Efficiency: 3.49% of 01:57:20 core-walltime Job Wall-clock time: 00:03:40 Memory Utilized: 34.37 GB Memory Efficiency: 11.93% of 288.00 GB GPUs per node: 2 Total GPUs: 2 GPU Memory Utilized: 77.17 GB GPU Memory Efficiency: 41.49% of 186.01 GB GPU Utilization: 00:01:14 GPU Efficiency: 17.03% of 00:07:20 gpu-walltime
To get more detailed GPU resource information, use the gpu-seff command.