OSC Custom Commands

There are some commands that OSC has created custom versions of to be more useful to OSC users.

 

OSCfinger

Introduction

OSCfinger is a command developed at OSC for use on OSC's systems and is similar to the standard finger command. It allows various account information to be viewed.

Availability

owens PITZER
X

X

 

Usage

OSCfinger takes the following options and parameters.

$ OSCfinger -h
usage: OSCfinger.py [-h] [-e] [-g] USER

positional arguments:
  USER

optional arguments:
  -h, --help   show this help message and exit
  -e           Extend search to include gecos/full name (user) or
               category/institution (group)
  -g, --group  Query group instead of users

Query user:
    OSCfinger foobar

Query by first or last name:
    OSCfinger -e Foo
    OSCfinger -e Bar

Query group:
    OSCfinger -g PZS0001

Query group by category or insitituion:
    OSCfinger -e -g OSC    

 

View information by username

The OSCfinger command can be used to view account information given a username.

$ OSCfinger jsmith
Login: xxx                                   Name: John Smith
Directory: xxx                               Shell: /bin/bash
E-mail: xxx
Primary Group: PPP1234
Groups: 

Project Information by Project ID

The OSCfinger command can also reveal details about a project using the -g flag.

$ OSCfinger -g PPP1234
Group: PPP1234                                    GID: 1234
Status: ACTIVE                                    Type: Academic
Principal Investigator: xxx                       Admins: NA
Members: xxx
Category: NA
Institution: OHIO SUPERCOMPUTER CENTER
Description: xxx
---

Search for a user via first and/or last name

If the username is not known, a lookup can be initiated using the -e flag.

This example is shown using the lookup for a first and last name.

$ OSCfinger -e "John Smith"
Login: jsmith                                     Name: John Smith
Directory: xxx                                    Shell: /bin/bash
E-mail: NA
Primary Group: PPP1234
Groups: xxx
Password Changed: Jul 04 1776 15:47 (calculated)  Password Expires: Aug 21 1778 12:05 AM
Login Disabled: FALSE                             Password Expired: FALSE
---

 

One can also lookup users with only the last name:

$ OSCfinger -e smith
Login: jsmith                                      Name: John Smith
Directory: xxx                                    Shell: /bin/bash
E-mail: NA
Primary Group: PPP1234
Groups:
---
Login: asmith                                     Name: Anne Smith
Directory: xxx                                    Shell: /bin/bash
E-mail: xxx
Primary Group: xxx
Groups: 
---

 

Only the first name can also be used, but many accounts are likely to be returned.

$ OSCfinger -e John
Login: jsmith                                     Name: John Smith
Directory: xxx                                    Shell: /bin/bash
E-mail: xxx
Primary Group: PPP1234
Groups:
---
Login: xxx                                        Name: John XXX
Directory: xxx                                    Shell: /bin/bash
E-mail: xxx
Primary Group: xxx
Groups:
---
Login: xxx                                        Name: John XXX
Directory: xxx                                    Shell: /bin/ksh
E-mail: xxx
Primary Group: xxx
Groups:
---
...(more accounts below)...

Slurm usage

While in a slurm environment, the OSCfinger command shows some additional information:

$ OSCfinger jsmith 
Login: xxx Name: John Smith 
Directory: xxx Shell: /bin/bash 
E-mail: xxx 
Primary Group: PPP1234 
Groups:
SLURM Enabled: TRUE
SLURM Clusters: pitzer
SLURM Accounts: PPP1234, PPP4321
SLURM Default Account: PPPP1234

It's important to note that the default account in slurm will be used if an account is not specified at job submission.

Supercomputer: 
Service: 

OSCgetent

Introduction

OSCgetent is a command developed at OSC for use on OSC's systems and is similar to the standard getent command. It lets one view group information.

Availability

owens PITZER
X

X

 

Usage

OSCgetent takes the following options and parameters.

$ OSCgetent -h
usage: OSCgetent.py [-h] {group} [name [name ...]]

positional arguments:
  {group}
  name

optional arguments:
  -h, --help  show this help message and exit

Query group:
    OSCgetent.py group PZS0708

Query multiple groups:
    OSCgetent.py group PZS0708 PZS0709
    

View group information

The OSCgetent command can be used to view group(s) members:

$ OSCgetent group PZS0712
PZS0712:*:5513:amarcum,amarcumtest,amarcumtest2,guilfoos,hhamblin,kcahill,xwang 

View information on multiple groups

$ OSCgetent group PZS0712 PZS0708
PZS0708:*:5509:djohnson,ewahl,kearley,kyriacou,linli,soottikkal,tdockendorf,troy
PZS0712:*:5513:amarcum,amarcumtest,amarcumtest2,guilfoos,hhamblin,kcahill,xwang

 

Supercomputer: 
Service: 

OSCprojects

Introduction

OSCprojects is a command developed at OSC for use on OSC's systems and is used to view your logged in accounts project information.

Availability

owens PITZER
X

X

 

Usage

OSCprojects does not take any arguments or options:

$ OSCprojects
OSC projects for user amarcumtest2:

Project         Status          Members
-------         ------          -------
PZS0712         ACTIVE          amarcumtest2,amarcumtest,guilfoos,amarcum,xwang
PZS0726         ACTIVE          amarcumtest2,xwangtest,amarcum

This command returns the current users projects, whether those projects are active/restricted and the current members of the projects.

Supercomputer: 
Service: 

OSCusage

Introduction

OSCusage is command developed at OSC for use on OSC's systems.  It allows for a user to see information on their project's usage, including different users and their jobs.

Availability

owens PITZER
X X

 

Usage

OSCusage takes the following options and parameters.

$ OSCusage --help
usage: OSCusage.py [-h] [-u USER]
                   [-s {opt,pitzer,glenn,bale,oak,oakley,owens,ruby}] [-A]
                   [-P PROJECT] [-q] [-H] [-r] [-n] [-v]
                   [start_date] [end_date]

positional arguments:
  start_date            start date (default: 2021-03-16)
  end_date              end date (default: 2021-03-17)

optional arguments:
  -h, --help            show this help message and exit
  -u USER, --user USER  username to run as. Be sure to include -P or -A.
                        (default: amarcum)
  -s {opt,pitzer,glenn,bale,oak,oakley,owens,ruby}, --system {opt,pitzer,glenn,bale,oak,oakley,owens,ruby}
  -A                    Show all
  -P PROJECT, --project PROJECT
                        project to query (default: PZS0712)
  -q                    show user data
  -H                    show hours
  -r                    show raw
  -n                    show job ID
  -v                    do not summarize

Usage Examples:

    Specify start time:
        OSCusage 2018-01-24

    Specify start and end time:
        OSCusage 2018-01-24 2018-01-25

Today's Usage

Running OSCusage with no options or parameters specified will provide the usage information in Dollars for the current day.

$ OSCusage
----------------  ------------------------------------
                  Usage Statistics for project PZS0712
Time              2021-03-16 to 2021-03-17
PI                guilfoos@osc.edu
Remaining Budget  -1.15
----------------  ------------------------------------

User          Jobs    Dollars    Status
------------  ------  ---------  --------
amarcum       0       0.0        ACTIVE
amarcumtest   0       0.0        ACTIVE
amarcumtest2  0       0.0        ACTIVE
guilfoos      0       0.0        ACTIVE
hhamblin      0       0.0        ACTIVE
kcahill       0       0.0        ACTIVE
wouma         0       0.0        ACTIVE
xwang         12      0.0        ACTIVE
--            --      --
TOTAL         12      0.0

Usage in Timeframe

If you specify a timeframe you can get utilization information specifically for jobs that completed within that period.

$ OSCusage 2020-01-01 2020-07-01 -H
----------------  ------------------------------------
                  Usage Statistics for project PZS0712
Time              2020-01-01 to 2020-07-01
PI                Brian Guilfoos <guilfoos@osc.edu>
Remaining Budget  -1.15
----------------  ------------------------------------

User          Jobs    core-hours    Status
------------  ------  ------------  ----------
amarcum       86      260.3887      ACTIVE
amarcumtest   0       0.0           ACTIVE
amarcumtest2  0       0.0           RESTRICTED
guilfoos      9       29.187        ACTIVE
hhamblin      1       1.01          ACTIVE
kcahill       7       40.5812       ACTIVE
wouma         63      841.2503      ACTIVE
xwang         253     8148.2638     ACTIVE
--            --      --
TOTAL         419     9320.681

Show only a single user's usage

Specify -q to show only the current user's usage. This stacks with -u to specify which user you want to see.

$ OSCusage -u xwang -q 2020-01-01 2020-07-01 -H
----  -------------------------------
      Usage Statistics for user xwang
Time  2020-01-01 to 2020-07-01
----  -------------------------------

User    Jobs    core-hours    Status
------  ------  ------------  --------
xwang   253     8148.2638     -
--      --      --
TOTAL   253     8148.2638

Show a particular project

By default, the tool shows your default (first) project. You can use -P to specify which charge code to report on.

$ OSCusage -P PZS0200 -H
----------------  ------------------------------------
                  Usage Statistics for project PZS0200
Time              2020-09-13 to 2020-09-14
PI                David Hudak <dhudak@osc.edu>
Remaining Budget  0
----------------  ------------------------------------

User        Jobs    core-hours    Status
----------  ------  ------------  ----------
adraghi     0       0.0           ARCHIVED
airani      0       0.0           ARCHIVED
alingg      0       0.0           ARCHIVED

 

You can show all of your charge codes/projects at once, by using -A .

Select a particular cluster

By default, all charges are shown in the output. However, you can filter to show a particular system with -s .

$ OSCusage -s pitzer -H
----------------  ------------------------------------
                  Usage Statistics for project PZS0712
Time              2021-03-16 to 2021-03-17
PI                guilfoos@osc.edu
Remaining Budget  -1.15
----------------  ------------------------------------

User          Jobs    core-hours    Status
------------  ------  ------------  --------
amarcum       0       0.0           ACTIVE
amarcumtest   0       0.0           ACTIVE
amarcumtest2  0       0.0           ACTIVE
guilfoos      0       0.0           ACTIVE
hhamblin      0       0.0           ACTIVE
kcahill       0       0.0           ACTIVE
wouma         0       0.0           ACTIVE
xwang         0       0.0           ACTIVE
--            --      --
TOTAL         0       0.0

Changing the units reported

The report can show usage dollars. You can elect to get usage in core-hours using -H or raw seconds using -r

$ OSCusage 2020-01-01 2020-07-01 -r
----------------  ------------------------------------
                  Usage Statistics for project PZS0712
Time              2020-01-01 to 2020-07-01
PI                Brian Guilfoos <guilfoos@osc.edu>
Remaining Budget  -1.15
----------------  ------------------------------------

User          Jobs    raw_used    Status
------------  ------  ----------  ----------
amarcum       86      937397.0    ACTIVE
amarcumtest   0       0.0         ACTIVE
amarcumtest2  0       0.0         RESTRICTED
guilfoos      9       105073.0    ACTIVE
hhamblin      1       3636.0      ACTIVE
kcahill       7       146092.0    ACTIVE
wouma         63      3028500.0   ACTIVE
xwang         253     29333749.0  ACTIVE
--            --      --
TOTAL         419     33554447.0


Detailed Charges Breakdown

Specify -v to get detailed information jobs.

You can add the -n option to the -v option to add the job ID to the report output. OSCHelp will need the job ID to answer any questions about a particular job record.

Please contact OSC Help with questions. 

Supercomputer: 
Service: 

checkgpu Command

Introduction

checkgpu is a command developed at OSC for use on OSC’s systems to report information regarding the usage of OSC’s GPU nodes. It reports various information on the usage and availability of the GPU nodes.

Availability

owens PITZER
X

X

 

Usage

checkgpu takes the following options and parameters (viewable by passing  -h/--help)

$ checkgpu -h
usage: checkgpu.py [-h] [-j] [-r] [-q] [-n] [-u] [-a] [-v]

optional arguments:
  -h, --help     show this help message and exit
  -j, --jobs     Check status of gpu jobs
  -r, --run      Check status of running gpu jobs (use with --jobs)
  -q, --queued   Check status of queued gpu jobs (use with --jobs)
  -n, --node     Check status of gpu nodes
  -u, --used     Check status of used gpu nodes (use with --node)
  -a, --avail    Check status of available gpu nodes (use with --node)
  -v, --verbose  Display full job report

View information on GPU jobs

By using the flag -j/--jobs, checkgpu can report summaries for the total number of jobs using GPU nodes

$ checkgpu -j
*** SUMMARY ***
===================================================
Summary of Running GPU Jobs:

152 GPU jobs running using 152 GPU nodes. Among them:
 * 146 GPU jobs running using 146 GPU nodes from non-condo and non-debugging jobs
 * 0 GPU jobs running using 0 GPU nodes from debugging jobs
 * 6 GPU Jobs running using 6 GPU nodes from PCONXXXX
===================================================
Summary of Queued GPU Jobs:

485 GPU Jobs queued requesting 492 GPU nodes. Among them:
 * 382 GPU Jobs queued requesting 389 GPU nodes from non-condo and non-debugging jobs
 * 0 GPU Jobs queued requesting 0 debug GPU nodes
 * 103 GPU Jobs queued requesting 103 GPU nodes from PCONXXXX
===================================================

View information on GPU jobs (run-only)

By using the flag -r/--run in tandem with -j, checkgpu can report summaries for the total number of running jobs using GPU nodes

$ checkgpu -j --run
===================================================
Summary of Running GPU Jobs:
152 GPU jobs running using 152 GPU nodes. Among them:
 * 146 GPU jobs running using 146 GPU nodes from non-condo and non-debugging jobs
 * 0 GPU jobs running using 0 GPU nodes from debugging jobs
 * 6 GPU Jobs running using 6 GPU nodes from PCONXXXX

View information on GPU jobs (queue-only)

By using the flag-q/--queued in tandem with -j, checkgpu can report summaries for the total number of queued jobs using GPU nodes

$ checkgpu -j --queued
===================================================
Summary of Queued GPU jobs:
484 GPU Jobs queued requesting 491 GPU nodes. Among them:
 * 382 GPU Jobs queued requesting 389 GPU nodes from non-condo and non-debugging jobs
 * 0 GPU Jobs queued requesting 0 debug GPU nodes
 * 102 GPU Jobs queued requesting 102 GPU nodes from PCONXXXX

View information on GPU nodes

By using the flag -n/--node, checkgpu can report summaries for the usage of OSC’s GPU nodes

$ checkgpu --node
***USED NODES***
================================================================
================================================================
Summary of the current GPU nodes:
153 total jobs on gpu nodes
 * 153 GPU jobs using 153 GPU nodes. Among them:
  ** 147 GPU nodes used by non-condo and non-debug jobs
  ** 0 GPU nodes used by debug jobs
  ** 6 GPU nodes used by condo group PCONXXXX
 * 0 non-GPU jobs using 0 GPU nodes. Among them:
  ** 0 GPU nodes used by non-condo and non-debug non-gpu jobs
  ** 0 GPU nodes used by debug non-gpu jobs
  ** 0 GPU nodes used by non-gpu condo jobs

***AVAILABLE NODES***
================================================================
================================================================
Summary of the available GPU nodes:
144 available gpu nodes. Among them: 
 * 4 fully available gpu nodes.
 * 140 partly available gpu nodes.
 * 0 open gpus on partly available nodes.

View information on GPU nodes (used-only)

By using the flag -u/--used in tandem with -n, checkgpu can report summaries for the usage of used GPU nodes

$ checkgpu -n -u
================================================================
Summary of the current GPU nodes:
152 total jobs on gpu nodes
 * 152 GPU jobs using 152 GPU nodes. Among them:
  ** 147 GPU nodes used by non-condo and non-debug jobs
  ** 0 GPU nodes used by debug jobs
  ** 5 GPU nodes used by condo group PCONXXXX
 * 0 non-GPU jobs using 0 GPU nodes. Among them:
  ** 0 GPU nodes used by non-condo and non-debug non-gpu jobs
  ** 0 GPU nodes used by debug non-gpu jobs
  ** 0 GPU nodes used by non-gpu condo jobs

View information on GPU nodes (available-only)

By using the flag -a/--avail in tandem with -n, checkgpu can report summaries for the availability of GPU nodes

$ chekgpu -n -a
================================================================
Summary of the available GPU nodes:
144 available gpu nodes. Among them: 
 * 4 fully available gpu nodes.
 * 140 partly available gpu nodes.
 * 0 open gpus on partly available nodes.

View full job summary

By using the flag -v/--verbose in tandem with any of the above command options, checkgpu will report full summaries on the usage of GPU nodes

$ checkgpu --jobs --run -v
============================================
Job Details:
List of non-condo and non-debug GPU jobs
XXXXXXX.owens-batch.ten.osc.edu
o0678-gpu/0
XXXXXXX.owens-batch.ten.osc.edu
o0673-gpu/0
XXXXXXX.owens-batch.ten.osc.edu
o0695-gpu/0
XXXXXXX.owens-batch.ten.osc.edu
o0805-gpu/0
XXXXXXX.owens-batch.ten.osc.edu
o0763-gpu/0
...

List of debug GPU jobs

List of GPU jobs from PCON0005
XXXXXXX.owens-batch.ten.osc.edu
o0732-gpu/0
XXXXXXX.owens-batch.ten.osc.edu
o0779-gpu/0
XXXXXXX.owens-batch.ten.osc.edu
o0745-gpu/0
XXXXXXX.owens-batch.ten.osc.edu
o0712-gpu/0
XXXXXXX.owens-batch.ten.osc.edu
o0663-gpu/0
XXXXXXX.owens-batch.ten.osc.edu
o0750-gpu/0

===================================================
Summary of Running GPU Jobs:
153 GPU jobs running using 153 GPU nodes. Among them:
 * 147 GPU jobs running using 147 GPU nodes from non-condo and non-debugging jobs
 * 0 GPU jobs running using 0 GPU nodes from debugging jobs
 * 6 GPU Jobs running using 6 GPU nodes from PCON0005
Supercomputer: 
Service: