OnDemand

OnDemand is our "one-stop shop" for access to our High Performance Computing resources. With OnDemand, you can upload and download files; create, edit, submit, and monitor jobs; run GUI applications; and connect via SSH, all via a web broswer, with no client software to install and configure.

We've created a brief video explaining OnDemand's capabilities so users can better gauge if it is the right fit for them. Getting connected to OnDemand is also covered in this video. In addition, we've developed tutorial videos for OnDemand's job client and file management client.

 

 

System Requirements

Currently, the site is confirmed to work with newer versions of Chrome (87+), Firefox (87+), and Internet Explorer (11+). We are still working to expand availability to additional clients and are planning on including compatibility for mobile devices (phones and tablets) in the future. 

Connecting

To connect to OnDemand, visit https://ondemand.osc.edu. The first page of the OnDemand site displays instructions on how to log in to OSC OnDemand, which are shown in the image below:

There are two options for logging in.

Option 1: Using OSC HPC Credentials

Follow the steps below to use OSC HPC credentials to directly log in.

ondemand-osc-login.png

Option 2: Using Third-Party Credentials

You can also map third-party credentials to an OSC account and log in using the third-party credentials.

ondemand-cilogin.png

Click the button labeled "Log in with third party though CILogon" at the bottom to continue to step 1 shown below:

  • Step 1 requires you to choose an identity provider to log in with. This can be any institute you're affiliated with or even services like Google.
    If you check "Remember this selection" you will need to delete cookies from the cilogon domain to make a new choice. You can do this through your browser's settings or at https://cilogon.org/me
  • Step 2 has you log in using your credentials for the provider you picked.
  • Step 3 has you log in with your OSC credentials in order to link your identity provider to your OSC account.
These three steps are only needed the first time you log in to OSC OnDemand.
Only one third-party credential can be mapped to an osc account at a time. An osc account can be disassociated from an external account by navigating to:

https://idp.osc.edu/realms/osc/account/identity

ondemand-unmap-CILogin2.png

and clicking Remove.

Once completed you will be presented with this page:

screenshot of OnDemand displaying the above text

Or this page if your browser window is more compact:

screenshot of OnDemand displaying the above text

The three-lined icon to the right of the compact page expands when clicked to show the full toolbar that the non-compact page has. At this point, you can begin accessing the tools in OnDemand.

File Transfer

Move data on and off of OSC storage services. To use the file transfer client, select "Files" in the main menu. For more information, please visit the file transfer and management page.

Job Management

Create, edit, submit, and monitor jobs. To access the job management tools, use the "Jobs" menu. For more information about the tools, please visit the job management page.

screenshot of OnDemand displaying the above text

Shell Access

OnDemand provides the ability to SSH to Pitzer or Owens from inside your web browser. To get to a terminal, select the "Clusters" menu and choose either Pitzer or Owens.

screenshot of OnDemand displaying the above text

Please note that this action will open a new tab with shell access to the cluster. 

System Status

System status can be accessed by clicking on the "Clusters" tab and selecting "System Status" on the drop-down menu. This page shows the current status of the entire system (nodes in use, cores in use, and number of running, queued, and blocked jobs). 

screenshot of OnDemand displaying the above text

Screen Shot 2021-03-25 at 12.06.02 PM.png

If you click the image, you'll get a detailed report including the total system load, the CPU usage, the total cluster memory use, and the total network traffic on the cluster. The "CPU Report" showing less than 100% use means that there are some cores not currently being used by a job, but that doesn't necessarily mean that they are available to be scheduled. There may be a system reservation that is preventing the scheduler from utilizing all of those cores.

Get a Virtual Desktop

Request a desktop to run GUI applications without the hassle or performance problems associated with X11 forwarding. To access one, please select either "Interactive HPC" or "Virtual Desktop Interface" from the "Desktops" menu.

Virtual Desktop Interface

Select one VDI app and you will arrive on the page shown below. Please modify the resolution information if needed and click "Launch." You may need to wait for a few minutes for the interface to launch.

The maximum walltime for VDI is 24 hours. 
Here it also provides access to Files ("Access Files") and Shell (">_Launch Shell") and allows you to go back to the OnDemand main page ("Go to Dashboard").

screenshot of OnDemand displaying the above text

 

Once the interface is ready to be used, your session manager will have a job that looks like the image below:

screenshot of OnDemand displaying the above text

 

To access the interface, you can choose from "noVNC Instructions" or "Native Client Instructions." If you use "noVNC," you can choose from "Launch noVNC in New Tab" to get your desktop shown below and work accordingly, or "View Only (Shareable Link)" to get your desktop or share your desktop with colleagues in "View Only" mode. If you use "Native Client Instructions", see this page for more information on how to use OSC Connect.  

screenshot of OnDemand displaying the above text

 

The Virtual Desktop Interfaces should not be used for computationally or memory intensive processes because it is a shared resource and there are other users on the same node. Such processes will be terminated. Please perform such work on compute node as discussed below.

Interactive HPC

Request a desktop on compute node through the batch system without the hassle or performance problems associated with X11 forwarding. To access one, please select "Pitzer Desktop" or others under "Interactive" from the "Desktops" menu.  

If you choose "Pitzer Desktop," you will arrive on the page shown below. Please enter the information as desired and click "Launch." You may need to wait for a few minutes for the interface to launch. If you'd like to recieve an email notifiying you of when the interface has launched, please select "I would like to recieve an email when the session starts."

screenshot of OnDemand displaying the above text

 

Once the interface is ready to be used, your session manager will have a job that looks like the image below:

screenshot of OnDemand displaying the above text

 

Similarly, you can choose from "noVNC Instructions" or "Native Client Instructions" to get the interface. If you use "noVNC," you can choose from "Launch noVNC in New Tab" to get your desktop and work accordingly, or "View Only (Shareable Link)" to get your desktop or share your desktop with colleagues in "View Only" mode. If you use "Native Client Instructions," see this page for more information on how to use OSC Connect. 

screenshot of OnDemand displaying the above text

Here you can run computationally or memory intensive processes since you do not share the node(s) with other clients.

Access Application on Compute Node

Request an application on compute node through the batch system without the hassle or performance problems associated with X11 forwarding. To access one, please select any app from the "Desktop Apps" menu. See "Desktop App Catalog" for more information. 

Support Tickets

The OnDemand dashboard now supports the ability to send a support ticket to our Help Desk system. 

To do so from your OnDemand Dashboard, simply go to > Help > Submit Support Ticket and fill in the following fields on the form that comes up:

Username: Logged in user. Username will be added to support ticket body for reference.

Email: Email address for communication regarding this ticket. Only a single email address is supported.

CC: Additional email address to copy onto this ticket. Only a single email address is supported.

Subject: Brief description of the problem.

Expected behavior: Detailed description of what was expected to happen.

Actual behavior: Detailed description of the unintended outcome.

Steps to reproduce: Detailed description of steps that led to the problem.

 

Service: 

Desktop App Catalog

OSC OnDemand provides access to applications on compute nodes through the batch system, without the hassle or performance problems associated with X11 forwarding. To access one, please select an application under "Interactive HPC" from the "Desktop Apps" menu. For more information on each product, please go to its page provided below.

Supercomputer: 
Service: 

Accessing Parallel R tutorial

This  document will guide you on how to launch Rstudio App and acess Parallel R workshop material through OSC onDemand.

Step 1: Log on to https://ondemand.osc.edu. Please see this guide on how to use OnDemand service.

Step 2: Launch Rstudio App

RStudio is an integrated development environment (IDE) for R. It includes a console, syntax-highlighting editor that supports direct code execution, as well as tools for plotting, history, debugging and workspace management.

Accessing through OnDemand

All the interactive apps can be found within the 'Interactive Apps' dropdown in our OnDemand web portal as shown in the image below:

Image of OnDemand Desktop Apps Dropdown

After selecting RStudio Server, you will arrive at this job submission page:

Image of how to setup RStudio

Here you can select the resources you would like your RStudio session to have using the menu on the lefthand side of the page. Please pick R/3.6.1 to access the tutorial material. Please provide your project ID as PZS0687 if you have registered for the workshop. If not, use your default project ID. Please contact oschelp@osc.edu if you dont have a project ID yet. Please make sure you check on the box that says Include access to OSC tutorial/workshop material. After you click "Launch", a new session will be queued. Once the session is active you page will look like the image below:

Image of launchable RStudio Session

Click on "Connect to RStudio Server" to begin using RStudio.

Using RStudio

Once you have entered your session, you should see the RStudio page below:

Image of an active RStudio session

This is a typical RStudio interface that should be familiar to most users. Please note that  Rstudio session for accessing the tutorial materials will start with a clean environment. While running in tutorial mode you will not have access to the files your $HOME.

  • Top Left panel: code editer
  • Bottom Left panel: The console where you can type commands and see the output.
  • Top right panel:
    • Environment tab: Shows all the active objects
    • History tab: Shows a list of commands used so far.
  • Bottom right panel:
    • Files tab: Shows all the files and and folders in your home directory
    • Plots tab: Shows all your graphs
    • Packages tab: Lists a series of packages or add-ons needed to run certain processes
    • Help tab: Can be used to find additional info
    • Viewer tab: Used to view local web content

More info on RStudio and its use can be found on the RStudio official support page.

From the bottom right panel, please click on parallelR.rmd markdown file. This will open the training material in the code editer window as shown below.

Image of an active RStudio session

This ParallelR.rmd material is prepared using the “rmarkdown” library. To view this in html format, select “Preview in Viewer Pane” option in the setting of “Knit” button top of the code editer window and then press “Knit”.

Image of an active RStudio session

This will open html version of the rmd document in the Veiwer pane of bottom right panel. 

Image of an active RStudio session

You can read though the html document in the Viewer Pane and execute codes in the Code editor. Press the Run/play button on the top right of each code block to run the code. 

Please reachout to soottikkal@osc.edu if you have questions.

 

Supercomputer: 
Service: 

OnDemand Desktop App: MATLAB

MATLAB allows matrix manipulations, plotting of functions and data, implementation of algorithms, creation of user interfaces, and interfacing with programs written in other languages, including C, C++, C#, Java, Fortran and Python.

Accessing through OnDemand

All the desktop apps can be found within the 'Interactive Apps' dropdown in our OnDemand web portal as shown in the image below:

access

 

After selecting MATLAB, you will arrive at this page:

interface

Here you can select the resources you would like your MATLAB session to have using the menu on the righthand side of the page. Additionally you can adjust the window size of the MATLAB session using the same menu. After you click "Launch", a new session will be queued. Once the session is active you page will look like the image below:

matlab

You can choose from "noVNC Connection", or "Native Instructions" to get the interface. If you use "noVNC", you can choose from "Launch noVNC in New Tab" to get your desktop and work accordingly, or "View Only (Share-able Link)" to get your desktop or share your desktop with colleagues in "View Only" mode. If you use "Native Instructions", see this page for more information on how to use OSC Connect.

Using MATLAB

Once you have entered your session, you should see a new tab. Allow MATLAB a moment to start up before your screen eventually looks like the image below:

workspace

This is a typical MATLAB interface that should be familiar to most users.

The desktop includes these panels:

  •     Current Folder — Access your files.
  •     Command Window — Enter commands at the command line, indicated by the prompt (>>).
  •     Workspace — Explore data that you create or import from files.

More info on Matlab and its use can be found here

Further Reading

MathWorks support page

 

Supercomputer: 
Service: 

OnDemand Desktop App: RStudio Server

RStudio is an integrated development environment (IDE) for R. It includes a console, syntax-highlighting editor that supports direct code execution, as well as tools for plotting, history, debugging and workspace management.

Accessing through OnDemand

All the desktop apps can be found within the 'Desktop Apps' dropdown in our OnDemand web portal as shown in the image below:

Image of OnDemand Desktop Apps Dropdown

After selecting RStudio Server, you will arrive at this page:

Image of how to setup RStudio

Here you can select the resources you would like your RStudio session to have using the menu on the lefthand side of the page. You can also access your files or launch the shell using the options in the center. After you click "Launch", a new session will be queued. Once the session is active you page will look like the image below:

Image of launchable RStudio Session

Click on "Connect to RStudio Server" to begin using RStudio.

Using RStudio

Once you have entered your session, you should see the RStudio page below:

Image of an active RStudio session

This is a typical RStudio interface that should be familiar to most users.

  • Left panel: The console where you can type commands and see the output.
  • Top right panel:
    • Environment tab: Shows all the active objects
    • History tab: Shows a list of commands used so far.
  • Bottom right panel:
    • Files tab: Shows all the files and and folders in your home directory
    • Plots tab: Shows all your graphs
    • Packages tab: Lists a series of packages or add-ons needed to run certain processes
    • Help tab: Can be used to find additional info
    • Viewer tab: Used to view local web content

More info on RStudio and its use can be found on the RStudio official support page.

 

Supercomputer: 
Service: 

File Transfer and Management

OnDemand provides a web-based File Explorer that can be used to upload and download files to your OSC home directory or project directory, and copy, delete, rename, and edit files.

Here is a tutorial video that gives a overview of OnDemand's file management client's capabilities and how to utilize them.

 

 

Here you can see a screen shot of what the main interface looks like.

File Main Page

 

Navigating

Selecting which File System

File Navigation

By default, the file browser will be looking at your home directory. If you have access to a project space for GPFS directory, you can switch to those by backtracking through the path near the top of the window and selecting the file system you wish to browse. In addition, you can use the "Go To" option in the top menu to type the path you wish to navigate to. Both the directory path and "Go To" button are highlighted in red in the image above.

Switching Folders

On the left side of the screen is a tree view of the directory structure. You can navigate this tree the same way you might on a normal desktop GUI to select the folder you wish to use.

Uploading and Downloading Files

File Upload

There are several methods to upload files. The first is to simply navigate to the desired destination, and then drag the file from your local desktop or window navigator to the OnDemand window. You can also click the "Upload" button to open a dialog that will allow you to navigate your local computer and select which files you want to upload. This process is highlighted in the image above.

To download files, you must select the file or files you wish to download, and then use the "Download" button. If you select multiple files ("control+click" or "shift+click") or a folder, the server will zip the files up and download a single zip archive to your desktop machine.

You can also click "Globus" to perform data transfer between OneDrive and other storage via Globus. See this Globus page for general information, and see this OneDrive in Globus on data transfer service with OneDrive. 

There is a size limit for upload/download of data using ondemand file explorer of 10GB.
It is recommended to use an sftp client, globus, or command line tool such as rsync for larger data transfers.
Contact oschelp@osc.edu for questions and guidance.

File Operations

Most file operations can be accomplished by selecting the file in the main window pane and selecting the desired operation from the main menu.

File Operations

Viewing and/or Editing a File

The application also includes several built-in file editors. To access these editors and viewers, select the file you wish to view or edit, and select "Edit" from the main menu. This will open up a text editor in a new tab.

 

Service: 

Job Management

OnDemand provides two related job managment tools: one allows you to create and submit jobs via your web browser, and the other allows you to monitor queued and running jobs.

Here is a tutorial video that gives an overview of OnDemand's Job client's capabilities and how to utilize them.

Jobs Menu

My Jobs

Selecting "My Jobs" in the Jobs menu will open an application that allows you to create new jobs and submit them to the cluster, and inspect the results of jobs submitted via this tool.

myjobs

New Job

Please following the steps in order to create a job:

  1. Create a new job by copying from an existing job template directory (by clicking "+New Job" button under "Listing Jobs") or a previously run job directory (by clicking "+Copy Job" button under "Listing Jobs").
    • If you click the "+New Job" button, you will see the page as below. Select the job template to copy, then click "Create New Job". The template directory will be copied and a new job row will be added to the top of the jobs table on the page titled "Listing Jobs"new job
  2. Select the job, then modify and/or add any files to this job by clicking "Edit Files" editfile
  3. Select the job and click "Job Options" to change the cluster, job name, or job script, if necessary editjob
  4. At this point, you can submit the job using the green "submit" button from the "My Jobs" page; or return to this job later to modify it further. You can also monitor the job status here. Currently you must reload the page to see job status changes.

Job Management

On the main screen for "My Jobs" you can click on a job to examine it.

Submit Job

If the job has not been submitted, the "submit job" button will be active, and will submit the selected job to the queue.

Stop Job

The "stop" button will allow you to kill a running job.

Delete Job

The "delete" button will allow you to delete a job

Active Jobs

The "Active Jobs" application will show you all of jobs currently in the queue (running or queued), regardless of how the jobs were submitted. You can click the button on top to switch from:

  • All jobs from your OSC account
  • All jobs from your project/group
  • All jobs from OSC users

You can also use "Search" tool to filter jobs based on sytems (oakley or ruby), status (running, queued, or hold), OSC user accout, etc.

activejobs

Job Inspection

If you click the arrow to the left of each job you can get more information about this job. It also provides the information of each node on which this job is running on, over the duration of the entire job. Clicking on each image expands it to show more information.

job status

Service: