Using Jupyter for Classroom

OSC provide an isolated and custom Jupyter environment for each classroom project that requires Jupyter Notebook or JupyterLab. 

The instructor must apply for a classroom project that is unique for the course. More details on the classroom project can be found in our classroom project guide. Once we get the information, we will provide you a project ID and a course ID (which is commonly the course ID provided by instructor + school code, e.g. MATH_2530_OU).  The instructor can set up a Jupyter environment for the course using the information (see below). The Jupyter environment will be tied to the project ID.

Set up a Jupyter environment

The instructor can set up a Jupyter environment for the course once the project space is initialized:

1. Login to cluster as PI account of the classroom project.
2. Run setup script with the project ID and course ID:

~support/classroom/tools/setup_jupyter_classroom  /fs/ess/project_ID  course_ID

If the Jupyter environment is created successfully, please inform us so we can update you when your class is ready at class.osc.edu.

Upgrade a Jupyter environment

You may need to upgrade Jupyter kernels to the latest stable version for a security vulnerability or trying out new features. Please run upgrade script with the project ID and course ID:

~support/classroom/tools/upgrade_jupyter_classroom /fs/ess/project_ID course_ID

Manage the Jupyter environment

Install packages

When your class is ready, launch your class session at class.osc.edu. Then, open a notebook and use the following command to install packages:

pip install --no-cache-dir --ignore-installed [package-name]

Please note that using the --no-cache-dir and --ignore-installed flags can skip using the caches in the home directory, which may cause conflicts when installing classroom packages if you have previously used pip to install packages in multiple Python environments.

Pip install will install packages for all classroom participants - participants are disallowed from doing their own installs.  As a result of this, commands they run that attempt to download data to a central python installation location will fail.  Either install as classroom owner, or have participants set the download directory to a location they can write to. E.g.,

import os
nltk.download('stopwords', download_dir=os.environ['NOTEBOOK_ROOT'])

While pip install is recommended, OSC also supports the use of conda environments.  See How to use a conda virtual environment with jupyter for more details.

Install extensions

Jupyter Notebook

To enable or install nbextension, please use --sys-prefix to install into the classroom Jupyter environment, e.g.

!jupyter contrib nbextension install --sys-prefix

Please do not use --user, which install to your home directory and could mess up the Jupyter environment.

JupyterLab

To install labextension, simply click Extension Manager icon at the side bar

Enable local package access (optional)

By default this Jupyter environment is an isolated Python environment. Anyone launches python from this environment can only access packages installed inside unless PYTHONPATH is used. The instructor can change it by setting include-system-site-packages = true in /fs/ess/project_ID/course_ID/jupyter/pyvenv.cfg. This will allows students to access packages in home directory ~/.local/lib/pythonX.X/site-packages ,and install packages via pip install –user

Workspace

When a class session starts, we create a classroom workspace under the instructor's and students' home space: $HOME/osc_classes/course_ID, and launch Jupyter at the workspace. The root /  will appear in the landing page (Files) but everything can be found in $HOME/osc_classes/course_ID on OSC system.

Shared Access

Share class material

The instructor can upload class material to /fs/ess/project_ID/course_ID/materials . When a student launch a Jupyter session, the diretory will be copied to the student's worksapce $HOME/osc_classes/course_ID . The student will see the directory materials on the landing page. PI can add files to the material source directory. New files will be copied to the destination every time when a new Jupyter session starts. But If PI modifies existing files, the changes won't be copied as the files were copied before. Therefore we recommend renaming the file after the update so that it will be copied

Use data dir for large files

For large files, create a data dir to the classroom and place the large files there.

mkdir /fs/ess/project_ID/course_ID/data

Now the large data will not be copied to each user's home dir when they start a classroom job session. Make sure to reference this data properly in notebooks that will be copied to students home dirs from the /materials dir.

Access student workspace

The instructor and TAs can access a student's workspace with limited permissions. First, the instructor sends us a request with the information including the instructor's and TAs' OSC accounts. After a student launches a class session, you can access known files and directories in the student's workspace. For example, you cannot explore the student's workspace

ls /users/PZS1234/student1/osc_classes/course_ID
ls: cannot open directory /users/PZS1234/student1/osc_classes/course_ID: Permission denied

but you can access a known file or directory in the workspace

ls /users/PZS1234/student1/osc_classes/course_ID/homework