Open OnDemand
Open OnDemand is a web portal for accessing certain clusters operated by Research Computing Services. The web portal provides a convenient way to access the login node, your files, and certain graphical applications such as Jupyter Notebooks and remote desktops. This service is an additional way to access HPC resources in addition to the existing command-line based options.
Open OnDemand is an open source project actively developed by the Ohio Supercomputer Center.
Access
Get an account
Before using Open OnDemand, you will need to have an account on the cluster. If you do not already have an account, please review the cluster's quick start guide for information on getting started.
Connect to OnDemand
You may access Open OnDemand for the following clusters using your UCIT credentials. Sign-on is handled through the University's Single Sign-On mechanism and requires Multi-Factor Authentication enabled.
| Cluster | Open OnDemand Access |
|---|---|
| ARC Cluster | ood-arc.rcs.ucalgary.ca |
Open OnDemand Dashboard
After logging in, you will see the Open OnDemand dashboard. The Message Of The Day (MOTD) will show any news and announcements relating to the cluster. Any quota warnings will also be displayed on your dashboard.
Other components and applications can be accessed through the top navigation.
File Browser
The file browser interface allows you to manage, upload, or download files from your directories, drag & drop file management, and basic file viewing and editing. You can access all files across all filesystems available to the cluster with this interface.
There is a 128 MB limit on file uploads. Please do not use this interface for large file transfers and instead look at other methods for file transfers.
Job Explorer
You may view and manage your current jobs on the cluster through the Active Jobs page. This may be helpful for users new to visualize scheduled jobs in Slurm.
Shell Access
You can launch a SSH session via Shell Access to the login node of the cluster.
Interactive Applications
We have created a small selection of graphical applications launchable through Open OnDemand.
Desktop (Container)
The Desktop interactive app may be helpful when exploring data or to run certain GUI based applications. This app simplifies the process of launching a full Linux desktop session or other graphical based applications all within your web browser without the need to use SSH and X11 forwarding. We currently offer only XFCE as the desktop manager.
Your desktop will run as an interactive job on the Slurm cluster and will be subject to a maximum time limit before it is terminated. As a result, you will need to restart your desktop after the time limit is reached. When using this desktop feature, be sure to save your work frequently and be mindful of the remaining time left for your session to avoid loss of work.
To start a desktop, go to the 'Interactive Apps' menu, then click on 'Desktop (Container)'.
Specify the partition you wish to run the desktop on as well as the desired runtime and resources your session should need.
After you launch the desktop, the job will be sent to Slurm. Depending on the partition and resources you selected, the job may be queued for some time. Once the desktop is started, you will see the option to launch the Remote Desktop in green:
Click on the blue "Launch Desktop" button to connect to your desktop.
Connecting to CIFS/Samba (eg. ResearchFS)
You may connect to CIFS/Samba shares through the desktop. Do so by opening the Thunar file manager and entering a SMB address:
Additional Apps
Please be aware that the desktop runs within an Apptainer environment. Certain features that require containerisation or Linux namespaces may not be available. This is particularly an issue with applications based on webkit/chromium and may require workarounds.
The following are some applications that have been installed within the Desktop environment:
Applications / Development
- Visual Studio Code
- RStudio
Applications / Internet
- FileZilla
- RClone Browser
- Chromium
- Firefox
- Citrix Workspace (Use Firefox or Chromium to connect to myappmf.ucalgary.ca)
Applications / Office
- LibreOffice Suite
Applications / Other
- Visual Molecular Dynamics (VMD)
Jupyter Notebooks
Open OnDemand offers Jupyter Notebooks via containers or environment modules. For most users, we recommend using the containers way to launch your Jupyter notebooks as it is the simplest. We also offer the environment module based method for users that already have a working workflow based on environment modules.
Container-based Jupyter notebooks
We offer a selection of different Jupyter Notebooks derived from the official Jupyter Stacks Docker images. When launching a Jupyter Notebook using this method, your notebook will run within an Apptainer container environment.
To start a Jupyter Notebook, go to the 'Interactive Apps' menu, then click on 'Jupyter Notebook (Container)'. On the launch page, select the desired image in the drop-down menu or use a custom image by selecting 'Custom Singularity Image...'.
The following flavours of Jupyter notebook images are currently available. When launching a notebook, specify an image that best suits your needs.
| Image | Description | |
|---|---|---|
| Spark Notebook | all-spark-notebook_spark-#.#.#.sif | Includes Python, R, and Scala support for Apache Spark.
Spark version is listed in the filename. |
| Julia Data Science Notebook | datascience-notebook_julia-#.#.#.sif | Includes libraries for data analysis from the Julia, Python, and R communities.
Julia version is listed in the filename. |
| Python Scipy Notebook | scipy-notebook_python-#.#.#.sif | Includes Python support for Apache Spark.
Python version is listed in the filename |
| Tensorflow Notebook | tensorflow-notebook_tensorflow-#.#.#.sif | Based on the Scipy notebook but includes Tensorflow.
Tensorflow version is listed in the filename. |
| gpu-jupyter.sif | Includes NVIDIA CUDA along with the data science notebook. |
Create a custom Jupyter Notebook image
If additional packages are required in the notebook that is not available as part of an existing Jupyter Notebook container image, you can create an extended and customized container image instead.
To build a custom container image, create a Singularity build definition file that references the appropriate base Jupyter notebook image. All custom Jupyter images should be based from one of the official base Jupyter notebook image for compatibility and to ensure that it will start properly within our Open OnDemand environment. As all the Jupyter notebook images are based on Ubuntu with mamba, you may install additional packages using apt-get, mamba, and pip.
In the following example, we will extend the tensorflow-notebook image with a C++ compiler (g++) and an additional pip package (corels).
Create a Singularity build file custom-notebook.def with the following contents:
Bootstrap: docker
From: jupyter/tensorflow-notebook
%post
apt-get autoremove; \
apt-get update --yes; \
apt-get install --yes --no-install-recommends g++; \
pip install corels
To build custom-notebook.def, run: singularity build custom-notebook.sif custom-notebook.def. This will generate a custom-notebook.sif singularity image file.
To use this custom image in Open OnDemand, move the newly generated .sif file to $HOME/ondemand/jupyter. In Open OnDemand, when launching a new notebook, select 'Custom Singularity Image...' on the launch page and select the appropriate .sif file.
Environment module based Jupyter notebooks
This is the alternative way of launching Jupyter notebooks on Open OnDemand. This option automates our previous recommended way of running Jupyter on ARC (described on our Jupyter Notebooks page) by loading python from an Anaconda environment and creating a port forwarding. We recommend this option only if you have an existing workflow that uses Jupyter with modules.
When using this method, you will be asked for an initialization bash script that sets up and starts Jupyter. An example script is given below. You may modify and add to this example script to customize your Jupyter Notebook setup. If no script is given when launching the notebook or if the script is not found, this example script will be used instead.
#!/bin/bash
# TODO: Set up your environment as required.
# Purge the module environment to avoid conflicts
module purge
module load python/anaconda3-2018.12
# TODO: Add any additional modules desired here.
module list
# As per https://rcs.ucalgary.ca/index.php/Jupyter_Notebooks
unset XDG_RUNTIME_DIR
# Launch the Jupyter Notebook Server
set -x
jupyter notebook --config="${CONFIG_FILE}"
RStudio Server (Container)
RStudio Server is a web-based version of RStudio and functions similar to Jupyter Notebooks. In our instance of Open OnDemand, RStudio Server will run within a Apptainer container environment.
After requesting for a RStudio Server session through Open OnDemand, you will see a session information box similar to the one below. Each session will use a randomised password.
Because RStudio Server runs within an Apptainer environment, some features may not be working as expected. We are still testing this app -- If you notice any issues, please reach out to RCS.
VS Code Server
Open OnDemand offers VS Code Server as an Interactive App. VS Code Server is a web-based version of Visual Studio Code and runs directly on the remote compute node. This is an alternative to running Visual Studio Code as a native app within the Open OnDemand Desktop.
To start a VS Code Server, go to the 'Interactive Apps' menu, then click on 'VS Code Server'.
Visual Studio Code
As an alternative to Visual Studio Code Server, you can run Visual Studio Code from the Desktop app. This will run Visual Studio Code as a desktop application within a Linux desktop environment.
To start a VS Code, go to the 'Interactive Apps' menu, then click on 'Desktop (Container)'. After launching the desktop, navigate to 'Application' -> 'Development' -> 'Visual Studio Code'
Troubleshooting
VNC session cannot reconnect
If your VNC session closed, you may need to re-connect by going back to the Open OnDemand dashboard, listing your sessions under "My Interactive Sessions" page, then click on the blue "Launch Desktop" button.
|
|
Do you need assistance, have questions or concerns? Please do not hesitate to contact us at support@hpc.ucalgary.ca.
|
