RCSWiki - User contributions [en]

ARC Cluster Status

2026-09-10T15:23:02Z

Dschulz:

{{ARC Cluster Status}}

== System Messages ==
{{Message of the day item
| title = ⚠️ January System Updates
| date = 2023/01/01
| message =
Beginning January 16, 2023, the ARC cluster will undergo operating system updates. We shall do our utmost to minimize disruption and allow ongoing jobs to be completed. New jobs may be temporarily held from scheduling.

The ARC login node will reboot on the morning of January 16. Please save your work and log out if possible.

The upgrade is planned to be fully complete by January 20.

If you encounter any system issues, do not hesitate to let us know.

Thank you for your cooperation.
}}

{{Message of the day item
| title = System Updates Completed
| date = 2023/01/24
| message =
The upgrade has been completed. The following has been changed:
* OS Updated to Rocky Linux 8.7
* Slurm updated to 22.05.7
* Apptainer replaces Singularity
* Each job will have its own /tmp, /dev/shm, /run/user/$uid mounted

If you encounter any system issues, do not hesitate to let us know.

Thank you for your cooperation.
}}

{{Message of the day item
| title = Filesystem Issues
| date = 2023/02/28
| message =
We are currently investigating a filesystem issue that is causing filesystem slowdowns across ARC.

We will update you with more information as it becomes available.

Thank you for your patience.
}}

{{Message of the day item
| title = Filesystem Issues
| date = 2023/03/1
| message =
We are still currently investigating a filesystem issue that is causing filesystem slowdowns across ARC. Some jobs on ARC have been paused to help us find the root cause of the slowdowns.

We will update you with more information as it becomes available.

Thank you for your patience.
}}

{{Message of the day item
| title = ARC Login node reboot
| date = 2023/03/2
| message =
The ARC login node will be rebooted this afternoon for an emergency maintenance. This downtime is needed to help mitigate the filesystem slowdowns experienced on the login node. Jobs will continue running and scheduling during this time.

All logins to the ARC login node will be terminated at 3:00PM and will remain unavailable until 4:00PM.

We apologize for the inconvenience and thank you for your patience.
}}

{{Message of the day item
| title = ⚠️ Filesystem Issues
| date = 2023/03/2
| message =
We are still currently investigating a filesystem issue that is causing filesystem slowdowns on specific nodes in our MSRDC location.

We will update you with more information as it becomes available.

We apologize for the inconvenience and thank you for your patience.
}}

{{Message of the day item
| title = Filesystem Issues Resolved
| date = 2023/03/10
| message =
We have upgraded the filesystem routers in our MSRDC location to address the performance issues.

Please let us know if you experience any issues with the filesystem performance.

Thank-you for your patience.
}}

{{Message of the day item
| title = Open OnDemand reboot
| date = 2023/05/01
| message =
On May 1, 2023, the ARC Open OnDemand node will be rebooted between 5PM and 6PM. Expected downtime will be approximately 15 minutes.

If you encounter any system issues, do not hesitate to let us know.

Thank you for your cooperation.
}}

{{Message of the day item
| title = Apptainer (Singularity) on ARC Login Node
| date = 2023/06/22
| message =
Apptainer (Singularity) containers may experience an error when
running on the Arc login node. If apptainer complains that a system
administrator needs to enable user namespaces, simply run your
containers inside a job.

This is a temporary measure due to security vulnerability that will be
patched soon.

}}

{{Message of the day item
| title = Lattice, Single, cpu2013 partition changes
| date = 2023/07/13
| message =
The Lattice and Single, and cpu2013 have all been decomissioned. The Single
partition will be replaced by the nodes formerly in the cpu2013 partition but
will be called single.

}}

{{Message of the day item
| title = Open OnDemand reboot
| date = 2023/10/17
| message =
Open OnDemand will be rebooted on October 17, 2023 for an update. It will be down for up to 30 minutes.

}}

{{Message of the day item
| title = Storage Upgrade MARC/ARC cluster
| date = 2023/10/23
| message =
We will be performing storage upgrades on the MARC/ARC cluster on
November 16 and 17, 2023. To facilitate this, we will be throttling
down the number of jobs on both clusters while the upgrades are
performed
}}

{{Message of the day item
| title = Systems Operating Normally
| date = 2024/05/3
| message =
}}

{{Message of the day item
| title = Power Interruption
| date = 2024/05/07
| message = Arc Experienced an brief power outage around 11AM May 7, 2024.
Most compute nodes have or are rebooting. Most jobs running at this time
were lost. Arc administrators are actively working on restarting compute
nodes. Sorry for the inconvenience.
}}

{{Message of the day item
| title = GPU a100 Node Reservation
| date = 2024/06/03
| message = Job submissions targeted to the GPU a100 partition will be
affected by a temporary reservation on the nodes to accommodate the RCS
summer school class taking place on 2024/Jun/10. Reservation will end
shortly after. Please submit your jobs normally and the scheduler will
start them as soon as the nodes are available. Sorry for the inconvenience.
}}

{{Message of the day item
| title = GPU a100 Node Reservation Removed
| date = 2024/06/11
| message = GPU a100 Nodes in ARC have been returned to normal scheduling.
}}

{{Message of the day item
| title = Notice of Upcoming Partial Outage
| date = 2024/08/23
| message = Several compute nodes from the ARC cluster will be unavailable
between Sept 23 to Sept 27 inclusive (subject to change). All compute nodes
in cpu2019, cpu2021/2, gpu-v100 most nodes from bigmem and gpu-a100 will be
affected. These nodes will return to service as soon as the work is complete.
}}

{{Message of the day item
| title = Partial Outage Update I
| date = 2024/09/25
| message = Due to hardware issues that is blocking our original maintenance window, most compute nodes that were taken offline on Monday has been brought back online today. An additional partial outage will occur again starting next Tuesday for the same nodes.

On Tuesday, October 1, 2024, the compute nodes in cpu2019, cpu2021, cpu2022, gpu-v100, gpu-a100, and most nodes from bigmem will be unavailable until Friday October 4, 2024.

We apologise for the inconvenience.
}}

{{Message of the day item
| title = Partial Outage Update II
| date = 2024/10/04
| message = The maintenance window will be extended until at least Monday, October 7, 2024 due to a power distribution issue in our renovated data centre.

Currently, the compute nodes in cpu2019, cpu2021, cpu2022, gpu-v100, gpu-a100, and most nodes from bigmem will be unavailable until at least Monday, October 7, 2024. Affected WDF-Altis GPU nodes include: wdfgpu[1-2,6,8-12].

We apologize for the extended downtime and will update you as soon as we have additional information from our operations team.
}}

{{Message of the day item
| title = Partial Outage Update III
| date = 2024/10/07
| message = Due to technical issues beyond our control the maintenance window will be extended until at least Tuesday, October 15, 2024.

Currently, the compute nodes in cpu2019, cpu2021, cpu2022, gpu-v100, gpu-a100, and most nodes from bigmem will be unavailable until at least Tuesday, October 15, 2024. Affected WDF-Altis GPU nodes include: wdfgpu[1-2,6,8-12].

We apologize for the extended downtime and will update you as soon as we have additional information from our operations team.
}}

{{Message of the day item
| title = Normal Scheduling has resumed.
| date = 2024/10/08
| message = The ARC cluster has been successfully brought online and nodes are running jobs normally. We apologize for the extended downtime.

Please reach out to support@hpc.ucalgary.ca with any issues or concerns.
}}

{{Message of the day item
| title = Scheduled Maintenance
| date = 2024/12/11
| message = The ARC login node will be rebooted on Tuesday December 17 for scheduled maintenance. It will be down for a few minutes and return shortly. Job scheduling and jobs running on the cluster will not be affected. Thank you for understanding.

Please reach out to support@hpc.ucalgary.ca with any issues or concerns.
}}

{{Message of the day item
| title = Scheduled Maintenance and OS Update
| date = 2025/01/07
| message = The ARC cluster will be rebooted for OS updates on Monday January 20, 2025. Please make sure to save your work and log out before the reboot happens. Scheduling will be paused until the cluster is back, but queued jobs will remain in the queue and nodes will start scheduling when the cluster is ready. Thank you for understanding.

Please reach out to support@hpc.ucalgary.ca with any issues or concerns.
}}

{{Message of the day item
| title = ⚠️ Scheduled Maintenance and OS Update
| date = 2025/01/15
| message = The ARC cluster will be down for maintenance and upgrades starting 9AM Monday, January 20, 2025 through Wednesday, January 22, 2025.

For the duration of the upgrade window:
* Scheduling will be paused and new jobs will be queued. Any queued jobs will start scheduling only after the upgrade is complete.
* Access to files via the login node and arc-dtn will generally be available but intermittent. File transfers on the DTN node, including Globus file transfers, may be interrupted during this window.

Please make sure to save your work prior to this outage window to avoid any loss of work.

During this time the following changes will happen:

1. Ethernet will replace the 11 year old, unsupported Infiniband on the following partitions:
* cpu2023 (temporary)
* Parallel
* Theia/Synergy/cpu2017-bf05
* Single
Any multi-node jobs (MPI) running on these partitions will have increased latency going forward. If you run multi-node jobs, make sure to run on a partition such as cpu2019, cpu2021, cpu2022.

2. A component of the NetApp filer will be replaced. Access to /bulk will be unavailable on Wednesday, January 22, 2025.

3. The compute node operating system will be updated to Rocky Linux 8.10.

4. The Slurm scheduling system will be upgraded.

5. The Open OnDemand web portal will be upgraded.

Please reach out to support@hpc.calgary.ca with any issues or concerns.

⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️
Update Jan 18, 2025

Around 10AM Arc experienced an electrical power brownout. Some percentage (how many is unknown at this time) of the nodes lost electrical power during this time causing a loss of a number of running jobs.

Sorry for the inconvenience.

Since Arc is shutting down for maintenance on Monday Jan 20, replacement jobs will likely not start unless they request a timelimit less than the time until 8AM Monday.
⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️
}}

{{Message of the day item
| title = Maintenance Complete
| date = 2025/01/22
| message = The ARC cluster upgrade is complete

During this time the following changes happened:

1. Ethernet will replace the 11 year old, unsupported Infiniband on the following partitions:
* cpu2023 (temporary)
* Parallel
* Theia/Synergy/cpu2017-bf05
* Single
Any multi-node jobs (MPI) running on these partitions will have increased latency going forward. If you run multi-node jobs, make sure to run on a partition such as cpu2019, cpu2021, cpu2022.

2. A component of the NetApp filer was replaced successfully.

3. The compute node operating was updated to Rocky Linux 8.10.

4. The Slurm scheduling system was upgraded.

5. The Open OnDemand web portal was upgraded.

6. The Parallel partition was renamed to Legacy to show the lack of an interconnect for parallel MPI work and was restricted to maximum 4 node jobs.

Please reach out to support@hpc.calgary.ca with any issues or concerns.

Jan 23, 9:08AM

Remount complete. arc is back in full service.
}}

{{Message of the day item
| title = Module Command Upgrade
| date = 2025/02/03
| message = Upgrade of the module command

On Tuesday, February 11, 2025 the module command will be upgraded to a new verson on Arc. This should result in new capabilities and a slightly different visual experience when using the module command. Loading modules is not expected to change.
}}

{{Message of the day item
| title = Module Command Upgraded
| date = 2025/02/12
| message = The module command was upgraded

On Tuesday, February 12, 2025 the module command was upgraded to a new verson on Arc. This should result in new capabilities and a slightly different visual experience when using the module command. Loading modules should not have changed.

}}

{{Message of the day item
| title = Support email address down
| date = 2025/03/07
| message = support@hpc.ucalgary.ca Unavailable

Please be informed that our support email address (support@hpc.ucalgary.ca) for RCS is currently not working. We are working to bring it back as soon as possible. Please keep an eye on this space for updates. The clusters are working normally, but support will not receive your messages at this time. We will begin responding as soon as we can get it back.

Apologies for the inconvenience.

}}

{{Message of the day item
| title = Support email address functional
| date = 2025/03/07
| message = support@hpc.ucalgary.ca is back

support@hpc.ucalgary.ca has been repaired and RCS can be contacted there. If you had reached out for assistance in recent days without response please follow up as we may not have received your initial email.

Apologies for the inconvenience.
}}

{{Message of the day item
| title = Interactive Job Timelimit will be Enforced
| date = 2025/04/11
| message = In order to improve the scheduling and job throughput efficiency of ARC, interactive jobs will be limited to a maximum of 5 hours of runtime. Interactive jobs that are submitted with a timelimit over 5 hours will be rejected at submission time. This change will be made on Monday, April 28, 2025.
}}

{{Message of the day item
| title = Interactive Job Timelimit Is Now Enforced
| date = 2025/04/28
| message = In order to improve the scheduling and job throughput efficiency of ARC, interactive jobs are now limited to a maximum of 5 hours of runtime. Interactive jobs that are submitted with a timelimit over 5 hours will be rejected at submission time.

Apr 29, 2025
To increase the security posture of the Arc cluster administrators will be installing Trend Micro on cluster login nodes over the week starting Apr 30. Please report any inconsistencies to support@hpc.ucalgary.ca
}}

{{Message of the day item
| title = ⚠️ Bulk Filesystem Emergency Maintenance
| date = 2025/07/29
| message = The filer that provides the /bulk filesystem will be down for emergency repairs at 12 Noon on Thursday July 31. No access to files on /bulk will be possible for the duration of the multi-hour outage. Any jobs running that access /bulk will start and then pause when access to /bulk is attempted. Jobs should continue once service is restored. Sorry for the inconvenience.
}}

{{Message of the day item
| title = Bulk Filesystem Maintenance Complete
| date = 2025/08/01
| message = Maintenance on /bulk was completed successfully and all filesystems are back in service on Arc.
}}

{{Message of the day item
| title = GPU partition changes on ARC
| date = 2025/08/15
| message = New gpu-h100 and gpu-l40 partitions are available for general scheduling with new gpu hardware. gpu-v100 also has 6 fewer nodes. You can view more details about the node specs by running the arc.hardware script on the login node.
}}

{{Message of the day item
| title = Update and Bulk outage on ARC
| date = 2025/09/15
| message = ⚠️ Arc Update September 15-19

An update of the Arc cluster will begin on Sep 15. This will result in fewer resources while the compute nodes are restarted. The login node and Scheduler will be restarted on Sep 17.

⚠️ Bulk Filesystem Maintenance September 17
The filer that provides the /bulk filesystem will be down for emergency repairs at 9 AM on Wednesday Sep 17th. No access to files on /bulk will be possible for the duration of the multi-hour outage. Any jobs running that access /bulk will start and then pause when access to /bulk is attempted. Jobs should continue once service is restored. We apologize for any inconvenience.

}}

{{Message of the day item
| title = Update and Bulk outage on ARC
| date = 2025/09/18
| message = Arc update complete

The update to arc is complete.

}}

{{Message of the day item
| title = GPU partition changes on ARC
| date = 2025/09/23
| message = gpu-v100 has 6 more nodes. You can view more details about the node specs by running the arc.hardware script on the login node.
}}

{{Message of the day item
| title = Systems Operating Normally
| date = 2025/11/4
| message =
}}

{{Message of the day item
| title = Legacy compute nodes are being retired
| date = 2025/11/20
| message = ARC nodes cn[0513-1096] are being removed from the arc cluster. They will be removed from scheduling and removed from the cluster over the next while.
}}

{{Message of the day item
| title = Planned outage in Jan 2026
| date = 2025/12/26
| message = ⚠️ Arc Update Week of January 12

ARC will be going down periodically during the week of 2026/Jan/12 to allow for system maintenance. All nodes and filesystems will be unavailable for use during the outage, and filesystem performance may be variable following the upgrade for a time until the changes finish.

If you have questions or concerns, please contact support@hpc.ucalgary.ca
}}

{{Message of the day item
| title = Arc Filesystem Returned to service, scheduling outages continue
| date = 2025/01/13
| message = ⚠️ Data access now possible ⚠️

The first stage of the maintenance is complete and the filesystems are once again available.

There is still significant work to do on the cluster so jobs will not start until this maintenance is complete but clients will be able to access data via the login node and DTN.

If you have questions or concerns, please contact support@hpc.ucalgary.ca
}}

{{Message of the day item
| title = Arc Returned to service, scheduling resumed
| date = 2026/01/16
| message = Maintenance Complete

Arc has been returned to service.

If you have questions or concerns, please contact support@hpc.ucalgary.ca
}}

{{Message of the day item
| title = New Compute nodes added to Arc
| date = 2026/01/22
| message = cpu2025 partition is now available.

New nodes have been installed to replace the aging parallel nodes. They have 1TiB of memory and 128 cpus.

If you have questions or concerns, please contact support@hpc.ucalgary.ca
}}

{{Message of the day item
| title = Filesystem Outage Feb 23, 2026 Starting at 7AM
| date = 2026/02/17
| message = ⚠️ Filesystem Outage Affecting Arc, Marc, Talc, Cloudstack

Filesystems will be unavailable Feb 23 starting at 7AM MST. This outage is expected to end prior to 8PM. No logins or file access will be possible until the outage is complete.

Any jobs queued must request timelimits less then the remaining time until Feb 23, 7AM or they will wait until the outage is complete.

Sorry for the inconvenience.

As of Friday Feb 20, 2026 - Due to some configuration issues on the network Fluent is not available on ARC at this time.
We are working to fix the issues and the software will return as soon as possible. Please try resubmitting any affected jobs later.

If you have questions or concerns, please contact support@hpc.ucalgary.ca
}}

{{Message of the day item
| title = Filesystem Outage Complete
| date = 2026/02/23
| message = ✅ Filesystem Outage Complete

All clusters are back in production.

As of Friday Feb 20, 2026 - Due to some configuration issues on the network Fluent is not available on ARC at this time.
We are working to fix the issues and the software will return as soon as possible. Please try resubmitting any affected jobs later.

If you have questions or concerns, please contact support@hpc.ucalgary.ca
}}

{{Message of the day item
| title = Arc Login and Scheduling Outage May 7, 2023, Starting at 9AM
| date = 2026/05/07
| message = ⚠️ Urgent Security Update

The Arc Support nodes (Arc login node, DTN, OOD, and scheduling system) will be unavailable starting at 9 AM on May 7 to install an urgent kernel update. You will be unable to login to Arc but queued jobs will continue to run. The compute nodes are actively being rebooted as they become idle resulting in fewer available resources for the next week or so.

Sorry for the inconvenience.

}}

{{Message of the day item
| title = Emergency Security Mitigation
| date = 2026/05/15
| message = ptrace has been disable on clusters

The ptrace system call has been temporarily disabled while we await a patch for CVE-2026-46333. This means that software such as gdb or strace will not work.

If you have questions or concerns, please contact support@hpc.ucalgary.ca
}}

{{Message of the day item
| title = Arc Login and Scheduling Outage June 3, 2026, Starting at 9AM
| date = 2026/06/03
| message = ⚠️ Urgent Security Update

The Arc Support nodes (Arc login node, DTN, OOD, and scheduling system) will be unavailable starting at 9 AM on June 3, 2026 to install an urgent kernel update. You will be unable to login to Arc but queued jobs will continue to run. The compute nodes are actively being rebooted as they become idle resulting in fewer available resources for the next week or so.

Sorry for the inconvenience.

}}

{{Message of the day item
| title = Systems Operating Normally
| date = 2026/06/22
| message =
}}

{{Message of the day item
| title = Upcoming File Server Update July 2, 2026. Starting at 9AM
| date = 2026/06/23
| message = File Server Update

We will be updating the file servers on July 2, 2026, beginning at 9AM continuing throughout the day. We do not expect any service interruptions or degradation during the process.

If you have any questions ahead of time, or experience any symptoms on the date, please contact RCS via support@hpc.ucalgary.ca
}}

{{Message of the day item
| title = Power Outage Caused loss of most or all jobs
| date = 2026/09/10
| message = Main campus experienced a power outage around 06:30 on Sep 10, 2026 resulting in the loss of most possibly all jobs. Staff are restarting the clusters. Please submit replacement jobs and they will run as resources return to scheduling.
}}

{{Navbox ARC}}
[[Category:ARC]]

Open OnDemand

2026-09-01T23:48:57Z

Dschulz: /* Custom RStudio Container Images */

Open OnDemand is a web portal for accessing certain clusters operated by Research Computing Services. The web portal provides a convenient way to access the login node, your files, and certain graphical applications such as Jupyter Notebooks and remote desktops. This service is an additional way to access HPC resources in addition to the existing command-line based options.

Open OnDemand is an open source project actively developed by the Ohio Supercomputer Center.

== Access ==

=== Get an account ===
Before using Open OnDemand, you will need to have an account on the cluster. If you do not already have an account, please review the cluster's quick start guide for information on getting started.

=== Connect to OnDemand ===
You may access Open OnDemand for the following clusters using your UCIT credentials. Sign-on is handled through the University's Single Sign-On mechanism and requires Multi-Factor Authentication enabled.
{| class="wikitable"
!Cluster
!Open OnDemand Access
|-
|[[ARC Cluster Guide|ARC Cluster]]
|[https://ood-arc.rcs.ucalgary.ca ood-arc.rcs.ucalgary.ca]
|}

== Open OnDemand Dashboard ==
After logging in, you will see the Open OnDemand dashboard. The Message Of The Day (MOTD) will show any news and announcements relating to the cluster. Any quota warnings will also be displayed on your dashboard.
[[File:Open OnDemand Dashboard.jpg|none|thumb|Open OnDemand Dashboard]]Other components and applications can be accessed through the top navigation.

=== File Browser ===
The file browser interface allows you to manage, upload, or download files from your directories, drag & drop file management, and basic file viewing and editing. You can access all files across all filesystems available to the cluster with this interface.

There is a 128 MB limit on file uploads. Please do not use this interface for large file transfers and instead look at [[How to transfer data|other methods for file transfers]].

=== Job Explorer ===
You may view and manage your current jobs on the cluster through the Active Jobs page. This may be helpful for users new to visualize scheduled jobs in Slurm.

=== Shell Access ===
You can launch a SSH session via Shell Access to the login node of the cluster.

== Interactive Applications ==
We have created a small selection of graphical applications launchable through Open OnDemand.

=== Desktop (Container) ===
The Desktop interactive app may be helpful when exploring data or to run certain GUI based applications. This app simplifies the process of launching a full Linux desktop session or other graphical based applications all within your web browser without the need to use SSH and X11 forwarding. We currently offer only XFCE as the desktop manager.

Your desktop will run as an interactive job on the Slurm cluster and will be subject to a maximum time limit before it is terminated. As a result, you will need to restart your desktop after the time limit is reached. When using this desktop feature, be sure to save your work frequently and be mindful of the remaining time left for your session to avoid loss of work.

To start a desktop, go to the 'Interactive Apps' menu, then click on 'Desktop (Container)'.
[[File:Open OnDemand Desktop.jpg|alt=Open OnDemand Desktop|none|thumb|Open OnDemand Desktop]]
Specify the partition you wish to run the desktop on as well as the desired runtime and resources your session should need.

After you launch the desktop, the job will be sent to Slurm. Depending on the partition and resources you selected, the job may be queued for some time. Once the desktop is started, you will see the option to launch the Remote Desktop in green:
[[File:Open OnDemand Desktop Launch.png|alt=Open OnDemand Desktop Launch|none|thumb|Open OnDemand Desktop Launch]]Click on the blue "Launch Desktop" button to connect to your desktop.
[[File:Remote Desktop on Open OnDemand.png|alt=Remote Desktop on Open OnDemand|none|thumb|Remote Desktop on Open OnDemand]]

==== Connecting to CIFS/Samba (eg. ResearchFS) ====
You may connect to CIFS/Samba shares through the desktop. Do so by opening the Thunar file manager and entering a SMB address:
[[File:Thunar SMB connection.png|alt=Thunar SMB connection|none|thumb|Thunar SMB connection]]

==== Additional Apps ====
Please be aware that the desktop runs within an Apptainer environment. Certain features that require containerisation or Linux namespaces may not be available. This is particularly an issue with applications based on webkit/chromium and may require workarounds.

The following are some applications that have been installed within the Desktop environment:

===== Applications / Development =====
* Visual Studio Code
* RStudio

===== Applications / Internet =====
* FileZilla
* RClone Browser
* Chromium
* Firefox
* Citrix Workspace (Use Firefox or Chromium to connect to myappmf.ucalgary.ca)

===== Applications / Office =====
* LibreOffice Suite

===== Applications / Other =====
* Visual Molecular Dynamics (VMD)
=== Jupyter Notebooks ===
Open OnDemand offers Jupyter Notebooks via containers or environment modules. For most users, we recommend using the containers way to launch your Jupyter notebooks as it is the simplest. We also offer the environment module based method for users that already have a working workflow based on environment modules.

==== Container-based Jupyter notebooks ====
[[File:Open OnDemand Jupyter.jpg|alt=Open OnDemand Jupyter Notebook|thumb|Open OnDemand Jupyter Notebook ]]
We offer a selection of different Jupyter Notebooks derived from the [https://jupyter-docker-stacks.readthedocs.io/en/latest/using/selecting.html. official Jupyter Stacks Docker images]. When launching a Jupyter Notebook using this method, your notebook will run within an Apptainer container environment.

To start a Jupyter Notebook, go to the 'Interactive Apps' menu, then click on 'Jupyter Notebook (Container)'. On the launch page, select the desired image in the drop-down menu or use a custom image by selecting 'Custom Singularity Image...'.

The following flavours of Jupyter notebook images are currently available. When launching a notebook, specify an image that best suits your needs.
{| class="wikitable"
!
!Image
!Description
|-
|Spark Notebook
|all-spark-notebook_spark-#.#.#.sif
|Includes Python, R, and Scala support for Apache Spark.
Spark version is listed in the filename.
|-
|Julia Data Science Notebook
|datascience-notebook_julia-#.#.#.sif
|Includes libraries for data analysis from the Julia, Python, and R communities.
Julia version is listed in the filename.
|-
|Python Scipy Notebook
|scipy-notebook_python-#.#.#.sif
|Includes Python support for Apache Spark.
Python version is listed in the filename
|-
|Tensorflow Notebook
|tensorflow-notebook_tensorflow-#.#.#.sif
|Based on the Scipy notebook but includes Tensorflow.
Tensorflow version is listed in the filename.
|-
|
|gpu-jupyter.sif
|Includes NVIDIA CUDA along with the data science notebook.
Based on https://github.com/iot-salzburg/gpu-jupyter/
|}

===== Create a custom Jupyter Notebook image =====
If additional packages are required in the notebook that is not available as part of an existing Jupyter Notebook container image, you can create an extended and customized container image instead.

To build a custom container image, create a Singularity build definition file that references the appropriate base Jupyter notebook image. All custom Jupyter images should be based from one of the official base Jupyter notebook image for compatibility and to ensure that it will start properly within our Open OnDemand environment. As all the Jupyter notebook images are based on Ubuntu with mamba, you may install additional packages using <code>apt-get</code>, <code>mamba</code>, and <code>pip</code>.

In the following example, we will extend the <code>tensorflow-notebook</code> image with a C++ compiler (g++) and an additional pip package (corels).

Create a Singularity build file <code>custom-notebook.def</code> with the following contents:
{{Highlight|code=Bootstrap: docker
From: jupyter/tensorflow-notebook

%post
apt-get autoremove; \
apt-get update --yes; \
apt-get install --yes --no-install-recommends g++; \
pip install corels|lang=text}}
To build <code>custom-notebook.def</code>, run: <code>singularity build custom-notebook.sif custom-notebook.def</code>. This will generate a <code>custom-notebook.sif</code> singularity image file.

To use this custom image in Open OnDemand, move the newly generated <code>.sif</code> file to <code>$HOME/ondemand/jupyter</code>. In Open OnDemand, when launching a new notebook, select 'Custom Singularity Image...' on the launch page and select the appropriate <code>.sif</code> file.

==== Environment module based Jupyter notebooks ====
This is the alternative way of launching Jupyter notebooks on Open OnDemand. This option automates our previous recommended way of running Jupyter on ARC (described on our [[Jupyter Notebooks|Jupyter Notebooks page]]) by loading python from an Anaconda environment and creating a port forwarding. We recommend this option '''only''' if you have an existing workflow that uses Jupyter with modules.

When using this method, you will be asked for an initialization bash script that sets up and starts Jupyter. An example script is given below. You may modify and add to this example script to customize your Jupyter Notebook setup. If no script is given when launching the notebook or if the script is not found, this example script will be used instead.

{{Highlight|code=#!/bin/bash

# TODO: Set up your environment as required.

# Purge the module environment to avoid conflicts
module purge
module load python/anaconda3-2018.12
# TODO: Add any additional modules desired here.
module list

# As per https://rcs.ucalgary.ca/index.php/Jupyter_Notebooks
unset XDG_RUNTIME_DIR

# Launch the Jupyter Notebook Server
set -x
jupyter notebook --config="${CONFIG_FILE}"|lang=bash}}
=== RStudio Server (Container) ===
RStudio Server is a web-based version of RStudio and functions similar to Jupyter Notebooks. In our instance of Open OnDemand, RStudio Server will run within a Apptainer container environment.

After requesting for a RStudio Server session through Open OnDemand, you will see a session information box similar to the one below. Each session will use a randomised password.
[[File:RStudio Server launch on OOD.png|alt=RStudio Server launch on Open OnDemand|none|thumb|RStudio Server launch on Open OnDemand]]
Because RStudio Server runs within an Apptainer environment, some features may not work as expected.

==== Custom RStudio Container Images ====
At times R libraries need system dependencies to function correctly. This procedure is used to take a common image that is already on Arc and modify it to contain the various libraries that are required. Apptainer allows us to install packages into containers that would normally not be possible to install without administrative privileges.
* Create an apptainer definition file. This file is similar to a Dockerfile but with a different format that is specific to apptainer. Here's an example that both installs a library using apt and installs an R package that requires that library:
* Create a file modifyrstudio.def
<PRE>
Bootstrap: localimage
From: /global/software/ondemand/rstudio/rstudio_4.4.3.sif

%post
apt update
apt install -y libglpk40
Rscript -e "install.packages(c('Seurat'), repos='https://cran.rstudio.com')"
</PRE>
* Modify the file modifyrstudio.def to meet your needs
** The From value is the name of an rstudio image that is currently on the system. There are other versions available in that same directory. This is the starting point so you don't have to install Rstudio from first principles.
** The section after %post is simply a script that virtually runs as root inside the build container. You can express any configuration changes, software installations, etc. Just make sure that nothing asks for input (for example the -y to apt install means say "yes" to any questions)
* Build the image like the following. The .sif file's filename is not critical but it must be in ~/ondemand/rstudio and end with .sif or it will not appear as an image in the RStudio App:
<PRE>
apptainer build ~/ondemand/rstudio/rstudio-4.4.3-libglpk40-Seurat.sif modifyrstudio.def
</PRE>
* Once you've created the image successfully in your home directory, you can use the image by starting a new instance of the Rstudio app (you'll need to refresh the webpage with the parameters of the app before the image shows up).
* Choose "Custom Singularity Image...." from the "RStudio Server Image" dropdown box and a new dropdown will appear named "Custom Rstudio Image". In the Custom Rstudio Image dropdown you should see the file you just created with the apptainer build step above.
* Choose "Launch" as normal and after your job starts you should be met with an Rstudio interface that contains the changes you have introduced.

=== VS Code Server ===
Open OnDemand offers VS Code Server as an Interactive App. VS Code Server is a web-based version of Visual Studio Code and runs directly on the remote compute node. This is an alternative to running Visual Studio Code as a native app within the Open OnDemand Desktop.

To start a VS Code Server, go to the 'Interactive Apps' menu, then click on 'VS Code Server'.
[[File:Open OnDemand VSCodeServer.png|alt=Open OnDemand VSCodeServer|none|thumb|Open OnDemand VS Code Server]]

=== Visual Studio Code ===
As an alternative to Visual Studio Code Server, you can run Visual Studio Code from the Desktop app. This will run Visual Studio Code as a desktop application within a Linux desktop environment.

To start a VS Code, go to the 'Interactive Apps' menu, then click on 'Desktop (Container)'. After launching the desktop, navigate to 'Application' -> 'Development' -> 'Visual Studio Code'
[[File:Visual Studio Code on OOD Desktop.png|alt=Visual Studio Code on OOD Desktop|none|thumb|Visual Studio Code on OOD Desktop]]

== Troubleshooting ==

=== VNC session cannot reconnect ===
If your VNC session closed, you may need to re-connect by going back to the Open OnDemand dashboard, listing your sessions under "My Interactive Sessions" page, then click on the blue "Launch Desktop" button.

{{Support}}

[[Category:ARC]]
{{Navbox ARC}}

Open OnDemand

2026-09-01T23:46:23Z

Dschulz: /* RStudio Server (Container) */

Open OnDemand is a web portal for accessing certain clusters operated by Research Computing Services. The web portal provides a convenient way to access the login node, your files, and certain graphical applications such as Jupyter Notebooks and remote desktops. This service is an additional way to access HPC resources in addition to the existing command-line based options.

Open OnDemand is an open source project actively developed by the Ohio Supercomputer Center.

== Access ==

=== Get an account ===
Before using Open OnDemand, you will need to have an account on the cluster. If you do not already have an account, please review the cluster's quick start guide for information on getting started.

=== Connect to OnDemand ===
You may access Open OnDemand for the following clusters using your UCIT credentials. Sign-on is handled through the University's Single Sign-On mechanism and requires Multi-Factor Authentication enabled.
{| class="wikitable"
!Cluster
!Open OnDemand Access
|-
|[[ARC Cluster Guide|ARC Cluster]]
|[https://ood-arc.rcs.ucalgary.ca ood-arc.rcs.ucalgary.ca]
|}

== Open OnDemand Dashboard ==
After logging in, you will see the Open OnDemand dashboard. The Message Of The Day (MOTD) will show any news and announcements relating to the cluster. Any quota warnings will also be displayed on your dashboard.
[[File:Open OnDemand Dashboard.jpg|none|thumb|Open OnDemand Dashboard]]Other components and applications can be accessed through the top navigation.

=== File Browser ===
The file browser interface allows you to manage, upload, or download files from your directories, drag & drop file management, and basic file viewing and editing. You can access all files across all filesystems available to the cluster with this interface.

There is a 128 MB limit on file uploads. Please do not use this interface for large file transfers and instead look at [[How to transfer data|other methods for file transfers]].

=== Job Explorer ===
You may view and manage your current jobs on the cluster through the Active Jobs page. This may be helpful for users new to visualize scheduled jobs in Slurm.

=== Shell Access ===
You can launch a SSH session via Shell Access to the login node of the cluster.

== Interactive Applications ==
We have created a small selection of graphical applications launchable through Open OnDemand.

=== Desktop (Container) ===
The Desktop interactive app may be helpful when exploring data or to run certain GUI based applications. This app simplifies the process of launching a full Linux desktop session or other graphical based applications all within your web browser without the need to use SSH and X11 forwarding. We currently offer only XFCE as the desktop manager.

Your desktop will run as an interactive job on the Slurm cluster and will be subject to a maximum time limit before it is terminated. As a result, you will need to restart your desktop after the time limit is reached. When using this desktop feature, be sure to save your work frequently and be mindful of the remaining time left for your session to avoid loss of work.

To start a desktop, go to the 'Interactive Apps' menu, then click on 'Desktop (Container)'.
[[File:Open OnDemand Desktop.jpg|alt=Open OnDemand Desktop|none|thumb|Open OnDemand Desktop]]
Specify the partition you wish to run the desktop on as well as the desired runtime and resources your session should need.

After you launch the desktop, the job will be sent to Slurm. Depending on the partition and resources you selected, the job may be queued for some time. Once the desktop is started, you will see the option to launch the Remote Desktop in green:
[[File:Open OnDemand Desktop Launch.png|alt=Open OnDemand Desktop Launch|none|thumb|Open OnDemand Desktop Launch]]Click on the blue "Launch Desktop" button to connect to your desktop.
[[File:Remote Desktop on Open OnDemand.png|alt=Remote Desktop on Open OnDemand|none|thumb|Remote Desktop on Open OnDemand]]

==== Connecting to CIFS/Samba (eg. ResearchFS) ====
You may connect to CIFS/Samba shares through the desktop. Do so by opening the Thunar file manager and entering a SMB address:
[[File:Thunar SMB connection.png|alt=Thunar SMB connection|none|thumb|Thunar SMB connection]]

==== Additional Apps ====
Please be aware that the desktop runs within an Apptainer environment. Certain features that require containerisation or Linux namespaces may not be available. This is particularly an issue with applications based on webkit/chromium and may require workarounds.

The following are some applications that have been installed within the Desktop environment:

===== Applications / Development =====
* Visual Studio Code
* RStudio

===== Applications / Internet =====
* FileZilla
* RClone Browser
* Chromium
* Firefox
* Citrix Workspace (Use Firefox or Chromium to connect to myappmf.ucalgary.ca)

===== Applications / Office =====
* LibreOffice Suite

===== Applications / Other =====
* Visual Molecular Dynamics (VMD)
=== Jupyter Notebooks ===
Open OnDemand offers Jupyter Notebooks via containers or environment modules. For most users, we recommend using the containers way to launch your Jupyter notebooks as it is the simplest. We also offer the environment module based method for users that already have a working workflow based on environment modules.

==== Container-based Jupyter notebooks ====
[[File:Open OnDemand Jupyter.jpg|alt=Open OnDemand Jupyter Notebook|thumb|Open OnDemand Jupyter Notebook ]]
We offer a selection of different Jupyter Notebooks derived from the [https://jupyter-docker-stacks.readthedocs.io/en/latest/using/selecting.html. official Jupyter Stacks Docker images]. When launching a Jupyter Notebook using this method, your notebook will run within an Apptainer container environment.

To start a Jupyter Notebook, go to the 'Interactive Apps' menu, then click on 'Jupyter Notebook (Container)'. On the launch page, select the desired image in the drop-down menu or use a custom image by selecting 'Custom Singularity Image...'.

The following flavours of Jupyter notebook images are currently available. When launching a notebook, specify an image that best suits your needs.
{| class="wikitable"
!
!Image
!Description
|-
|Spark Notebook
|all-spark-notebook_spark-#.#.#.sif
|Includes Python, R, and Scala support for Apache Spark.
Spark version is listed in the filename.
|-
|Julia Data Science Notebook
|datascience-notebook_julia-#.#.#.sif
|Includes libraries for data analysis from the Julia, Python, and R communities.
Julia version is listed in the filename.
|-
|Python Scipy Notebook
|scipy-notebook_python-#.#.#.sif
|Includes Python support for Apache Spark.
Python version is listed in the filename
|-
|Tensorflow Notebook
|tensorflow-notebook_tensorflow-#.#.#.sif
|Based on the Scipy notebook but includes Tensorflow.
Tensorflow version is listed in the filename.
|-
|
|gpu-jupyter.sif
|Includes NVIDIA CUDA along with the data science notebook.
Based on https://github.com/iot-salzburg/gpu-jupyter/
|}

===== Create a custom Jupyter Notebook image =====
If additional packages are required in the notebook that is not available as part of an existing Jupyter Notebook container image, you can create an extended and customized container image instead.

To build a custom container image, create a Singularity build definition file that references the appropriate base Jupyter notebook image. All custom Jupyter images should be based from one of the official base Jupyter notebook image for compatibility and to ensure that it will start properly within our Open OnDemand environment. As all the Jupyter notebook images are based on Ubuntu with mamba, you may install additional packages using <code>apt-get</code>, <code>mamba</code>, and <code>pip</code>.

In the following example, we will extend the <code>tensorflow-notebook</code> image with a C++ compiler (g++) and an additional pip package (corels).

Create a Singularity build file <code>custom-notebook.def</code> with the following contents:
{{Highlight|code=Bootstrap: docker
From: jupyter/tensorflow-notebook

%post
apt-get autoremove; \
apt-get update --yes; \
apt-get install --yes --no-install-recommends g++; \
pip install corels|lang=text}}
To build <code>custom-notebook.def</code>, run: <code>singularity build custom-notebook.sif custom-notebook.def</code>. This will generate a <code>custom-notebook.sif</code> singularity image file.

To use this custom image in Open OnDemand, move the newly generated <code>.sif</code> file to <code>$HOME/ondemand/jupyter</code>. In Open OnDemand, when launching a new notebook, select 'Custom Singularity Image...' on the launch page and select the appropriate <code>.sif</code> file.

==== Environment module based Jupyter notebooks ====
This is the alternative way of launching Jupyter notebooks on Open OnDemand. This option automates our previous recommended way of running Jupyter on ARC (described on our [[Jupyter Notebooks|Jupyter Notebooks page]]) by loading python from an Anaconda environment and creating a port forwarding. We recommend this option '''only''' if you have an existing workflow that uses Jupyter with modules.

When using this method, you will be asked for an initialization bash script that sets up and starts Jupyter. An example script is given below. You may modify and add to this example script to customize your Jupyter Notebook setup. If no script is given when launching the notebook or if the script is not found, this example script will be used instead.

{{Highlight|code=#!/bin/bash

# TODO: Set up your environment as required.

# Purge the module environment to avoid conflicts
module purge
module load python/anaconda3-2018.12
# TODO: Add any additional modules desired here.
module list

# As per https://rcs.ucalgary.ca/index.php/Jupyter_Notebooks
unset XDG_RUNTIME_DIR

# Launch the Jupyter Notebook Server
set -x
jupyter notebook --config="${CONFIG_FILE}"|lang=bash}}
=== RStudio Server (Container) ===
RStudio Server is a web-based version of RStudio and functions similar to Jupyter Notebooks. In our instance of Open OnDemand, RStudio Server will run within a Apptainer container environment.

After requesting for a RStudio Server session through Open OnDemand, you will see a session information box similar to the one below. Each session will use a randomised password.
[[File:RStudio Server launch on OOD.png|alt=RStudio Server launch on Open OnDemand|none|thumb|RStudio Server launch on Open OnDemand]]
Because RStudio Server runs within an Apptainer environment, some features may not work as expected.

==== Custom RStudio Container Images ====
At times R libraries need system dependencies to function correctly. This procedure is used to take a common image that is already on Arc and modify it to contain the various libraries that are required. Apptainer allows us to install packages into containers that would normally not be possible to install without administrative privileges.
* Create an apptainer definition file. This file is similar to a Dockerfile but with a different format that is specific to apptainer. Here's an example that both installs a library using apt and installs an R package that requires that library:
* Create a file modifyrstudio.def
<PRE>
Bootstrap: localimage
From: /global/software/ondemand/rstudio/rstudio_4.4.3.sif
%post
apt update
apt install -y libglpk40
Rscript -e "install.packages(c('Seurat'), repos='https://cran.rstudio.com')"
</PRE>
* Modify the file modifyrstudio.def to meet your needs
** The From value is the name of an rstudio image that is currently on the system. There are other versions available in that same directory. This is the starting point so you don't have to install Rstudio from first principles.
** The section after %post is simply a script that virtually runs as root inside the build container. You can express any configuration changes, software installations, etc. Just make sure that nothing asks for input (for example the -y to apt install means say "yes" to any questions)
* Build the image like the following. The .sif file's filename is not critical but it must be in ~/ondemand/rstudio and end with .sif or it will not appear as an image in the RStudio App:
<PRE>
apptainer build ~/ondemand/rstudio/rstudio-4.4.3-libglpk40-Seurat.sif modifyrstudio.def
</PRE>
* Once you've created the image successfully in your home directory, you can use the image by starting a new instance of the Rstudio app (you'll need to refresh the webpage with the parameters of the app before the image shows up).
* Choose "Custom Singularity Image...." from the "RStudio Server Image" dropdown box and a new dropdown will appear named "Custom Rstudio Image". In the Custom Rstudio Image dropdown you should see the file you just created with the apptainer build step above.
* Choose "Launch" as normal and after your job starts you should be met with an Rstudio interface that contains the changes you have introduced.

=== VS Code Server ===
Open OnDemand offers VS Code Server as an Interactive App. VS Code Server is a web-based version of Visual Studio Code and runs directly on the remote compute node. This is an alternative to running Visual Studio Code as a native app within the Open OnDemand Desktop.

To start a VS Code Server, go to the 'Interactive Apps' menu, then click on 'VS Code Server'.
[[File:Open OnDemand VSCodeServer.png|alt=Open OnDemand VSCodeServer|none|thumb|Open OnDemand VS Code Server]]

=== Visual Studio Code ===
As an alternative to Visual Studio Code Server, you can run Visual Studio Code from the Desktop app. This will run Visual Studio Code as a desktop application within a Linux desktop environment.

To start a VS Code, go to the 'Interactive Apps' menu, then click on 'Desktop (Container)'. After launching the desktop, navigate to 'Application' -> 'Development' -> 'Visual Studio Code'
[[File:Visual Studio Code on OOD Desktop.png|alt=Visual Studio Code on OOD Desktop|none|thumb|Visual Studio Code on OOD Desktop]]

== Troubleshooting ==

=== VNC session cannot reconnect ===
If your VNC session closed, you may need to re-connect by going back to the Open OnDemand dashboard, listing your sessions under "My Interactive Sessions" page, then click on the blue "Launch Desktop" button.

{{Support}}

[[Category:ARC]]
{{Navbox ARC}}

ARC Cluster Status

2026-06-01T17:25:26Z

Dschulz:

2026-01-22T20:25:44Z

Dschulz:

ARC Cluster Status

2026-01-16T19:49:32Z

Dschulz:

ARC Cluster Status

2026-01-16T19:47:37Z

Dschulz:

{{ARC Cluster Status}}

== System Messages ==
{{Message of the day item
| title = ⚠️ January System Updates
| date = 2023/01/01
| message =
Beginning January 16, 2023, the ARC cluster will undergo operating system updates. We shall do our utmost to minimize disruption and allow ongoing jobs to be completed. New jobs may be temporarily held from scheduling.

The ARC login node will reboot on the morning of January 16. Please save your work and log out if possible.

The upgrade is planned to be fully complete by January 20.

If you encounter any system issues, do not hesitate to let us know.

Thank you for your cooperation.
}}

{{Message of the day item
| title = System Updates Completed
| date = 2023/01/24
| message =
The upgrade has been completed. The following has been changed:
* OS Updated to Rocky Linux 8.7
* Slurm updated to 22.05.7
* Apptainer replaces Singularity
* Each job will have its own /tmp, /dev/shm, /run/user/$uid mounted

If you encounter any system issues, do not hesitate to let us know.

Thank you for your cooperation.
}}

{{Message of the day item
| title = Filesystem Issues
| date = 2023/02/28
| message =
We are currently investigating a filesystem issue that is causing filesystem slowdowns across ARC.

We will update you with more information as it becomes available.

Thank you for your patience.
}}

{{Message of the day item
| title = Filesystem Issues
| date = 2023/03/1
| message =
We are still currently investigating a filesystem issue that is causing filesystem slowdowns across ARC. Some jobs on ARC have been paused to help us find the root cause of the slowdowns.

We will update you with more information as it becomes available.

Thank you for your patience.
}}

{{Message of the day item
| title = ARC Login node reboot
| date = 2023/03/2
| message =
The ARC login node will be rebooted this afternoon for an emergency maintenance. This downtime is needed to help mitigate the filesystem slowdowns experienced on the login node. Jobs will continue running and scheduling during this time.

All logins to the ARC login node will be terminated at 3:00PM and will remain unavailable until 4:00PM.

We apologize for the inconvenience and thank you for your patience.
}}

{{Message of the day item
| title = ⚠️ Filesystem Issues
| date = 2023/03/2
| message =
We are still currently investigating a filesystem issue that is causing filesystem slowdowns on specific nodes in our MSRDC location.

We will update you with more information as it becomes available.

We apologize for the inconvenience and thank you for your patience.
}}

{{Message of the day item
| title = Filesystem Issues Resolved
| date = 2023/03/10
| message =
We have upgraded the filesystem routers in our MSRDC location to address the performance issues.

Please let us know if you experience any issues with the filesystem performance.

Thank-you for your patience.
}}

{{Message of the day item
| title = Open OnDemand reboot
| date = 2023/05/01
| message =
On May 1, 2023, the ARC Open OnDemand node will be rebooted between 5PM and 6PM. Expected downtime will be approximately 15 minutes.

If you encounter any system issues, do not hesitate to let us know.

Thank you for your cooperation.
}}

{{Message of the day item
| title = Apptainer (Singularity) on ARC Login Node
| date = 2023/06/22
| message =
Apptainer (Singularity) containers may experience an error when
running on the Arc login node. If apptainer complains that a system
administrator needs to enable user namespaces, simply run your
containers inside a job.

This is a temporary measure due to security vulnerability that will be
patched soon.

}}

{{Message of the day item
| title = Lattice, Single, cpu2013 partition changes
| date = 2023/07/13
| message =
The Lattice and Single, and cpu2013 have all been decomissioned. The Single
partition will be replaced by the nodes formerly in the cpu2013 partition but
will be called single.

}}

{{Message of the day item
| title = Open OnDemand reboot
| date = 2023/10/17
| message =
Open OnDemand will be rebooted on October 17, 2023 for an update. It will be down for up to 30 minutes.

}}

{{Message of the day item
| title = Storage Upgrade MARC/ARC cluster
| date = 2023/10/23
| message =
We will be performing storage upgrades on the MARC/ARC cluster on
November 16 and 17, 2023. To facilitate this, we will be throttling
down the number of jobs on both clusters while the upgrades are
performed
}}

{{Message of the day item
| title = Systems Operating Normally
| date = 2024/05/3
| message =
}}

{{Message of the day item
| title = Power Interruption
| date = 2024/05/07
| message = Arc Experienced an brief power outage around 11AM May 7, 2024.
Most compute nodes have or are rebooting. Most jobs running at this time
were lost. Arc administrators are actively working on restarting compute
nodes. Sorry for the inconvenience.
}}

{{Message of the day item
| title = GPU a100 Node Reservation
| date = 2024/06/03
| message = Job submissions targeted to the GPU a100 partition will be
affected by a temporary reservation on the nodes to accommodate the RCS
summer school class taking place on 2024/Jun/10. Reservation will end
shortly after. Please submit your jobs normally and the scheduler will
start them as soon as the nodes are available. Sorry for the inconvenience.
}}

{{Message of the day item
| title = GPU a100 Node Reservation Removed
| date = 2024/06/11
| message = GPU a100 Nodes in ARC have been returned to normal scheduling.
}}

{{Message of the day item
| title = Notice of Upcoming Partial Outage
| date = 2024/08/23
| message = Several compute nodes from the ARC cluster will be unavailable
between Sept 23 to Sept 27 inclusive (subject to change). All compute nodes
in cpu2019, cpu2021/2, gpu-v100 most nodes from bigmem and gpu-a100 will be
affected. These nodes will return to service as soon as the work is complete.
}}

{{Message of the day item
| title = Partial Outage Update I
| date = 2024/09/25
| message = Due to hardware issues that is blocking our original maintenance window, most compute nodes that were taken offline on Monday has been brought back online today. An additional partial outage will occur again starting next Tuesday for the same nodes.

On Tuesday, October 1, 2024, the compute nodes in cpu2019, cpu2021, cpu2022, gpu-v100, gpu-a100, and most nodes from bigmem will be unavailable until Friday October 4, 2024.

We apologise for the inconvenience.
}}

{{Message of the day item
| title = Partial Outage Update II
| date = 2024/10/04
| message = The maintenance window will be extended until at least Monday, October 7, 2024 due to a power distribution issue in our renovated data centre.

Currently, the compute nodes in cpu2019, cpu2021, cpu2022, gpu-v100, gpu-a100, and most nodes from bigmem will be unavailable until at least Monday, October 7, 2024. Affected WDF-Altis GPU nodes include: wdfgpu[1-2,6,8-12].

We apologize for the extended downtime and will update you as soon as we have additional information from our operations team.
}}

{{Message of the day item
| title = Partial Outage Update III
| date = 2024/10/07
| message = Due to technical issues beyond our control the maintenance window will be extended until at least Tuesday, October 15, 2024.

Currently, the compute nodes in cpu2019, cpu2021, cpu2022, gpu-v100, gpu-a100, and most nodes from bigmem will be unavailable until at least Tuesday, October 15, 2024. Affected WDF-Altis GPU nodes include: wdfgpu[1-2,6,8-12].

We apologize for the extended downtime and will update you as soon as we have additional information from our operations team.
}}

{{Message of the day item
| title = Normal Scheduling has resumed.
| date = 2024/10/08
| message = The ARC cluster has been successfully brought online and nodes are running jobs normally. We apologize for the extended downtime.

Please reach out to support@hpc.ucalgary.ca with any issues or concerns.
}}

{{Message of the day item
| title = Scheduled Maintenance
| date = 2024/12/11
| message = The ARC login node will be rebooted on Tuesday December 17 for scheduled maintenance. It will be down for a few minutes and return shortly. Job scheduling and jobs running on the cluster will not be affected. Thank you for understanding.

Please reach out to support@hpc.ucalgary.ca with any issues or concerns.
}}

{{Message of the day item
| title = Scheduled Maintenance and OS Update
| date = 2025/01/07
| message = The ARC cluster will be rebooted for OS updates on Monday January 20, 2025. Please make sure to save your work and log out before the reboot happens. Scheduling will be paused until the cluster is back, but queued jobs will remain in the queue and nodes will start scheduling when the cluster is ready. Thank you for understanding.

Please reach out to support@hpc.ucalgary.ca with any issues or concerns.
}}

{{Message of the day item
| title = ⚠️ Scheduled Maintenance and OS Update
| date = 2025/01/15
| message = The ARC cluster will be down for maintenance and upgrades starting 9AM Monday, January 20, 2025 through Wednesday, January 22, 2025.

For the duration of the upgrade window:
* Scheduling will be paused and new jobs will be queued. Any queued jobs will start scheduling only after the upgrade is complete.
* Access to files via the login node and arc-dtn will generally be available but intermittent. File transfers on the DTN node, including Globus file transfers, may be interrupted during this window.

Please make sure to save your work prior to this outage window to avoid any loss of work.

During this time the following changes will happen:

1. Ethernet will replace the 11 year old, unsupported Infiniband on the following partitions:
* cpu2023 (temporary)
* Parallel
* Theia/Synergy/cpu2017-bf05
* Single
Any multi-node jobs (MPI) running on these partitions will have increased latency going forward. If you run multi-node jobs, make sure to run on a partition such as cpu2019, cpu2021, cpu2022.

2. A component of the NetApp filer will be replaced. Access to /bulk will be unavailable on Wednesday, January 22, 2025.

3. The compute node operating system will be updated to Rocky Linux 8.10.

4. The Slurm scheduling system will be upgraded.

5. The Open OnDemand web portal will be upgraded.

Please reach out to support@hpc.calgary.ca with any issues or concerns.

⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️
Update Jan 18, 2025

Around 10AM Arc experienced an electrical power brownout. Some percentage (how many is unknown at this time) of the nodes lost electrical power during this time causing a loss of a number of running jobs.

Sorry for the inconvenience.

Since Arc is shutting down for maintenance on Monday Jan 20, replacement jobs will likely not start unless they request a timelimit less than the time until 8AM Monday.
⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️
}}

{{Message of the day item
| title = Maintenance Complete
| date = 2025/01/22
| message = The ARC cluster upgrade is complete

During this time the following changes happened:

1. Ethernet will replace the 11 year old, unsupported Infiniband on the following partitions:
* cpu2023 (temporary)
* Parallel
* Theia/Synergy/cpu2017-bf05
* Single
Any multi-node jobs (MPI) running on these partitions will have increased latency going forward. If you run multi-node jobs, make sure to run on a partition such as cpu2019, cpu2021, cpu2022.

2. A component of the NetApp filer was replaced successfully.

3. The compute node operating was updated to Rocky Linux 8.10.

4. The Slurm scheduling system was upgraded.

5. The Open OnDemand web portal was upgraded.

6. The Parallel partition was renamed to Legacy to show the lack of an interconnect for parallel MPI work and was restricted to maximum 4 node jobs.

Please reach out to support@hpc.calgary.ca with any issues or concerns.

Jan 23, 9:08AM

Remount complete. arc is back in full service.
}}

{{Message of the day item
| title = Module Command Upgrade
| date = 2025/02/03
| message = Upgrade of the module command

On Tuesday, February 11, 2025 the module command will be upgraded to a new verson on Arc. This should result in new capabilities and a slightly different visual experience when using the module command. Loading modules is not expected to change.
}}

{{Message of the day item
| title = Module Command Upgraded
| date = 2025/02/12
| message = The module command was upgraded

On Tuesday, February 12, 2025 the module command was upgraded to a new verson on Arc. This should result in new capabilities and a slightly different visual experience when using the module command. Loading modules should not have changed.

}}

{{Message of the day item
| title = Support email address down
| date = 2025/03/07
| message = support@hpc.ucalgary.ca Unavailable

Please be informed that our support email address (support@hpc.ucalgary.ca) for RCS is currently not working. We are working to bring it back as soon as possible. Please keep an eye on this space for updates. The clusters are working normally, but support will not receive your messages at this time. We will begin responding as soon as we can get it back.

Apologies for the inconvenience.

}}

{{Message of the day item
| title = Support email address functional
| date = 2025/03/07
| message = support@hpc.ucalgary.ca is back

support@hpc.ucalgary.ca has been repaired and RCS can be contacted there. If you had reached out for assistance in recent days without response please follow up as we may not have received your initial email.

Apologies for the inconvenience.
}}

{{Message of the day item
| title = Interactive Job Timelimit will be Enforced
| date = 2025/04/11
| message = In order to improve the scheduling and job throughput efficiency of ARC, interactive jobs will be limited to a maximum of 5 hours of runtime. Interactive jobs that are submitted with a timelimit over 5 hours will be rejected at submission time. This change will be made on Monday, April 28, 2025.
}}

{{Message of the day item
| title = Interactive Job Timelimit Is Now Enforced
| date = 2025/04/28
| message = In order to improve the scheduling and job throughput efficiency of ARC, interactive jobs are now limited to a maximum of 5 hours of runtime. Interactive jobs that are submitted with a timelimit over 5 hours will be rejected at submission time.

Apr 29, 2025
To increase the security posture of the Arc cluster administrators will be installing Trend Micro on cluster login nodes over the week starting Apr 30. Please report any inconsistencies to support@hpc.ucalgary.ca
}}

{{Message of the day item
| title = ⚠️ Bulk Filesystem Emergency Maintenance
| date = 2025/07/29
| message = The filer that provides the /bulk filesystem will be down for emergency repairs at 12 Noon on Thursday July 31. No access to files on /bulk will be possible for the duration of the multi-hour outage. Any jobs running that access /bulk will start and then pause when access to /bulk is attempted. Jobs should continue once service is restored. Sorry for the inconvenience.
}}

{{Message of the day item
| title = Bulk Filesystem Maintenance Complete
| date = 2025/08/01
| message = Maintenance on /bulk was completed successfully and all filesystems are back in service on Arc.
}}

{{Message of the day item
| title = GPU partition changes on ARC
| date = 2025/08/15
| message = New gpu-h100 and gpu-l40 partitions are available for general scheduling with new gpu hardware. gpu-v100 also has 6 fewer nodes. You can view more details about the node specs by running the arc.hardware script on the login node.
}}

{{Message of the day item
| title = Update and Bulk outage on ARC
| date = 2025/09/15
| message = ⚠️ Arc Update September 15-19

An update of the Arc cluster will begin on Sep 15. This will result in fewer resources while the compute nodes are restarted. The login node and Scheduler will be restarted on Sep 17.

⚠️ Bulk Filesystem Maintenance September 17
The filer that provides the /bulk filesystem will be down for emergency repairs at 9 AM on Wednesday Sep 17th. No access to files on /bulk will be possible for the duration of the multi-hour outage. Any jobs running that access /bulk will start and then pause when access to /bulk is attempted. Jobs should continue once service is restored. We apologize for any inconvenience.

}}

{{Message of the day item
| title = Update and Bulk outage on ARC
| date = 2025/09/18
| message = Arc update complete

The update to arc is complete.

}}

{{Message of the day item
| title = GPU partition changes on ARC
| date = 2025/09/23
| message = gpu-v100 has 6 more nodes. You can view more details about the node specs by running the arc.hardware script on the login node.
}}

{{Message of the day item
| title = Systems Operating Normally
| date = 2025/11/4
| message =
}}

{{Message of the day item
| title = Legacy compute nodes are being retired
| date = 2025/11/20
| message = ARC nodes cn[0513-1096] are being removed from the arc cluster. They will be removed from scheduling and removed from the cluster over the next while.
}}

{{Message of the day item
| title = Planned outage in Jan 2026
| date = 2025/12/26
| message = ⚠️ Arc Update Week of January 12

ARC will be going down periodically during the week of 2026/Jan/12 to allow for system maintenance. All nodes and filesystems will be unavailable for use during the outage, and filesystem performance may be variable following the upgrade for a time until the changes finish.

If you have questions or concerns, please contact support@hpc.ucalgary.ca
}}

{{Message of the day item
| title = Arc Filesystem Returned to service, scheduling outages continue
| date = 2025/01/13
| message = ⚠️ Data access now possible ⚠️

The first stage of the maintenance is complete and the filesystems are once again available.

There is still significant work to do on the cluster so jobs will not start until this maintenance is complete but clients will be able to access data via the login node and DTN.

If you have questions or concerns, please contact support@hpc.ucalgary.ca
}}

{{Message of the day item
| title = Arc Returned to service, scheduling resumed
| date = 2025/01/16
| message = Maintenance Complete

Arc has been returned to service.

If you have questions or concerns, please contact support@hpc.ucalgary.ca
}}

{{Navbox ARC}}
[[Category:ARC]]

TALC Cluster Status

2026-01-13T20:49:40Z

Dschulz:

TALC Cluster Status

2026-01-13T20:04:59Z

Dschulz:

ARC Cluster Status

2026-01-13T19:54:55Z

Dschulz:

Running alphafold3

2025-11-12T17:55:55Z

Dschulz: Updated Alphafold3 usage instructions

= Running Alphafold3 on Arc =

Alphafold3 has been compiled into an Apptainer image for use on Arc.

Alphafold3 has two separate steps, one that requires cpus only, and reads much of the large public_databases directory. The second step runs the inference and requires the use of a GPU. The pipeline step takes much longer than the inference step. It is wasteful to occupy a GPU node for the duration of the pipeline step. To run Alphafold3 efficiently we have to run it two separate times in different modes on compute nodes that have appropriate resources.

== Prerequisites ==

* Due to licensing reasons, every client who would like to use alphafold3 must register for and download the model parameters. See https://forms.gle/svvpY4u2jsHEwWYS6 to register. The modelparameters are relatively small and can be stored anywhere on Arc that you have access to and can be used by multiple jobs at the same time (you do not have to have a separate copy of it for every job).
* The alphafold3 container is stored in /global/software/alphafold3. There are example job scripts in that directory that are referenced below.
* Due to the nature of the pipeline stage, we need to split each alphafold run into separate pipeline and inference stages. If we don't do this, a valuable GPU node is tied up for a long time preventing others from using it. In our testing using an input of 120 proteins the pipeline stage took a day and the inference stage took only an hour
* The public databases that Alphafold uses have been pre-downloaded and are stored in a location for anyone on Arc to use. This location, /bulk/public/alphafold3/public_databases, is reflected in the example job scripts.

== The Pipeline stage ==

* This stage is very time consuming. Much of the time is spent waiting to load a very large amount of data from the filesystem. The example jobscript in /global/software/alphafold3 copies the /bulk/public/alphafold3/public_databases/mmcif_files to /dev/shm which is a filesystem that exists entirely in the compute node's memory (this makes it very fast). This staging of data takes 20-30 min so you'll want to run a fairly large number of proteins in one job to amortize the cost of copying the mmcif_files to the compute node.
* Download the model and put it in ./models (This is a legal/license requirement)
* Generate input data. Make sure you've split your input sequences into 5 roughly equally sized files (by number of input sequences). For the examples we'll use filenames starting with xa and ending with .json so xa*.json. The example job script runs 5 copies of alphafold at the same time on the compute node as alphafold3's pipeline stage can really only effectively make use of 4 cpus for a single job. This way the system is running multiple 4 cpu jobs at the same time.
* Create ./pipelineoutputs for the resulting files
* Run the pipeline stage on cpus by submitting runpipeline.slurm. Required Job resources:
** --mem=500GB # We need to request enough memory for the public_databases/mmcif_files plus whatever the job needs.
** -c 20 # 20 cpus
** -N 1 # 1 node

== The Inference stage ==

* This is the part of alphafold that requires a GPU.
* Compared to the pipeline stage, the inference is fairly quick.
* copy the runinference.slurm file to a directory you have write access to.
* Take the outputs from the pipeline stage (which were put in ./pipelineoutputs in the example, move/copy the files into a directory called something like inference_inputs. Note that the pipeline stage makes a directory for each of the files but the inference stage expects them to all be directly in the inference_inputs. You'll have to copy them all. You can use find to help:
cd pipelineoutputs
cp $(find -type f -name \*.json) ../inferenceinputs
* submit the runpipeline.slurm to the scheduler. Required job resources:
** -p gpu-h100
** --gres=gpu:1 # Alphafold can only use 1 gpu
** --mem=120G # This might need to be increased depending on the inputs.
** --ntasks-per-node=4

== References ==
* https://github.com/google-deepmind/alphafold3/blob/main/docs/performance.md
* https://docs.alliancecan.ca/wiki/AlphaFold3

Template:ARC Cluster Status

2025-09-18T19:55:03Z

Dschulz: /* Software pages */

Research Computing Services (RCS) is a group within the wider University of Calgary Information Technologies team that plans, manages, and supports high performance computing (HPC) systems in use by researchers throughout the University of Calgary. Our primary focus is to meet the increasing demand for engineering and scientific computation by offering a wide range of specialized services to help researchers solve highly complex real-world problems or run large scale computationally intensive workloads on our high-end HPC resources.

This RCS Wiki contains technical documentation for use by users of HPC systems operated by RCS


=== Contact us for support ===

* For general RCS/HPC inquiries, please email: [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca]
* For IT related issues (networking, VPN, email), please email: [mailto:it@ucalgary.ca it@ucalgary.ca]
* For Compute Canada specific questions: [mailto:support@tech.alliancecan.ca support@tech.alliancecan.ca]

{{Clear}}
<div class="row">
<div class="col-md-6">

== General information ==
* [[General Cluster Guidelines and Policies]]
* [[How to get an account]]
* [[Data ownership]]
* [[Connecting to RCS HPC Systems]]
* [[External collaborators]]

* [[CloudStack|Cloud/Virtual Machine Infrastructure (CloudStack)]]

* [[On-line resources for new Linux and ARC users]]
* [[Acknowledging Research Computing Services Group]]

== Cluster Guides ==
* [[ ARC Cluster Guide]] - ARC is a general purpose cluster for University of Calgary researchers.
* [[GLaDOS Cluster Guide]] - GLaDOS is a researcher-owned cluster maintained by Research Computing Services.
* [[TALC Cluster Guide]] - Teaching and Learning Cluster (TALC) is a cluster created by Research Computing Services to support academic courses and workshops.
* [[MARC Cluster Guide]] -- Medical Advanced Research Computing cluster at the University of Calgary created by Research Computing Services in 2020.

== Other services ==

* [[Jupyter Notebooks]]
* [[Open OnDemand | Open OnDemand portal]]

== Software pages ==
* [[Managing software on ARC]]
* [[Gaussian on ARC]] -- How to use Gaussian 16 on ARC.
* [[Apache Spark on ARC]]
* [[ARC Software pages]]
* [[Bioinformatics applications]]
* [[Running_alphafold3]]
</div>
<div class="col-md-6">

== Running courses on HPC resources ==
* [[TALC Cluster|TALC]] - Teaching and Learning Cluster (TALC) is a cluster created by Research Computing Services to support academic courses and workshops.
* [[TALC Terms of Use]] - Terms of use to which TALC account holders must agree to use the cluster.
* [[List of courses on TALC]] - A list of current and historical courses taught using TALC.

== Training ==
* Our [[HPC Systems]]
* [[HPC Linux topics]] - A list of topics on which RCS technical support staff can provide one-on-one or group training
* [[Courses]]
* [[Linux Introduction]]
* [[What is a scheduler?]]
* [[Running jobs]]
* [[Data storage options for UofC researchers]]
* [[Security and privacy]]
* [[How to transfer data]]

* [[UofC Services]]

* [[Book online training sessions]]

* [[How-Tos | More How-Tos]]
</div>
</div>

{{Clear}}

__NOTOC__

Running alphafold3

2025-07-29T18:35:37Z

Dschulz:

Running alphafold3

2025-07-29T17:22:14Z

Dschulz: Created page with "= Running Alphafold3 on Arc = Alphafold3 has been compiled into an Apptainer image for use on Arc. Alphafold3 has two separate steps, one that requires cpus only, and reads much of the large public_databases directory. The second step runs the inference and requires the use of a GPU. The pipeline step takes much longer than the inference step. It is wasteful to occupy a GPU node for the duration of the pipeline step. To run Alphafold3 efficiently we have to run it t..."

MARC Cluster Status

2025-06-17T19:38:54Z

Dschulz: