https://rcs.ucalgary.ca/api.php?action=feedcontributions&user=Darcy&feedformat=atomRCSWiki - User contributions [en]2024-03-29T12:42:57ZUser contributionsMediaWiki 1.39.6https://rcs.ucalgary.ca/index.php?title=ARC_Cluster_Guide&diff=2641ARC Cluster Guide2023-09-14T23:13:01Z<p>Darcy: /* ARC Cluster Storage */</p>
<hr />
<div>{{ARC Cluster Status}}<br />
<br />
{{Message Box<br />
|title=[[Support|Need Help or have other ARC Related Questions?]]<br />
|message=For all general RCS related issues, questions, or comments, please contact us at support@hpc.ucalgary.ca.<br />
|icon=Support Icon.png}}<br />
<br />
This guide gives an overview of the Advanced Research Computing (ARC) cluster at the University of Calgary and is intended to be read by new account holders getting started on ARC. This guide covers topics such as the hardware and performance characteristics, available software, usage policies and how to log in and run jobs. ARC can be used with data that a Researcher has classified as Lv1 and Lv2 as described in the UCalgary [https://www.ucalgary.ca/legal-services/sites/default/files/teams/1/Standards-Legal-Information-Security-Classification-Standard.pdf Information Security Classification Standard] <br />
<br />
== Introduction ==<br />
The ARC is a high performance compute (HPC) cluster that is available for research projects based at the University of Calgary. This compute cluster is comprised of hundreds of severs interconnected with a high bandwidth interconnect. Special resources within the cluster include nodes with large memory installed and GPUs are also available. You may learn more about ARC's hardware in the [[ARC Cluster Guide#Hardware|hardware section below]]. ARC can be accessed through a [[Linux Introduction|command line interface]] or via a web interface called Open OnDemand.<br />
<br />
This cluster can be used for running large numbers (hundreds) of concurrent serial (one core) jobs, OpenMP or other thread-based jobs, shared-memory parallel code using up to 40 or 80 threads per job (depending on the partition), distributed-memory (MPI-based) parallel code using up to hundreds of cores, or jobs that take advantage of Graphics Processing Units (GPUs).<br />
<br />
Historically, ARC is primarily comprised of older, disparate Linux-based clusters that were formerly offered to researchers from across Canada such as Breezy, Lattice, and Parallel. In addition, a large-memory compute node (Bigbyte) was salvaged from the now-retired local Storm cluster. In January 2019, a major addition to ARC with modern hardware was purchased. In 2020, compute clusters from CHGI have been migrated into ARC.<br />
<br />
=== How to Get Started ===<br />
If you have a project you think would be appropriate for ARC, please email support@hpc.ucalgary.ca and mention the intended research and software you plan to use. You must have a University of Calgary IT account in order to use ARC.<br />
* For users that do not have a University of IT account or email address, please register for one at https://itregport.ucalgary.ca/.<br />
* For users external to the University, such as for users collaborating on a research project at the University of Calgary, please contact us and mention the project leader you are collaborating with.<br />
<br />
Once your access to ARC has been granted, you will be able to immediately make use of the cluster using your University of Calgary IT account by following the [[ARC_Cluster_Guide#Using_ARC|usage guide outlined below]].<br />
<br />
== Using ARC ==<br />
<br />
{{Message Box<br />
|icon=Security Icon.png<br />
|title=Cybersecurity awareness at the U of C<br />
|message=Please note that there are typically about 950 phishing attempts targeting University of Calgary accounts each month. This is just a reminder to be careful about computer security issues, both at home and at the University. Please visit https://it.ucalgary.ca/it-security for more information, tips on secure computing, and how to report suspected security problems.}}<br />
<br />
=== Logging in ===<br />
To log in to ARC, connect using SSH to <code>arc.ucalgary.ca</code> on port <code>22</code>. Connections to ARC are accepted only from the University of Calgary network (on campus) or through the University of Calgary General VPN (off campus).<br />
<br />
See [[Connecting to RCS HPC Systems]] for more information.<br />
=== How to interact with ARC ===<br />
<br />
ARC cluster is a collection of several compute nodes connected by a high-speed network. On ARC, computations get submitted as jobs. Once submitted, the jobs are then assigned to compute nodes by the job scheduler as resources become available.<br />
<br />
[[File:Cluster.png]]<br />
<br />
You can access ARC with your UCalgary IT user credentials. Once connected, you will get placed in the ARC login node, for basic tasks such as job submission, monitor job status, manage files, edit text, etc. It is a shared resource where multiple users get connected at the same time. Thus, any intensive tasks is not allowed on the login node as it may block other potential users to connect/submit their computations. <br />
[tannistha.nandi@arc ~]$ <br />
The job scheduling system on ARC is called SLURM. On ARC, there are two SLURM commands that can allocate resources to a job under appropriate conditions: ‘salloc’ and ‘sbatch’. They both accept the same set of command line options with respect to resource allocation. <br />
<br />
'''‘salloc’''' is to launch an interactive session, typically for tasks under 5 hours. <br />
Once an interactive job session is created, you can do things like explore research datasets, start R or python sessions to test your code, compile software applications etc.<br />
<br />
a. Example 1: The following command requests for 1 cpu on 1 node for 1 task along with 1 GB of RAM for an hour. <br />
[tannistha.nandi@arc ~]$ salloc --mem=1G -c 1 -N 1 -n 1 -t 01:00:00<br />
salloc: Granted job allocation 6758015<br />
salloc: Waiting for resource configuration<br />
salloc: Nodes fc4 are ready for job<br />
[tannistha.nandi@fc4 ~]$ <br />
<br />
<br />
b. Example 2: The following command requests for 1 GPU to be used from 1 node belonging to the gpu-v100 partition along with 1 GB of RAM for 1 hour. Generic resource scheduling (--gres) is used to request for GPU resources.<br />
[tannistha.nandi@arc ~]$ salloc --mem=1G -t 01:00:00 -p gpu-v100 --gres=gpu:1<br />
salloc: Granted job allocation 6760460<br />
salloc: Waiting for resource configuration<br />
salloc: Nodes fg3 are ready for job<br />
[tannistha.nandi@fg3 ~]$<br />
<br />
Once you finish the work, type 'exit' at the command prompt to end the interactive session,<br />
[tannistha.nandi@fg3 ~]$ exit<br />
[tannistha.nandi@fg3 ~]$ salloc: Relinquishing job allocation 6760460<br />
It is to ensure that the allocated resources are released from your job and now available to other users.<br />
<br />
'''‘sbatch’''' is to submit computations as jobs to run on the cluster. You can submit a job-script.slurm via 'sbatch' for execution. <br />
[tannistha.nandi@arc ~]$ sbatch job-script.slurm<br />
When resources become available, they get allocated to this task. Batch jobs are suited for tasks that run for long periods of time without any user supervision. When the job-script terminates, the allocation is released. <br />
Please review the section on how to prepare job scripts for more information.<br />
<br />
=== Prepare job scripts ===<br />
Job scripts are text files saved with an extension '.slurm', for example, 'job-script.slurm'. <br />
A job script looks something like this:<br />
''#!/bin/bash''<br />
####### Reserve computing resources #############<br />
#SBATCH --nodes=1<br />
#SBATCH --ntasks=1<br />
#SBATCH --cpus-per-task=1<br />
#SBATCH --time=01:00:00<br />
#SBATCH --mem=1G<br />
#SBATCH --partition=cpu2019<br><br />
####### Set environment variables ###############<br />
module load python/anaconda3-2018.12<br><br />
####### Run your script #########################<br />
python myscript.py<br />
<br />
The first line contains the text "#!/bin/bash" to interpret it as a bash script.<br />
<br />
It is followed by lines that start with a '#SBATCH' to communicate with 'SLURM'. You may add as many #SBATCH directives as needed to reserve computing resources for your task. The above example requests for one cpu on a single node for 1 task along with 1GB RAM for an hour on cpu2019 partition.<br />
<br />
Next, you have to set up environment variables either by loading the modules centrally installed on ARC or export path to the software in your home directory. The above example loads an available python module.<br />
<br />
Finally, include the Linux command to execute the local script.<br />
<br />
Note that failing to specify part of a resource allocation request (most notably '''time''' and '''memory''') will result in bad resource requests as the defaults are not appropriate to most cases. Please refer to the section 'Running non-interactive jobs' for more examples.<br />
<br />
== Hardware ==<br />
Since the ARC cluster is a conglomeration of many different compute clusters, the hardware within ARC can vary widely in terms of performance and capabilities. To mitigate any compatibility issues with different hardware, we combine similar hardware into their own Slurm partition to ensure your workload runs as consistently as possible within one partition. Please carefully review the hardware specs for each of the partitions below to avoid any surprises.<br />
<br />
=== Partition Hardware Specs ===<br />
When submitting jobs to ARC, you may specify a partition that your job will run on. Please choose a partition that is most appropriate for your work.<br />
<br />
* See also [[How to find available partitions on ARC]].<br />
<br />
A few things to keep in mind when choosing a partition:<br />
* Specific workloads requiring special Intel Instruction Set Extensions may only work on newer Intel CPUs. <br />
* If working with multi-node parallel processing, ensure your software and libraries support the partition's interconnect networking.<br />
* While older partitions may be slower, they may be less busy and have little to no wait times.<br />
<br />
If you are unsure which partition to use or need assistance on selecting an appropriate partition, please see [[#Selecting_a_Partition|the Selecting a Partition Section]] below. <br />
<br />
{| class="wikitable"<br />
! Partition<br />
! Description<br />
! Nodes<br />
! CPU Cores, Model, and Year<br />
! Memory<br />
! GPU<br />
! Network<br />
|-<br />
| -<br />
| ARC Login Node<br />
| 1<br />
| 16 cores, 2x Intel(R) Xeon(R) CPU E5620 @ 2.40GHz (Westmere, 2010)<br />
| 48 GB<br />
| N/A<br />
| 40 Gbit/s InfiniBand<br />
|-<br />
| gpu-v100<br />
| GPU Parition<br />
| 13<br />
| 80 cores, 2x Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz (Skylake, 2019)<br />
| 754 GB<br />
| 2x Tesla V100-PCIE-16GB<br />
| 100 Gbit/s Omni-Path<br />
|-<br />
|gpu-a100<br />
|GPU Partition<br />
|5<br />
|40 cores, 1x Intel(R) Xeon(R) Silver 4316 CPU @ 2.30GHz (Ice Lake, 2021)<br />
|512 GB<br />
|2x GA100 A100 PCIe 80GB<br />
|100 Gbit/s Mellanox Infiniband<br />
|-<br />
|cpu2022<br />
|General Purpose Compute<br />
|52<br />
|52 cores, 2x Intel(R) Xeon(R) Gold 5320 CPU @ 2.20GHz (Ice Lake)<br />
|256 GB<br />
|N/A<br />
|40 Gbit/s InfiniBand<br />
|-<br />
| cpu2021<br />
| General Purpose Compute<br />
| 48<br />
| 48 cores, 2x Intel(R) Xeon(R) Gold 6240R CPU @ 2.40GHz (Cascade Lake, 2021)<br />
| 185 GB<br />
| N/A <br />
| 100 Gbit/s Mellanox Infiniband<br />
|-<br />
| cpu2019<br />
| General Purpose Compute<br />
| 14<br />
| 40 cores, 2x Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz (Skylake, 2019)<br />
| 190 GB<br />
| N/A <br />
| 100 Gbit/s Omni-Path<br />
|-<br />
| apophis<br />
| General Purpose Compute<br />
| 21<br />
| 40 cores, 2x Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz (Skylake, 2019)<br />
| 190 GB<br />
| N/A <br />
| 100 Gbit/s Omni-Path<br />
|-<br />
| razi<br />
| General Purpose Compute<br />
| 41<br />
| 40 cores, 2x Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz (Skylake, 2019)<br />
| 190 GB<br />
| N/A <br />
| 100 Gbit/s Omni-Path<br />
|-<br />
| bigmem<br />
| Big Memory Nodes<br />
| 2<br />
| 80 cores, 4x Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz (Skylake, 2019)<br />
| 3022 GB<br />
| N/A <br />
| 100 Gbit/s Omni-Path<br />
|-<br />
| pawson<br />
| General Purpose Compute<br />
| 13<br />
| 40 cores, 2x Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz (Skylake, 2019)<br />
| 190 GB<br />
| N/A<br />
| 100 Gbit/s Omni-Path<br />
|-<br />
|cpu2017<br />
|General Purpose Compute<br />
|14<br />
|56 cores, 2x Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz (Sandy Bridge, 2012)<br />
|256 GB<br />
|N/A<br />
|40 Gbit/s InfiniBand<br />
|-<br />
| theia<br />
| Former Theia cluster<br />
| 20<br />
| 56 cores, 2x Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz (Sandy Bridge, 2012)<br />
| 188 GB<br />
| N/A <br />
| 40 Gbit/s InfiniBand<br />
|-<br />
| cpu2013<br />
| Former hyperion cluster<br />
| 12<br />
| 32 cores, 2x Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz (Sandy Bridge, 2012)<br />
| 126 GB<br />
| N/A<br />
| 40 Gbit/s InfiniBand<br />
|-<br />
| lattice<br />
| Former Lattice cluster<br />
| 307<br />
| 8 cores, 2x Intel(R) Xeon(R) CPU L5520 @ 2.27GHz (Nehalem, 2009)<br />
| 12 GB<br />
| N/A<br />
| 40 Gbit/s InfiniBand<br />
|-<br />
| single<br />
| Former Lattice cluster<br />
| 168<br />
| 8 cores, 2x Intel(R) Xeon(R) CPU L5520 @ 2.27GHz (Nehalem, 2009)<br />
| 12 GB<br />
| N/A<br />
| 40 Gbit/s InfiniBand<br />
|-<br />
| parallel<br />
| Former Parallel Cluster<br />
| 576<br />
| 12 cores, 2x Intel(R) Xeon(R) CPU E5649 @ 2.53GHz (Westmere, 2011)<br />
| 24 GB<br />
| N/A<br />
| 40 Gbit/s InfiniBand<br />
|}<br />
<br />
===ARC Cluster Storage===<br />
Usage of ARC cluster storage is outlined by our [[ARC Storage Terms of Use]] page.<br />
<br />
{{Warning Box<br />
| title=Data Storage<br />
| message=ARC storage is not suitable for long-term or archival storage. It is not backed-up and does not have sufficient redundancy to be used as a primary storage system. It is not guaranteed to be available for the time periods that are typical of archiving.<br />
<br />
Please ensure that the only data you keep on ARC is used for active computations.<br />
<br />
For information on available campus storage options, please see [[Storage Options]].<br />
}}<br />
<br />
{{Message Box<br />
| title=No Backup Policy!<br />
| message=You are responsible for your own backups. Many researchers will have accounts with Compute Canada and may choose to back up their data there (the Project file system accessible through the Cedar cluster would often be used). <br />
<br />
Please contact us at support@hpc.ucalgary.ca if you want more information about this option.<br />
<br />
You can also back up data to your UofC OneDrive for business allocation see: https://rcs.ucalgary.ca/How_to_transfer_data#rclone:_rsync_for_cloud_storage This allocation starts at 5TB. Contact the support center for questions regarding OneDrive for Business.<br />
}}<br />
<br />
The ARC cluster has around 2 petabyte of shared disk storage available across the entire cluster as well as temporary storage local to each of the compute nodes. Please refer to the individual sections below on the capacity limitations and usage policies. <br />
<br />
Use the <code>arc.quota</code> command on ARC to determine the available space on your various volumes and home directory.<br />
<br />
{| class="wikitable"<br />
!Partition<br />
!Description<br />
!Capacity<br />
|-<br />
|<code>/home</code><br />
|User home directories<br />
|500 GB (per user)<br />
|-<br />
|<code>/work</code><br />
|Research project storage<br />
|Up to 100's of TB<br />
|-<br />
|<code>/scratch</code><br />
|Scratch space for temporary files<br />
|Up to 15 TB<br />
|-<br />
|<code>/tmp</code><br />
|Temporary space local to the compute cluster<br />
|Dependent on available storage on nodes. Verify with <code>df -h</code>.<br />
|-<br />
|<code>/dev/shm</code><br />
|Small temporary in-memory disk space local to the compute cluster<br />
|Dependent on memory size set in your Slurm job.<br />
|}<br />
====<code>/home</code>: Home file system====<br />
Each user has a directory under /home and is the default working directory when logging in to ARC. Each home directory has a per-user quota of 500 GB. This limit is fixed and cannot be increased. Researchers requiring additional storage exceeding what is available on their home directory may use <code>/work</code> and <code>/scratch</code>.<br />
<br />
Note on file sharing: Due to security concerns, permissions set using <code>chmod</code> on your home directory to allow other users to read/write to your home directory be automatically reverted by an automated system process unless an explicit exception is made. If you need to share files with other researchers on the ARC cluster, please write to support@hpc.ucalgary.ca to ask for such an exception.<br />
<br />
====<code>/scratch</code>: Scratch file system for large job-oriented storage====<br />
Associated with each job, under the <code>/scratch</code> directory, a subdirectory is created that can be referenced in job scripts as <code>/scratch/${SLURM_JOB_ID}</code>. You can use that directory for temporary files needed during the course of a job. Up to 15 TB of storage may be used, per user (total for all your jobs) in the <code>/scratch</code> file system. <br />
<br />
Data in <code>/scratch</code> associated with a given job will be deleted automatically, without exception, five days after the job finishes.<br />
<br />
====<code>/work</code>: Work file system for larger projects====<br />
If you need more space than provided in <code>/home</code> and the <code>/scratch</code> job-oriented space is not appropriate for you case, please write to support@hpc.ucalgary.ca with an explanation, including an indication of how much storage you expect to need and for how long. If approved, you will then be assigned a directory under <code>/work</code> with an appropriately large quota.<br />
<br />
====<code>/tmp</code>,<code>/var/tmp</code>: Temporary files====<br />
You may use <code>/tmp</code> or <code>/var/tmp</code> for storing temporary files generated by your job. The <code>/tmp</code> is stored on a disk local to the compute node and is not shared across the cluster. The files stored here will be removed immediately after your job terminates.<br />
<br />
==== <code>/dev/shm</code>, <code>/run/user/$uid</code>: In-memory temporary files ====<br />
<code>/dev/shm</code> and <code>/run/user/$UID</code> is writable location for temporary files backed by virtual memory. This can be used if faster I/O is required. This is ideal for workloads that require many small read/writes to share data between processes or as a fast cache. The amount of data you can write here is dependent on the amount of free memory available to your job. The files stored at these locations will be removed immediately after your job terminates.<br />
<br />
== Software ==<br />
All ARC nodes run the latest version of Rocky Linux 8 with the same set of base software packages. To maintain the stability and consistency of all nodes, any additional dependencies that your software requires must be installed under your account. For your convenience, we have packaged commonly used software packages and dependencies as modules available under <code>/global/software</code>. If your software package is not available as a module, you may also try Anaconda which allows users to manage and install custom packages in an isolated environment.<br />
<br />
For a list of available packages that have been made available, please see [[ARC Software pages]]. <br />
<br />
Please contact us at support@hpc.ucalgary.ca if you need additional software installed.<br />
<br />
==== Modules ====<br />
The setup of the environment for using some of the installed software is through the <code>module</code> command. An overview of [https://www.westgrid.ca//support/modules modules on WestGrid (external link)] is largely applicable to ARC.<br />
<br />
Software packages bundled as a module will be available under <code>/global/software</code> and can be listed with the <code>module avail</code> command.<br />
<syntaxhighlight lang="bash"><br />
$ module avail<br />
</syntaxhighlight><br />
<br />
To enable Python, load the Python module by running:<br />
<syntaxhighlight lang="bash"><br />
$ module load python/anaconda-3.6-5.1.0<br />
</syntaxhighlight><br />
<br />
To unload the Python module, run:<br />
<syntaxhighlight lang="bash"><br />
$ module remove python/anaconda-3.6-5.1.0<br />
</syntaxhighlight><br />
<br />
To see currently loaded modules, run:<br />
<syntaxhighlight lang="bash"><br />
$ module list<br />
</syntaxhighlight><br />
<br />
By default, no modules are loaded on ARC. If you wish to use a specific module, such as the Intel compilers or the Open MPI parallel programming packages, you must load the appropriate module.<br />
<br />
== Job submission ==<br />
<br />
=== Interactive Jobs ===<br />
The ARC login node may be used for such tasks as editing files, compiling programs and running short tests while developing programs. We suggest CPU intensive workloads on the login node be restricted to under 15 minutes as per [[General Cluster Guidelines and Policies|our cluster guidelines]]. For interactive workloads exceeding 15 minutes, use the '''[[Running_jobs#Interactive_jobs|salloc command]]''' to allocate an interactive session on a compute node.<br />
<br />
The default salloc allocation is 1 CPU and 1 GB of memory. Adjust this by specifying <code>-n CPU#</code> and <code>--mem Megabytes</code>. You may request up to 5 hours of CPU time for interactive jobs.<br />
salloc --time=5:00:00 --partition=cpu2019<br />
<br />
Always use salloc or srun to start an interactive job. Do not SSH directly to a compute node as SSH sessions will be refused without an active job running.<br />
<br />
<!-- This information doesn't seem that useful or relevant to running interactive jobs. Move to getting started section?<br />
ARC uses the Linux operating system. The program that responds to your typed commands and allows you to run other programs is called the Linux shell. There are several different shells available, but, by default you will use one called bash. It is useful to have some knowledge of the shell and a variety of other command-line programs that you can use to manipulate files. If you are new to Linux systems, we recommend that you work through one of the many online tutorials that are available, such as the [http://www.ee.surrey.ac.uk/Teaching/Unix/index.html UNIX Tutorial for Beginners (external link)] provided by the University of Surrey. The tutorial covers such fundamental topics, among others, as creating, renaming and deleting files and directories, how to produce a listing of your files and how to tell how much disk space you are using. For a more comprehensive introduction to Linux, see [http://linuxcommand.sourceforge.net/tlcl.php The Linux Command Line (external link)].<br />
--><br />
<br />
=== Running non-interactive jobs (batch processing) ===<br />
Production runs and longer test runs should be submitted as (non-interactive) batch jobs, in which commands to be executed are listed in a script (text file). Batch jobs scripts are submitted using the <code>sbatch</code> command, part of the Slurm job management and scheduling software. #SBATCH directive lines at the beginning of the script are used to specify the resources needed for the job (cores, memory, run time limit and any specialized hardware needed).<br />
<br />
Most of the information on the [https://docs.computecanada.ca/wiki/Running_jobs Running Jobs (external link)] page on the Compute Canada web site is also relevant for submitting and managing batch jobs and reserving processors for interactive work on ARC. One major difference between running jobs on the ARC and Compute Canada clusters is in selecting the type of hardware that should be used for a job. On ARC, you choose the hardware to use primarily by specifying a partition, as described below.<br />
<br />
=== Selecting a Partition ===<br />
There are some aspects to consider when selecting a partition including:<br />
* Resource requirements in terms of memory and CPU cores<br />
* Hardware specific requirements, such as GPU or CPU Instruction Set Extensions<br />
* Partition resource limits and potential wait time<br />
* Software support parallel processing using Message Passing Interface (MPI), OpenMP, etc.<br />
** Eg. MPI for parallel processing can distribute memory across multiple nodes, per-node memory requirements could be lower. Whereas, OpenMP or single process code that is restricted to one node would require a higher memory node.<br />
** Note: MPI code running on hardware with Omni-Path networking should be compiled with Omni-Path networking support. This is provided by loading the <code>openmpi/2.1.3-opa</code> or <code>openmpi/3.1.2-opa</code> modules prior to compiling.<br />
<br />
Since resources that are requested are reserved for your job, please request only as much CPU and memory as your job requires to avoid reducing the cluster efficiency. If you are unsure which partition to use or the specific resource requests that are appropriate for your jobs, please contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] and we would be happy to work with you.<br />
<br />
{| class="wikitable" style="width: 100%;"<br />
!Partition<br />
!Description<br />
!Cores/node<br />
!Memory Request Limit<br />
!Time Limit<br />
!GPU<br />
!Networking<br />
|-<br />
|cpu2021<br />
|General Purpose Compute<br />
|48<br />
|185,000 MB<br />
|7 days ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|cpu2019<br />
|General Purpose Compute<br />
|40<br />
|185,000 MB<br />
|7 days ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|bigmem<br />
|Big Memory Compute<br />
|80<br />
|3,000,000 MB<br />
|24 hours ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|gpu-v100<br />
|GPU Compute<br />
|80<br />
|753,000 MB<br />
|24 hours ‡<br />
|2<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|apophis&dagger;<br />
|Private Research Partition<br />
|40<br />
|185,000 MB<br />
|7 days ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|razi&dagger;<br />
|Private Research Partition<br />
|40<br />
|185,000 MB<br />
|7 days ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|pawson&dagger;<br />
|Private Research Partition<br />
|40<br />
|185,000 MB<br />
|7 days ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|sherlock&dagger;<br />
|Private Research Partition<br />
|7<br />
|185,000 MB<br />
|7 days ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|theia&dagger;<br />
|Private Research Partition<br />
|28<br />
|188,000 MB<br />
|7 days ‡<br />
|<br />
|40 Gbit/s InfiniBand<br />
|-<br />
|synergy&dagger;<br />
|Private Research Partition<br />
|14<br />
|245,000 MB<br />
|7 days ‡<br />
|<br />
|40 Gbit/s InfiniBand<br />
|-<br />
|cpu2013<br />
|Legacy General Purpose Compute<br />
|16<br />
|120000<br />
|7 days ‡<br />
|<br />
|40 Gbit/s InfiniBand<br />
|-<br />
|lattice<br />
|Legacy General Purpose Compute<br />
|8<br />
|12000<br />
|7 days ‡<br />
|<br />
|40 Gbit/s InfiniBand<br />
|-<br />
|parallel<br />
|Legacy General Purpose Compute<br />
|12<br />
|23000<br />
|7 days ‡<br />
|<br />
|40 Gbit/s InfiniBand<br />
|-<br />
|single<br />
|Legacy Single-Node Job Compute<br />
|8<br />
|12000<br />
|7 days ‡<br />
|<br />
|40 Gbit/s InfiniBand<br />
|-<br />
|cpu2021-bf24<br />
|Back-fill Compute (2021-era hardware, 24h)<br />
|48<br />
|185,000 MB<br />
|24 hours ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|cpu2019-bf05<br />
|Back-fill Compute (2019-era hardware, 5h)<br />
|40<br />
|185,000 MB<br />
|5 hours ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|cpu2017-bf05<br />
|Back-fill Compute (2017-era hardware, 5h)<br />
|14<br />
|245,000 MB<br />
|5 hours ‡<br />
|<br />
|40 Gbit/s InfiniBand<br />
|-<br />
|+ style="caption-side: bottom; text-align: left; font-weight: normal;" | &dagger; These partitions contain hardware contributed to ARC by particular researchers and should only be used by members of their research groups. However, they have generously allowed their compute nodes to be shared with others outside their research groups for short jobs. A special 'back-fill' or -bf partition is available for use by all ARC users for jobs shorter than 5 hours.<br />‡ As time limits may be changed by administrators to adjust to maintenance schedules or system load, the values given in the tables are not definitive. See the Time limits section below for commands you can use on ARC itself to determine current limits.<br />
|}<br />
<br />
==== Backfill partitions ====<br />
Backfill partitions can be used by all users on ARC for short-term jobs. The hardware backing these partitions are generously contributed by researchers. We recommend including the backfill partitions for short term jobs as it may help reduce your job's wait time and increase the overall cluster throughput.<br />
<br />
Previously, each contributing research group had their own backfill partition. Since June 2021, we have merged:<br />
<br />
* apophis-bf, pawson-bf, and razi-bf into cpu2019-bf05 <br />
* theia-bf and synergy-bf into cpu2017-bf05<br />
<br />
The naming scheme of the backfill partitions is the CPU generation year, followed by -bf and the time limit in hours. For example, cpu2017-bf05 would represent a backfill partition containing processors from 2017 with a time limit of 5 hours.<br />
<br />
==== Hardware resource and job policy limits ====<br />
In addition to the hardware limitations, please be aware that there may also be policy limits imposed on your account for each partition. These limits restrict the number of cores, nodes, or GPUs that can be used at any given time. Since the limits are applied on a partition-by-partition basis, using resources in one partition should not affect the available resources you can use in another partition.<br />
<br />
These limits can be listed by running:<br />
<syntaxhighlight lang="bash"><br />
$ sacctmgr show qos format=Name,MaxWall,MaxTRESPU%20,MaxSubmitJobs<br />
Name MaxWall MaxTRESPU MaxSubmit<br />
---------- ----------- -------------------- ---------<br />
normal 7-00:00:00 2000<br />
breezy 3-00:00:00 cpu=384 2000<br />
gpu 7-00:00:00 13000<br />
cpu2019 7-00:00:00 cpu=240 2000<br />
gpu-v100 1-00:00:00 cpu=80,gres/gpu=4 2000<br />
single 7-00:00:00 cpu=408,node=75 2000<br />
razi 7-00:00:00 2000<br />
</syntaxhighlight><br />
<br />
==== Specifying a partition in a job ====<br />
One you have decided which partitions best suits your computation, you can select one or more partition on a job-by-job basis by including the <code>partition</code> keyword for an <code>SBATCH</code> directive in your batch job. Multiple partitions should be comma separated. If you omit the partition specification, the system will try to assign your job to appropriate hardware based on other aspects of your request. <br />
<br />
In some cases, you really should specify the partition explicitly. For example, if you are running single-node jobs with thread-based parallel processing requesting 8 cores you could use:<br />
<syntaxhighlight lang="bash"><br />
#SBATCH --mem=0 ❶<br />
#SBATCH --nodes=1 ❷<br />
#SBATCH --ntasks=1 ❸<br />
#SBATCH --cpus-per-task=8 ❹<br />
#SBATCH --partition=single,lattice ❺ <br />
</syntaxhighlight><br />
<br />
A few things to mention in this example:<br />
# <code>--mem=0</code> allocates all available memory on the compute node for the job. This effectively allocates the entire node for your job.<br />
# <code>--nodes=1</code> allocates 1 node for the job<br />
# <code>--ntasks=1</code> your job has a single task<br />
# <code>--cpus-per-task=8</code> asks for 8 CPUs per task. This job in total will request 8 * 1, or 8 CPUs.<br />
# <code>--partition=single,lattice</code> specifies that this job can run on either single or lattice.<br />
Suppose that your job requires at most 8 CPU cores and 10 GB of memory. The above Slurm request would be valid and optimal since your job fits neatly in a single node on the single and parallel partition. However, if you failed to specify the partition, Slurm may try to schedule your job to a partition with larger nodes, such as cpu2019 where each node has 40 cores and 190 GB of memory. If your job is scheduled on such a node, your job will be effectively wasting 32 cores and 180 GB of memory because <code>--mem=0</code> not only requests for 190 GB on this node, but also prevents other jobs from being scheduled on the same node.<br />
<br />
If you don't specify a partition, please give greater thought to the memory specification to make sure that the scheduler will not assign your job more resources than are needed.<br />
<br />
Parameters such as '''--ntasks-per-cpu''', '''--cpus-per-task''', '''--mem''' and '''--mem-per-cpu>''' have to be adjusted according to the capabilities of the hardware also. The product of --ntasks-per-cpu and --cpus-per-task should be less than or equal to the number given in the "Cores/node" column. The '''--mem>''' parameter (or the product of '''--mem-per-cpu''' and '''--cpus-per-task''') should be less than the "Memory limit" shown. If using whole nodes, you can specify '''--mem=0''' to request the maximum amount of memory per node.<br />
<br />
===== Examples =====<br />
Here are some examples of specifying the various partitions.<br />
<br />
As mentioned in the [[#Hardware|Hardware]] section above, the ARC cluster was expanded in January 2019. To select the 40-core general purpose nodes specify:<br />
<br />
#SBATCH --partition=cpu2019<br />
<br />
To run on the Tesla V100 GPU-enabled nodes, use the '''gpu-v100''' partition. You will also need to include an SBATCH directive in the form '''--gres=gpu:n''' to specify the number of GPUs, n, that you need. For example, if the software you are running can make use of both GPUs on a gpu-v100 partition compute node, use:<br />
<br />
#SBATCH --partition=gpu-v100 --gres=gpu:2<br />
<br />
For very large memory jobs (more than 185000 MB), specify the bigmem partition:<br />
<br />
#SBATCH --partition=bigmem<br />
<br />
If the more modern computers are too busy or you have a job well-suited to run on the compute nodes described in the legacy hardware section above, choose the cpu2013, Lattice or Parallel compute nodes by specifying the corresponding partition keyword:<br />
<br />
#SBATCH --partition=cpu2013<br />
#SBATCH --partition=lattice<br />
#SBATCH --partition=parallel<br />
<br />
There is an additional partition called '''single''' that provides nodes similar to the lattice partition, but, is intended for single-node jobs. Select the single partition with<br />
<br />
#SBATCH --partition=single<br />
<br />
=== Time limits ===<br />
Use the <code>--time</code> directive to tell the job scheduler the maximum time that your job might run. For example:<br />
#SBATCH --time=hh:mm:ss<br />
<br />
You can use <code>scontrol show partitions</code> or <code>sinfo</code> to see the current maximum time that a job can run.<br />
<syntaxhighlight lang="bash" highlight="6"><br />
$ scontrol show partitions<br />
PartitionName=single <br />
AllowGroups=ALL AllowAccounts=ALL AllowQos=ALL <br />
AllocNodes=ALL Default=NO QoS=single <br />
DefaultTime=NONE DisableRootJobs=NO ExclusiveUser=NO GraceTime=0 Hidden=NO <br />
MaxNodes=UNLIMITED MaxTime=7-00:00:00 MinNodes=1 LLN=NO MaxCPUsPerNode=UNLIMITED <br />
Nodes=cn[001-168] <br />
PriorityJobFactor=1 PriorityTier=1 RootOnly=NO ReqResv=NO OverSubscribe=NO <br />
OverTimeLimit=NONE PreemptMode=OFF <br />
State=UP TotalCPUs=1344 TotalNodes=168 SelectTypeParameters=NONE <br />
DefMemPerNode=UNLIMITED MaxMemPerNode=UNLIMITED <br />
</syntaxhighlight><br />
<br />
Alternatively, with <code>sinfo</code> under the <code>TIMELIMIT</code> column:<br />
<syntaxhighlight lang="bash"><br />
$ sinfo <br />
PARTITION AVAIL TIMELIMIT NODES STATE NODELIST <br />
single up 7-00:00:00 1 drain* cn097 <br />
single up 7-00:00:00 1 maint cn002 <br />
single up 7-00:00:00 4 drain* cn[001,061,133,154] <br />
...<br />
</syntaxhighlight><br />
<br />
== Support ==<br />
{{Message Box<br />
|title=[[Support|Need Help or have other ARC Related Questions?]]<br />
|message=For all general RCS related issues, questions, or comments, please contact us at support@hpc.ucalgary.ca.<br />
|icon=Support Icon.png}}<br />
<br />
Please don't hesitate to [[Support|contact us]] directly by email if you need help using ARC or require guidance on migrating and running your workflows to ARC.<br />
<br />
[[Category:ARC]]<br />
[[Category:Guides]]</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=ARC_Cluster_Guide&diff=2640ARC Cluster Guide2023-09-14T23:12:49Z<p>Darcy: /* ARC Cluster Storage */</p>
<hr />
<div>{{ARC Cluster Status}}<br />
<br />
{{Message Box<br />
|title=[[Support|Need Help or have other ARC Related Questions?]]<br />
|message=For all general RCS related issues, questions, or comments, please contact us at support@hpc.ucalgary.ca.<br />
|icon=Support Icon.png}}<br />
<br />
This guide gives an overview of the Advanced Research Computing (ARC) cluster at the University of Calgary and is intended to be read by new account holders getting started on ARC. This guide covers topics such as the hardware and performance characteristics, available software, usage policies and how to log in and run jobs. ARC can be used with data that a Researcher has classified as Lv1 and Lv2 as described in the UCalgary [https://www.ucalgary.ca/legal-services/sites/default/files/teams/1/Standards-Legal-Information-Security-Classification-Standard.pdf Information Security Classification Standard] <br />
<br />
== Introduction ==<br />
The ARC is a high performance compute (HPC) cluster that is available for research projects based at the University of Calgary. This compute cluster is comprised of hundreds of severs interconnected with a high bandwidth interconnect. Special resources within the cluster include nodes with large memory installed and GPUs are also available. You may learn more about ARC's hardware in the [[ARC Cluster Guide#Hardware|hardware section below]]. ARC can be accessed through a [[Linux Introduction|command line interface]] or via a web interface called Open OnDemand.<br />
<br />
This cluster can be used for running large numbers (hundreds) of concurrent serial (one core) jobs, OpenMP or other thread-based jobs, shared-memory parallel code using up to 40 or 80 threads per job (depending on the partition), distributed-memory (MPI-based) parallel code using up to hundreds of cores, or jobs that take advantage of Graphics Processing Units (GPUs).<br />
<br />
Historically, ARC is primarily comprised of older, disparate Linux-based clusters that were formerly offered to researchers from across Canada such as Breezy, Lattice, and Parallel. In addition, a large-memory compute node (Bigbyte) was salvaged from the now-retired local Storm cluster. In January 2019, a major addition to ARC with modern hardware was purchased. In 2020, compute clusters from CHGI have been migrated into ARC.<br />
<br />
=== How to Get Started ===<br />
If you have a project you think would be appropriate for ARC, please email support@hpc.ucalgary.ca and mention the intended research and software you plan to use. You must have a University of Calgary IT account in order to use ARC.<br />
* For users that do not have a University of IT account or email address, please register for one at https://itregport.ucalgary.ca/.<br />
* For users external to the University, such as for users collaborating on a research project at the University of Calgary, please contact us and mention the project leader you are collaborating with.<br />
<br />
Once your access to ARC has been granted, you will be able to immediately make use of the cluster using your University of Calgary IT account by following the [[ARC_Cluster_Guide#Using_ARC|usage guide outlined below]].<br />
<br />
== Using ARC ==<br />
<br />
{{Message Box<br />
|icon=Security Icon.png<br />
|title=Cybersecurity awareness at the U of C<br />
|message=Please note that there are typically about 950 phishing attempts targeting University of Calgary accounts each month. This is just a reminder to be careful about computer security issues, both at home and at the University. Please visit https://it.ucalgary.ca/it-security for more information, tips on secure computing, and how to report suspected security problems.}}<br />
<br />
=== Logging in ===<br />
To log in to ARC, connect using SSH to <code>arc.ucalgary.ca</code> on port <code>22</code>. Connections to ARC are accepted only from the University of Calgary network (on campus) or through the University of Calgary General VPN (off campus).<br />
<br />
See [[Connecting to RCS HPC Systems]] for more information.<br />
=== How to interact with ARC ===<br />
<br />
ARC cluster is a collection of several compute nodes connected by a high-speed network. On ARC, computations get submitted as jobs. Once submitted, the jobs are then assigned to compute nodes by the job scheduler as resources become available.<br />
<br />
[[File:Cluster.png]]<br />
<br />
You can access ARC with your UCalgary IT user credentials. Once connected, you will get placed in the ARC login node, for basic tasks such as job submission, monitor job status, manage files, edit text, etc. It is a shared resource where multiple users get connected at the same time. Thus, any intensive tasks is not allowed on the login node as it may block other potential users to connect/submit their computations. <br />
[tannistha.nandi@arc ~]$ <br />
The job scheduling system on ARC is called SLURM. On ARC, there are two SLURM commands that can allocate resources to a job under appropriate conditions: ‘salloc’ and ‘sbatch’. They both accept the same set of command line options with respect to resource allocation. <br />
<br />
'''‘salloc’''' is to launch an interactive session, typically for tasks under 5 hours. <br />
Once an interactive job session is created, you can do things like explore research datasets, start R or python sessions to test your code, compile software applications etc.<br />
<br />
a. Example 1: The following command requests for 1 cpu on 1 node for 1 task along with 1 GB of RAM for an hour. <br />
[tannistha.nandi@arc ~]$ salloc --mem=1G -c 1 -N 1 -n 1 -t 01:00:00<br />
salloc: Granted job allocation 6758015<br />
salloc: Waiting for resource configuration<br />
salloc: Nodes fc4 are ready for job<br />
[tannistha.nandi@fc4 ~]$ <br />
<br />
<br />
b. Example 2: The following command requests for 1 GPU to be used from 1 node belonging to the gpu-v100 partition along with 1 GB of RAM for 1 hour. Generic resource scheduling (--gres) is used to request for GPU resources.<br />
[tannistha.nandi@arc ~]$ salloc --mem=1G -t 01:00:00 -p gpu-v100 --gres=gpu:1<br />
salloc: Granted job allocation 6760460<br />
salloc: Waiting for resource configuration<br />
salloc: Nodes fg3 are ready for job<br />
[tannistha.nandi@fg3 ~]$<br />
<br />
Once you finish the work, type 'exit' at the command prompt to end the interactive session,<br />
[tannistha.nandi@fg3 ~]$ exit<br />
[tannistha.nandi@fg3 ~]$ salloc: Relinquishing job allocation 6760460<br />
It is to ensure that the allocated resources are released from your job and now available to other users.<br />
<br />
'''‘sbatch’''' is to submit computations as jobs to run on the cluster. You can submit a job-script.slurm via 'sbatch' for execution. <br />
[tannistha.nandi@arc ~]$ sbatch job-script.slurm<br />
When resources become available, they get allocated to this task. Batch jobs are suited for tasks that run for long periods of time without any user supervision. When the job-script terminates, the allocation is released. <br />
Please review the section on how to prepare job scripts for more information.<br />
<br />
=== Prepare job scripts ===<br />
Job scripts are text files saved with an extension '.slurm', for example, 'job-script.slurm'. <br />
A job script looks something like this:<br />
''#!/bin/bash''<br />
####### Reserve computing resources #############<br />
#SBATCH --nodes=1<br />
#SBATCH --ntasks=1<br />
#SBATCH --cpus-per-task=1<br />
#SBATCH --time=01:00:00<br />
#SBATCH --mem=1G<br />
#SBATCH --partition=cpu2019<br><br />
####### Set environment variables ###############<br />
module load python/anaconda3-2018.12<br><br />
####### Run your script #########################<br />
python myscript.py<br />
<br />
The first line contains the text "#!/bin/bash" to interpret it as a bash script.<br />
<br />
It is followed by lines that start with a '#SBATCH' to communicate with 'SLURM'. You may add as many #SBATCH directives as needed to reserve computing resources for your task. The above example requests for one cpu on a single node for 1 task along with 1GB RAM for an hour on cpu2019 partition.<br />
<br />
Next, you have to set up environment variables either by loading the modules centrally installed on ARC or export path to the software in your home directory. The above example loads an available python module.<br />
<br />
Finally, include the Linux command to execute the local script.<br />
<br />
Note that failing to specify part of a resource allocation request (most notably '''time''' and '''memory''') will result in bad resource requests as the defaults are not appropriate to most cases. Please refer to the section 'Running non-interactive jobs' for more examples.<br />
<br />
== Hardware ==<br />
Since the ARC cluster is a conglomeration of many different compute clusters, the hardware within ARC can vary widely in terms of performance and capabilities. To mitigate any compatibility issues with different hardware, we combine similar hardware into their own Slurm partition to ensure your workload runs as consistently as possible within one partition. Please carefully review the hardware specs for each of the partitions below to avoid any surprises.<br />
<br />
=== Partition Hardware Specs ===<br />
When submitting jobs to ARC, you may specify a partition that your job will run on. Please choose a partition that is most appropriate for your work.<br />
<br />
* See also [[How to find available partitions on ARC]].<br />
<br />
A few things to keep in mind when choosing a partition:<br />
* Specific workloads requiring special Intel Instruction Set Extensions may only work on newer Intel CPUs. <br />
* If working with multi-node parallel processing, ensure your software and libraries support the partition's interconnect networking.<br />
* While older partitions may be slower, they may be less busy and have little to no wait times.<br />
<br />
If you are unsure which partition to use or need assistance on selecting an appropriate partition, please see [[#Selecting_a_Partition|the Selecting a Partition Section]] below. <br />
<br />
{| class="wikitable"<br />
! Partition<br />
! Description<br />
! Nodes<br />
! CPU Cores, Model, and Year<br />
! Memory<br />
! GPU<br />
! Network<br />
|-<br />
| -<br />
| ARC Login Node<br />
| 1<br />
| 16 cores, 2x Intel(R) Xeon(R) CPU E5620 @ 2.40GHz (Westmere, 2010)<br />
| 48 GB<br />
| N/A<br />
| 40 Gbit/s InfiniBand<br />
|-<br />
| gpu-v100<br />
| GPU Parition<br />
| 13<br />
| 80 cores, 2x Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz (Skylake, 2019)<br />
| 754 GB<br />
| 2x Tesla V100-PCIE-16GB<br />
| 100 Gbit/s Omni-Path<br />
|-<br />
|gpu-a100<br />
|GPU Partition<br />
|5<br />
|40 cores, 1x Intel(R) Xeon(R) Silver 4316 CPU @ 2.30GHz (Ice Lake, 2021)<br />
|512 GB<br />
|2x GA100 A100 PCIe 80GB<br />
|100 Gbit/s Mellanox Infiniband<br />
|-<br />
|cpu2022<br />
|General Purpose Compute<br />
|52<br />
|52 cores, 2x Intel(R) Xeon(R) Gold 5320 CPU @ 2.20GHz (Ice Lake)<br />
|256 GB<br />
|N/A<br />
|40 Gbit/s InfiniBand<br />
|-<br />
| cpu2021<br />
| General Purpose Compute<br />
| 48<br />
| 48 cores, 2x Intel(R) Xeon(R) Gold 6240R CPU @ 2.40GHz (Cascade Lake, 2021)<br />
| 185 GB<br />
| N/A <br />
| 100 Gbit/s Mellanox Infiniband<br />
|-<br />
| cpu2019<br />
| General Purpose Compute<br />
| 14<br />
| 40 cores, 2x Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz (Skylake, 2019)<br />
| 190 GB<br />
| N/A <br />
| 100 Gbit/s Omni-Path<br />
|-<br />
| apophis<br />
| General Purpose Compute<br />
| 21<br />
| 40 cores, 2x Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz (Skylake, 2019)<br />
| 190 GB<br />
| N/A <br />
| 100 Gbit/s Omni-Path<br />
|-<br />
| razi<br />
| General Purpose Compute<br />
| 41<br />
| 40 cores, 2x Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz (Skylake, 2019)<br />
| 190 GB<br />
| N/A <br />
| 100 Gbit/s Omni-Path<br />
|-<br />
| bigmem<br />
| Big Memory Nodes<br />
| 2<br />
| 80 cores, 4x Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz (Skylake, 2019)<br />
| 3022 GB<br />
| N/A <br />
| 100 Gbit/s Omni-Path<br />
|-<br />
| pawson<br />
| General Purpose Compute<br />
| 13<br />
| 40 cores, 2x Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz (Skylake, 2019)<br />
| 190 GB<br />
| N/A<br />
| 100 Gbit/s Omni-Path<br />
|-<br />
|cpu2017<br />
|General Purpose Compute<br />
|14<br />
|56 cores, 2x Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz (Sandy Bridge, 2012)<br />
|256 GB<br />
|N/A<br />
|40 Gbit/s InfiniBand<br />
|-<br />
| theia<br />
| Former Theia cluster<br />
| 20<br />
| 56 cores, 2x Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz (Sandy Bridge, 2012)<br />
| 188 GB<br />
| N/A <br />
| 40 Gbit/s InfiniBand<br />
|-<br />
| cpu2013<br />
| Former hyperion cluster<br />
| 12<br />
| 32 cores, 2x Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz (Sandy Bridge, 2012)<br />
| 126 GB<br />
| N/A<br />
| 40 Gbit/s InfiniBand<br />
|-<br />
| lattice<br />
| Former Lattice cluster<br />
| 307<br />
| 8 cores, 2x Intel(R) Xeon(R) CPU L5520 @ 2.27GHz (Nehalem, 2009)<br />
| 12 GB<br />
| N/A<br />
| 40 Gbit/s InfiniBand<br />
|-<br />
| single<br />
| Former Lattice cluster<br />
| 168<br />
| 8 cores, 2x Intel(R) Xeon(R) CPU L5520 @ 2.27GHz (Nehalem, 2009)<br />
| 12 GB<br />
| N/A<br />
| 40 Gbit/s InfiniBand<br />
|-<br />
| parallel<br />
| Former Parallel Cluster<br />
| 576<br />
| 12 cores, 2x Intel(R) Xeon(R) CPU E5649 @ 2.53GHz (Westmere, 2011)<br />
| 24 GB<br />
| N/A<br />
| 40 Gbit/s InfiniBand<br />
|}<br />
<br />
===ARC Cluster Storage===<br />
Usage of ARC cluster storage is outlined by our [[ARC Storage Terms of Use]] page.<br />
<br />
{{Warning Box<br />
| title=Data Storage<br />
| message=ARC storage is not suitable for long-term or archival storage. It is not backed-up and does not have sufficient redundancy to be used as a primary storage system. It is not guaranteed to be available for the time periods that are typical of archiving.<br />
<br />
Please ensure that the only data you keep on ARC is used for active computations.<br />
<br />
<br />
For information on available campus storage options, please see [[Storage Options]].<br />
}}<br />
<br />
{{Message Box<br />
| title=No Backup Policy!<br />
| message=You are responsible for your own backups. Many researchers will have accounts with Compute Canada and may choose to back up their data there (the Project file system accessible through the Cedar cluster would often be used). <br />
<br />
Please contact us at support@hpc.ucalgary.ca if you want more information about this option.<br />
<br />
You can also back up data to your UofC OneDrive for business allocation see: https://rcs.ucalgary.ca/How_to_transfer_data#rclone:_rsync_for_cloud_storage This allocation starts at 5TB. Contact the support center for questions regarding OneDrive for Business.<br />
}}<br />
<br />
The ARC cluster has around 2 petabyte of shared disk storage available across the entire cluster as well as temporary storage local to each of the compute nodes. Please refer to the individual sections below on the capacity limitations and usage policies. <br />
<br />
Use the <code>arc.quota</code> command on ARC to determine the available space on your various volumes and home directory.<br />
<br />
{| class="wikitable"<br />
!Partition<br />
!Description<br />
!Capacity<br />
|-<br />
|<code>/home</code><br />
|User home directories<br />
|500 GB (per user)<br />
|-<br />
|<code>/work</code><br />
|Research project storage<br />
|Up to 100's of TB<br />
|-<br />
|<code>/scratch</code><br />
|Scratch space for temporary files<br />
|Up to 15 TB<br />
|-<br />
|<code>/tmp</code><br />
|Temporary space local to the compute cluster<br />
|Dependent on available storage on nodes. Verify with <code>df -h</code>.<br />
|-<br />
|<code>/dev/shm</code><br />
|Small temporary in-memory disk space local to the compute cluster<br />
|Dependent on memory size set in your Slurm job.<br />
|}<br />
====<code>/home</code>: Home file system====<br />
Each user has a directory under /home and is the default working directory when logging in to ARC. Each home directory has a per-user quota of 500 GB. This limit is fixed and cannot be increased. Researchers requiring additional storage exceeding what is available on their home directory may use <code>/work</code> and <code>/scratch</code>.<br />
<br />
Note on file sharing: Due to security concerns, permissions set using <code>chmod</code> on your home directory to allow other users to read/write to your home directory be automatically reverted by an automated system process unless an explicit exception is made. If you need to share files with other researchers on the ARC cluster, please write to support@hpc.ucalgary.ca to ask for such an exception.<br />
<br />
====<code>/scratch</code>: Scratch file system for large job-oriented storage====<br />
Associated with each job, under the <code>/scratch</code> directory, a subdirectory is created that can be referenced in job scripts as <code>/scratch/${SLURM_JOB_ID}</code>. You can use that directory for temporary files needed during the course of a job. Up to 15 TB of storage may be used, per user (total for all your jobs) in the <code>/scratch</code> file system. <br />
<br />
Data in <code>/scratch</code> associated with a given job will be deleted automatically, without exception, five days after the job finishes.<br />
<br />
====<code>/work</code>: Work file system for larger projects====<br />
If you need more space than provided in <code>/home</code> and the <code>/scratch</code> job-oriented space is not appropriate for you case, please write to support@hpc.ucalgary.ca with an explanation, including an indication of how much storage you expect to need and for how long. If approved, you will then be assigned a directory under <code>/work</code> with an appropriately large quota.<br />
<br />
====<code>/tmp</code>,<code>/var/tmp</code>: Temporary files====<br />
You may use <code>/tmp</code> or <code>/var/tmp</code> for storing temporary files generated by your job. The <code>/tmp</code> is stored on a disk local to the compute node and is not shared across the cluster. The files stored here will be removed immediately after your job terminates.<br />
<br />
==== <code>/dev/shm</code>, <code>/run/user/$uid</code>: In-memory temporary files ====<br />
<code>/dev/shm</code> and <code>/run/user/$UID</code> is writable location for temporary files backed by virtual memory. This can be used if faster I/O is required. This is ideal for workloads that require many small read/writes to share data between processes or as a fast cache. The amount of data you can write here is dependent on the amount of free memory available to your job. The files stored at these locations will be removed immediately after your job terminates.<br />
<br />
== Software ==<br />
All ARC nodes run the latest version of Rocky Linux 8 with the same set of base software packages. To maintain the stability and consistency of all nodes, any additional dependencies that your software requires must be installed under your account. For your convenience, we have packaged commonly used software packages and dependencies as modules available under <code>/global/software</code>. If your software package is not available as a module, you may also try Anaconda which allows users to manage and install custom packages in an isolated environment.<br />
<br />
For a list of available packages that have been made available, please see [[ARC Software pages]]. <br />
<br />
Please contact us at support@hpc.ucalgary.ca if you need additional software installed.<br />
<br />
==== Modules ====<br />
The setup of the environment for using some of the installed software is through the <code>module</code> command. An overview of [https://www.westgrid.ca//support/modules modules on WestGrid (external link)] is largely applicable to ARC.<br />
<br />
Software packages bundled as a module will be available under <code>/global/software</code> and can be listed with the <code>module avail</code> command.<br />
<syntaxhighlight lang="bash"><br />
$ module avail<br />
</syntaxhighlight><br />
<br />
To enable Python, load the Python module by running:<br />
<syntaxhighlight lang="bash"><br />
$ module load python/anaconda-3.6-5.1.0<br />
</syntaxhighlight><br />
<br />
To unload the Python module, run:<br />
<syntaxhighlight lang="bash"><br />
$ module remove python/anaconda-3.6-5.1.0<br />
</syntaxhighlight><br />
<br />
To see currently loaded modules, run:<br />
<syntaxhighlight lang="bash"><br />
$ module list<br />
</syntaxhighlight><br />
<br />
By default, no modules are loaded on ARC. If you wish to use a specific module, such as the Intel compilers or the Open MPI parallel programming packages, you must load the appropriate module.<br />
<br />
== Job submission ==<br />
<br />
=== Interactive Jobs ===<br />
The ARC login node may be used for such tasks as editing files, compiling programs and running short tests while developing programs. We suggest CPU intensive workloads on the login node be restricted to under 15 minutes as per [[General Cluster Guidelines and Policies|our cluster guidelines]]. For interactive workloads exceeding 15 minutes, use the '''[[Running_jobs#Interactive_jobs|salloc command]]''' to allocate an interactive session on a compute node.<br />
<br />
The default salloc allocation is 1 CPU and 1 GB of memory. Adjust this by specifying <code>-n CPU#</code> and <code>--mem Megabytes</code>. You may request up to 5 hours of CPU time for interactive jobs.<br />
salloc --time=5:00:00 --partition=cpu2019<br />
<br />
Always use salloc or srun to start an interactive job. Do not SSH directly to a compute node as SSH sessions will be refused without an active job running.<br />
<br />
<!-- This information doesn't seem that useful or relevant to running interactive jobs. Move to getting started section?<br />
ARC uses the Linux operating system. The program that responds to your typed commands and allows you to run other programs is called the Linux shell. There are several different shells available, but, by default you will use one called bash. It is useful to have some knowledge of the shell and a variety of other command-line programs that you can use to manipulate files. If you are new to Linux systems, we recommend that you work through one of the many online tutorials that are available, such as the [http://www.ee.surrey.ac.uk/Teaching/Unix/index.html UNIX Tutorial for Beginners (external link)] provided by the University of Surrey. The tutorial covers such fundamental topics, among others, as creating, renaming and deleting files and directories, how to produce a listing of your files and how to tell how much disk space you are using. For a more comprehensive introduction to Linux, see [http://linuxcommand.sourceforge.net/tlcl.php The Linux Command Line (external link)].<br />
--><br />
<br />
=== Running non-interactive jobs (batch processing) ===<br />
Production runs and longer test runs should be submitted as (non-interactive) batch jobs, in which commands to be executed are listed in a script (text file). Batch jobs scripts are submitted using the <code>sbatch</code> command, part of the Slurm job management and scheduling software. #SBATCH directive lines at the beginning of the script are used to specify the resources needed for the job (cores, memory, run time limit and any specialized hardware needed).<br />
<br />
Most of the information on the [https://docs.computecanada.ca/wiki/Running_jobs Running Jobs (external link)] page on the Compute Canada web site is also relevant for submitting and managing batch jobs and reserving processors for interactive work on ARC. One major difference between running jobs on the ARC and Compute Canada clusters is in selecting the type of hardware that should be used for a job. On ARC, you choose the hardware to use primarily by specifying a partition, as described below.<br />
<br />
=== Selecting a Partition ===<br />
There are some aspects to consider when selecting a partition including:<br />
* Resource requirements in terms of memory and CPU cores<br />
* Hardware specific requirements, such as GPU or CPU Instruction Set Extensions<br />
* Partition resource limits and potential wait time<br />
* Software support parallel processing using Message Passing Interface (MPI), OpenMP, etc.<br />
** Eg. MPI for parallel processing can distribute memory across multiple nodes, per-node memory requirements could be lower. Whereas, OpenMP or single process code that is restricted to one node would require a higher memory node.<br />
** Note: MPI code running on hardware with Omni-Path networking should be compiled with Omni-Path networking support. This is provided by loading the <code>openmpi/2.1.3-opa</code> or <code>openmpi/3.1.2-opa</code> modules prior to compiling.<br />
<br />
Since resources that are requested are reserved for your job, please request only as much CPU and memory as your job requires to avoid reducing the cluster efficiency. If you are unsure which partition to use or the specific resource requests that are appropriate for your jobs, please contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] and we would be happy to work with you.<br />
<br />
{| class="wikitable" style="width: 100%;"<br />
!Partition<br />
!Description<br />
!Cores/node<br />
!Memory Request Limit<br />
!Time Limit<br />
!GPU<br />
!Networking<br />
|-<br />
|cpu2021<br />
|General Purpose Compute<br />
|48<br />
|185,000 MB<br />
|7 days ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|cpu2019<br />
|General Purpose Compute<br />
|40<br />
|185,000 MB<br />
|7 days ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|bigmem<br />
|Big Memory Compute<br />
|80<br />
|3,000,000 MB<br />
|24 hours ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|gpu-v100<br />
|GPU Compute<br />
|80<br />
|753,000 MB<br />
|24 hours ‡<br />
|2<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|apophis&dagger;<br />
|Private Research Partition<br />
|40<br />
|185,000 MB<br />
|7 days ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|razi&dagger;<br />
|Private Research Partition<br />
|40<br />
|185,000 MB<br />
|7 days ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|pawson&dagger;<br />
|Private Research Partition<br />
|40<br />
|185,000 MB<br />
|7 days ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|sherlock&dagger;<br />
|Private Research Partition<br />
|7<br />
|185,000 MB<br />
|7 days ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|theia&dagger;<br />
|Private Research Partition<br />
|28<br />
|188,000 MB<br />
|7 days ‡<br />
|<br />
|40 Gbit/s InfiniBand<br />
|-<br />
|synergy&dagger;<br />
|Private Research Partition<br />
|14<br />
|245,000 MB<br />
|7 days ‡<br />
|<br />
|40 Gbit/s InfiniBand<br />
|-<br />
|cpu2013<br />
|Legacy General Purpose Compute<br />
|16<br />
|120000<br />
|7 days ‡<br />
|<br />
|40 Gbit/s InfiniBand<br />
|-<br />
|lattice<br />
|Legacy General Purpose Compute<br />
|8<br />
|12000<br />
|7 days ‡<br />
|<br />
|40 Gbit/s InfiniBand<br />
|-<br />
|parallel<br />
|Legacy General Purpose Compute<br />
|12<br />
|23000<br />
|7 days ‡<br />
|<br />
|40 Gbit/s InfiniBand<br />
|-<br />
|single<br />
|Legacy Single-Node Job Compute<br />
|8<br />
|12000<br />
|7 days ‡<br />
|<br />
|40 Gbit/s InfiniBand<br />
|-<br />
|cpu2021-bf24<br />
|Back-fill Compute (2021-era hardware, 24h)<br />
|48<br />
|185,000 MB<br />
|24 hours ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|cpu2019-bf05<br />
|Back-fill Compute (2019-era hardware, 5h)<br />
|40<br />
|185,000 MB<br />
|5 hours ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|cpu2017-bf05<br />
|Back-fill Compute (2017-era hardware, 5h)<br />
|14<br />
|245,000 MB<br />
|5 hours ‡<br />
|<br />
|40 Gbit/s InfiniBand<br />
|-<br />
|+ style="caption-side: bottom; text-align: left; font-weight: normal;" | &dagger; These partitions contain hardware contributed to ARC by particular researchers and should only be used by members of their research groups. However, they have generously allowed their compute nodes to be shared with others outside their research groups for short jobs. A special 'back-fill' or -bf partition is available for use by all ARC users for jobs shorter than 5 hours.<br />‡ As time limits may be changed by administrators to adjust to maintenance schedules or system load, the values given in the tables are not definitive. See the Time limits section below for commands you can use on ARC itself to determine current limits.<br />
|}<br />
<br />
==== Backfill partitions ====<br />
Backfill partitions can be used by all users on ARC for short-term jobs. The hardware backing these partitions are generously contributed by researchers. We recommend including the backfill partitions for short term jobs as it may help reduce your job's wait time and increase the overall cluster throughput.<br />
<br />
Previously, each contributing research group had their own backfill partition. Since June 2021, we have merged:<br />
<br />
* apophis-bf, pawson-bf, and razi-bf into cpu2019-bf05 <br />
* theia-bf and synergy-bf into cpu2017-bf05<br />
<br />
The naming scheme of the backfill partitions is the CPU generation year, followed by -bf and the time limit in hours. For example, cpu2017-bf05 would represent a backfill partition containing processors from 2017 with a time limit of 5 hours.<br />
<br />
==== Hardware resource and job policy limits ====<br />
In addition to the hardware limitations, please be aware that there may also be policy limits imposed on your account for each partition. These limits restrict the number of cores, nodes, or GPUs that can be used at any given time. Since the limits are applied on a partition-by-partition basis, using resources in one partition should not affect the available resources you can use in another partition.<br />
<br />
These limits can be listed by running:<br />
<syntaxhighlight lang="bash"><br />
$ sacctmgr show qos format=Name,MaxWall,MaxTRESPU%20,MaxSubmitJobs<br />
Name MaxWall MaxTRESPU MaxSubmit<br />
---------- ----------- -------------------- ---------<br />
normal 7-00:00:00 2000<br />
breezy 3-00:00:00 cpu=384 2000<br />
gpu 7-00:00:00 13000<br />
cpu2019 7-00:00:00 cpu=240 2000<br />
gpu-v100 1-00:00:00 cpu=80,gres/gpu=4 2000<br />
single 7-00:00:00 cpu=408,node=75 2000<br />
razi 7-00:00:00 2000<br />
</syntaxhighlight><br />
<br />
==== Specifying a partition in a job ====<br />
One you have decided which partitions best suits your computation, you can select one or more partition on a job-by-job basis by including the <code>partition</code> keyword for an <code>SBATCH</code> directive in your batch job. Multiple partitions should be comma separated. If you omit the partition specification, the system will try to assign your job to appropriate hardware based on other aspects of your request. <br />
<br />
In some cases, you really should specify the partition explicitly. For example, if you are running single-node jobs with thread-based parallel processing requesting 8 cores you could use:<br />
<syntaxhighlight lang="bash"><br />
#SBATCH --mem=0 ❶<br />
#SBATCH --nodes=1 ❷<br />
#SBATCH --ntasks=1 ❸<br />
#SBATCH --cpus-per-task=8 ❹<br />
#SBATCH --partition=single,lattice ❺ <br />
</syntaxhighlight><br />
<br />
A few things to mention in this example:<br />
# <code>--mem=0</code> allocates all available memory on the compute node for the job. This effectively allocates the entire node for your job.<br />
# <code>--nodes=1</code> allocates 1 node for the job<br />
# <code>--ntasks=1</code> your job has a single task<br />
# <code>--cpus-per-task=8</code> asks for 8 CPUs per task. This job in total will request 8 * 1, or 8 CPUs.<br />
# <code>--partition=single,lattice</code> specifies that this job can run on either single or lattice.<br />
Suppose that your job requires at most 8 CPU cores and 10 GB of memory. The above Slurm request would be valid and optimal since your job fits neatly in a single node on the single and parallel partition. However, if you failed to specify the partition, Slurm may try to schedule your job to a partition with larger nodes, such as cpu2019 where each node has 40 cores and 190 GB of memory. If your job is scheduled on such a node, your job will be effectively wasting 32 cores and 180 GB of memory because <code>--mem=0</code> not only requests for 190 GB on this node, but also prevents other jobs from being scheduled on the same node.<br />
<br />
If you don't specify a partition, please give greater thought to the memory specification to make sure that the scheduler will not assign your job more resources than are needed.<br />
<br />
Parameters such as '''--ntasks-per-cpu''', '''--cpus-per-task''', '''--mem''' and '''--mem-per-cpu>''' have to be adjusted according to the capabilities of the hardware also. The product of --ntasks-per-cpu and --cpus-per-task should be less than or equal to the number given in the "Cores/node" column. The '''--mem>''' parameter (or the product of '''--mem-per-cpu''' and '''--cpus-per-task''') should be less than the "Memory limit" shown. If using whole nodes, you can specify '''--mem=0''' to request the maximum amount of memory per node.<br />
<br />
===== Examples =====<br />
Here are some examples of specifying the various partitions.<br />
<br />
As mentioned in the [[#Hardware|Hardware]] section above, the ARC cluster was expanded in January 2019. To select the 40-core general purpose nodes specify:<br />
<br />
#SBATCH --partition=cpu2019<br />
<br />
To run on the Tesla V100 GPU-enabled nodes, use the '''gpu-v100''' partition. You will also need to include an SBATCH directive in the form '''--gres=gpu:n''' to specify the number of GPUs, n, that you need. For example, if the software you are running can make use of both GPUs on a gpu-v100 partition compute node, use:<br />
<br />
#SBATCH --partition=gpu-v100 --gres=gpu:2<br />
<br />
For very large memory jobs (more than 185000 MB), specify the bigmem partition:<br />
<br />
#SBATCH --partition=bigmem<br />
<br />
If the more modern computers are too busy or you have a job well-suited to run on the compute nodes described in the legacy hardware section above, choose the cpu2013, Lattice or Parallel compute nodes by specifying the corresponding partition keyword:<br />
<br />
#SBATCH --partition=cpu2013<br />
#SBATCH --partition=lattice<br />
#SBATCH --partition=parallel<br />
<br />
There is an additional partition called '''single''' that provides nodes similar to the lattice partition, but, is intended for single-node jobs. Select the single partition with<br />
<br />
#SBATCH --partition=single<br />
<br />
=== Time limits ===<br />
Use the <code>--time</code> directive to tell the job scheduler the maximum time that your job might run. For example:<br />
#SBATCH --time=hh:mm:ss<br />
<br />
You can use <code>scontrol show partitions</code> or <code>sinfo</code> to see the current maximum time that a job can run.<br />
<syntaxhighlight lang="bash" highlight="6"><br />
$ scontrol show partitions<br />
PartitionName=single <br />
AllowGroups=ALL AllowAccounts=ALL AllowQos=ALL <br />
AllocNodes=ALL Default=NO QoS=single <br />
DefaultTime=NONE DisableRootJobs=NO ExclusiveUser=NO GraceTime=0 Hidden=NO <br />
MaxNodes=UNLIMITED MaxTime=7-00:00:00 MinNodes=1 LLN=NO MaxCPUsPerNode=UNLIMITED <br />
Nodes=cn[001-168] <br />
PriorityJobFactor=1 PriorityTier=1 RootOnly=NO ReqResv=NO OverSubscribe=NO <br />
OverTimeLimit=NONE PreemptMode=OFF <br />
State=UP TotalCPUs=1344 TotalNodes=168 SelectTypeParameters=NONE <br />
DefMemPerNode=UNLIMITED MaxMemPerNode=UNLIMITED <br />
</syntaxhighlight><br />
<br />
Alternatively, with <code>sinfo</code> under the <code>TIMELIMIT</code> column:<br />
<syntaxhighlight lang="bash"><br />
$ sinfo <br />
PARTITION AVAIL TIMELIMIT NODES STATE NODELIST <br />
single up 7-00:00:00 1 drain* cn097 <br />
single up 7-00:00:00 1 maint cn002 <br />
single up 7-00:00:00 4 drain* cn[001,061,133,154] <br />
...<br />
</syntaxhighlight><br />
<br />
== Support ==<br />
{{Message Box<br />
|title=[[Support|Need Help or have other ARC Related Questions?]]<br />
|message=For all general RCS related issues, questions, or comments, please contact us at support@hpc.ucalgary.ca.<br />
|icon=Support Icon.png}}<br />
<br />
Please don't hesitate to [[Support|contact us]] directly by email if you need help using ARC or require guidance on migrating and running your workflows to ARC.<br />
<br />
[[Category:ARC]]<br />
[[Category:Guides]]</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=ARC_Cluster_Guide&diff=2639ARC Cluster Guide2023-09-14T23:12:20Z<p>Darcy: /* ARC Cluster Storage */</p>
<hr />
<div>{{ARC Cluster Status}}<br />
<br />
{{Message Box<br />
|title=[[Support|Need Help or have other ARC Related Questions?]]<br />
|message=For all general RCS related issues, questions, or comments, please contact us at support@hpc.ucalgary.ca.<br />
|icon=Support Icon.png}}<br />
<br />
This guide gives an overview of the Advanced Research Computing (ARC) cluster at the University of Calgary and is intended to be read by new account holders getting started on ARC. This guide covers topics such as the hardware and performance characteristics, available software, usage policies and how to log in and run jobs. ARC can be used with data that a Researcher has classified as Lv1 and Lv2 as described in the UCalgary [https://www.ucalgary.ca/legal-services/sites/default/files/teams/1/Standards-Legal-Information-Security-Classification-Standard.pdf Information Security Classification Standard] <br />
<br />
== Introduction ==<br />
The ARC is a high performance compute (HPC) cluster that is available for research projects based at the University of Calgary. This compute cluster is comprised of hundreds of severs interconnected with a high bandwidth interconnect. Special resources within the cluster include nodes with large memory installed and GPUs are also available. You may learn more about ARC's hardware in the [[ARC Cluster Guide#Hardware|hardware section below]]. ARC can be accessed through a [[Linux Introduction|command line interface]] or via a web interface called Open OnDemand.<br />
<br />
This cluster can be used for running large numbers (hundreds) of concurrent serial (one core) jobs, OpenMP or other thread-based jobs, shared-memory parallel code using up to 40 or 80 threads per job (depending on the partition), distributed-memory (MPI-based) parallel code using up to hundreds of cores, or jobs that take advantage of Graphics Processing Units (GPUs).<br />
<br />
Historically, ARC is primarily comprised of older, disparate Linux-based clusters that were formerly offered to researchers from across Canada such as Breezy, Lattice, and Parallel. In addition, a large-memory compute node (Bigbyte) was salvaged from the now-retired local Storm cluster. In January 2019, a major addition to ARC with modern hardware was purchased. In 2020, compute clusters from CHGI have been migrated into ARC.<br />
<br />
=== How to Get Started ===<br />
If you have a project you think would be appropriate for ARC, please email support@hpc.ucalgary.ca and mention the intended research and software you plan to use. You must have a University of Calgary IT account in order to use ARC.<br />
* For users that do not have a University of IT account or email address, please register for one at https://itregport.ucalgary.ca/.<br />
* For users external to the University, such as for users collaborating on a research project at the University of Calgary, please contact us and mention the project leader you are collaborating with.<br />
<br />
Once your access to ARC has been granted, you will be able to immediately make use of the cluster using your University of Calgary IT account by following the [[ARC_Cluster_Guide#Using_ARC|usage guide outlined below]].<br />
<br />
== Using ARC ==<br />
<br />
{{Message Box<br />
|icon=Security Icon.png<br />
|title=Cybersecurity awareness at the U of C<br />
|message=Please note that there are typically about 950 phishing attempts targeting University of Calgary accounts each month. This is just a reminder to be careful about computer security issues, both at home and at the University. Please visit https://it.ucalgary.ca/it-security for more information, tips on secure computing, and how to report suspected security problems.}}<br />
<br />
=== Logging in ===<br />
To log in to ARC, connect using SSH to <code>arc.ucalgary.ca</code> on port <code>22</code>. Connections to ARC are accepted only from the University of Calgary network (on campus) or through the University of Calgary General VPN (off campus).<br />
<br />
See [[Connecting to RCS HPC Systems]] for more information.<br />
=== How to interact with ARC ===<br />
<br />
ARC cluster is a collection of several compute nodes connected by a high-speed network. On ARC, computations get submitted as jobs. Once submitted, the jobs are then assigned to compute nodes by the job scheduler as resources become available.<br />
<br />
[[File:Cluster.png]]<br />
<br />
You can access ARC with your UCalgary IT user credentials. Once connected, you will get placed in the ARC login node, for basic tasks such as job submission, monitor job status, manage files, edit text, etc. It is a shared resource where multiple users get connected at the same time. Thus, any intensive tasks is not allowed on the login node as it may block other potential users to connect/submit their computations. <br />
[tannistha.nandi@arc ~]$ <br />
The job scheduling system on ARC is called SLURM. On ARC, there are two SLURM commands that can allocate resources to a job under appropriate conditions: ‘salloc’ and ‘sbatch’. They both accept the same set of command line options with respect to resource allocation. <br />
<br />
'''‘salloc’''' is to launch an interactive session, typically for tasks under 5 hours. <br />
Once an interactive job session is created, you can do things like explore research datasets, start R or python sessions to test your code, compile software applications etc.<br />
<br />
a. Example 1: The following command requests for 1 cpu on 1 node for 1 task along with 1 GB of RAM for an hour. <br />
[tannistha.nandi@arc ~]$ salloc --mem=1G -c 1 -N 1 -n 1 -t 01:00:00<br />
salloc: Granted job allocation 6758015<br />
salloc: Waiting for resource configuration<br />
salloc: Nodes fc4 are ready for job<br />
[tannistha.nandi@fc4 ~]$ <br />
<br />
<br />
b. Example 2: The following command requests for 1 GPU to be used from 1 node belonging to the gpu-v100 partition along with 1 GB of RAM for 1 hour. Generic resource scheduling (--gres) is used to request for GPU resources.<br />
[tannistha.nandi@arc ~]$ salloc --mem=1G -t 01:00:00 -p gpu-v100 --gres=gpu:1<br />
salloc: Granted job allocation 6760460<br />
salloc: Waiting for resource configuration<br />
salloc: Nodes fg3 are ready for job<br />
[tannistha.nandi@fg3 ~]$<br />
<br />
Once you finish the work, type 'exit' at the command prompt to end the interactive session,<br />
[tannistha.nandi@fg3 ~]$ exit<br />
[tannistha.nandi@fg3 ~]$ salloc: Relinquishing job allocation 6760460<br />
It is to ensure that the allocated resources are released from your job and now available to other users.<br />
<br />
'''‘sbatch’''' is to submit computations as jobs to run on the cluster. You can submit a job-script.slurm via 'sbatch' for execution. <br />
[tannistha.nandi@arc ~]$ sbatch job-script.slurm<br />
When resources become available, they get allocated to this task. Batch jobs are suited for tasks that run for long periods of time without any user supervision. When the job-script terminates, the allocation is released. <br />
Please review the section on how to prepare job scripts for more information.<br />
<br />
=== Prepare job scripts ===<br />
Job scripts are text files saved with an extension '.slurm', for example, 'job-script.slurm'. <br />
A job script looks something like this:<br />
''#!/bin/bash''<br />
####### Reserve computing resources #############<br />
#SBATCH --nodes=1<br />
#SBATCH --ntasks=1<br />
#SBATCH --cpus-per-task=1<br />
#SBATCH --time=01:00:00<br />
#SBATCH --mem=1G<br />
#SBATCH --partition=cpu2019<br><br />
####### Set environment variables ###############<br />
module load python/anaconda3-2018.12<br><br />
####### Run your script #########################<br />
python myscript.py<br />
<br />
The first line contains the text "#!/bin/bash" to interpret it as a bash script.<br />
<br />
It is followed by lines that start with a '#SBATCH' to communicate with 'SLURM'. You may add as many #SBATCH directives as needed to reserve computing resources for your task. The above example requests for one cpu on a single node for 1 task along with 1GB RAM for an hour on cpu2019 partition.<br />
<br />
Next, you have to set up environment variables either by loading the modules centrally installed on ARC or export path to the software in your home directory. The above example loads an available python module.<br />
<br />
Finally, include the Linux command to execute the local script.<br />
<br />
Note that failing to specify part of a resource allocation request (most notably '''time''' and '''memory''') will result in bad resource requests as the defaults are not appropriate to most cases. Please refer to the section 'Running non-interactive jobs' for more examples.<br />
<br />
== Hardware ==<br />
Since the ARC cluster is a conglomeration of many different compute clusters, the hardware within ARC can vary widely in terms of performance and capabilities. To mitigate any compatibility issues with different hardware, we combine similar hardware into their own Slurm partition to ensure your workload runs as consistently as possible within one partition. Please carefully review the hardware specs for each of the partitions below to avoid any surprises.<br />
<br />
=== Partition Hardware Specs ===<br />
When submitting jobs to ARC, you may specify a partition that your job will run on. Please choose a partition that is most appropriate for your work.<br />
<br />
* See also [[How to find available partitions on ARC]].<br />
<br />
A few things to keep in mind when choosing a partition:<br />
* Specific workloads requiring special Intel Instruction Set Extensions may only work on newer Intel CPUs. <br />
* If working with multi-node parallel processing, ensure your software and libraries support the partition's interconnect networking.<br />
* While older partitions may be slower, they may be less busy and have little to no wait times.<br />
<br />
If you are unsure which partition to use or need assistance on selecting an appropriate partition, please see [[#Selecting_a_Partition|the Selecting a Partition Section]] below. <br />
<br />
{| class="wikitable"<br />
! Partition<br />
! Description<br />
! Nodes<br />
! CPU Cores, Model, and Year<br />
! Memory<br />
! GPU<br />
! Network<br />
|-<br />
| -<br />
| ARC Login Node<br />
| 1<br />
| 16 cores, 2x Intel(R) Xeon(R) CPU E5620 @ 2.40GHz (Westmere, 2010)<br />
| 48 GB<br />
| N/A<br />
| 40 Gbit/s InfiniBand<br />
|-<br />
| gpu-v100<br />
| GPU Parition<br />
| 13<br />
| 80 cores, 2x Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz (Skylake, 2019)<br />
| 754 GB<br />
| 2x Tesla V100-PCIE-16GB<br />
| 100 Gbit/s Omni-Path<br />
|-<br />
|gpu-a100<br />
|GPU Partition<br />
|5<br />
|40 cores, 1x Intel(R) Xeon(R) Silver 4316 CPU @ 2.30GHz (Ice Lake, 2021)<br />
|512 GB<br />
|2x GA100 A100 PCIe 80GB<br />
|100 Gbit/s Mellanox Infiniband<br />
|-<br />
|cpu2022<br />
|General Purpose Compute<br />
|52<br />
|52 cores, 2x Intel(R) Xeon(R) Gold 5320 CPU @ 2.20GHz (Ice Lake)<br />
|256 GB<br />
|N/A<br />
|40 Gbit/s InfiniBand<br />
|-<br />
| cpu2021<br />
| General Purpose Compute<br />
| 48<br />
| 48 cores, 2x Intel(R) Xeon(R) Gold 6240R CPU @ 2.40GHz (Cascade Lake, 2021)<br />
| 185 GB<br />
| N/A <br />
| 100 Gbit/s Mellanox Infiniband<br />
|-<br />
| cpu2019<br />
| General Purpose Compute<br />
| 14<br />
| 40 cores, 2x Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz (Skylake, 2019)<br />
| 190 GB<br />
| N/A <br />
| 100 Gbit/s Omni-Path<br />
|-<br />
| apophis<br />
| General Purpose Compute<br />
| 21<br />
| 40 cores, 2x Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz (Skylake, 2019)<br />
| 190 GB<br />
| N/A <br />
| 100 Gbit/s Omni-Path<br />
|-<br />
| razi<br />
| General Purpose Compute<br />
| 41<br />
| 40 cores, 2x Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz (Skylake, 2019)<br />
| 190 GB<br />
| N/A <br />
| 100 Gbit/s Omni-Path<br />
|-<br />
| bigmem<br />
| Big Memory Nodes<br />
| 2<br />
| 80 cores, 4x Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz (Skylake, 2019)<br />
| 3022 GB<br />
| N/A <br />
| 100 Gbit/s Omni-Path<br />
|-<br />
| pawson<br />
| General Purpose Compute<br />
| 13<br />
| 40 cores, 2x Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz (Skylake, 2019)<br />
| 190 GB<br />
| N/A<br />
| 100 Gbit/s Omni-Path<br />
|-<br />
|cpu2017<br />
|General Purpose Compute<br />
|14<br />
|56 cores, 2x Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz (Sandy Bridge, 2012)<br />
|256 GB<br />
|N/A<br />
|40 Gbit/s InfiniBand<br />
|-<br />
| theia<br />
| Former Theia cluster<br />
| 20<br />
| 56 cores, 2x Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz (Sandy Bridge, 2012)<br />
| 188 GB<br />
| N/A <br />
| 40 Gbit/s InfiniBand<br />
|-<br />
| cpu2013<br />
| Former hyperion cluster<br />
| 12<br />
| 32 cores, 2x Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz (Sandy Bridge, 2012)<br />
| 126 GB<br />
| N/A<br />
| 40 Gbit/s InfiniBand<br />
|-<br />
| lattice<br />
| Former Lattice cluster<br />
| 307<br />
| 8 cores, 2x Intel(R) Xeon(R) CPU L5520 @ 2.27GHz (Nehalem, 2009)<br />
| 12 GB<br />
| N/A<br />
| 40 Gbit/s InfiniBand<br />
|-<br />
| single<br />
| Former Lattice cluster<br />
| 168<br />
| 8 cores, 2x Intel(R) Xeon(R) CPU L5520 @ 2.27GHz (Nehalem, 2009)<br />
| 12 GB<br />
| N/A<br />
| 40 Gbit/s InfiniBand<br />
|-<br />
| parallel<br />
| Former Parallel Cluster<br />
| 576<br />
| 12 cores, 2x Intel(R) Xeon(R) CPU E5649 @ 2.53GHz (Westmere, 2011)<br />
| 24 GB<br />
| N/A<br />
| 40 Gbit/s InfiniBand<br />
|}<br />
<br />
===ARC Cluster Storage===<br />
Usage of ARC cluster storage is outlined by our [[ARC Storage Terms of Use]] page.<br />
<br />
{{Warning Box<br />
| title=Data Storage<br />
| message=ARC storage is not suitable for long-term or archival storage. It is not backed-up and does not have sufficient redundancy to be used as a primary storage system. It is not guaranteed to be available for the time periods that are typical of archiving. For information on available campus storage options, please see [[Storage Options]].<br />
<br />
Please ensure that the only data you keep on ARC is used for active computations.<br />
<br />
}}<br />
<br />
{{Message Box<br />
| title=No Backup Policy!<br />
| message=You are responsible for your own backups. Many researchers will have accounts with Compute Canada and may choose to back up their data there (the Project file system accessible through the Cedar cluster would often be used). <br />
<br />
Please contact us at support@hpc.ucalgary.ca if you want more information about this option.<br />
<br />
You can also back up data to your UofC OneDrive for business allocation see: https://rcs.ucalgary.ca/How_to_transfer_data#rclone:_rsync_for_cloud_storage This allocation starts at 5TB. Contact the support center for questions regarding OneDrive for Business.<br />
}}<br />
<br />
The ARC cluster has around 2 petabyte of shared disk storage available across the entire cluster as well as temporary storage local to each of the compute nodes. Please refer to the individual sections below on the capacity limitations and usage policies. <br />
<br />
Use the <code>arc.quota</code> command on ARC to determine the available space on your various volumes and home directory.<br />
<br />
{| class="wikitable"<br />
!Partition<br />
!Description<br />
!Capacity<br />
|-<br />
|<code>/home</code><br />
|User home directories<br />
|500 GB (per user)<br />
|-<br />
|<code>/work</code><br />
|Research project storage<br />
|Up to 100's of TB<br />
|-<br />
|<code>/scratch</code><br />
|Scratch space for temporary files<br />
|Up to 15 TB<br />
|-<br />
|<code>/tmp</code><br />
|Temporary space local to the compute cluster<br />
|Dependent on available storage on nodes. Verify with <code>df -h</code>.<br />
|-<br />
|<code>/dev/shm</code><br />
|Small temporary in-memory disk space local to the compute cluster<br />
|Dependent on memory size set in your Slurm job.<br />
|}<br />
====<code>/home</code>: Home file system====<br />
Each user has a directory under /home and is the default working directory when logging in to ARC. Each home directory has a per-user quota of 500 GB. This limit is fixed and cannot be increased. Researchers requiring additional storage exceeding what is available on their home directory may use <code>/work</code> and <code>/scratch</code>.<br />
<br />
Note on file sharing: Due to security concerns, permissions set using <code>chmod</code> on your home directory to allow other users to read/write to your home directory be automatically reverted by an automated system process unless an explicit exception is made. If you need to share files with other researchers on the ARC cluster, please write to support@hpc.ucalgary.ca to ask for such an exception.<br />
<br />
====<code>/scratch</code>: Scratch file system for large job-oriented storage====<br />
Associated with each job, under the <code>/scratch</code> directory, a subdirectory is created that can be referenced in job scripts as <code>/scratch/${SLURM_JOB_ID}</code>. You can use that directory for temporary files needed during the course of a job. Up to 15 TB of storage may be used, per user (total for all your jobs) in the <code>/scratch</code> file system. <br />
<br />
Data in <code>/scratch</code> associated with a given job will be deleted automatically, without exception, five days after the job finishes.<br />
<br />
====<code>/work</code>: Work file system for larger projects====<br />
If you need more space than provided in <code>/home</code> and the <code>/scratch</code> job-oriented space is not appropriate for you case, please write to support@hpc.ucalgary.ca with an explanation, including an indication of how much storage you expect to need and for how long. If approved, you will then be assigned a directory under <code>/work</code> with an appropriately large quota.<br />
<br />
====<code>/tmp</code>,<code>/var/tmp</code>: Temporary files====<br />
You may use <code>/tmp</code> or <code>/var/tmp</code> for storing temporary files generated by your job. The <code>/tmp</code> is stored on a disk local to the compute node and is not shared across the cluster. The files stored here will be removed immediately after your job terminates.<br />
<br />
==== <code>/dev/shm</code>, <code>/run/user/$uid</code>: In-memory temporary files ====<br />
<code>/dev/shm</code> and <code>/run/user/$UID</code> is writable location for temporary files backed by virtual memory. This can be used if faster I/O is required. This is ideal for workloads that require many small read/writes to share data between processes or as a fast cache. The amount of data you can write here is dependent on the amount of free memory available to your job. The files stored at these locations will be removed immediately after your job terminates.<br />
<br />
== Software ==<br />
All ARC nodes run the latest version of Rocky Linux 8 with the same set of base software packages. To maintain the stability and consistency of all nodes, any additional dependencies that your software requires must be installed under your account. For your convenience, we have packaged commonly used software packages and dependencies as modules available under <code>/global/software</code>. If your software package is not available as a module, you may also try Anaconda which allows users to manage and install custom packages in an isolated environment.<br />
<br />
For a list of available packages that have been made available, please see [[ARC Software pages]]. <br />
<br />
Please contact us at support@hpc.ucalgary.ca if you need additional software installed.<br />
<br />
==== Modules ====<br />
The setup of the environment for using some of the installed software is through the <code>module</code> command. An overview of [https://www.westgrid.ca//support/modules modules on WestGrid (external link)] is largely applicable to ARC.<br />
<br />
Software packages bundled as a module will be available under <code>/global/software</code> and can be listed with the <code>module avail</code> command.<br />
<syntaxhighlight lang="bash"><br />
$ module avail<br />
</syntaxhighlight><br />
<br />
To enable Python, load the Python module by running:<br />
<syntaxhighlight lang="bash"><br />
$ module load python/anaconda-3.6-5.1.0<br />
</syntaxhighlight><br />
<br />
To unload the Python module, run:<br />
<syntaxhighlight lang="bash"><br />
$ module remove python/anaconda-3.6-5.1.0<br />
</syntaxhighlight><br />
<br />
To see currently loaded modules, run:<br />
<syntaxhighlight lang="bash"><br />
$ module list<br />
</syntaxhighlight><br />
<br />
By default, no modules are loaded on ARC. If you wish to use a specific module, such as the Intel compilers or the Open MPI parallel programming packages, you must load the appropriate module.<br />
<br />
== Job submission ==<br />
<br />
=== Interactive Jobs ===<br />
The ARC login node may be used for such tasks as editing files, compiling programs and running short tests while developing programs. We suggest CPU intensive workloads on the login node be restricted to under 15 minutes as per [[General Cluster Guidelines and Policies|our cluster guidelines]]. For interactive workloads exceeding 15 minutes, use the '''[[Running_jobs#Interactive_jobs|salloc command]]''' to allocate an interactive session on a compute node.<br />
<br />
The default salloc allocation is 1 CPU and 1 GB of memory. Adjust this by specifying <code>-n CPU#</code> and <code>--mem Megabytes</code>. You may request up to 5 hours of CPU time for interactive jobs.<br />
salloc --time=5:00:00 --partition=cpu2019<br />
<br />
Always use salloc or srun to start an interactive job. Do not SSH directly to a compute node as SSH sessions will be refused without an active job running.<br />
<br />
<!-- This information doesn't seem that useful or relevant to running interactive jobs. Move to getting started section?<br />
ARC uses the Linux operating system. The program that responds to your typed commands and allows you to run other programs is called the Linux shell. There are several different shells available, but, by default you will use one called bash. It is useful to have some knowledge of the shell and a variety of other command-line programs that you can use to manipulate files. If you are new to Linux systems, we recommend that you work through one of the many online tutorials that are available, such as the [http://www.ee.surrey.ac.uk/Teaching/Unix/index.html UNIX Tutorial for Beginners (external link)] provided by the University of Surrey. The tutorial covers such fundamental topics, among others, as creating, renaming and deleting files and directories, how to produce a listing of your files and how to tell how much disk space you are using. For a more comprehensive introduction to Linux, see [http://linuxcommand.sourceforge.net/tlcl.php The Linux Command Line (external link)].<br />
--><br />
<br />
=== Running non-interactive jobs (batch processing) ===<br />
Production runs and longer test runs should be submitted as (non-interactive) batch jobs, in which commands to be executed are listed in a script (text file). Batch jobs scripts are submitted using the <code>sbatch</code> command, part of the Slurm job management and scheduling software. #SBATCH directive lines at the beginning of the script are used to specify the resources needed for the job (cores, memory, run time limit and any specialized hardware needed).<br />
<br />
Most of the information on the [https://docs.computecanada.ca/wiki/Running_jobs Running Jobs (external link)] page on the Compute Canada web site is also relevant for submitting and managing batch jobs and reserving processors for interactive work on ARC. One major difference between running jobs on the ARC and Compute Canada clusters is in selecting the type of hardware that should be used for a job. On ARC, you choose the hardware to use primarily by specifying a partition, as described below.<br />
<br />
=== Selecting a Partition ===<br />
There are some aspects to consider when selecting a partition including:<br />
* Resource requirements in terms of memory and CPU cores<br />
* Hardware specific requirements, such as GPU or CPU Instruction Set Extensions<br />
* Partition resource limits and potential wait time<br />
* Software support parallel processing using Message Passing Interface (MPI), OpenMP, etc.<br />
** Eg. MPI for parallel processing can distribute memory across multiple nodes, per-node memory requirements could be lower. Whereas, OpenMP or single process code that is restricted to one node would require a higher memory node.<br />
** Note: MPI code running on hardware with Omni-Path networking should be compiled with Omni-Path networking support. This is provided by loading the <code>openmpi/2.1.3-opa</code> or <code>openmpi/3.1.2-opa</code> modules prior to compiling.<br />
<br />
Since resources that are requested are reserved for your job, please request only as much CPU and memory as your job requires to avoid reducing the cluster efficiency. If you are unsure which partition to use or the specific resource requests that are appropriate for your jobs, please contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] and we would be happy to work with you.<br />
<br />
{| class="wikitable" style="width: 100%;"<br />
!Partition<br />
!Description<br />
!Cores/node<br />
!Memory Request Limit<br />
!Time Limit<br />
!GPU<br />
!Networking<br />
|-<br />
|cpu2021<br />
|General Purpose Compute<br />
|48<br />
|185,000 MB<br />
|7 days ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|cpu2019<br />
|General Purpose Compute<br />
|40<br />
|185,000 MB<br />
|7 days ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|bigmem<br />
|Big Memory Compute<br />
|80<br />
|3,000,000 MB<br />
|24 hours ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|gpu-v100<br />
|GPU Compute<br />
|80<br />
|753,000 MB<br />
|24 hours ‡<br />
|2<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|apophis&dagger;<br />
|Private Research Partition<br />
|40<br />
|185,000 MB<br />
|7 days ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|razi&dagger;<br />
|Private Research Partition<br />
|40<br />
|185,000 MB<br />
|7 days ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|pawson&dagger;<br />
|Private Research Partition<br />
|40<br />
|185,000 MB<br />
|7 days ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|sherlock&dagger;<br />
|Private Research Partition<br />
|7<br />
|185,000 MB<br />
|7 days ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|theia&dagger;<br />
|Private Research Partition<br />
|28<br />
|188,000 MB<br />
|7 days ‡<br />
|<br />
|40 Gbit/s InfiniBand<br />
|-<br />
|synergy&dagger;<br />
|Private Research Partition<br />
|14<br />
|245,000 MB<br />
|7 days ‡<br />
|<br />
|40 Gbit/s InfiniBand<br />
|-<br />
|cpu2013<br />
|Legacy General Purpose Compute<br />
|16<br />
|120000<br />
|7 days ‡<br />
|<br />
|40 Gbit/s InfiniBand<br />
|-<br />
|lattice<br />
|Legacy General Purpose Compute<br />
|8<br />
|12000<br />
|7 days ‡<br />
|<br />
|40 Gbit/s InfiniBand<br />
|-<br />
|parallel<br />
|Legacy General Purpose Compute<br />
|12<br />
|23000<br />
|7 days ‡<br />
|<br />
|40 Gbit/s InfiniBand<br />
|-<br />
|single<br />
|Legacy Single-Node Job Compute<br />
|8<br />
|12000<br />
|7 days ‡<br />
|<br />
|40 Gbit/s InfiniBand<br />
|-<br />
|cpu2021-bf24<br />
|Back-fill Compute (2021-era hardware, 24h)<br />
|48<br />
|185,000 MB<br />
|24 hours ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|cpu2019-bf05<br />
|Back-fill Compute (2019-era hardware, 5h)<br />
|40<br />
|185,000 MB<br />
|5 hours ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|cpu2017-bf05<br />
|Back-fill Compute (2017-era hardware, 5h)<br />
|14<br />
|245,000 MB<br />
|5 hours ‡<br />
|<br />
|40 Gbit/s InfiniBand<br />
|-<br />
|+ style="caption-side: bottom; text-align: left; font-weight: normal;" | &dagger; These partitions contain hardware contributed to ARC by particular researchers and should only be used by members of their research groups. However, they have generously allowed their compute nodes to be shared with others outside their research groups for short jobs. A special 'back-fill' or -bf partition is available for use by all ARC users for jobs shorter than 5 hours.<br />‡ As time limits may be changed by administrators to adjust to maintenance schedules or system load, the values given in the tables are not definitive. See the Time limits section below for commands you can use on ARC itself to determine current limits.<br />
|}<br />
<br />
==== Backfill partitions ====<br />
Backfill partitions can be used by all users on ARC for short-term jobs. The hardware backing these partitions are generously contributed by researchers. We recommend including the backfill partitions for short term jobs as it may help reduce your job's wait time and increase the overall cluster throughput.<br />
<br />
Previously, each contributing research group had their own backfill partition. Since June 2021, we have merged:<br />
<br />
* apophis-bf, pawson-bf, and razi-bf into cpu2019-bf05 <br />
* theia-bf and synergy-bf into cpu2017-bf05<br />
<br />
The naming scheme of the backfill partitions is the CPU generation year, followed by -bf and the time limit in hours. For example, cpu2017-bf05 would represent a backfill partition containing processors from 2017 with a time limit of 5 hours.<br />
<br />
==== Hardware resource and job policy limits ====<br />
In addition to the hardware limitations, please be aware that there may also be policy limits imposed on your account for each partition. These limits restrict the number of cores, nodes, or GPUs that can be used at any given time. Since the limits are applied on a partition-by-partition basis, using resources in one partition should not affect the available resources you can use in another partition.<br />
<br />
These limits can be listed by running:<br />
<syntaxhighlight lang="bash"><br />
$ sacctmgr show qos format=Name,MaxWall,MaxTRESPU%20,MaxSubmitJobs<br />
Name MaxWall MaxTRESPU MaxSubmit<br />
---------- ----------- -------------------- ---------<br />
normal 7-00:00:00 2000<br />
breezy 3-00:00:00 cpu=384 2000<br />
gpu 7-00:00:00 13000<br />
cpu2019 7-00:00:00 cpu=240 2000<br />
gpu-v100 1-00:00:00 cpu=80,gres/gpu=4 2000<br />
single 7-00:00:00 cpu=408,node=75 2000<br />
razi 7-00:00:00 2000<br />
</syntaxhighlight><br />
<br />
==== Specifying a partition in a job ====<br />
One you have decided which partitions best suits your computation, you can select one or more partition on a job-by-job basis by including the <code>partition</code> keyword for an <code>SBATCH</code> directive in your batch job. Multiple partitions should be comma separated. If you omit the partition specification, the system will try to assign your job to appropriate hardware based on other aspects of your request. <br />
<br />
In some cases, you really should specify the partition explicitly. For example, if you are running single-node jobs with thread-based parallel processing requesting 8 cores you could use:<br />
<syntaxhighlight lang="bash"><br />
#SBATCH --mem=0 ❶<br />
#SBATCH --nodes=1 ❷<br />
#SBATCH --ntasks=1 ❸<br />
#SBATCH --cpus-per-task=8 ❹<br />
#SBATCH --partition=single,lattice ❺ <br />
</syntaxhighlight><br />
<br />
A few things to mention in this example:<br />
# <code>--mem=0</code> allocates all available memory on the compute node for the job. This effectively allocates the entire node for your job.<br />
# <code>--nodes=1</code> allocates 1 node for the job<br />
# <code>--ntasks=1</code> your job has a single task<br />
# <code>--cpus-per-task=8</code> asks for 8 CPUs per task. This job in total will request 8 * 1, or 8 CPUs.<br />
# <code>--partition=single,lattice</code> specifies that this job can run on either single or lattice.<br />
Suppose that your job requires at most 8 CPU cores and 10 GB of memory. The above Slurm request would be valid and optimal since your job fits neatly in a single node on the single and parallel partition. However, if you failed to specify the partition, Slurm may try to schedule your job to a partition with larger nodes, such as cpu2019 where each node has 40 cores and 190 GB of memory. If your job is scheduled on such a node, your job will be effectively wasting 32 cores and 180 GB of memory because <code>--mem=0</code> not only requests for 190 GB on this node, but also prevents other jobs from being scheduled on the same node.<br />
<br />
If you don't specify a partition, please give greater thought to the memory specification to make sure that the scheduler will not assign your job more resources than are needed.<br />
<br />
Parameters such as '''--ntasks-per-cpu''', '''--cpus-per-task''', '''--mem''' and '''--mem-per-cpu>''' have to be adjusted according to the capabilities of the hardware also. The product of --ntasks-per-cpu and --cpus-per-task should be less than or equal to the number given in the "Cores/node" column. The '''--mem>''' parameter (or the product of '''--mem-per-cpu''' and '''--cpus-per-task''') should be less than the "Memory limit" shown. If using whole nodes, you can specify '''--mem=0''' to request the maximum amount of memory per node.<br />
<br />
===== Examples =====<br />
Here are some examples of specifying the various partitions.<br />
<br />
As mentioned in the [[#Hardware|Hardware]] section above, the ARC cluster was expanded in January 2019. To select the 40-core general purpose nodes specify:<br />
<br />
#SBATCH --partition=cpu2019<br />
<br />
To run on the Tesla V100 GPU-enabled nodes, use the '''gpu-v100''' partition. You will also need to include an SBATCH directive in the form '''--gres=gpu:n''' to specify the number of GPUs, n, that you need. For example, if the software you are running can make use of both GPUs on a gpu-v100 partition compute node, use:<br />
<br />
#SBATCH --partition=gpu-v100 --gres=gpu:2<br />
<br />
For very large memory jobs (more than 185000 MB), specify the bigmem partition:<br />
<br />
#SBATCH --partition=bigmem<br />
<br />
If the more modern computers are too busy or you have a job well-suited to run on the compute nodes described in the legacy hardware section above, choose the cpu2013, Lattice or Parallel compute nodes by specifying the corresponding partition keyword:<br />
<br />
#SBATCH --partition=cpu2013<br />
#SBATCH --partition=lattice<br />
#SBATCH --partition=parallel<br />
<br />
There is an additional partition called '''single''' that provides nodes similar to the lattice partition, but, is intended for single-node jobs. Select the single partition with<br />
<br />
#SBATCH --partition=single<br />
<br />
=== Time limits ===<br />
Use the <code>--time</code> directive to tell the job scheduler the maximum time that your job might run. For example:<br />
#SBATCH --time=hh:mm:ss<br />
<br />
You can use <code>scontrol show partitions</code> or <code>sinfo</code> to see the current maximum time that a job can run.<br />
<syntaxhighlight lang="bash" highlight="6"><br />
$ scontrol show partitions<br />
PartitionName=single <br />
AllowGroups=ALL AllowAccounts=ALL AllowQos=ALL <br />
AllocNodes=ALL Default=NO QoS=single <br />
DefaultTime=NONE DisableRootJobs=NO ExclusiveUser=NO GraceTime=0 Hidden=NO <br />
MaxNodes=UNLIMITED MaxTime=7-00:00:00 MinNodes=1 LLN=NO MaxCPUsPerNode=UNLIMITED <br />
Nodes=cn[001-168] <br />
PriorityJobFactor=1 PriorityTier=1 RootOnly=NO ReqResv=NO OverSubscribe=NO <br />
OverTimeLimit=NONE PreemptMode=OFF <br />
State=UP TotalCPUs=1344 TotalNodes=168 SelectTypeParameters=NONE <br />
DefMemPerNode=UNLIMITED MaxMemPerNode=UNLIMITED <br />
</syntaxhighlight><br />
<br />
Alternatively, with <code>sinfo</code> under the <code>TIMELIMIT</code> column:<br />
<syntaxhighlight lang="bash"><br />
$ sinfo <br />
PARTITION AVAIL TIMELIMIT NODES STATE NODELIST <br />
single up 7-00:00:00 1 drain* cn097 <br />
single up 7-00:00:00 1 maint cn002 <br />
single up 7-00:00:00 4 drain* cn[001,061,133,154] <br />
...<br />
</syntaxhighlight><br />
<br />
== Support ==<br />
{{Message Box<br />
|title=[[Support|Need Help or have other ARC Related Questions?]]<br />
|message=For all general RCS related issues, questions, or comments, please contact us at support@hpc.ucalgary.ca.<br />
|icon=Support Icon.png}}<br />
<br />
Please don't hesitate to [[Support|contact us]] directly by email if you need help using ARC or require guidance on migrating and running your workflows to ARC.<br />
<br />
[[Category:ARC]]<br />
[[Category:Guides]]</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=ARC_Cluster_Guide&diff=2638ARC Cluster Guide2023-09-14T23:07:50Z<p>Darcy: /* ARC Cluster Storage */</p>
<hr />
<div>{{ARC Cluster Status}}<br />
<br />
{{Message Box<br />
|title=[[Support|Need Help or have other ARC Related Questions?]]<br />
|message=For all general RCS related issues, questions, or comments, please contact us at support@hpc.ucalgary.ca.<br />
|icon=Support Icon.png}}<br />
<br />
This guide gives an overview of the Advanced Research Computing (ARC) cluster at the University of Calgary and is intended to be read by new account holders getting started on ARC. This guide covers topics such as the hardware and performance characteristics, available software, usage policies and how to log in and run jobs. ARC can be used with data that a Researcher has classified as Lv1 and Lv2 as described in the UCalgary [https://www.ucalgary.ca/legal-services/sites/default/files/teams/1/Standards-Legal-Information-Security-Classification-Standard.pdf Information Security Classification Standard] <br />
<br />
== Introduction ==<br />
The ARC is a high performance compute (HPC) cluster that is available for research projects based at the University of Calgary. This compute cluster is comprised of hundreds of severs interconnected with a high bandwidth interconnect. Special resources within the cluster include nodes with large memory installed and GPUs are also available. You may learn more about ARC's hardware in the [[ARC Cluster Guide#Hardware|hardware section below]]. ARC can be accessed through a [[Linux Introduction|command line interface]] or via a web interface called Open OnDemand.<br />
<br />
This cluster can be used for running large numbers (hundreds) of concurrent serial (one core) jobs, OpenMP or other thread-based jobs, shared-memory parallel code using up to 40 or 80 threads per job (depending on the partition), distributed-memory (MPI-based) parallel code using up to hundreds of cores, or jobs that take advantage of Graphics Processing Units (GPUs).<br />
<br />
Historically, ARC is primarily comprised of older, disparate Linux-based clusters that were formerly offered to researchers from across Canada such as Breezy, Lattice, and Parallel. In addition, a large-memory compute node (Bigbyte) was salvaged from the now-retired local Storm cluster. In January 2019, a major addition to ARC with modern hardware was purchased. In 2020, compute clusters from CHGI have been migrated into ARC.<br />
<br />
=== How to Get Started ===<br />
If you have a project you think would be appropriate for ARC, please email support@hpc.ucalgary.ca and mention the intended research and software you plan to use. You must have a University of Calgary IT account in order to use ARC.<br />
* For users that do not have a University of IT account or email address, please register for one at https://itregport.ucalgary.ca/.<br />
* For users external to the University, such as for users collaborating on a research project at the University of Calgary, please contact us and mention the project leader you are collaborating with.<br />
<br />
Once your access to ARC has been granted, you will be able to immediately make use of the cluster using your University of Calgary IT account by following the [[ARC_Cluster_Guide#Using_ARC|usage guide outlined below]].<br />
<br />
== Using ARC ==<br />
<br />
{{Message Box<br />
|icon=Security Icon.png<br />
|title=Cybersecurity awareness at the U of C<br />
|message=Please note that there are typically about 950 phishing attempts targeting University of Calgary accounts each month. This is just a reminder to be careful about computer security issues, both at home and at the University. Please visit https://it.ucalgary.ca/it-security for more information, tips on secure computing, and how to report suspected security problems.}}<br />
<br />
=== Logging in ===<br />
To log in to ARC, connect using SSH to <code>arc.ucalgary.ca</code> on port <code>22</code>. Connections to ARC are accepted only from the University of Calgary network (on campus) or through the University of Calgary General VPN (off campus).<br />
<br />
See [[Connecting to RCS HPC Systems]] for more information.<br />
=== How to interact with ARC ===<br />
<br />
ARC cluster is a collection of several compute nodes connected by a high-speed network. On ARC, computations get submitted as jobs. Once submitted, the jobs are then assigned to compute nodes by the job scheduler as resources become available.<br />
<br />
[[File:Cluster.png]]<br />
<br />
You can access ARC with your UCalgary IT user credentials. Once connected, you will get placed in the ARC login node, for basic tasks such as job submission, monitor job status, manage files, edit text, etc. It is a shared resource where multiple users get connected at the same time. Thus, any intensive tasks is not allowed on the login node as it may block other potential users to connect/submit their computations. <br />
[tannistha.nandi@arc ~]$ <br />
The job scheduling system on ARC is called SLURM. On ARC, there are two SLURM commands that can allocate resources to a job under appropriate conditions: ‘salloc’ and ‘sbatch’. They both accept the same set of command line options with respect to resource allocation. <br />
<br />
'''‘salloc’''' is to launch an interactive session, typically for tasks under 5 hours. <br />
Once an interactive job session is created, you can do things like explore research datasets, start R or python sessions to test your code, compile software applications etc.<br />
<br />
a. Example 1: The following command requests for 1 cpu on 1 node for 1 task along with 1 GB of RAM for an hour. <br />
[tannistha.nandi@arc ~]$ salloc --mem=1G -c 1 -N 1 -n 1 -t 01:00:00<br />
salloc: Granted job allocation 6758015<br />
salloc: Waiting for resource configuration<br />
salloc: Nodes fc4 are ready for job<br />
[tannistha.nandi@fc4 ~]$ <br />
<br />
<br />
b. Example 2: The following command requests for 1 GPU to be used from 1 node belonging to the gpu-v100 partition along with 1 GB of RAM for 1 hour. Generic resource scheduling (--gres) is used to request for GPU resources.<br />
[tannistha.nandi@arc ~]$ salloc --mem=1G -t 01:00:00 -p gpu-v100 --gres=gpu:1<br />
salloc: Granted job allocation 6760460<br />
salloc: Waiting for resource configuration<br />
salloc: Nodes fg3 are ready for job<br />
[tannistha.nandi@fg3 ~]$<br />
<br />
Once you finish the work, type 'exit' at the command prompt to end the interactive session,<br />
[tannistha.nandi@fg3 ~]$ exit<br />
[tannistha.nandi@fg3 ~]$ salloc: Relinquishing job allocation 6760460<br />
It is to ensure that the allocated resources are released from your job and now available to other users.<br />
<br />
'''‘sbatch’''' is to submit computations as jobs to run on the cluster. You can submit a job-script.slurm via 'sbatch' for execution. <br />
[tannistha.nandi@arc ~]$ sbatch job-script.slurm<br />
When resources become available, they get allocated to this task. Batch jobs are suited for tasks that run for long periods of time without any user supervision. When the job-script terminates, the allocation is released. <br />
Please review the section on how to prepare job scripts for more information.<br />
<br />
=== Prepare job scripts ===<br />
Job scripts are text files saved with an extension '.slurm', for example, 'job-script.slurm'. <br />
A job script looks something like this:<br />
''#!/bin/bash''<br />
####### Reserve computing resources #############<br />
#SBATCH --nodes=1<br />
#SBATCH --ntasks=1<br />
#SBATCH --cpus-per-task=1<br />
#SBATCH --time=01:00:00<br />
#SBATCH --mem=1G<br />
#SBATCH --partition=cpu2019<br><br />
####### Set environment variables ###############<br />
module load python/anaconda3-2018.12<br><br />
####### Run your script #########################<br />
python myscript.py<br />
<br />
The first line contains the text "#!/bin/bash" to interpret it as a bash script.<br />
<br />
It is followed by lines that start with a '#SBATCH' to communicate with 'SLURM'. You may add as many #SBATCH directives as needed to reserve computing resources for your task. The above example requests for one cpu on a single node for 1 task along with 1GB RAM for an hour on cpu2019 partition.<br />
<br />
Next, you have to set up environment variables either by loading the modules centrally installed on ARC or export path to the software in your home directory. The above example loads an available python module.<br />
<br />
Finally, include the Linux command to execute the local script.<br />
<br />
Note that failing to specify part of a resource allocation request (most notably '''time''' and '''memory''') will result in bad resource requests as the defaults are not appropriate to most cases. Please refer to the section 'Running non-interactive jobs' for more examples.<br />
<br />
== Hardware ==<br />
Since the ARC cluster is a conglomeration of many different compute clusters, the hardware within ARC can vary widely in terms of performance and capabilities. To mitigate any compatibility issues with different hardware, we combine similar hardware into their own Slurm partition to ensure your workload runs as consistently as possible within one partition. Please carefully review the hardware specs for each of the partitions below to avoid any surprises.<br />
<br />
=== Partition Hardware Specs ===<br />
When submitting jobs to ARC, you may specify a partition that your job will run on. Please choose a partition that is most appropriate for your work.<br />
<br />
* See also [[How to find available partitions on ARC]].<br />
<br />
A few things to keep in mind when choosing a partition:<br />
* Specific workloads requiring special Intel Instruction Set Extensions may only work on newer Intel CPUs. <br />
* If working with multi-node parallel processing, ensure your software and libraries support the partition's interconnect networking.<br />
* While older partitions may be slower, they may be less busy and have little to no wait times.<br />
<br />
If you are unsure which partition to use or need assistance on selecting an appropriate partition, please see [[#Selecting_a_Partition|the Selecting a Partition Section]] below. <br />
<br />
{| class="wikitable"<br />
! Partition<br />
! Description<br />
! Nodes<br />
! CPU Cores, Model, and Year<br />
! Memory<br />
! GPU<br />
! Network<br />
|-<br />
| -<br />
| ARC Login Node<br />
| 1<br />
| 16 cores, 2x Intel(R) Xeon(R) CPU E5620 @ 2.40GHz (Westmere, 2010)<br />
| 48 GB<br />
| N/A<br />
| 40 Gbit/s InfiniBand<br />
|-<br />
| gpu-v100<br />
| GPU Parition<br />
| 13<br />
| 80 cores, 2x Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz (Skylake, 2019)<br />
| 754 GB<br />
| 2x Tesla V100-PCIE-16GB<br />
| 100 Gbit/s Omni-Path<br />
|-<br />
|gpu-a100<br />
|GPU Partition<br />
|5<br />
|40 cores, 1x Intel(R) Xeon(R) Silver 4316 CPU @ 2.30GHz (Ice Lake, 2021)<br />
|512 GB<br />
|2x GA100 A100 PCIe 80GB<br />
|100 Gbit/s Mellanox Infiniband<br />
|-<br />
|cpu2022<br />
|General Purpose Compute<br />
|52<br />
|52 cores, 2x Intel(R) Xeon(R) Gold 5320 CPU @ 2.20GHz (Ice Lake)<br />
|256 GB<br />
|N/A<br />
|40 Gbit/s InfiniBand<br />
|-<br />
| cpu2021<br />
| General Purpose Compute<br />
| 48<br />
| 48 cores, 2x Intel(R) Xeon(R) Gold 6240R CPU @ 2.40GHz (Cascade Lake, 2021)<br />
| 185 GB<br />
| N/A <br />
| 100 Gbit/s Mellanox Infiniband<br />
|-<br />
| cpu2019<br />
| General Purpose Compute<br />
| 14<br />
| 40 cores, 2x Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz (Skylake, 2019)<br />
| 190 GB<br />
| N/A <br />
| 100 Gbit/s Omni-Path<br />
|-<br />
| apophis<br />
| General Purpose Compute<br />
| 21<br />
| 40 cores, 2x Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz (Skylake, 2019)<br />
| 190 GB<br />
| N/A <br />
| 100 Gbit/s Omni-Path<br />
|-<br />
| razi<br />
| General Purpose Compute<br />
| 41<br />
| 40 cores, 2x Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz (Skylake, 2019)<br />
| 190 GB<br />
| N/A <br />
| 100 Gbit/s Omni-Path<br />
|-<br />
| bigmem<br />
| Big Memory Nodes<br />
| 2<br />
| 80 cores, 4x Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz (Skylake, 2019)<br />
| 3022 GB<br />
| N/A <br />
| 100 Gbit/s Omni-Path<br />
|-<br />
| pawson<br />
| General Purpose Compute<br />
| 13<br />
| 40 cores, 2x Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz (Skylake, 2019)<br />
| 190 GB<br />
| N/A<br />
| 100 Gbit/s Omni-Path<br />
|-<br />
|cpu2017<br />
|General Purpose Compute<br />
|14<br />
|56 cores, 2x Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz (Sandy Bridge, 2012)<br />
|256 GB<br />
|N/A<br />
|40 Gbit/s InfiniBand<br />
|-<br />
| theia<br />
| Former Theia cluster<br />
| 20<br />
| 56 cores, 2x Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz (Sandy Bridge, 2012)<br />
| 188 GB<br />
| N/A <br />
| 40 Gbit/s InfiniBand<br />
|-<br />
| cpu2013<br />
| Former hyperion cluster<br />
| 12<br />
| 32 cores, 2x Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz (Sandy Bridge, 2012)<br />
| 126 GB<br />
| N/A<br />
| 40 Gbit/s InfiniBand<br />
|-<br />
| lattice<br />
| Former Lattice cluster<br />
| 307<br />
| 8 cores, 2x Intel(R) Xeon(R) CPU L5520 @ 2.27GHz (Nehalem, 2009)<br />
| 12 GB<br />
| N/A<br />
| 40 Gbit/s InfiniBand<br />
|-<br />
| single<br />
| Former Lattice cluster<br />
| 168<br />
| 8 cores, 2x Intel(R) Xeon(R) CPU L5520 @ 2.27GHz (Nehalem, 2009)<br />
| 12 GB<br />
| N/A<br />
| 40 Gbit/s InfiniBand<br />
|-<br />
| parallel<br />
| Former Parallel Cluster<br />
| 576<br />
| 12 cores, 2x Intel(R) Xeon(R) CPU E5649 @ 2.53GHz (Westmere, 2011)<br />
| 24 GB<br />
| N/A<br />
| 40 Gbit/s InfiniBand<br />
|}<br />
<br />
===ARC Cluster Storage===<br />
Usage of ARC cluster storage is outlined by our [[ARC Storage Terms of Use]] page.<br />
<br />
{{Warning Box<br />
| title=Data Storage<br />
| message=ARC storage is not suitable for long-term or archival storage. It is not backed-up and does not have sufficient redundancy to be used as a primary storage system. It is not guaranteed to be available for the time periods that are typical of archiving.<br />
<br />
Please ensure that the only data you keep on ARC is used for active computations.<br />
<br />
}}<br />
<br />
{{Message Box<br />
| title=No Backup Policy!<br />
| message=You are responsible for your own backups. Many researchers will have accounts with Compute Canada and may choose to back up their data there (the Project file system accessible through the Cedar cluster would often be used). <br />
<br />
Please contact us at support@hpc.ucalgary.ca if you want more information about this option.<br />
<br />
You can also back up data to your UofC OneDrive for business allocation see: https://rcs.ucalgary.ca/How_to_transfer_data#rclone:_rsync_for_cloud_storage This allocation starts at 5TB. Contact the support center for questions regarding OneDrive for Business.<br />
}}<br />
<br />
The ARC cluster has around 2 petabyte of shared disk storage available across the entire cluster as well as temporary storage local to each of the compute nodes. Please refer to the individual sections below on the capacity limitations and usage policies. <br />
<br />
Use the <code>arc.quota</code> command on ARC to determine the available space on your various volumes and home directory.<br />
<br />
{| class="wikitable"<br />
!Partition<br />
!Description<br />
!Capacity<br />
|-<br />
|<code>/home</code><br />
|User home directories<br />
|500 GB (per user)<br />
|-<br />
|<code>/work</code><br />
|Research project storage<br />
|Up to 100's of TB<br />
|-<br />
|<code>/scratch</code><br />
|Scratch space for temporary files<br />
|Up to 15 TB<br />
|-<br />
|<code>/tmp</code><br />
|Temporary space local to the compute cluster<br />
|Dependent on available storage on nodes. Verify with <code>df -h</code>.<br />
|-<br />
|<code>/dev/shm</code><br />
|Small temporary in-memory disk space local to the compute cluster<br />
|Dependent on memory size set in your Slurm job.<br />
|}<br />
====<code>/home</code>: Home file system====<br />
Each user has a directory under /home and is the default working directory when logging in to ARC. Each home directory has a per-user quota of 500 GB. This limit is fixed and cannot be increased. Researchers requiring additional storage exceeding what is available on their home directory may use <code>/work</code> and <code>/scratch</code>.<br />
<br />
Note on file sharing: Due to security concerns, permissions set using <code>chmod</code> on your home directory to allow other users to read/write to your home directory be automatically reverted by an automated system process unless an explicit exception is made. If you need to share files with other researchers on the ARC cluster, please write to support@hpc.ucalgary.ca to ask for such an exception.<br />
<br />
====<code>/scratch</code>: Scratch file system for large job-oriented storage====<br />
Associated with each job, under the <code>/scratch</code> directory, a subdirectory is created that can be referenced in job scripts as <code>/scratch/${SLURM_JOB_ID}</code>. You can use that directory for temporary files needed during the course of a job. Up to 15 TB of storage may be used, per user (total for all your jobs) in the <code>/scratch</code> file system. <br />
<br />
Data in <code>/scratch</code> associated with a given job will be deleted automatically, without exception, five days after the job finishes.<br />
<br />
====<code>/work</code>: Work file system for larger projects====<br />
If you need more space than provided in <code>/home</code> and the <code>/scratch</code> job-oriented space is not appropriate for you case, please write to support@hpc.ucalgary.ca with an explanation, including an indication of how much storage you expect to need and for how long. If approved, you will then be assigned a directory under <code>/work</code> with an appropriately large quota.<br />
<br />
====<code>/tmp</code>,<code>/var/tmp</code>: Temporary files====<br />
You may use <code>/tmp</code> or <code>/var/tmp</code> for storing temporary files generated by your job. The <code>/tmp</code> is stored on a disk local to the compute node and is not shared across the cluster. The files stored here will be removed immediately after your job terminates.<br />
<br />
==== <code>/dev/shm</code>, <code>/run/user/$uid</code>: In-memory temporary files ====<br />
<code>/dev/shm</code> and <code>/run/user/$UID</code> is writable location for temporary files backed by virtual memory. This can be used if faster I/O is required. This is ideal for workloads that require many small read/writes to share data between processes or as a fast cache. The amount of data you can write here is dependent on the amount of free memory available to your job. The files stored at these locations will be removed immediately after your job terminates.<br />
<br />
== Software ==<br />
All ARC nodes run the latest version of Rocky Linux 8 with the same set of base software packages. To maintain the stability and consistency of all nodes, any additional dependencies that your software requires must be installed under your account. For your convenience, we have packaged commonly used software packages and dependencies as modules available under <code>/global/software</code>. If your software package is not available as a module, you may also try Anaconda which allows users to manage and install custom packages in an isolated environment.<br />
<br />
For a list of available packages that have been made available, please see [[ARC Software pages]]. <br />
<br />
Please contact us at support@hpc.ucalgary.ca if you need additional software installed.<br />
<br />
==== Modules ====<br />
The setup of the environment for using some of the installed software is through the <code>module</code> command. An overview of [https://www.westgrid.ca//support/modules modules on WestGrid (external link)] is largely applicable to ARC.<br />
<br />
Software packages bundled as a module will be available under <code>/global/software</code> and can be listed with the <code>module avail</code> command.<br />
<syntaxhighlight lang="bash"><br />
$ module avail<br />
</syntaxhighlight><br />
<br />
To enable Python, load the Python module by running:<br />
<syntaxhighlight lang="bash"><br />
$ module load python/anaconda-3.6-5.1.0<br />
</syntaxhighlight><br />
<br />
To unload the Python module, run:<br />
<syntaxhighlight lang="bash"><br />
$ module remove python/anaconda-3.6-5.1.0<br />
</syntaxhighlight><br />
<br />
To see currently loaded modules, run:<br />
<syntaxhighlight lang="bash"><br />
$ module list<br />
</syntaxhighlight><br />
<br />
By default, no modules are loaded on ARC. If you wish to use a specific module, such as the Intel compilers or the Open MPI parallel programming packages, you must load the appropriate module.<br />
<br />
== Job submission ==<br />
<br />
=== Interactive Jobs ===<br />
The ARC login node may be used for such tasks as editing files, compiling programs and running short tests while developing programs. We suggest CPU intensive workloads on the login node be restricted to under 15 minutes as per [[General Cluster Guidelines and Policies|our cluster guidelines]]. For interactive workloads exceeding 15 minutes, use the '''[[Running_jobs#Interactive_jobs|salloc command]]''' to allocate an interactive session on a compute node.<br />
<br />
The default salloc allocation is 1 CPU and 1 GB of memory. Adjust this by specifying <code>-n CPU#</code> and <code>--mem Megabytes</code>. You may request up to 5 hours of CPU time for interactive jobs.<br />
salloc --time=5:00:00 --partition=cpu2019<br />
<br />
Always use salloc or srun to start an interactive job. Do not SSH directly to a compute node as SSH sessions will be refused without an active job running.<br />
<br />
<!-- This information doesn't seem that useful or relevant to running interactive jobs. Move to getting started section?<br />
ARC uses the Linux operating system. The program that responds to your typed commands and allows you to run other programs is called the Linux shell. There are several different shells available, but, by default you will use one called bash. It is useful to have some knowledge of the shell and a variety of other command-line programs that you can use to manipulate files. If you are new to Linux systems, we recommend that you work through one of the many online tutorials that are available, such as the [http://www.ee.surrey.ac.uk/Teaching/Unix/index.html UNIX Tutorial for Beginners (external link)] provided by the University of Surrey. The tutorial covers such fundamental topics, among others, as creating, renaming and deleting files and directories, how to produce a listing of your files and how to tell how much disk space you are using. For a more comprehensive introduction to Linux, see [http://linuxcommand.sourceforge.net/tlcl.php The Linux Command Line (external link)].<br />
--><br />
<br />
=== Running non-interactive jobs (batch processing) ===<br />
Production runs and longer test runs should be submitted as (non-interactive) batch jobs, in which commands to be executed are listed in a script (text file). Batch jobs scripts are submitted using the <code>sbatch</code> command, part of the Slurm job management and scheduling software. #SBATCH directive lines at the beginning of the script are used to specify the resources needed for the job (cores, memory, run time limit and any specialized hardware needed).<br />
<br />
Most of the information on the [https://docs.computecanada.ca/wiki/Running_jobs Running Jobs (external link)] page on the Compute Canada web site is also relevant for submitting and managing batch jobs and reserving processors for interactive work on ARC. One major difference between running jobs on the ARC and Compute Canada clusters is in selecting the type of hardware that should be used for a job. On ARC, you choose the hardware to use primarily by specifying a partition, as described below.<br />
<br />
=== Selecting a Partition ===<br />
There are some aspects to consider when selecting a partition including:<br />
* Resource requirements in terms of memory and CPU cores<br />
* Hardware specific requirements, such as GPU or CPU Instruction Set Extensions<br />
* Partition resource limits and potential wait time<br />
* Software support parallel processing using Message Passing Interface (MPI), OpenMP, etc.<br />
** Eg. MPI for parallel processing can distribute memory across multiple nodes, per-node memory requirements could be lower. Whereas, OpenMP or single process code that is restricted to one node would require a higher memory node.<br />
** Note: MPI code running on hardware with Omni-Path networking should be compiled with Omni-Path networking support. This is provided by loading the <code>openmpi/2.1.3-opa</code> or <code>openmpi/3.1.2-opa</code> modules prior to compiling.<br />
<br />
Since resources that are requested are reserved for your job, please request only as much CPU and memory as your job requires to avoid reducing the cluster efficiency. If you are unsure which partition to use or the specific resource requests that are appropriate for your jobs, please contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] and we would be happy to work with you.<br />
<br />
{| class="wikitable" style="width: 100%;"<br />
!Partition<br />
!Description<br />
!Cores/node<br />
!Memory Request Limit<br />
!Time Limit<br />
!GPU<br />
!Networking<br />
|-<br />
|cpu2021<br />
|General Purpose Compute<br />
|48<br />
|185,000 MB<br />
|7 days ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|cpu2019<br />
|General Purpose Compute<br />
|40<br />
|185,000 MB<br />
|7 days ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|bigmem<br />
|Big Memory Compute<br />
|80<br />
|3,000,000 MB<br />
|24 hours ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|gpu-v100<br />
|GPU Compute<br />
|80<br />
|753,000 MB<br />
|24 hours ‡<br />
|2<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|apophis&dagger;<br />
|Private Research Partition<br />
|40<br />
|185,000 MB<br />
|7 days ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|razi&dagger;<br />
|Private Research Partition<br />
|40<br />
|185,000 MB<br />
|7 days ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|pawson&dagger;<br />
|Private Research Partition<br />
|40<br />
|185,000 MB<br />
|7 days ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|sherlock&dagger;<br />
|Private Research Partition<br />
|7<br />
|185,000 MB<br />
|7 days ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|theia&dagger;<br />
|Private Research Partition<br />
|28<br />
|188,000 MB<br />
|7 days ‡<br />
|<br />
|40 Gbit/s InfiniBand<br />
|-<br />
|synergy&dagger;<br />
|Private Research Partition<br />
|14<br />
|245,000 MB<br />
|7 days ‡<br />
|<br />
|40 Gbit/s InfiniBand<br />
|-<br />
|cpu2013<br />
|Legacy General Purpose Compute<br />
|16<br />
|120000<br />
|7 days ‡<br />
|<br />
|40 Gbit/s InfiniBand<br />
|-<br />
|lattice<br />
|Legacy General Purpose Compute<br />
|8<br />
|12000<br />
|7 days ‡<br />
|<br />
|40 Gbit/s InfiniBand<br />
|-<br />
|parallel<br />
|Legacy General Purpose Compute<br />
|12<br />
|23000<br />
|7 days ‡<br />
|<br />
|40 Gbit/s InfiniBand<br />
|-<br />
|single<br />
|Legacy Single-Node Job Compute<br />
|8<br />
|12000<br />
|7 days ‡<br />
|<br />
|40 Gbit/s InfiniBand<br />
|-<br />
|cpu2021-bf24<br />
|Back-fill Compute (2021-era hardware, 24h)<br />
|48<br />
|185,000 MB<br />
|24 hours ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|cpu2019-bf05<br />
|Back-fill Compute (2019-era hardware, 5h)<br />
|40<br />
|185,000 MB<br />
|5 hours ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|cpu2017-bf05<br />
|Back-fill Compute (2017-era hardware, 5h)<br />
|14<br />
|245,000 MB<br />
|5 hours ‡<br />
|<br />
|40 Gbit/s InfiniBand<br />
|-<br />
|+ style="caption-side: bottom; text-align: left; font-weight: normal;" | &dagger; These partitions contain hardware contributed to ARC by particular researchers and should only be used by members of their research groups. However, they have generously allowed their compute nodes to be shared with others outside their research groups for short jobs. A special 'back-fill' or -bf partition is available for use by all ARC users for jobs shorter than 5 hours.<br />‡ As time limits may be changed by administrators to adjust to maintenance schedules or system load, the values given in the tables are not definitive. See the Time limits section below for commands you can use on ARC itself to determine current limits.<br />
|}<br />
<br />
==== Backfill partitions ====<br />
Backfill partitions can be used by all users on ARC for short-term jobs. The hardware backing these partitions are generously contributed by researchers. We recommend including the backfill partitions for short term jobs as it may help reduce your job's wait time and increase the overall cluster throughput.<br />
<br />
Previously, each contributing research group had their own backfill partition. Since June 2021, we have merged:<br />
<br />
* apophis-bf, pawson-bf, and razi-bf into cpu2019-bf05 <br />
* theia-bf and synergy-bf into cpu2017-bf05<br />
<br />
The naming scheme of the backfill partitions is the CPU generation year, followed by -bf and the time limit in hours. For example, cpu2017-bf05 would represent a backfill partition containing processors from 2017 with a time limit of 5 hours.<br />
<br />
==== Hardware resource and job policy limits ====<br />
In addition to the hardware limitations, please be aware that there may also be policy limits imposed on your account for each partition. These limits restrict the number of cores, nodes, or GPUs that can be used at any given time. Since the limits are applied on a partition-by-partition basis, using resources in one partition should not affect the available resources you can use in another partition.<br />
<br />
These limits can be listed by running:<br />
<syntaxhighlight lang="bash"><br />
$ sacctmgr show qos format=Name,MaxWall,MaxTRESPU%20,MaxSubmitJobs<br />
Name MaxWall MaxTRESPU MaxSubmit<br />
---------- ----------- -------------------- ---------<br />
normal 7-00:00:00 2000<br />
breezy 3-00:00:00 cpu=384 2000<br />
gpu 7-00:00:00 13000<br />
cpu2019 7-00:00:00 cpu=240 2000<br />
gpu-v100 1-00:00:00 cpu=80,gres/gpu=4 2000<br />
single 7-00:00:00 cpu=408,node=75 2000<br />
razi 7-00:00:00 2000<br />
</syntaxhighlight><br />
<br />
==== Specifying a partition in a job ====<br />
One you have decided which partitions best suits your computation, you can select one or more partition on a job-by-job basis by including the <code>partition</code> keyword for an <code>SBATCH</code> directive in your batch job. Multiple partitions should be comma separated. If you omit the partition specification, the system will try to assign your job to appropriate hardware based on other aspects of your request. <br />
<br />
In some cases, you really should specify the partition explicitly. For example, if you are running single-node jobs with thread-based parallel processing requesting 8 cores you could use:<br />
<syntaxhighlight lang="bash"><br />
#SBATCH --mem=0 ❶<br />
#SBATCH --nodes=1 ❷<br />
#SBATCH --ntasks=1 ❸<br />
#SBATCH --cpus-per-task=8 ❹<br />
#SBATCH --partition=single,lattice ❺ <br />
</syntaxhighlight><br />
<br />
A few things to mention in this example:<br />
# <code>--mem=0</code> allocates all available memory on the compute node for the job. This effectively allocates the entire node for your job.<br />
# <code>--nodes=1</code> allocates 1 node for the job<br />
# <code>--ntasks=1</code> your job has a single task<br />
# <code>--cpus-per-task=8</code> asks for 8 CPUs per task. This job in total will request 8 * 1, or 8 CPUs.<br />
# <code>--partition=single,lattice</code> specifies that this job can run on either single or lattice.<br />
Suppose that your job requires at most 8 CPU cores and 10 GB of memory. The above Slurm request would be valid and optimal since your job fits neatly in a single node on the single and parallel partition. However, if you failed to specify the partition, Slurm may try to schedule your job to a partition with larger nodes, such as cpu2019 where each node has 40 cores and 190 GB of memory. If your job is scheduled on such a node, your job will be effectively wasting 32 cores and 180 GB of memory because <code>--mem=0</code> not only requests for 190 GB on this node, but also prevents other jobs from being scheduled on the same node.<br />
<br />
If you don't specify a partition, please give greater thought to the memory specification to make sure that the scheduler will not assign your job more resources than are needed.<br />
<br />
Parameters such as '''--ntasks-per-cpu''', '''--cpus-per-task''', '''--mem''' and '''--mem-per-cpu>''' have to be adjusted according to the capabilities of the hardware also. The product of --ntasks-per-cpu and --cpus-per-task should be less than or equal to the number given in the "Cores/node" column. The '''--mem>''' parameter (or the product of '''--mem-per-cpu''' and '''--cpus-per-task''') should be less than the "Memory limit" shown. If using whole nodes, you can specify '''--mem=0''' to request the maximum amount of memory per node.<br />
<br />
===== Examples =====<br />
Here are some examples of specifying the various partitions.<br />
<br />
As mentioned in the [[#Hardware|Hardware]] section above, the ARC cluster was expanded in January 2019. To select the 40-core general purpose nodes specify:<br />
<br />
#SBATCH --partition=cpu2019<br />
<br />
To run on the Tesla V100 GPU-enabled nodes, use the '''gpu-v100''' partition. You will also need to include an SBATCH directive in the form '''--gres=gpu:n''' to specify the number of GPUs, n, that you need. For example, if the software you are running can make use of both GPUs on a gpu-v100 partition compute node, use:<br />
<br />
#SBATCH --partition=gpu-v100 --gres=gpu:2<br />
<br />
For very large memory jobs (more than 185000 MB), specify the bigmem partition:<br />
<br />
#SBATCH --partition=bigmem<br />
<br />
If the more modern computers are too busy or you have a job well-suited to run on the compute nodes described in the legacy hardware section above, choose the cpu2013, Lattice or Parallel compute nodes by specifying the corresponding partition keyword:<br />
<br />
#SBATCH --partition=cpu2013<br />
#SBATCH --partition=lattice<br />
#SBATCH --partition=parallel<br />
<br />
There is an additional partition called '''single''' that provides nodes similar to the lattice partition, but, is intended for single-node jobs. Select the single partition with<br />
<br />
#SBATCH --partition=single<br />
<br />
=== Time limits ===<br />
Use the <code>--time</code> directive to tell the job scheduler the maximum time that your job might run. For example:<br />
#SBATCH --time=hh:mm:ss<br />
<br />
You can use <code>scontrol show partitions</code> or <code>sinfo</code> to see the current maximum time that a job can run.<br />
<syntaxhighlight lang="bash" highlight="6"><br />
$ scontrol show partitions<br />
PartitionName=single <br />
AllowGroups=ALL AllowAccounts=ALL AllowQos=ALL <br />
AllocNodes=ALL Default=NO QoS=single <br />
DefaultTime=NONE DisableRootJobs=NO ExclusiveUser=NO GraceTime=0 Hidden=NO <br />
MaxNodes=UNLIMITED MaxTime=7-00:00:00 MinNodes=1 LLN=NO MaxCPUsPerNode=UNLIMITED <br />
Nodes=cn[001-168] <br />
PriorityJobFactor=1 PriorityTier=1 RootOnly=NO ReqResv=NO OverSubscribe=NO <br />
OverTimeLimit=NONE PreemptMode=OFF <br />
State=UP TotalCPUs=1344 TotalNodes=168 SelectTypeParameters=NONE <br />
DefMemPerNode=UNLIMITED MaxMemPerNode=UNLIMITED <br />
</syntaxhighlight><br />
<br />
Alternatively, with <code>sinfo</code> under the <code>TIMELIMIT</code> column:<br />
<syntaxhighlight lang="bash"><br />
$ sinfo <br />
PARTITION AVAIL TIMELIMIT NODES STATE NODELIST <br />
single up 7-00:00:00 1 drain* cn097 <br />
single up 7-00:00:00 1 maint cn002 <br />
single up 7-00:00:00 4 drain* cn[001,061,133,154] <br />
...<br />
</syntaxhighlight><br />
<br />
== Support ==<br />
{{Message Box<br />
|title=[[Support|Need Help or have other ARC Related Questions?]]<br />
|message=For all general RCS related issues, questions, or comments, please contact us at support@hpc.ucalgary.ca.<br />
|icon=Support Icon.png}}<br />
<br />
Please don't hesitate to [[Support|contact us]] directly by email if you need help using ARC or require guidance on migrating and running your workflows to ARC.<br />
<br />
[[Category:ARC]]<br />
[[Category:Guides]]</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=Template:Warning_Box&diff=2637Template:Warning Box2023-09-14T23:05:22Z<p>Darcy: Created page with "<table role="presentation" style="border: 1px solid #a2a9b1; background-color: #f8f9fa; width: 80%; text-align: left; margin: 1em auto 1em auto; padding-right: 15px;"> <tr> <td style="width: 5em; text-align: center;"> 40px </td> <td> <div style="padding-top: 6px; padding-bottom: 5px;"> <b>{{{title|{{{1|Default Title}}}}}}</b><br> <div style="font-size: 90%;">{{{message|{{{2|Default message}}}}}}</div> </div> </td> </table>"</p>
<hr />
<div><table role="presentation" style="border: 1px solid #a2a9b1; background-color: #f8f9fa; width: 80%; text-align: left; margin: 1em auto 1em auto; padding-right: 15px;"><br />
<tr><br />
<br />
<td style="width: 5em; text-align: center;"><br />
[[File:{{{icon|{{{3|Attention Icon.png}}}}}}|40px]]<br />
</td><br />
<br />
<td><br />
<div style="padding-top: 6px; padding-bottom: 5px;"><br />
<b>{{{title|{{{1|Default Title}}}}}}</b><br><br />
<div style="font-size: 90%;">{{{message|{{{2|Default message}}}}}}</div><br />
</div><br />
<br />
</td><br />
</table></div>Darcyhttps://rcs.ucalgary.ca/index.php?title=File:Attention_Icon.png&diff=2636File:Attention Icon.png2023-09-14T22:51:14Z<p>Darcy: Direct the user to pay attention to this because it is important</p>
<hr />
<div>== Summary ==<br />
Direct the user to pay attention to this because it is important</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=RCS_Data_Sheet&diff=2462RCS Data Sheet2023-05-01T21:00:29Z<p>Darcy: </p>
<hr />
<div>Specifications<br />
<br />
HPC CPU/GPU Service<br />
<br />
L4 HPC CPU/GPU/VM Service<br />
<br />
HPC Desktop Service<br />
<br />
L4 HPC Storage Service<br />
<br />
CloudStack VM Service<br />
<br />
OneFS Storage Service<br />
<br />
Instrument Storage Service<br />
<br />
HPC Access Methods<br />
<br />
CloudStack Access Methods<br />
<br />
HPC Desktop Access Methods<br />
<br />
L4 Services Access Methods<br />
<br />
OneFS Access Methods<br />
<br />
Instrument Storage Access Methods</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=RCS_Data_Sheet&diff=2461RCS Data Sheet2023-05-01T21:00:03Z<p>Darcy: Created page with "Specifications HPC CPU/GPU Service L4 HPC CPU/GPU/VM Service HPC Desktop Service L4 HPC Storage Service CloudStack VM Service OneFS Storage Service Instrument Storage Service HPC Access Methods CloudStack Access Methods HPC Desktop Access Methods L4 Services Access Methods OneFS Access Methods Instrument Storage Access Methods"</p>
<hr />
<div>Specifications<br />
<br />
HPC CPU/GPU Service<br />
L4 HPC CPU/GPU/VM Service<br />
HPC Desktop Service<br />
L4 HPC Storage Service<br />
CloudStack VM Service<br />
OneFS Storage Service<br />
Instrument Storage Service<br />
<br />
HPC Access Methods<br />
CloudStack Access Methods<br />
HPC Desktop Access Methods<br />
L4 Services Access Methods<br />
OneFS Access Methods<br />
Instrument Storage Access Methods</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=CloudStack_End_User_Agreement&diff=2089CloudStack End User Agreement2022-08-30T22:12:57Z<p>Darcy: /* Important Notes */</p>
<hr />
<div>==Introduction==<br />
CloudStack is an Infrastructure as a Service provided to University of Calgary researchers. It allows them to quickly deploy Virtual Machines to support their research.<br />
<br />
Researchers have a great degree of freedom in how they use CloudStack. They are solely responsible for maintaining any VMs they deploy.<br />
<br />
CloudStack is a research environment. While the system is available 24/7, support is only available during University business hours.<br />
<br />
Researchers are asked to follow the best practices listed below to ensure that the system remains available for the campus community.<br />
<br />
==Best Practices==<br />
<br />
# Please stay abreast of security updates for your OS and apply them.<br />
# Backup your data to non-CloudStack hosted storage (RCS does not provide data backups of VMs).<br />
# Do not run Windows (This infrastructure is not licensed to run Windows).<br />
# The University's "Acceptable Use of Electronic Resources and Information Policy" applies to your work using a VM. Please see [https://www.ucalgary.ca/legal-services/university-policies-procedures/ University Policies and Procedures] and search for "Electronic".<br />
# This infrastructure is only rated to handle Level 1 and Level 2 data. Please see [https://www.ucalgary.ca/legal-services/university-legal-services/operating-standards-guidelines-forms/ University Legal Services] and select "Information Security Classification Standard" for details.<br />
# You are responsible for the appropriate use of the VM by any accounts you have created.<br />
# You should remove/disable accounts that are no longer required.<br />
# All user accounts on the VM must have good passwords. See [https://it.ucalgary.ca/it-security/passwords-do-i-have-change-them/ here] for details on creating strong passwords.<br />
# CloudStack is not meant as a High Performance Computing (HPC) number cruncher. If you have HPC needs, please see "[[How to get an account]]" for details on how to apply for an account on ARC. Not sure what you need? Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca].<br />
# If your VM faces the outside world, please consider using appropriate security tools. Contact [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] for assistance.<br />
<br />
==Important Notes==<br />
In the event of a security incident, IT Operations/Security will shut down affected VMs.<br />
<br />
Non-critical patches/upgrades to CloudStack that are required will happen on Tuesday of each week. Running VMs should not be affected.<br />
<br />
Non-critical patches/upgrades that require a complete restart of CloudStack will occur after 1 day email notice.<br />
<br />
Urgent security patches to CloudStack may happen with little to no notice.<br />
<br />
CloudStack is provided as-is, with best effort support. It is not suitable for mission critical, high availability services.<br />
<br />
Please check back with this document as changes may occur.</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=CloudStack_End_User_Agreement&diff=2082CloudStack End User Agreement2022-08-24T17:15:41Z<p>Darcy: /* Best Practices */</p>
<hr />
<div>==Introduction==<br />
CloudStack is an Infrastructure as a Service provided to University of Calgary researchers. It allows them to quickly deploy Virtual Machines to support their research.<br />
<br />
Researchers have a great degree of freedom in how they use CloudStack. They are solely responsible for maintaining any VMs they deploy.<br />
<br />
CloudStack is a research environment. While the system is available 24/7, support is only available during University business hours.<br />
<br />
Researchers are asked to follow the best practices listed below to ensure that the system remains available for the campus community.<br />
<br />
==Best Practices==<br />
<br />
# Please stay abreast of security updates for your OS and apply them.<br />
# Backup your data to non-CloudStack hosted storage (RCS does not provide data backups of VMs).<br />
# Do not run Windows (This infrastructure is not licensed to run Windows).<br />
# The University's "Acceptable Use of Electronic Resources and Information Policy" applies to your work using a VM. Please see [https://www.ucalgary.ca/legal-services/university-policies-procedures/ University Policies and Procedures] and search for "Electronic".<br />
# This infrastructure is only rated to handle Level 1 and Level 2 data. Please see [https://www.ucalgary.ca/legal-services/university-legal-services/operating-standards-guidelines-forms/ University Legal Services] and select "Information Security Classification Standard" for details.<br />
# You are responsible for the appropriate use of the VM by any accounts you have created.<br />
# You should remove/disable accounts that are no longer required.<br />
# All user accounts on the VM must have good passwords. See [https://it.ucalgary.ca/it-security/passwords-do-i-have-change-them/ here] for details on creating strong passwords.<br />
# CloudStack is not meant as a High Performance Computing (HPC) number cruncher. If you have HPC needs, please see "[[How to get an account]]" for details on how to apply for an account on ARC. Not sure what you need? Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca].<br />
# If your VM faces the outside world, please consider using appropriate security tools. Contact [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] for assistance.<br />
<br />
==Important Notes==<br />
In the event of a security incident, IT Operations/Security will shut down affected VMs.<br />
<br />
Non-critical patches/upgrades to CloudStack that are required will happen on Tuesday of each week. Running VMs should not be affected.<br />
<br />
Non-critical patches/upgrades that require a complete restart of CloudStack will occur after 1 day email notice.<br />
<br />
Urgent security patches to CloudStack may happen with little to no notice.<br />
<br />
Please check back with this document as changes may occur.</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=CloudStack_End_User_Agreement&diff=2081CloudStack End User Agreement2022-08-24T17:14:46Z<p>Darcy: /* Best Practices */</p>
<hr />
<div>==Introduction==<br />
CloudStack is an Infrastructure as a Service provided to University of Calgary researchers. It allows them to quickly deploy Virtual Machines to support their research.<br />
<br />
Researchers have a great degree of freedom in how they use CloudStack. They are solely responsible for maintaining any VMs they deploy.<br />
<br />
CloudStack is a research environment. While the system is available 24/7, support is only available during University business hours.<br />
<br />
Researchers are asked to follow the best practices listed below to ensure that the system remains available for the campus community.<br />
<br />
==Best Practices==<br />
<br />
# Please stay abreast of security updates for your OS and apply them.<br />
# Backup your data to some other destination (RCS does not provide data backups of VMs).<br />
# Do not run Windows (This infrastructure is not licensed to run Windows).<br />
# The University's "Acceptable Use of Electronic Resources and Information Policy" applies to your work using a VM. Please see [https://www.ucalgary.ca/legal-services/university-policies-procedures/ University Policies and Procedures] and search for "Electronic".<br />
# This infrastructure is only rated to handle Level 1 and Level 2 data. Please see [https://www.ucalgary.ca/legal-services/university-legal-services/operating-standards-guidelines-forms/ University Legal Services] and select "Information Security Classification Standard" for details.<br />
# You are responsible for the appropriate use of the VM by any accounts you have created.<br />
# You should remove/disable accounts that are no longer required.<br />
# All user accounts on the VM must have good passwords. See [https://it.ucalgary.ca/it-security/passwords-do-i-have-change-them/ here] for details on creating strong passwords.<br />
# CloudStack is not meant as a High Performance Computing (HPC) number cruncher. If you have HPC needs, please see "[[How to get an account]]" for details on how to apply for an account on ARC. Not sure what you need? Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca].<br />
# If your VM faces the outside world, please consider using appropriate security tools. Contact [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] for assistance.<br />
<br />
==Important Notes==<br />
In the event of a security incident, IT Operations/Security will shut down affected VMs.<br />
<br />
Non-critical patches/upgrades to CloudStack that are required will happen on Tuesday of each week. Running VMs should not be affected.<br />
<br />
Non-critical patches/upgrades that require a complete restart of CloudStack will occur after 1 day email notice.<br />
<br />
Urgent security patches to CloudStack may happen with little to no notice.<br />
<br />
Please check back with this document as changes may occur.</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=CloudStack_End_User_Agreement&diff=2080CloudStack End User Agreement2022-08-24T17:12:51Z<p>Darcy: </p>
<hr />
<div>==Introduction==<br />
CloudStack is an Infrastructure as a Service provided to University of Calgary researchers. It allows them to quickly deploy Virtual Machines to support their research.<br />
<br />
Researchers have a great degree of freedom in how they use CloudStack. They are solely responsible for maintaining any VMs they deploy.<br />
<br />
CloudStack is a research environment. While the system is available 24/7, support is only available during University business hours.<br />
<br />
Researchers are asked to follow the best practices listed below to ensure that the system remains available for the campus community.<br />
<br />
==Best Practices==<br />
<br />
# Please stay abreast of security updates for your OS and apply them.<br />
# Backup your data (RCS does not provide data backups of VMs).<br />
# Do not run Windows (This infrastructure is not licensed to run Windows).<br />
# The University's "Acceptable Use of Electronic Resources and Information Policy" applies to your work using a VM. Please see [https://www.ucalgary.ca/legal-services/university-policies-procedures/ University Policies and Procedures] and search for "Electronic".<br />
# This infrastructure is only rated to handle Level 1 and Level 2 data. Please see [https://www.ucalgary.ca/legal-services/university-legal-services/operating-standards-guidelines-forms/ University Legal Services] and select "Information Security Classification Standard" for details.<br />
# You are responsible for the appropriate use of the VM by any accounts you have created.<br />
# You should remove/disable accounts that are no longer required.<br />
# All user accounts on the VM must have good passwords. See [https://it.ucalgary.ca/it-security/passwords-do-i-have-change-them/ here] for details on creating strong passwords.<br />
# CloudStack is not meant as a High Performance Computing (HPC) number cruncher. If you have HPC needs, please see "[[How to get an account]]" for details on how to apply for an account on ARC. Not sure what you need? Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca].<br />
# If your VM faces the outside world, please consider using appropriate security tools. Contact [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] for assistance.<br />
<br />
==Important Notes==<br />
In the event of a security incident, IT Operations/Security will shut down affected VMs.<br />
<br />
Non-critical patches/upgrades to CloudStack that are required will happen on Tuesday of each week. Running VMs should not be affected.<br />
<br />
Non-critical patches/upgrades that require a complete restart of CloudStack will occur after 1 day email notice.<br />
<br />
Urgent security patches to CloudStack may happen with little to no notice.<br />
<br />
Please check back with this document as changes may occur.</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=ARC_Cluster_Guide&diff=1905ARC Cluster Guide2022-06-10T21:18:01Z<p>Darcy: /* Storage */</p>
<hr />
<div>{{Message Box<br />
|icon=Security Icon.png<br />
|title=Cybersecurity awareness at the U of C<br />
|message=Please note that there are typically about 950 phishing attempts targeting University of Calgary accounts each month. This is just a reminder to be careful about computer security issues, both at home and at the University. Please visit https://it.ucalgary.ca/it-security for more information, tips on secure computing, and how to report suspected security problems.}}<br />
<br />
{{Message Box<br />
|title=[[Support|Need Help or have other ARC Related Questions?]]<br />
|message=For all general RCS related issues, questions, or comments, please contact us at support@hpc.ucalgary.ca.<br />
|icon=Support Icon.png}}<br />
<br />
This guide gives an overview of the Advanced Research Computing (ARC) cluster at the University of Calgary and is intended to be read by new account holders getting started on ARC. This guide covers topics such as the hardware and performance characteristics, available software, usage policies and how to log in and run jobs. <br />
<br />
== Introduction ==<br />
The ARC compute cluster can be used for running large numbers (hundreds) of concurrent serial (one core) jobs, OpenMP or other thread-based jobs, shared-memory parallel code using up to 40 or 80 threads per job (depending on the partition), distributed-memory (MPI-based) parallel code using up to hundreds of cores, or jobs that take advantage of Graphics Processing Units (GPUs). Almost all work on ARC is done through a [[Linux Introduction|command line interface]]. This computational resource is available for research projects based at the University of Calgary and is meant to supplement the resources available to researchers through Compute Canada.<br />
<br />
Historically, ARC is primarily comprised of older, disparate Linux-based clusters that were formerly offered to researchers from across Canada such as Breezy, Lattice, and Parallel. In addition, a large-memory compute node (Bigbyte) was salvaged from the now-retired local Storm cluster. In January 2019, a major addition to ARC with modern hardware was purchased. In 2020, compute clusters from CHGI have been migrated into ARC.<br />
<br />
=== How to Get Started ===<br />
If you have a project you think would be appropriate for ARC, please write to support@hpc.ucalgary.ca and mention the intended research and software you plan to use. You must have a University of Calgary IT account in order to use ARC.<br />
* For users that do not have a University of IT account or email address, please register for one at https://itregport.ucalgary.ca/.<br />
* For users external to the University, such as for users collaborating on a research project at the University of Calgary, please contact us and mention the project leader you are collaborating with.<br />
<br />
Once your access to ARC has been granted, you will be able to immediately make use of the cluster using your University of Calgary IT account by following the [[ARC_Cluster_Guide#Using_ARC|usage guide outlined below]].<br />
<br />
== Hardware ==<br />
Since the ARC cluster is a conglomeration of many different compute clusters, the hardware within ARC can vary widely in terms of performance and capabilities. To mitigate any compatibility issues with different hardware, we combine similar hardware into their own Slurm partition to ensure your workload runs as consistently as possible within one partition. Please carefully review the hardware specs for each of the partitions below to avoid any surprises.<br />
<br />
=== Partition Hardware Specs ===<br />
When submitting jobs to ARC, you may specify a partition that your job will run on. Please choose a partition that is most appropriate for your work.<br />
<br />
A few things to keep in mind when choosing a partition:<br />
* Specific workloads requiring special Intel Instruction Set Extensions may only work on newer Intel CPUs. <br />
* If working with multi-node parallel processing, ensure your software and libraries support the partition's interconnect networking.<br />
* While older partitions may be slower, they may be less busy and have little to no wait times.<br />
<br />
If you are unsure which partition to use or need assistance on selecting an appropriate partition, please see [[#Selecting_a_Partition|the Selecting a Partition Section]] below. <br />
<br />
{| class="wikitable"<br />
! Partition<br />
! Description<br />
! Nodes<br />
! CPU Cores, Model, and Year<br />
! Memory<br />
! GPU<br />
! Network<br />
|-<br />
| -<br />
| ARC Login Node<br />
| 1<br />
| 16 cores, 2x Intel(R) Xeon(R) CPU E5620 @ 2.40GHz (Westmere, 2010)<br />
| 48 GB<br />
| N/A<br />
| 40 Gbit/s InfiniBand<br />
|-<br />
| gpu-v100<br />
| GPU Parition<br />
| 13<br />
| 80 cores, 2x Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz (Skylake, 2019)<br />
| 754 GB<br />
| 2x Tesla V100-PCIE-16GB<br />
| 100 Gbit/s Omni-Path<br />
|-<br />
| cpu2021<br />
| General Purpose Compute<br />
| 48<br />
| 48 cores, 2x Intel(R) Xeon(R) Gold 6240R CPU @ 2.40GHz (Cascade Lake, 2021)<br />
| 185 GB<br />
| N/A <br />
| 100 Gbit/s Mellanox Infiniband<br />
|-<br />
| cpu2019<br />
| General Purpose Compute<br />
| 14<br />
| 40 cores, 2x Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz (Skylake, 2019)<br />
| 190 GB<br />
| N/A <br />
| 100 Gbit/s Omni-Path<br />
|-<br />
| apophis<br />
| General Purpose Compute<br />
| 21<br />
| 40 cores, 2x Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz (Skylake, 2019)<br />
| 190 GB<br />
| N/A <br />
| 100 Gbit/s Omni-Path<br />
|-<br />
| razi<br />
| General Purpose Compute<br />
| 41<br />
| 40 cores, 2x Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz (Skylake, 2019)<br />
| 190 GB<br />
| N/A <br />
| 100 Gbit/s Omni-Path<br />
|-<br />
| bigmem<br />
| Big Memory Nodes<br />
| 2<br />
| 80 cores, 4x Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz (Skylake, 2019)<br />
| 3022 GB<br />
| N/A <br />
| 100 Gbit/s Omni-Path<br />
|-<br />
| pawson<br />
| General Purpose Compute<br />
| 13<br />
| 40 cores, 2x Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz (Skylake, 2019)<br />
| 190 GB<br />
| N/A<br />
| 100 Gbit/s Omni-Path<br />
|-<br />
|cpu2017<br />
|General Purpose Compute<br />
|14<br />
|56 cores, 2x Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz (Sandy Bridge, 2012)<br />
|256 GB<br />
|N/A<br />
|40 Gbit/s InfiniBand<br />
|-<br />
| theia<br />
| Former Theia cluster<br />
| 20<br />
| 56 cores, 2x Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz (Sandy Bridge, 2012)<br />
| 188 GB<br />
| N/A <br />
| 40 Gbit/s InfiniBand<br />
|-<br />
| cpu2013<br />
| Former hyperion cluster<br />
| 12<br />
| 32 cores, 2x Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz (Sandy Bridge, 2012)<br />
| 126 GB<br />
| N/A<br />
| 40 Gbit/s InfiniBand<br />
|-<br />
| lattice<br />
| Former Lattice cluster<br />
| 307<br />
| 8 cores, 2x Intel(R) Xeon(R) CPU L5520 @ 2.27GHz (Nehalem, 2009)<br />
| 12 GB<br />
| N/A<br />
| 40 Gbit/s InfiniBand<br />
|-<br />
| single<br />
| Former Lattice cluster<br />
| 168<br />
| 8 cores, 2x Intel(R) Xeon(R) CPU L5520 @ 2.27GHz (Nehalem, 2009)<br />
| 12 GB<br />
| N/A<br />
| 40 Gbit/s InfiniBand<br />
|-<br />
| parallel<br />
| Former Parallel Cluster<br />
| 576<br />
| 12 cores, 2x Intel(R) Xeon(R) CPU E5649 @ 2.53GHz (Westmere, 2011)<br />
| 24 GB<br />
| N/A<br />
| 40 Gbit/s InfiniBand<br />
|}<br />
<br />
===ARC Cluster Storage===<br />
Usage of ARC cluster storage is outlined by our [[ARC Storage Terms of Use]] page.<br />
<br />
{{Message Box<br />
| title=No Backup Policy!<br />
| message=You are responsible for your own backups. Many researchers will have accounts with Compute Canada and may choose to back up their data there (the Project file system accessible through the Cedar cluster would often be used). <br />
<br />
Please contact us at support@hpc.ucalgary.ca if you want more information about this option.<br />
<br />
You can also back up data to your UofC OneDrive for business allocation see: https://rcs.ucalgary.ca/How_to_transfer_data#rclone:_rsync_for_cloud_storage This allocation starts at 5TB. Contact the support center for questions regarding OneDrive for Business.<br />
}}<br />
<br />
The ARC cluster has around 2 petabyte of shared disk storage available across the entire cluster as well as temporary storage local to each of the compute nodes. Please refer to the individual sections below on the capacity limitations and usage policies. <br />
<br />
Use the <code>arc.quota</code> command on ARC to determine the available space on your various volumes and home directory.<br />
<br />
{| class="wikitable"<br />
!Partition<br />
!Description<br />
!Capacity<br />
|-<br />
|<code>/home</code><br />
|User home directories<br />
|500 GB (per user)<br />
|-<br />
|<code>/work</code><br />
|Research project storage<br />
|Up to 100's of TB<br />
|-<br />
|<code>/scratch</code><br />
|Scratch space for temporary files<br />
|Up to 15 TB<br />
|-<br />
|<code>/tmp</code><br />
|Temporary space local to the compute cluster<br />
|Dependent on nodes, use <code>df -h</code>.<br />
|-<br />
|<code>/dev/shm</code><br />
|Small temporary in-memory disk space local to the compute cluster<br />
|Dependent on nodes, use <code>df -h</code>.<br />
|}<br />
====<code>/home</code>: Home file system====<br />
Each user has a directory under /home and is the default working directory when logging in to ARC. Each home directory has a per-user quota of 500 GB. This limit is fixed and cannot be increased. Researchers requiring additional storage exceeding what is available on their home directory may use <code>/work</code> and <code>/scratch</code>.<br />
<br />
Note on file sharing: Due to security concerns, permissions set using <code>chmod</code> on your home directory to allow other users to read/write to your home directory be automatically reverted by an automated system process unless an explicit exception is made. If you need to share files with other researchers on the ARC cluster, please write to support@hpc.ucalgary.ca to ask for such an exception.<br />
<br />
====<code>/scratch</code>: Scratch file system for large job-oriented storage====<br />
Associated with each job, under the <code>/scratch</code> directory, a subdirectory is created that can be referenced in job scripts as <code>/scratch/${SLURM_JOB_ID}</code>. You can use that directory for temporary files needed during the course of a job. Up to 15 TB of storage may be used, per user (total for all your jobs) in the <code>/scratch</code> file system. <br />
<br />
Data in <code>/scratch</code> associated with a given job will be deleted automatically, without exception, five days after the job finishes.<br />
<br />
====<code>/work</code>: Work file system for larger projects====<br />
If you need more space than provided in <code>/home</code> and the <code>/scratch</code> job-oriented space is not appropriate for you case, please write to support@hpc.ucalgary.ca with an explanation, including an indication of how much storage you expect to need and for how long. If approved, you will then be assigned a directory under <code>/work</code> with an appropriately large quota.<br />
<br />
====<code>/tmp</code>, <code>/dev/shm</code>: Temporary files====<br />
You may use <code>/tmp</code> for temporary files generated by your job. The <code>/tmp</code> is stored on a disk local to the compute node and is not shared across the cluster. The files stored here may be removed immediately after your job terminates.<br />
<br />
<code>/dev/shm</code> is similar to <code>/tmp</code> but the storage is backed by virtual memory for higher IOPS. This is ideal for workloads that require many small read/writes to share data between processes or as a fast cache. The files stored here may be removed immediately after your job terminates.<br />
<br />
== Using ARC ==<br />
=== Logging in ===<br />
To log in to ARC, connect using SSH to <code>arc.ucalgary.ca</code> on port <code>22</code>. Connections to ARC are accepted only from the University of Calgary network (on campus) or through the University of Calgary General VPN (off campus).<br />
<br />
See [[Connecting to RCS HPC Systems]] for more information.<br />
=== How to interact with ARC ===<br />
<br />
ARC cluster is a collection of several compute nodes connected by a high-speed network. On ARC, computations get submitted as jobs. Once submitted, the jobs are then assigned to compute nodes by the job scheduler as resources become available.<br />
[[File:Cluster.png]]<br />
<br />
You can access ARC with your UCalgary IT user credentials. Once connected, you will get placed in the ARC login node, for basic tasks such as job submission, monitor job status, manage files, edit text, etc. It is a shared resource where multiple users get connected at the same time. Thus, any intensive tasks is not allowed on the login node as it may block other potential users to connect/submit their computations. <br />
[tannistha.nandi@arc ~]$ <br />
The job scheduling system on ARC is called SLURM. On ARC, there are two SLURM commands that can allocate resources to a job under appropriate conditions: ‘salloc’ and ‘sbatch’. They both accept the same set of command line options with respect to resource allocation. <br />
<br />
'''‘salloc’''' is to launch an interactive session, typically for tasks under 5 hours. <br />
Once an interactive job session is created, you can do things like explore research datasets, start R or python sessions to test your code, compile software applications etc.<br />
<br />
a. Example 1: The following command requests for 1 cpu on 1 node for 1 task along with 1 GB of RAM for an hour. <br />
[tannistha.nandi@arc ~]$ salloc --mem=1G -c 1 -N 1 -n 1 -t 01:00:00<br />
salloc: Granted job allocation 6758015<br />
salloc: Waiting for resource configuration<br />
salloc: Nodes fc4 are ready for job<br />
[tannistha.nandi@fc4 ~]$ <br />
<br />
<br />
b. Example 2: The following command requests for 1 GPU to be used from 1 node belonging to the gpu-v100 partition along with 1 GB of RAM for 1 hour. Generic resource scheduling (--gres) is used to request for GPU resources.<br />
[tannistha.nandi@arc ~]$ salloc --mem=1G -t 01:00:00 -p gpu-v100 --gres=gpu:1<br />
salloc: Granted job allocation 6760460<br />
salloc: Waiting for resource configuration<br />
salloc: Nodes fg3 are ready for job<br />
[tannistha.nandi@fg3 ~]$<br />
<br />
Once you finish the work, type 'exit' at the command prompt to end the interactive session,<br />
[tannistha.nandi@fg3 ~]$ exit<br />
[tannistha.nandi@fg3 ~]$ salloc: Relinquishing job allocation 6760460<br />
It is to ensure that the allocated resources are released from your job and now available to other users.<br />
<br />
'''‘sbatch’''' is to submit computations as jobs to run on the cluster. You can submit a job-script.slurm via 'sbatch' for execution. <br />
[tannistha.nandi@arc ~]$ sbatch job-script.slurm<br />
When resources become available, they get allocated to this task. Batch jobs are suited for tasks that run for long periods of time without any user supervision. When the job-script terminates, the allocation is released. <br />
Please review the section on how to prepare job scripts for more information.<br />
<br />
=== Prepare job scripts ===<br />
Job scripts are text files saved with an extension '.slurm', for example, 'job-script.slurm'. <br />
A job script looks something like this:<br />
''#!/bin/bash''<br />
####### Reserve computing resources #############<br />
#SBATCH --nodes=1<br />
#SBATCH --ntasks=1<br />
#SBATCH --cpus-per-task=1<br />
#SBATCH --time=01:00:00<br />
#SBATCH --mem=1G<br />
#SBATCH --partition=cpu2019<br><br />
####### Set environment variables ###############<br />
module load python/anaconda3-2018.12<br><br />
####### Run your script #########################<br />
python myscript.py<br />
<br />
The first line contains the text "#!/bin/bash" to interpret it as a bash script.<br />
<br />
It is followed by lines that start with a '#SBATCH' to communicate with 'SLURM'. You may add as many #SBATCH directives as needed to reserve computing resources for your task. The above example requests for one cpu on a single node for 1 task along with 1GB RAM for an hour on cpu2019 partition.<br />
<br />
Next, you have to set up environment variables either by loading the modules centrally installed on ARC or export path to the software in your home directory. The above example loads an available python module.<br />
<br />
Finally, include the Linux command to execute the local script.<br />
<br />
Note that failing to specify part of a resource allocation request (most notably '''time''' and '''memory''') will result in bad resource requests as the defaults are not appropriate to most cases. Please refer to the section 'Running non-interactive jobs' for more examples.<br />
<br />
=== Software ===<br />
All ARC nodes run the latest version of CentOS 7 with the same set of base software packages. To maintain the stability and consistency of all nodes, any additional dependencies that your software requires must be installed under your account. For your convenience, we have packaged commonly used software packages and dependencies as modules available under <code>/global/software</code>. If your software package is not available as a module, you may also try Anaconda which allows users to manage and install custom packages in an isolated environment.<br />
<br />
For a list of available packages that have been made available, please see [[ARC Software pages]]. <br />
<br />
Please contact us at support@hpc.ucalgary.ca if you need additional software installed.<br />
<br />
==== Modules ====<br />
The setup of the environment for using some of the installed software is through the <code>module</code> command. An overview of [https://www.westgrid.ca//support/modules modules on WestGrid (external link)] is largely applicable to ARC.<br />
<br />
Software packages bundled as a module will be available under <code>/global/software</code> and can be listed with the <code>module avail</code> command.<br />
<syntaxhighlight lang="bash"><br />
$ module avail<br />
</syntaxhighlight><br />
<br />
To enable Python, load the Python module by running:<br />
<syntaxhighlight lang="bash"><br />
$ module load python/anaconda-3.6-5.1.0<br />
</syntaxhighlight><br />
<br />
To unload the Python module, run:<br />
<syntaxhighlight lang="bash"><br />
$ module remove python/anaconda-3.6-5.1.0<br />
</syntaxhighlight><br />
<br />
To see currently loaded modules, run:<br />
<syntaxhighlight lang="bash"><br />
$ module list<br />
</syntaxhighlight><br />
<br />
By default, no modules are loaded on ARC. If you wish to use a specific module, such as the Intel compilers or the Open MPI parallel programming packages, you must load the appropriate module.<br />
<br />
=== Storage ===<br />
Please review the [[#ARC Cluster Storage|ARC Cluster Storage]] section above for important policies and advice regarding file storage and file sharing.<br />
<br />
=== Interactive Jobs ===<br />
The ARC login node may be used for such tasks as editing files, compiling programs and running short tests while developing programs. We suggest CPU intensive workloads on the login node be restricted to under 15 minutes as per [[General Cluster Guidelines and Policies|our cluster guidelines]]. For interactive workloads exceeding 15 minutes, use the '''[[Running_jobs#Interactive_jobs|salloc command]]''' to allocate an interactive session on a compute node.<br />
<br />
The default salloc allocation is 1 CPU and 1 GB of memory. Adjust this by specifying <code>-n CPU#</code> and <code>--mem Megabytes</code>. You may request up to 5 hours of CPU time for interactive jobs.<br />
salloc --time 5:00:00 --partition cpu2019<br />
<br />
Always use salloc or srun to start an interactive job. Do not SSH directly to a compute node as SSH sessions will be refused without an active job running.<br />
<br />
<!-- This information doesn't seem that useful or relevant to running interactive jobs. Move to getting started section?<br />
ARC uses the Linux operating system. The program that responds to your typed commands and allows you to run other programs is called the Linux shell. There are several different shells available, but, by default you will use one called bash. It is useful to have some knowledge of the shell and a variety of other command-line programs that you can use to manipulate files. If you are new to Linux systems, we recommend that you work through one of the many online tutorials that are available, such as the [http://www.ee.surrey.ac.uk/Teaching/Unix/index.html UNIX Tutorial for Beginners (external link)] provided by the University of Surrey. The tutorial covers such fundamental topics, among others, as creating, renaming and deleting files and directories, how to produce a listing of your files and how to tell how much disk space you are using. For a more comprehensive introduction to Linux, see [http://linuxcommand.sourceforge.net/tlcl.php The Linux Command Line (external link)].<br />
--><br />
<br />
=== Running non-interactive jobs (batch processing) ===<br />
Production runs and longer test runs should be submitted as (non-interactive) batch jobs, in which commands to be executed are listed in a script (text file). Batch jobs scripts are submitted using the <code>sbatch</code> command, part of the Slurm job management and scheduling software. #SBATCH directive lines at the beginning of the script are used to specify the resources needed for the job (cores, memory, run time limit and any specialized hardware needed).<br />
<br />
Most of the information on the [https://docs.computecanada.ca/wiki/Running_jobs Running Jobs (external link)] page on the Compute Canada web site is also relevant for submitting and managing batch jobs and reserving processors for interactive work on ARC. One major difference between running jobs on the ARC and Compute Canada clusters is in selecting the type of hardware that should be used for a job. On ARC, you choose the hardware to use primarily by specifying a partition, as described below.<br />
<br />
=== Selecting a Partition ===<br />
There are some aspects to consider when selecting a partition including:<br />
* Resource requirements in terms of memory and CPU cores<br />
* Hardware specific requirements, such as GPU or CPU Instruction Set Extensions<br />
* Partition resource limits and potential wait time<br />
* Software support parallel processing using Message Passing Interface (MPI), OpenMP, etc.<br />
** Eg. MPI for parallel processing can distribute memory across multiple nodes, per-node memory requirements could be lower. Whereas, OpenMP or single process code that is restricted to one node would require a higher memory node.<br />
** Note: MPI code running on hardware with Omni-Path networking should be compiled with Omni-Path networking support. This is provided by loading the <code>openmpi/2.1.3-opa</code> or <code>openmpi/3.1.2-opa</code> modules prior to compiling.<br />
<br />
Since resources that are requested are reserved for your job, please request only as much CPU and memory as your job requires to avoid reducing the cluster efficiency. If you are unsure which partition to use or the specific resource requests that are appropriate for your jobs, please contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] and we would be happy to work with you.<br />
<br />
{| class="wikitable" style="width: 100%;"<br />
!Partition<br />
!Description<br />
!Cores/node<br />
!Memory Request Limit<br />
!Time Limit<br />
!GPU<br />
!Networking<br />
|-<br />
|cpu2021<br />
|General Purpose Compute<br />
|48<br />
|185,000 MB<br />
|7 days ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|cpu2019<br />
|General Purpose Compute<br />
|40<br />
|185,000 MB<br />
|7 days ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|bigmem<br />
|Big Memory Compute<br />
|80<br />
|3,000,000 MB<br />
|24 hours ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|gpu-v100<br />
|GPU Compute<br />
|80<br />
|753,000 MB<br />
|24 hours ‡<br />
|2<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|apophis&dagger;<br />
|Private Research Partition<br />
|40<br />
|185,000 MB<br />
|7 days ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|razi&dagger;<br />
|Private Research Partition<br />
|40<br />
|185,000 MB<br />
|7 days ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|pawson&dagger;<br />
|Private Research Partition<br />
|40<br />
|185,000 MB<br />
|7 days ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|sherlock&dagger;<br />
|Private Research Partition<br />
|7<br />
|185,000 MB<br />
|7 days ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|theia&dagger;<br />
|Private Research Partition<br />
|28<br />
|188,000 MB<br />
|7 days ‡<br />
|<br />
|40 Gbit/s InfiniBand<br />
|-<br />
|synergy&dagger;<br />
|Private Research Partition<br />
|14<br />
|245,000 MB<br />
|7 days ‡<br />
|<br />
|40 Gbit/s InfiniBand<br />
|-<br />
|cpu2013<br />
|Legacy General Purpose Compute<br />
|16<br />
|120000<br />
|7 days ‡<br />
|<br />
|40 Gbit/s InfiniBand<br />
|-<br />
|lattice<br />
|Legacy General Purpose Compute<br />
|8<br />
|12000<br />
|7 days ‡<br />
|<br />
|40 Gbit/s InfiniBand<br />
|-<br />
|parallel<br />
|Legacy General Purpose Compute<br />
|12<br />
|23000<br />
|7 days ‡<br />
|<br />
|40 Gbit/s InfiniBand<br />
|-<br />
|single<br />
|Legacy Single-Node Job Compute<br />
|8<br />
|12000<br />
|7 days ‡<br />
|<br />
|40 Gbit/s InfiniBand<br />
|-<br />
|cpu2021-bf24<br />
|Back-fill Compute (2021-era hardware, 24h)<br />
|48<br />
|185,000 MB<br />
|24 hours ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|cpu2019-bf05<br />
|Back-fill Compute (2019-era hardware, 5h)<br />
|40<br />
|185,000 MB<br />
|5 hours ‡<br />
|<br />
|100 Gbit/s Omni-Path<br />
|-<br />
|cpu2017-bf05<br />
|Back-fill Compute (2017-era hardware, 5h)<br />
|14<br />
|245,000 MB<br />
|5 hours ‡<br />
|<br />
|40 Gbit/s InfiniBand<br />
|-<br />
|+ style="caption-side: bottom; text-align: left; font-weight: normal;" | &dagger; These partitions contain hardware contributed to ARC by particular researchers and should only be used by members of their research groups. However, they have generously allowed their compute nodes to be shared with others outside their research groups for short jobs. A special 'back-fill' or -bf partition is available for use by all ARC users for jobs shorter than 5 hours.<br />‡ As time limits may be changed by administrators to adjust to maintenance schedules or system load, the values given in the tables are not definitive. See the Time limits section below for commands you can use on ARC itself to determine current limits.<br />
|}<br />
<br />
==== Backfill partitions ====<br />
Backfill partitions can be used by all users on ARC for short-term jobs. The hardware backing these partitions are generously contributed by researchers. We recommend including the backfill partitions for short term jobs as it may help reduce your job's wait time and increase the overall cluster throughput.<br />
<br />
Previously, each contributing research group had their own backfill partition. Since June 2021, we have merged:<br />
<br />
* apophis-bf, pawson-bf, and razi-bf into cpu2019-bf05 <br />
* theia-bf and synergy-bf into cpu2017-bf05<br />
<br />
The naming scheme of the backfill partitions is the CPU generation year, followed by -bf and the time limit in hours. For example, cpu2017-bf05 would represent a backfill partition containing processors from 2017 with a time limit of 5 hours.<br />
<br />
==== Hardware resource and job policy limits ====<br />
In addition to the hardware limitations, please be aware that there may also be policy limits imposed on your account for each partition. These limits restrict the number of cores, nodes, or GPUs that can be used at any given time. Since the limits are applied on a partition-by-partition basis, using resources in one partition should not affect the available resources you can use in another partition.<br />
<br />
These limits can be listed by running:<br />
<syntaxhighlight lang="bash"><br />
$ sacctmgr show qos format=Name,MaxWall,MaxTRESPU%20,MaxSubmitJobs<br />
Name MaxWall MaxTRESPU MaxSubmit<br />
---------- ----------- -------------------- ---------<br />
normal 7-00:00:00 2000<br />
breezy 3-00:00:00 cpu=384 2000<br />
gpu 7-00:00:00 13000<br />
cpu2019 7-00:00:00 cpu=240 2000<br />
gpu-v100 1-00:00:00 cpu=80,gres/gpu=4 2000<br />
single 7-00:00:00 cpu=408,node=75 2000<br />
razi 7-00:00:00 2000<br />
</syntaxhighlight><br />
<br />
==== Specifying a partition in a job ====<br />
One you have decided which partitions best suits your computation, you can select one or more partition on a job-by-job basis by including the <code>partition</code> keyword for an <code>SBATCH</code> directive in your batch job. Multiple partitions should be comma separated. If you omit the partition specification, the system will try to assign your job to appropriate hardware based on other aspects of your request. <br />
<br />
In some cases, you really should specify the partition explicitly. For example, if you are running single-node jobs with thread-based parallel processing requesting 8 cores you could use:<br />
<syntaxhighlight lang="bash"><br />
#SBATCH --mem=0 ❶<br />
#SBATCH --nodes=1 ❷<br />
#SBATCH --ntasks=1 ❸<br />
#SBATCH --cpus-per-task=8 ❹<br />
#SBATCH --partition=single,lattice ❺ <br />
</syntaxhighlight><br />
<br />
A few things to mention in this example:<br />
# <code>--mem=0</code> allocates all available memory on the compute node for the job. This effectively allocates the entire node for your job.<br />
# <code>--nodes=1</code> allocates 1 node for the job<br />
# <code>--ntasks=1</code> your job has a single task<br />
# <code>--cpus-per-task=8</code> asks for 8 CPUs per task. This job in total will request 8 * 1, or 8 CPUs.<br />
# <code>--partition=single,lattice</code> specifies that this job can run on either single or lattice.<br />
Suppose that your job requires at most 8 CPU cores and 10 GB of memory. The above Slurm request would be valid and optimal since your job fits neatly in a single node on the single and parallel partition. However, if you failed to specify the partition, Slurm may try to schedule your job to a partition with larger nodes, such as cpu2019 where each node has 40 cores and 190 GB of memory. If your job is scheduled on such a node, your job will be effectively wasting 32 cores and 180 GB of memory because <code>--mem=0</code> not only requests for 190 GB on this node, but also prevents other jobs from being scheduled on the same node.<br />
<br />
If you don't specify a partition, please give greater thought to the memory specification to make sure that the scheduler will not assign your job more resources than are needed.<br />
<br />
Parameters such as '''--ntasks-per-cpu''', '''--cpus-per-task''', '''--mem''' and '''--mem-per-cpu>''' have to be adjusted according to the capabilities of the hardware also. The product of --ntasks-per-cpu and --cpus-per-task should be less than or equal to the number given in the "Cores/node" column. The '''--mem>''' parameter (or the product of '''--mem-per-cpu''' and '''--cpus-per-task''') should be less than the "Memory limit" shown. If using whole nodes, you can specify '''--mem=0''' to request the maximum amount of memory per node.<br />
<br />
===== Examples =====<br />
Here are some examples of specifying the various partitions.<br />
<br />
As mentioned in the [[#Hardware|Hardware]] section above, the ARC cluster was expanded in January 2019. To select the 40-core general purpose nodes specify:<br />
<br />
#SBATCH --partition=cpu2019<br />
<br />
To run on the Tesla V100 GPU-enabled nodes, use the '''gpu-v100''' partition. You will also need to include an SBATCH directive in the form '''--gres=gpu:n''' to specify the number of GPUs, n, that you need. For example, if the software you are running can make use of both GPUs on a gpu-v100 partition compute node, use:<br />
<br />
#SBATCH --partition=gpu-v100 --gres=gpu:2<br />
<br />
For very large memory jobs (more than 185000 MB), specify the bigmem partition:<br />
<br />
#SBATCH --partition=bigmem<br />
<br />
If the more modern computers are too busy or you have a job well-suited to run on the compute nodes described in the legacy hardware section above, choose the cpu2013, Lattice or Parallel compute nodes by specifying the corresponding partition keyword:<br />
<br />
#SBATCH --partition=cpu2013<br />
#SBATCH --partition=lattice<br />
#SBATCH --partition=parallel<br />
<br />
There is an additional partition called '''single''' that provides nodes similar to the lattice partition, but, is intended for single-node jobs. Select the single partition with<br />
<br />
#SBATCH --partition=single<br />
<br />
=== Time limits ===<br />
Use the <code>--time</code> directive to tell the job scheduler the maximum time that your job might run. For example:<br />
#SBATCH --time=hh:mm:ss<br />
<br />
You can use <code>scontrol show partitions</code> or <code>sinfo</code> to see the current maximum time that a job can run.<br />
<syntaxhighlight lang="bash" highlight="6"><br />
$ scontrol show partitions<br />
PartitionName=single <br />
AllowGroups=ALL AllowAccounts=ALL AllowQos=ALL <br />
AllocNodes=ALL Default=NO QoS=single <br />
DefaultTime=NONE DisableRootJobs=NO ExclusiveUser=NO GraceTime=0 Hidden=NO <br />
MaxNodes=UNLIMITED MaxTime=7-00:00:00 MinNodes=1 LLN=NO MaxCPUsPerNode=UNLIMITED <br />
Nodes=cn[001-168] <br />
PriorityJobFactor=1 PriorityTier=1 RootOnly=NO ReqResv=NO OverSubscribe=NO <br />
OverTimeLimit=NONE PreemptMode=OFF <br />
State=UP TotalCPUs=1344 TotalNodes=168 SelectTypeParameters=NONE <br />
DefMemPerNode=UNLIMITED MaxMemPerNode=UNLIMITED <br />
</syntaxhighlight><br />
<br />
Alternatively, with <code>sinfo</code> under the <code>TIMELIMIT</code> column:<br />
<syntaxhighlight lang="bash"><br />
$ sinfo <br />
PARTITION AVAIL TIMELIMIT NODES STATE NODELIST <br />
single up 7-00:00:00 1 drain* cn097 <br />
single up 7-00:00:00 1 maint cn002 <br />
single up 7-00:00:00 4 drain* cn[001,061,133,154] <br />
...<br />
</syntaxhighlight><br />
<br />
== Support ==<br />
{{Message Box<br />
|title=[[Support|Need Help or have other ARC Related Questions?]]<br />
|message=For all general RCS related issues, questions, or comments, please contact us at support@hpc.ucalgary.ca.<br />
|icon=Support Icon.png}}<br />
<br />
Please don't hesitate to [[Support|contact us]] directly by email if you need help using ARC or require guidance on migrating and running your workflows to ARC.<br />
<br />
[[Category:ARC]]<br />
[[Category:Guides]]</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=CloudStack_User_Guide&diff=1904CloudStack User Guide2022-06-09T20:35:15Z<p>Darcy: /* Introduction */</p>
<hr />
<div>This is a user's guide on using CloudStack provided by Research Computing Services.<br />
<br />
== Introduction==<br />
Apache CloudStack is an Infrastructure as a Service (IaaS) platform that allows users to quickly spin up Linux/Non-Windows based virtual machines. RCS is providing this service to help researchers quickly set up and prototype research related software on premises. CloudStack is not appropriate for workloads that depend on Windows. Services set up on CloudStack virtual machines can be accessed from the campus network and also the internet if required.<br />
<br />
Access to CloudStack can be requested via [https://ucalgary.service-now.com/it?id=sc_cat_item&sys_id=e3c1d6e91be48554cca5ecefbd4bcb6c ServiceNow].<br />
<br />
Please refer to our [[CloudStack End User Agreement]] for acceptable uses and requirements.<br />
<br />
== Using your virtual machine ==<br />
You will be able to run whatever virtual machine you wish (with the exception of Windows). Clearly we cannot provide specific management advice on each and every operating system available. We can provide you with some suggestions on important considerations to be aware of.<br />
<br />
=== Educate yourself ===<br />
All operating systems (OS) have user groups, web sites, wikis, or mailing lists somewhere on the internet. They can be a valuable resource. Most OS providers have on-line documentation that describes using their product. For example Rocky Linux, used by RCS, has a [https://docs.rockylinux.org/ documentation site]. These are excellent resources and can help you understand how to manage your virtual machine.<br />
=== Keep security in mind===<br />
To help keep our network and infrastructure safe from cyber attacks, it is critical that your VMs are properly configured to reduce the number of ways that hackers could exploit it. Here are some common tasks that you can do to help harden your VM:<br />
*Ensure that the only services running on your VM are the ones you must run. Each OS has a way of managing what services are running (sysinit, systemd etc). Please ensure that unnecessary services have been disabled.<br />
<br />
*Disable or delete any unused accounts. Many OSs will have pre-configured accounts, and many applications will have pre-configured accounts. Make sure they are either disabled or not allowed to login.<br />
<br />
*All accounts should have strong [https://it.ucalgary.ca/it-security/passwords-do-i-have-change-them/ passwords].<br />
<br />
*Many OS's have the ability to automatically update themselves. If possible please consider doing this. Updates can also be configured to skip certain software if it will interfere with your research, but please be advised that doing so could place your system at risk.<br />
<br />
*If your VM must be exposed to the internet, consider using some kind of end-point security tool to help monitor for and block cyber attacks.<br />
<br />
== Accessing CloudStack ==<br />
===Accessing the CloudStack management console===<br />
<br />
The CloudStack management console is a web-based portal that allows you to view and manage your cloud infrastructure including virtual machines, storage, and network. Any modern web browsers including Chrome, Firefox, Edge, and Safari is supported. <br />
<br />
Access the CloudStack management console is possible only from an IT-managed computer or through the IT General VPN when working on unmanaged machines (eg. AirUC) or when working off campus (eg. at home). Please review the IT [https://ucalgary.service-now.com/it?id=kb_article&sys_id=52a169d6dbe5bc506ad32637059619cd knowledge base article on connecting to the General VPN] or contact IT support if you need assistance connecting to the General VPN. <br />
[[File:CloudStack VPN Connection.png|alt=CloudStack VPN Connection|none|thumb|CloudStack VPN Connection]]<br />
=== Login to CloudStack===<br />
<br />
To log in to CloudStack, navigate to https://cloudstack.rcs.ucalgary.ca/. If this site fails to load, please make sure you are either on a IT managed computer or connected to the General VPN.<br />
<br />
Sign in to CloudStack using the Single Sign-On option as shown in the image below. This method will require you to authenticate through our central authentication service using your University of Calgary IT credentials and will require multi-factor authentication. You must have multi-factor authentication set up either via your phone or with the Microsoft Authenticator app.<br />
[[File:CloudStack Login Page.png|alt=CloudStack Login Page|none|thumb|CloudStack Login Page]]<br />
<br />
'''Note:''' Due to a bug with the UI, if the Single Sign-On option is disabled, please refresh the login page and try again. This issue should be addressed in our next update for CloudStack.<br />
<br />
=== CloudStack Dashboard===<br />
<br />
After logging in, you will be presented with your CloudStack management console. The dashboard shows you a general overview of your account's status.<br />
[[File:CloudStack Dashboard.png|alt=CloudStack Dashboard|none|thumb|CloudStack Dashboard]]On the right hand side of the dashboard, you will also see recent activity and events that was done within your CloudStack account.<br />
<br />
If you wish to see your CloudStack account resource quota and allocation, navigate to: <code>Accounts -> Click on your account -> Resources</code>. <br />
[[File:CloudStack Resource Quota.png|alt=CloudStack Resource Quota|none|thumb|CloudStack Resource Quota]]<br />
<br />
== Working with virtual machines==<br />
<br />
CloudStack allows you to control the lifecycle of virtual machines within your cloud account. VMs may be started, stopped, rebooted, or destroyed within your management console.<br />
<br />
===Create a VM===<br />
<br />
To create a new VM, enter the CloudStack management console and navigate to: <code>Compute -> Instances -> Add Instance</code>[[File:CloudStack Instance Summary.png|alt=CloudStack Instance Summary|thumb|CloudStack Instance Summary|493x493px]]<br />
<br />
Virtual Machines require the following details:<br />
<br />
# '''Deployment zone'''. Your account will already be placed in the appropriate zone.<br />
# '''Boot template or ISO'''. You may choose either a pre-created template or boot from a custom CD-ROM ISO file.<br />
# '''Compute offering'''. You may select an appropriate size for your new VM. Resources will be counted against your account's quota.<br />
# '''Data Disk'''. You may choose to add an additional virtual disk to your VM to store your data. Alternatively, if you wish to use a single virtual disk for your VM, you may choose to override the size of your root disk in step 2 and select 'No thanks' in this step.<br />
# '''Networks'''. You may choose one or more networks your VM should connect to. All CloudStack accounts come with a default network already created and ready to be used.<br />
# '''SSH keypairs'''. For templates that support custom SSH key pairs, you may choose to use a custom SSH keypair to be installed as part of the deployment process.<br />
# '''Advanced settings'''. For templates that support custom user-data (Cloud-Init), you may choose to enable the advanced settings and provide your own Cloud-Init user-data payload. More on this in the advanced tasks section below.<br />
# '''Other VM details'''. You may give your new VM a friendly name and make it part of a group. Groups allow you to group related VMs together for better organization. You may change these details at a later time.<br />
<br />
When you are done, review the instance summary on the right hand side and then click on the 'Launch Virtual Machine' button.<br />
<br />
=== Selecting your VM Operating system ===<br />
Many OSs will provide various editions that are tailored to a specific use case. A desktop VM may not be appropriate when you need to run a database server. The OS provider will have guides on how to choose an edition.<br />
<br />
You may choose to install the operating system to your virtual machine using either pre-built templates or from scratch using an ISO image.<br />
<br />
====Install from a virtual machine template====<br />
<br />
We provide a Rocky Linux 8.5 and a Ubuntu Server 22.04 LTS template for your convenience. These templates are pre-built images with the operating system installed and ready for use. Our templates also support further automated setup configured using Cloud-Init configuration data that can be provided when deploying a new VM. Currently, we offer the following templates: <br />
{| class="wikitable"<br />
!Template<br />
!Cloud-Init Support<br />
!Password Support<br />
!Default Username<br />
|-<br />
|Rocky Linux 8.5<br />
|Yes<br />
|Yes<br />
|rocky<br />
|-<br />
|Ubuntu Server 22.04<br />
|Yes<br />
|Yes<br />
|ubuntu<br />
|}<br />
Rocky Linux is an open source Linux distribution that is binary-compatible with Red Hat Enterprise Linux and is what RCS recommends.<br />
<br />
For templates that support passwords, the generated password that appears after a VM is created is applied to the default username.<br />
<br />
Security note: All VM templates are configured with SSH password authentication enabled. You should be able to SSH to your VM from another system connected to the same guest network. Do not expose port 22 unless required and we highly recommend using key based authentication.<br />
<br />
===== Virtual machine credentials =====<br />
VM templates that have password support will have a randomly generated password set when the VM is first created or when a password reset request is made (available only when the VM is powered off). A randomly generated 6 character password will be displayed when a new password is set and appears as a notification in your CloudStack management console. <br />
[[File:CloudStack VM Password.png|alt=CloudStack VM Password|none|thumb|CloudStack VM Password]]<br />
This password is set on the default username for your template. For example, the Rocky Linux VM template will set this password to the ''''rocky'''<nowiki/>' user account. You may become the super user by logging in as the <code>rocky</code> user and then running <code>sudo su</code>.<br />
<br />
Note: If you specify a custom Cloud-Init config that creates additional users or sets account passwords, the displayed password will be overridden and have no effect.<br />
<br />
==== Install from an ISO image====<br />
We provide various ISO images for popular Linux distributions. You may select one of these ISO images instead of using a pre-built template when deploying a new virtual machine. We currently provide:<br />
{| class="wikitable"<br />
!Distribution<br />
!ISO<br />
|-<br />
|Ubuntu 20.04<br />
|ubuntu-20.04.4-desktop-amd64.iso<br />
<br />
ubuntu-20.04.4-live-server-amd64.iso<br />
|-<br />
|Ubuntu 21.10<br />
|ubuntu-21.10-desktop-amd64.iso<br />
<br />
ubuntu-21.10-live-server-amd64.iso<br />
|-<br />
|Ubuntu 22.04<br />
|ubuntu-22.04-live-server-amd64.iso<br />
|-<br />
|Rocky Linux 8.5<br />
|Rocky-8.5-x86_64-minimal.iso<br />
|-<br />
|Fedora 35<br />
|Fedora-Workstation-Live-x86_64-35-1.2.iso<br />
|}<br />
You may install custom ISO file into your CloudStack account either by directly uploading the ISO through the web console or by providing a URL to the ISO file on the internet.<br />
<br />
Please do not install Windows on our CloudStack infrastructure. It is against our user agreement to run Windows based systems in this infrastructure. If you need a Windows VM, please contact us for alternative solutions.<br />
<br />
===== Register a ISO with a URL=====<br />
[[File:CloudStack Download ISO.png|alt=CloudStack Download ISO|thumb|CloudStack Download ISO|190x190px]]<br />
<br />
To add a custom ISO file from the internet, enter the CloudStack management console and navigate to: <code>Images -> ISOs -> Register ISO</code><br />
<br />
You may check the state of the ISO file by clicking on it and verify the state of the file. If the file is successfully downloaded, its ready state should become ‘true’. The ISO file will only appear in the selection list when the file is downloaded successfully.<br />
[[File:CloudStack ISO Ready.png|alt=CloudStack ISO Ready|none|thumb|172x172px|CloudStack ISO Ready]]<br />
<br />
=====Upload a custom ISO=====<br />
<br />
To upload an ISO file, enter the CloudStack management console and navigate to: <code>Images -> ISOs -> Upload ISO from Local (icon)</code><br />
<br />
[[File:CloudStack Upload ISO.png|alt=CloudStack Upload ISO|none|thumb|CloudStack Upload ISO|217x217px]]<br />
<br />
===Connecting to your VM console===<br />
The CloudStack management console has a KVM (keyboard, video, mouse) feature built-in, allowing you to remotely connect to and interact with your virtual machine. To connect to your virtual machine's console, navigate to: <code>Compute -> Instances -> Your Instance -> View console</code>.<br />
[[File:CloudStack View Console.png|alt=CloudStack View Console|none|thumb|CloudStack View Console]]<br />
<br />
=== Expanding a VM disk ===<br />
[[File:CloudStack Expand Volume.png|alt=CloudStack Expand Volume|thumb|CloudStack Expand Volume]]<br />
Virtual machine disks can be expanded after they are created within CloudStack. However, you will need to expand the partitions and filesystems manually.<br />
<br />
To grow an existing disk:<br />
<br />
# Go into your VM details page and click on ‘Volumes’.<br />
# Click on the volume you wish to expand.<br />
# Click on the ‘Resize Volume’ icon in the top right.<br />
Once the volume has been expanded, you should be able to verify the disk volume has grown with <code>lsblk</code>. There should also be some messages by the kernel when this occurs. However, you will still need to expand any partitions, volumes, and filesystems on your system manually.<br />
<br />
To expand your partition, use the <code>growpart</code> command followed by your disk device and partition number. Eg: <code>/usr/bin/growpart /dev/vda 3</code><br />
<br />
For LVM volume sets, you can expand the volume using the <code>pvresize</code> and <code>lvresize</code> commands:<br />
<br />
* <code>/usr/sbin/pvresize -y -q /dev/vda3</code><br />
* <code>/usr/sbin/lvresize -y -q -r -l +100%FREE /dev/mapper/<volume-name></code><br />
<br />
To expand your filesystem:<br />
<br />
* XFS: <code>/usr/sbin/xfs_growfs <volume></code><br />
* EXT: <code>resize2fs <volume></code><br />
<br />
=== Destroying a VM ===<br />
If you need to delete a VM, click on the red garbage bin icon in the VM instance page. All deletions are irreversible, so please make sure you have a copy of any data you need before proceeding.<br />
[[File:CloudStack Delete VM.png|alt=CloudStack Delete VM|none|thumb|CloudStack Delete VM]]<br />
The VM root volume can be deleted immediately by enabling the 'Expunge' option in the dialog box. If left disabled, the VM root volume will linger for a day before it is deleted by the system. You may wish to expunge a volume if you are running low on space or volume quota.<br />
<br />
<br />
== Virtual machine networking ==<br />
The CloudStack platform allows you to define custom virtual private cloud (VPC) network which can contain any number of guest networks that your virtual machines connect to. Each guest network has its own private network address space and is not directly routable from campus or the internet. For virtual machines that require internet access, the VPC or guest network it is connected to must have a NAT IP address associated. The following diagram shows how a guest network connects to the internet and campus network.<br />
[[File:CloudStack Guest Networking.png|alt=CloudStack Guest Networking|none|thumb|CloudStack Guest Networking]]<br />
In order to expose a virtual machine's services to campus or the internet, the appropriate port forwardings must be set up on the VPC containing the guest network. More on this will be discussed in the next section.<br />
<br />
Having multiple guest networks allows for more advanced network setups but is not required. We recommend using a single flat network for most workloads. <br />
<br />
By default, all CloudStack accounts come with a default VPC and guest network set up with a NAT IP assigned.<br />
<br />
=== IP addresses ===<br />
Due to the design decisions made during the setup of the CloudStack platform, only internal 10.44.12X.X IPs can be assigned to your VPC. These IP addresses are accessible from the university campus network. However, there is a special section of IP addresses that can be accessed from the internet.<br />
{| class="wikitable"<br />
!IP address range<br />
!Accessible from<br />
!Internet IP mapping<br />
|-<br />
|10.44.120.3-128<br />
|Campus, Internet<br />
|10.44.120.X maps to 136.159.140.X (ports 80 and 443 only)<br />
|-<br />
|10.44.120.129-255<br />
|Campus only<br />
|N/A<br />
|-<br />
|10.44.121.0-255<br />
|Campus only<br />
|N/A<br />
|-<br />
|10.44.122.0-255<br />
|Campus only<br />
|N/A<br />
|-<br />
|10.44.123.0-255<br />
|Campus only<br />
|N/A<br />
|}<br />
If you need a service exposed to the internet, please request for a public IP address using our [https://ucalgary.service-now.com/it?id=sc_cat_item&sys_id=e3c1d6e91be48554cca5ecefbd4bcb6c ServiceNow request form]. Additionally, if your service is not port 80 or 443, you must also request for a firewall change request to allow the special port through. <br />
<br />
=== Exposing a network service to campus ===<br />
In order to make a virtual machine be visible to the campus network, you must first set up a port forwarding from a campus IP address to your virtual machine.<br />
<br />
To create a port forwarding, navigate to <code>Network -> VPC -> Select your VPC -> Public IP Addresses</code>. If you do not have any available IP addresses, you will need to click on 'Acquire New IP' and select an available IP address. Click on the IP address you wish to use to create a port forwarding on and then navigate to the 'Port Forwarding' tab. Enter the private port range, the public port range, the protocol, and select the target VM. <br />
<br />
For example, to port forward only HTTP (tcp/80) traffic, you would enter the following:<br />
[[File:CloudStack Port Forwarding.png|alt=CloudStack Port Forwarding|none|thumb|CloudStack Port Forwarding]]Once the port forwarding is created, you should be able to access the service from on campus. If for some reason access to your service does not work, there may be a firewall restriction on IT's network. In such circumstances, please contact us for assistance.<br />
<br />
=== Exposing a network to the internet ===<br />
Exposing a service to the internet is the same as exposing it to campus. However, you must create a port forwarding on an IP address that maps to an internet IP address outlined in the IP address table above. If your account does not have one of these IP addresses available, please request for one on the [https://ucalgary.service-now.com/it?id=sc_cat_item&sys_id=e3c1d6e91be48554cca5ecefbd4bcb6c ServiceNow request form].<br />
<br />
By default, only ports 80 and 443 are allowed through the Internet IP address. For all other ports, please [https://ucalgary.service-now.com/it?id=sc_cat_item&sys_id=47cd16d113153a00b5b4ff82e144b0bf create a firewall rule change request in ServiceNow].<br />
<br />
== Cloud-Init Automation ==<br />
<br />
=== Ubuntu ===<br />
The following Cloud-Init configs apply to Ubuntu VM templates.<br />
<br />
==== Ubuntu desktop ====<br />
Use the following Cloud-Init config with the Ubuntu Server template to set up an Ubuntu desktop environment. The setup step takes a up to 15 minutes to complete and you should see a login screen when the setup finishes. <br />
<br />
Adjust the root and user password as desired. Below, the test user password is set to blank, allowing you to login to Gnome without a password.<br />
{{Highlight|code=#cloud-config<br />
disable_root: false<br />
<br />
users:<br />
- name: user<br />
lock_passwd: false<br />
inactive: false<br />
gecos: Test User<br />
primary_group: user<br />
groups: wheel<br />
passwd: $1$ADUODeAy$eCJ1lPSxhSGmSvrmWxjLC1<br />
<br />
chpasswd:<br />
list: {{!}}<br />
root:password<br />
expire: false<br />
<br />
# Install a graphical desktop<br />
runcmd:<br />
- DEBIAN_FRONTEND=noninteractive apt -y upgrade<br />
- DEBIAN_FRONTEND=noninteractive apt -y install tasksel<br />
- tasksel install gnome-desktop<br />
- systemctl set-default graphical.target<br />
- systemctl isolate graphical.target|style=max-height: 300px;overflow:scroll;}}<br />
<br />
=== Rocky Linux ===<br />
The following Cloud-Init configs apply to Rocky Linux templates.<br />
<br />
==== Rocky Linux Desktop ====<br />
Use the following Cloud-Init config using the Rocky Linux template to set up a Rocky Linux desktop environment. The setup step takes up to 10 minutes to complete and you should see a login screen when the setup finishes. <br />
<br />
Adjust the root and user password as desired. Below, the test user password is set to blank, allowing you to login to Gnome without a password.<br />
{{Highlight|code=#cloud-config<br />
disable_root: false<br />
<br />
users:<br />
- name: user<br />
lock_passwd: false<br />
inactive: false<br />
gecos: Test User<br />
primary_group: user<br />
groups: wheel<br />
passwd: $1$ADUODeAy$eCJ1lPSxhSGmSvrmWxjLC1<br />
<br />
chpasswd:<br />
list: {{!}}<br />
root:password<br />
expire: false<br />
<br />
# Install a graphical desktop<br />
runcmd:<br />
- yum -y install "@Workstation"<br />
- systemctl set-default graphical.target<br />
- systemctl isolate graphical.target}}<br />
<br />
==== Rocky Linux Docker host ====<br />
Use the following Cloud-Init config using the Rocky Linux template to set up a new docker host. This server can then be used to run Docker containers. Also included are:<br />
<br />
# The docker-compose utility to help deploy container stacks more easily<br />
# A helper script to expand the <code>/var</code> and <code>/</code> filesystems on first startup based on the available space in the ROOT volume. <br />
<br />
Use the CloudStack generated password with the '<code>rocky</code>' default user account to log in.{{Highlight|code=#cloud-config<br />
<br />
write_files:<br />
- path: /usr/bin/expand_lvm_root<br />
content: {{!}}<br />
#!/bin/bash<br />
/usr/bin/growpart /dev/vda 3<br />
/usr/sbin/pvresize -y -q /dev/vda3<br />
/usr/sbin/lvresize -y -q -r -l +50%FREE /dev/mapper/*root<br />
/usr/sbin/lvresize -y -q -r -l +100%FREE /dev/mapper/*var/dev/mapper/*root<br />
/usr/sbin/xfs_growfs /<br />
/usr/sbin/xfs_growfs /var<br />
permissions: '0700'<br />
<br />
- path: /usr/bin/setup_docker<br />
content: {{!}}<br />
#!/bin/bash<br />
yum install -y yum-utils<br />
yum-config-manager \<br />
--add-repo \<br />
https://download.docker.com/linux/centos/docker-ce.repo<br />
yum install -y docker-ce docker-ce-cli containerd.io<br />
systemctl start docker<br />
systemctl enable docker<br />
<br />
curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose<br />
chmod +x /usr/local/bin/docker-compose<br />
permissions: '0700'<br />
<br />
- path: /root/docker-compose.yml<br />
permissions: '0700'<br />
content: {{!}}<br />
version: '3.3'<br />
services:<br />
web:<br />
image: php:7.4-apache<br />
restart: always<br />
user: "0:0"<br />
volumes:<br />
- /var/www/html:/var/www/html<br />
ports:<br />
- "80:80"<br />
<br />
- path: /var/www/html/index.php<br />
permissions: '0644'<br />
content: {{!}}<br />
<h1>Hello there!</h1><br />
<p>I see you from <?php echo $_SERVER['REMOTE_ADDR']; ?></p><br />
<<nowiki>pre</nowiki>><?php print_r($_SERVER); ?></pre><br />
<br />
# Ensure VM has the largest / possible<br />
runcmd:<br />
- /usr/bin/expand_lvm_root<br />
- /usr/bin/setup_docker<br />
- cd /root; docker-compose up -d|style=max-height: 300px;overflow:scroll;}}<br />
<br />
==== UC Authentication ====<br />
Use the following Cloud-Init config to allow UC-based authentication. Local accounts with the same username as the IT account may use the IT credential to log in.<br />
{{Highlight|code=#cloud-config<br />
<br />
write_files:<br />
- path: /usr/bin/setup_uc_auth<br />
permissions: '0700'<br />
content: {{!}}<br />
#!/bin/bash<br />
yum install -y sssd sssd-dbus sssd-krb5 krb5-workstation authselect-compat<br />
<br />
cat <<EOF > /etc/sssd/sssd.conf<br />
[sssd]<br />
config_file_version = 2<br />
services = nss, pam, ifp<br />
domains = uc.ucalgary.ca<br />
<br />
[domain/uc.ucalgary.ca]<br />
id_provider = files<br />
debug_level = 5<br />
auth_provider = krb5<br />
chpass_provider = krb5<br />
<br />
krb5_realm = UC.UCALGARY.CA<br />
krb5_server = ITSODCSRV14.UC.UCALGARY.CA:88<br />
krb5_validate = false<br />
EOF<br />
chmod 600 /etc/sssd/sssd.conf<br />
<br />
cat <<EOF > /etc/krb5.conf<br />
[logging]<br />
default = FILE:/var/log/krb5libs.log<br />
kdc = FILE:/var/log/krb5kdc.log<br />
admin_server = FILE:/var/log/kadmind.log<br />
<br />
[libdefaults]<br />
default_realm = UC.UCALGARY.CA<br />
dns_lookup_realm = false<br />
dns_lookup_kdc = false<br />
ticket_lifetime = 24h<br />
renew_lifetime = 7d<br />
forwardable = true<br />
<br />
[realms]<br />
UC.UCALGARY.CA = {<br />
kdc = itsodcsrv14.uc.ucalgary.ca<br />
}<br />
<br />
[domain_realm]<br />
uc.ucalgary.ca = UC.UCALGARY.CA<br />
.uc.ucalgary.ca = UC.UCALGARY.CA<br />
EOF<br />
<br />
mkdir -p /etc/authselect/custom/rcs<br />
cd /etc/authselect/custom/rcs<br />
echo H4sIAMYsbGIAA+0ca3PbuDGf+StQ34dLMpEtybLdUevO6BKl56mTuJGv7Uwmo4FISMKJBHQAaEfNub+9uwBJkXrLshxnjjuTyCKJfWF3sQsuFPhS9CtB79keoQpw2mjYT4DZz1r1uPGs1qhX6yf12vFx/Vm1Vqs1jp+R6j6ZSiHWhipCnikpzarn1t3/TuErj8bhhBzccjOs6Igq41MVHBDen71WUey3mCsWHNx5W4wKpT+qSAGjI3lDQxj8SarB0UDIiB2FcsBFRfuKMfHZY4L2QpYbS2MzZMJwnxouxfnXefQHTaNi9nufhprdpQj6XAyYGisuzHIUuYcWIhlTrW/lRkxMNdO0OH5HdHdeXlDNjAGCuhJQ0IM4GjPFx0Om4PGjDM9n8pULP4wDtpkik78q1Lec/WifcMr8kSyE7fB/a9ssYf8Q2PiPc6/3RmNN/K826mez8f/07LSM/48BR4uD8dGaWOytGbc8BK8buSTu5odtFEyPitGxDH2LYXai9kFjjf8fn4HPT/3/GPy/fnZ6Uvr/Y8BXiP/gTHHOL/KZ0Z2HRpGmD2meszi5mIExjbpM3BxquRuOgIkJICE/kE7qsKQYGwjXRbSrfD2Xxe7EVp/yEOMGsjZWzOLSPASuCHJ83iCxwPtdwyN2XqtXqwvYSpHMMKPjfp/7HHFtyIydrWBnXeeFQkT4fak4ZImulwh134n3POr7Mk50sTUWGMy0RoE2hnmRAJHDgzLtwk1ewffmJq/gXZiJBf9iTSaHZGvTA0ZoGGumdsakY1AxC7q8j9qJeUD+SmqwPhAQi5kC7k8B69M4NOc9GhA7TutzOSLIB4g1EvJWnHPIFhT77HBbE9hNXZBmRNxYi0yTlHvhyexaA9cYvhDkGCMZDTfGMmITLiw7wMGNHLECuq2ZCjnIZnVUyeHZmis90YZFNhDtJJ0Mgl9lrxuNhpDwBVxZi4ioHp1Xq2dni0bNu0k2GPwkz8yn1GBqJDWj1FSWCFUwTDCxG+4DHUF8JUXgzBNNrws2u9sspB65k+4SW99s/RdgyNz4w0OsA/eUY6yr/05rtdn8r35S1n+PAjaSBU1rPGA4pA8pjCaJG6NbsS8zbuWDxmTk6jTMogZKxmOHYDsMdiAgEMxkOKYIcp69DEE60CUaMsLQ3twSRzbQBgnr2XprRtKBiCMOJFMWRaoJXcAxn5rCiG9XcRbK7T3RWOP/pyen+fqvgf5/XC3rv0eBb17eYTIbsJDaGs/+cV53dlGWjfsuG+O6zWh8qP43hYW1UQUQ7aqYHC9CYh4P6fx9eanU+3SGn6xiqBGX7CU5X656mKZ/i6uRv52T+rQa2QV3sWbaaQZdwkjwrZSQxqlBxqYiYrCQ0UHTfd4RoybdPlfadDHiz0+V5mYjI1ivmm4i9W6CuSSW9KW6BfdbwnS5tVFubSxmJvUL0oM4wkRXD2kgb70fcnojt0CKUCB3S5IrYG3EZ8pQqO1sZqkJ+JLig6H5C/CgMUaPmQjgsiA6jFW0247Hg0yg5WNXfVkkXdDR2GzKzmacPLH9pPr3sZ20TTQe3/4W05Abm+0UwzyxyuyiGLorRTgpULr3YgPOdFKrE+dT2y49dqMEg6eRo93YSZaIZfjKfbkcmnJf7jvbl3ssGEtt7Gv4PdJYt/93Um/M1P81iC9l/f8YsKFFWydAP7eeZuvVTX13uaf+6X9kEEQv8991/DJZnwvYswJrLhTYGEmtDSNqWKlNNCa5Jj3LayV55KDpvv+uh/IWc00W3G3q1DOUkqIdatVxQGG1nmJ8Yh6+Gj62W2/etfdLY43/n1VPT3P+XwP/rzaOy/3/R4G27boinU7njS1/XKIwu1P2nIauGHcZpU2MsQgKJy+8813B8zosZD62dBEz5JqMlcStc1egsRyDVMMD4GoyVhAuZJ/wAJk0E4+Kud09wHID95U+BAI4OrmgCYWIY3C46x8DpJJEVNABSwtBg8lbJMGtIU0B1qTiTC8iEjF/SAXXkca6YYgcXr5pXb0i/2Cqx5QEJSnyVjF2cdU6JBcmYwKQkfedDkGcV613kLUYpvoUxDISdzycoMlkIF0yDuPBwOqiR32oaYPktgfM+lIIYNMKAnGSj0PkvN9nCmNUWqY4vaE+3uHuFBcwoZGTg/Ygc3dK9oGzHoPJjoEE7pIaOyO/Iv4x6KjpDY0Z6+bREXyLFTvk8ggHHkF+E3jeT5N0BXhVMBaK+2EQ6CHr6isZOVqKgpgKZRXJw+6dCYRvvO+BMbj5D8hziK/sxj08yU8DIzecZlnai0NyjTY0pDcMlQZKRSGFz7weE6zPQQ2QkbJUVn/InFE5wtgNhSqDJLIPsR4+KxEYgpq4R+1UmGGsPcgeY9AlEXEE82yNiesRrB9MWY2iln92HL9CcSYyJoG0hdItFXamRoyNEzXEQqD1g7bhOQXG6A+5YK/sKJgPzz5a8A1tXQb0ghz9GuPWBNdTT0mWVFQGw50LNAox8PAlL4dJc5Nu/Usb/B+KphEIr9DYwwkBtvG2GTpsCWpHLDep1iOSjRKLrIfEaDLD+Ql1bhROUl98/eH924u///KxdX3x4b1XmQPPa8EMOyFBb6hoiUpDH0MEP9qpmspySK5CRjWozNJ3j3iB9OMIptOJazDzYATsiDiXscPBDwx6P9TQwB75ADGtIGIqF72Bxd0qGFFNnRPmFm2Y4dsEIQn7wrWNZM6+CjyCUP9qXVy2frpskw9XKHnrkrxtt65/+djuLFBCThuFzbFm08OEJAndqXfjHSSM9Q9oAk3SSBvYJsiHZn5s+A2zQ2eiGCIGTYDN4k2vWKXNUMP3pTDKhyKLudFACB/Nh0q7VKSLBOqHK4vEbgMQW28cJkJlr1jm6OQ5tLaYPYoeC1Y3cLaQzj2aCITPITUWUfY0RObxWCqYZXQTnEpczNiXcch9bq09CTFIhAs7eKGNzXI82zecSHAJV4lraia3GNlojhf7zgmeZsE8urQsTfBs8t7qkLyXRNogGjEKSxlMxkzjNKJ67opwNI90i+RF5rGJ+Ck/ufbP9VOSe9h6no3Pbmpe5hoTX6bIkxc0KxFjOIdnwO3FIGRFfHB9Fhe+7Cniq0NUgqUUTPG+qItVQ4L9jQvfEIEx9OfrAdC91pg+BLFCHRcNPA5kgqFlN5vxAkYg8I8sraFpUmNzMHxAxXYhBOsJAp762REz/lHSYpBTg8taEiKvhwzsL9laRgNOuUoDBeoEvPS/eaOe7tsVhQXixF0HeRWNmMEZltNNEc97c9GxEa1z1X598fbiNaY1/764fv0zedO6bv3U6qyObdMY9x4XagjCr0jalAQJmqE9cG1tM7bexAbedAkEJ1K3ihu4C84XJw+BA2PgqRQam9w62Jeof7vQ2jJP21wHNO47G0yWGyO95P2dRbuINK57OH14TQqWECbzhD0uYJWgmXMVmmdA1ReO0EHSTZMJvIDoDArXMjPF4DphtkCQtd1McWQNNVugmXbeTPFMm2q2QJR130zxZH0126BJGnByWNyVdUja/2m9u7rMGavnvST58sitQeibOP1zy4vnkbwVpR+QFJMsDpBixF9OIV2ykJJdaNkmayzxCkvsOo6yJX7KIKRn7TZpXXY+TNXwEnMIO8ya9PM/v/iudjhKWAUf2//85eJj+137/XVnXzTWnv85qc7s/+C3cv/nMeAdHWGfinKJc6F4xDwzq5Jc7ZemiqTDkpBVLLEwJEUzewuuqNgPrOjYOrjzKjPhFgVK6+ZXENRSyfWKRDutDXguAjbX0CWkYpeYA8ySsJ/Avp8k5+RaxewAUX2CO5/hGXc08aHk3ZuaFzVqFA6JJZrOVwNLdJ0UBOttbEPKjynztNMtkTe5sExW61E2yaCRLTsiqBxCKzOmhZDrzr+CXEUV7cot+Ji32k2z6S7GiE3w3brbRLUkYZDfH+AmwNp3lKup7gvW9e8VtYwXH0rT6yh/A01nlB9P2/kX8ZXZlHCJphc1AaTavg9dQuymJtN2oxp3nC3yQoAoRoSHkdfGaLuD55vsJUNC+zClvQc9F+li7mEehuwausvW/+IB//3kGOvyv2L//wn2/zfqtTL/ewy45+//zJ8azyciT+JQwZPpsk/61GynaTfiGleOrqBQza/+LaHkr2kG+YBHD8oO6D9IB/RTbsZ9ooe7n37751Zsfdv2z7L/c8ruU+z/dJaxz9Of6/K/WrXWaMzkf7Wzk7My/3sMuF/+9yQSvH2cGt1n4vhA8U1IyLzdC57mIIiaX+DfCD9dy4GmN0w17Q+lVfJXRrkvxdB4n3OvmSwBvv4snnpMhQt4IlkuCZ7LaZOx9m/cNIZQazttyL2O4+7ww0kbmQhZyNbyH8wqT+R+hydyN1LQol+DWFYCZ9zU98jNyjcDy1131u9S1la4dM6d8WjX1JXvw0154jmjUZ54XoulrPeXMlOeeC5PPJcnnssTz+WJ52+95VXueGXsPsUdrxJKKKGEEkoooYQSSiihhBJKKKGEEkr448D/AT019aAAeAAA {{!}} base64 -d {{!}} tar -xzpf -<br />
<br />
if [[ "`authselect current -r`" != "custom/rcs" ]] ; then<br />
authselect select custom/rcs --force<br />
systemctl restart sssd<br />
fi<br />
<br />
systemctl enable sssd<br />
<br />
runcmd:<br />
- /usr/bin/setup_uc_auth|style=max-height: 300px;overflow:scroll;}}<br />
<br />
==== Rocky Linux NFS server ====<br />
Because there is no ability to share storage among multiple VMs, a local NFS server could be useful if you need to share data between multiple VMs.{{Highlight|code=#cloud-config<br />
<br />
write_files:<br />
- path: /usr/bin/expand_lvm_root<br />
content: {{!}}<br />
#!/bin/bash<br />
/usr/bin/growpart /dev/vda 3<br />
/usr/sbin/pvresize -y -q /dev/vda3<br />
/usr/sbin/lvresize -y -q -r -l +50%FREE /dev/mapper/*root<br />
/usr/sbin/lvresize -y -q -r -l +100%FREE /dev/mapper/*var<br />
/usr/sbin/xfs_growfs /<br />
/usr/sbin/xfs_growfs /var<br />
permissions: '0700'<br />
<br />
- path: /usr/bin/setup_nfs<br />
content: {{!}}<br />
#!/bin/bash<br />
yum install -y nfs-utils<br />
<br />
mkdir /export<br />
if [ -b /dev/vdb ] ; then<br />
mkfs.xfs /dev/vdb<br />
echo "/dev/vdb /export xfs defaults 1 2" >> /etc/fstab<br />
mount -a<br />
fi<br />
<br />
ip a {{!}} grep -w inet {{!}} awk '{print $2}' {{!}} while read subnet ; do<br />
echo "/export $subnet(rw,no_subtree_check,no_root_squash,async)" >> /etc/exports<br />
done<br />
<br />
systemctl start nfs-server<br />
systemctl enable nfs-server<br />
<br />
exportfs -ra<br />
<br />
permissions: '0700'<br />
<br />
runcmd:<br />
- /usr/bin/expand_lvm_root<br />
- /usr/bin/setup_nfs|style=max-height: 300px;overflow:scroll;}}NFS clients connected to the same network as the NFS server can then mount <code>/export</code> using a command similar to: <code>mount -t nfs nfs-server:/export /mnt</code>.<br />
<br />
== Infrastructure tools ==<br />
<br />
=== Generating a CloudStack API ===<br />
You can request for a CloudStack API key to automate infrastructure deployment using Terraform or CloudMonkey. A new API key can be generated by navigating to your profile page (top right) and then clicking on the 'Generate keys' button.<br />
[[File:CloudStack API Key.png|alt=CloudStack API Key|none|thumb|CloudStack API Key]]<br />
<br />
=== CloudMonkey ===<br />
CloudMonkey is a utility that makes it easier to interact with the CloudStack API. This tool may be used to help automate VM actions (such as start/stop/reboot), or infrastructure tasks (such as creating/destroying VMs, networks, or firewall rules). <br />
<br />
To get started with CloudMonkey, refer to the following resources:<br />
<br />
* Download from: <nowiki>https://github.com/apache/cloudstack-cloudmonkey/releases/tag/6.1.0</nowiki><br />
* Documentation at: <nowiki>https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+cloudmonkey+CLI</nowiki><br />
<br />
=== Terraform Integration ===<br />
Terraform allows you to define infrastructure as code and can be used in conjunction with CloudStack to configure your virtual machines and guest networks. Use the official CloudStack provider.<br />
<br />
The following is an example Terraform file for reference. Specify your CloudStack API keys either as a separate <code>vars.tf</code>.<br />
{{Highlight|code=# Configure the CloudStack Provider<br />
terraform {<br />
required_providers {<br />
cloudstack = {<br />
source = "cloudstack/cloudstack"<br />
version = "0.4.0"<br />
}<br />
}<br />
}<br />
<br />
provider "cloudstack" {<br />
api_url = "${var.cloudstack_api_url}"<br />
api_key = "${var.cloudstack_api_key}"<br />
secret_key = "${var.cloudstack_secret_key}"<br />
}<br />
<br />
# Create a new VPC<br />
resource "cloudstack_vpc" "default" {<br />
name = "wedgenet-vpc"<br />
display_text = "wedgenet-vpc"<br />
cidr = "100.64.0.0/20"<br />
vpc_offering = "Default VPC offering"<br />
zone = "zone1"<br />
}<br />
<br />
# Create a new ACL<br />
resource "cloudstack_network_acl" "default" {<br />
name = "vpc-acl"<br />
vpc_id = "${cloudstack_vpc.default.id}"<br />
}<br />
<br />
# One ingress and one egress rule for the ACL<br />
resource "cloudstack_network_acl_rule" "ingress" {<br />
acl_id = "${cloudstack_network_acl.default.id}"<br />
<br />
rule {<br />
action = "allow"<br />
cidr_list = ["0.0.0.0/0"]<br />
protocol = "tcp"<br />
ports = ["22", "80", "443"]<br />
traffic_type = "ingress"<br />
}<br />
}<br />
<br />
resource "cloudstack_network_acl_rule" "egress" {<br />
acl_id = "${cloudstack_network_acl.default.id}"<br />
<br />
rule {<br />
action = "allow"<br />
cidr_list = ["0.0.0.0/0"]<br />
protocol = "all"<br />
traffic_type = "egress"<br />
}<br />
}<br />
<br />
<br />
# Create a new network in the VPC<br />
resource "cloudstack_network" "primary" {<br />
name = "primary"<br />
display_text = "primary"<br />
cidr = "100.64.1.0/24"<br />
network_offering = "DefaultIsolatedNetworkOfferingForVpcNetworks"<br />
acl_id = "${cloudstack_network_acl.default.id}"<br />
vpc_id = "${cloudstack_vpc.default.id}"<br />
zone = "zone1"<br />
}<br />
<br />
# Create a new public IP address for this network<br />
resource "cloudstack_ipaddress" "public_ip" {<br />
vpc_id = "${cloudstack_vpc.default.id}"<br />
network_id = "${cloudstack_network.wosnet1.id}"<br />
}<br />
<br />
# Create VMs. <br />
resource "cloudstack_instance" "vm" {<br />
count = 1<br />
name = "vm${count.index+1}"<br />
zone = "zone1"<br />
service_offering = "rcs.c4"<br />
template = "RockyLinux 8.5"<br />
network_id = "${cloudstack_network.wosnet1.id}"<br />
<br />
# Cloud Init data can be used to configure your VM on first startup if your template supports Cloud Init<br />
user_data = <<EOF<br />
#cloud-config<br />
<br />
# Require specific packages<br />
packages:<br />
- tmux<br />
- git<br />
- tcpdump<br />
<br />
EOF<br />
}<br />
<br />
<br />
|style=max-height: 300px;overflow:scroll;}}<br />
<br />
= Troubleshooting =<br />
<br />
=== Cannot create a volume snapshot ===<br />
Volume snapshots can only be taken on VMs that are powered off.<br />
<br />
=== Cannot create a VM snapshot ===<br />
Disk-only VM snapshots cannot be taken when the VM is running. If you intend to snapshot a running system, you must also snapshot its memory.<br />
<br />
=== VM state is still running after shutdown ===<br />
After running 'shutdown' on a VM, the VM state reported by CloudStack is still running. <br />
<br />
Please try to do a force shutdown from the CloudStack management console. The VM state isn't updated by CloudStack and as a result, the state of a VM isn't properly reflected when power state changes outside of CloudStack (likely a bug?)</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=CloudStack&diff=1901CloudStack2022-06-09T16:57:44Z<p>Darcy: </p>
<hr />
<div>Apache CloudStack is an Infrastructure as a Service (IaaS) offering that allows researchers to quickly deploy virtual machines for research projects. This service is part of Research Computing Services' Digital Research Infrastructure (DRI) and is free for all University of Calgary researchers and principal investigators.<br />
<br />
== Use cases ==<br />
CloudStack allows you to create virtual machines for a wide range of workloads and use cases, including:<br />
<br />
* Running an internal or public facing web site<br />
* Running a database<br />
* Experiment with new software tools<br />
* Test out the latest release of a software package<br />
Please note that CloudStack is offered as a research environment and is supported as such. For workloads that demand high availability and high uptime, this may not be the appropriate choice. Researchers are solely responsible for maintaining any VMs they deploy. The expectation is that the owners of the VMs will be patching and doing other maintenance work on a regular basis. We will be more than happy to provide guidance on this process. <br />
<br />
Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] for details.<br />
<br />
=== Differences between RCS HPC and CloudStack ===<br />
There are some overlaps between the CloudStack offering and our existing High Performance Computing (HPC) cluster environment.<br />
{| class="wikitable"<br />
!<br />
!RCS HPC Cluster<br />
!CloudStack<br />
|-<br />
|CPU intensive workloads<br />
|Yes; 48 CPUs per node, 100's of nodes<br />
|No; 1-8 CPUs per VM<br />
|-<br />
|Memory intensive workloads<br />
|Yes; up to 2TB memory per node<br />
|No; up to 32GB memory per VM<br />
|-<br />
|High storage requirement workloads<br />
|Yes; shared multi-petabyte storage<br />
|No; up to 1TB per account<br />
|-<br />
|Data classification<br />
|Level 1 & 2 (ARC), Level 3 & 4 (MARC)<br />
|Level 1 & 2 only<br />
|-<br />
|Customized software requirements<br />
|Yes; use singularity containers<br />
|Yes<br />
|-<br />
|Custom OS configuration<br />
|No<br />
|Yes<br />
|-<br />
|Persistent software or services<br />
|No; time limited jobs only<br />
|Yes<br />
|-<br />
|Managed environment<br />
|Yes<br />
|No; self managed VMs only<br />
|-<br />
|Research support by analysts<br />
|Yes<br />
|Limited<br />
|}<br />
Not sure if you need a virtual machine or a compute cluster? Is this the "Cloud"? Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] and we will assist you in using this service to support your research goals.<br />
<br />
== Getting access to CloudStack ==<br />
If you are a researcher or principal investigator, please review our [[CloudStack End User Agreement]] and then request a CloudStack account through [https://ucalgary.service-now.com/it?id=sc_cat_item&sys_id=e3c1d6e91be48554cca5ecefbd4bcb6c Service Now].<br />
<br />
Once your account is ready, please refer to the [[CloudStack User Guide]] for details on using CloudStack.<br />
<br />
__NOTOC__</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=CloudStack_User_Guide&diff=1870CloudStack User Guide2022-06-03T14:04:48Z<p>Darcy: /* Keep security in mind */</p>
<hr />
<div>This is a user's guide on using CloudStack provided by Research Computing Services.<br />
<br />
== Introduction==<br />
Apache CloudStack is an Infrastructure as a Service (IaaS) platform that allows users to quickly spin up Linux/Non-Windows based virtual machines. RCS is providing this service to help researchers quickly set up and prototype short-term research related software on premises. CloudStack is not appropriate for workloads that depend on Windows. Services set up on CloudStack virtual machines can be accessed from the campus network and also the internet if required.<br />
<br />
Access to CloudStack can be requested via [https://ucalgary.service-now.com/it?id=sc_cat_item&sys_id=e3c1d6e91be48554cca5ecefbd4bcb6c ServiceNow].<br />
<br />
Please refer to our [[CloudStack End User Agreement]] for acceptable uses and requirements.<br />
<br />
== Using your virtual machine ==<br />
You will be able to run whatever virtual machine you wish (with the exception of Windows). Clearly we cannot provide specific management advice on each and every operating system available. We can provide you with some suggestions on important considerations to be aware of.<br />
<br />
=== Educate yourself ===<br />
All operating systems (OS) have user groups, web sites, wikis, or mailing lists somewhere on the internet. They can be a valuable resource. Most OS providers have on-line documentation that describes using their product. For example Rocky Linux, used by RCS, has a [https://docs.rockylinux.org/ documentation site]. These are excellent resources and can help you understand how to manage your virtual machine.<br />
=== Keep security in mind===<br />
To help keep our network and infrastructure safe from cyber attacks, it is critical that your VMs are properly configured to reduce the number of ways that hackers could exploit it. Here are some common tasks that you can do to help harden your VM:<br />
*Ensure that the only services running on your VM are the ones you must run. Each OS has a way of managing what services are running (sysinit, systemd etc). Please ensure that unnecessary services have been disabled.<br />
<br />
*Disable or delete any unused accounts. Many OSs will have pre-configured accounts, and many applications will have pre-configured accounts. Make sure they are either disabled or not allowed to login.<br />
<br />
*All accounts should have strong [https://it.ucalgary.ca/it-security/passwords-do-i-have-change-them/ passwords].<br />
<br />
*Many OS's have the ability to automatically update themselves. If possible please consider doing this. Updates can also be configured to skip certain software if it will interfere with your research, but please be advised that doing so could place your system at risk.<br />
<br />
*If your VM must be exposed to the internet, consider using some kind of end-point security tool to help monitor for and block cyber attacks.<br />
<br />
== Accessing CloudStack ==<br />
===Accessing the CloudStack management console===<br />
<br />
The CloudStack management console is a web-based portal that allows you to view and manage your cloud infrastructure including virtual machines, storage, and network. Any modern web browsers including Chrome, Firefox, Edge, and Safari is supported. <br />
<br />
Access the CloudStack management console is possible only from an IT-managed computer or through the IT General VPN when working on unmanaged machines (eg. AirUC) or when working off campus (eg. at home). Please review the IT [https://ucalgary.service-now.com/it?id=kb_article&sys_id=52a169d6dbe5bc506ad32637059619cd knowledge base article on connecting to the General VPN] or contact IT support if you need assistance connecting to the General VPN. <br />
[[File:CloudStack VPN Connection.png|alt=CloudStack VPN Connection|none|thumb|CloudStack VPN Connection]]<br />
=== Login to CloudStack===<br />
<br />
To log in to CloudStack, navigate to https://cloudstack.rcs.ucalgary.ca/. If this site fails to load, please make sure you are either on a IT managed computer or connected to the General VPN.<br />
<br />
Sign in to CloudStack using the Single Sign-On option as shown in the image below. This method will require you to authenticate through our central authentication service using your University of Calgary IT credentials and will require multi-factor authentication. You must have multi-factor authentication set up either via your phone or with the Microsoft Authenticator app.<br />
[[File:CloudStack Login Page.png|alt=CloudStack Login Page|none|thumb|CloudStack Login Page]]<br />
<br />
'''Note:''' Due to a bug with the UI, if the Single Sign-On option is disabled, please refresh the login page and try again. This issue should be addressed in our next update for CloudStack.<br />
<br />
=== CloudStack Dashboard===<br />
<br />
After logging in, you will be presented with your CloudStack management console. The dashboard shows you a general overview of your account's status.<br />
[[File:CloudStack Dashboard.png|alt=CloudStack Dashboard|none|thumb|CloudStack Dashboard]]On the right hand side of the dashboard, you will also see recent activity and events that was done within your CloudStack account.<br />
<br />
If you wish to see your CloudStack account resource quota and allocation, navigate to: <code>Accounts -> Click on your account -> Resources</code>. <br />
[[File:CloudStack Resource Quota.png|alt=CloudStack Resource Quota|none|thumb|CloudStack Resource Quota]]<br />
<br />
== Working with virtual machines==<br />
<br />
CloudStack allows you to control the lifecycle of virtual machines within your cloud account. VMs may be started, stopped, rebooted, or destroyed within your management console.<br />
<br />
===Create a VM===<br />
<br />
To create a new VM, enter the CloudStack management console and navigate to: <code>Compute -> Instances -> Add Instance</code>[[File:CloudStack Instance Summary.png|alt=CloudStack Instance Summary|thumb|CloudStack Instance Summary|493x493px]]<br />
<br />
Virtual Machines require the following details:<br />
<br />
# '''Deployment zone'''. Your account will already be placed in the appropriate zone.<br />
# '''Boot template or ISO'''. You may choose either a pre-created template or boot from a custom CD-ROM ISO file.<br />
# '''Compute offering'''. You may select an appropriate size for your new VM. Resources will be counted against your account's quota.<br />
# '''Data Disk'''. You may choose to add an additional virtual disk to your VM to store your data. Alternatively, if you wish to use a single virtual disk for your VM, you may choose to override the size of your root disk in step 2 and select 'No thanks' in this step.<br />
# '''Networks'''. You may choose one or more networks your VM should connect to. All CloudStack accounts come with a default network already created and ready to be used.<br />
# '''SSH keypairs'''. For templates that support custom SSH key pairs, you may choose to use a custom SSH keypair to be installed as part of the deployment process.<br />
# '''Advanced settings'''. For templates that support custom user-data (Cloud-Init), you may choose to enable the advanced settings and provide your own Cloud-Init user-data payload. More on this in the advanced tasks section below.<br />
# '''Other VM details'''. You may give your new VM a friendly name and make it part of a group. Groups allow you to group related VMs together for better organization. You may change these details at a later time.<br />
<br />
When you are done, review the instance summary on the right hand side and then click on the 'Launch Virtual Machine' button.<br />
<br />
=== Selecting your VM Operating system ===<br />
Many OSs will provide various editions that are tailored to a specific use case. A desktop VM may not be appropriate when you need to run a database server. The OS provider will have guides on how to choose an edition.<br />
<br />
You may choose to install the operating system to your virtual machine using either pre-built templates or from scratch using an ISO image.<br />
<br />
====Install from a virtual machine template====<br />
<br />
We provide a Rocky Linux 8.5 and a Ubuntu Server 22.04 LTS template for your convenience. These templates are pre-built images with the operating system installed and ready for use. Our templates also support further automated setup configured using Cloud-Init configuration data that can be provided when deploying a new VM. Currently, we offer the following templates: <br />
{| class="wikitable"<br />
!Template<br />
!Cloud-Init Support<br />
!Password Support<br />
!Default Username<br />
|-<br />
|Rocky Linux 8.5<br />
|Yes<br />
|Yes<br />
|rocky<br />
|-<br />
|Ubuntu Server 22.04<br />
|Yes<br />
|Yes<br />
|ubuntu<br />
|}<br />
Rocky Linux is an open source Linux distribution that is binary-compatible with Red Hat Enterprise Linux and is what RCS recommends.<br />
<br />
For templates that support passwords, the generated password that appears after a VM is created is applied to the default username.<br />
<br />
Security note: All VM templates are configured with SSH password authentication enabled. You should be able to SSH to your VM from another system connected to the same guest network. Do not expose port 22 unless required and we highly recommend using key based authentication.<br />
<br />
===== Virtual machine credentials =====<br />
VM templates that have password support will have a randomly generated password set when the VM is first created or when a password reset request is made (available only when the VM is powered off). A randomly generated 6 character password will be displayed when a new password is set and appears as a notification in your CloudStack management console. <br />
[[File:CloudStack VM Password.png|alt=CloudStack VM Password|none|thumb|CloudStack VM Password]]<br />
This password is set on the default username for your template. For example, the Rocky Linux VM template will set this password to the ''''rocky'''<nowiki/>' user account. You may become the super user by logging in as the <code>rocky</code> user and then running <code>sudo su</code>.<br />
<br />
Note: If you specify a custom Cloud-Init config that creates additional users or sets account passwords, the displayed password will be overridden and have no effect.<br />
<br />
==== Install from an ISO image====<br />
We provide various ISO images for popular Linux distributions. You may select one of these ISO images instead of using a pre-built template when deploying a new virtual machine. We currently provide:<br />
{| class="wikitable"<br />
!Distribution<br />
!ISO<br />
|-<br />
|Ubuntu 20.04<br />
|ubuntu-20.04.4-desktop-amd64.iso<br />
<br />
ubuntu-20.04.4-live-server-amd64.iso<br />
|-<br />
|Ubuntu 21.10<br />
|ubuntu-21.10-desktop-amd64.iso<br />
<br />
ubuntu-21.10-live-server-amd64.iso<br />
|-<br />
|Ubuntu 22.04<br />
|ubuntu-22.04-live-server-amd64.iso<br />
|-<br />
|Rocky Linux 8.5<br />
|Rocky-8.5-x86_64-minimal.iso<br />
|-<br />
|Fedora 35<br />
|Fedora-Workstation-Live-x86_64-35-1.2.iso<br />
|}<br />
You may install custom ISO file into your CloudStack account either by directly uploading the ISO through the web console or by providing a URL to the ISO file on the internet.<br />
<br />
Please do not install Windows on our CloudStack infrastructure. It is against our user agreement to run Windows based systems in this infrastructure. If you need a Windows VM, please contact us for alternative solutions.<br />
<br />
===== Register a ISO with a URL=====<br />
[[File:CloudStack Download ISO.png|alt=CloudStack Download ISO|thumb|CloudStack Download ISO|190x190px]]<br />
<br />
To add a custom ISO file from the internet, enter the CloudStack management console and navigate to: <code>Images -> ISOs -> Register ISO</code><br />
<br />
You may check the state of the ISO file by clicking on it and verify the state of the file. If the file is successfully downloaded, its ready state should become ‘true’. The ISO file will only appear in the selection list when the file is downloaded successfully.<br />
[[File:CloudStack ISO Ready.png|alt=CloudStack ISO Ready|none|thumb|172x172px|CloudStack ISO Ready]]<br />
<br />
=====Upload a custom ISO=====<br />
<br />
To upload an ISO file, enter the CloudStack management console and navigate to: <code>Images -> ISOs -> Upload ISO from Local (icon)</code><br />
<br />
[[File:CloudStack Upload ISO.png|alt=CloudStack Upload ISO|none|thumb|CloudStack Upload ISO|217x217px]]<br />
<br />
===Connecting to your VM console===<br />
The CloudStack management console has a KVM (keyboard, video, mouse) feature built-in, allowing you to remotely connect to and interact with your virtual machine. To connect to your virtual machine's console, navigate to: <code>Compute -> Instances -> Your Instance -> View console</code>.<br />
[[File:CloudStack View Console.png|alt=CloudStack View Console|none|thumb|CloudStack View Console]]<br />
<br />
=== Expanding a VM disk ===<br />
[[File:CloudStack Expand Volume.png|alt=CloudStack Expand Volume|thumb|CloudStack Expand Volume]]<br />
Virtual machine disks can be expanded after they are created within CloudStack. However, you will need to expand the partitions and filesystems manually.<br />
<br />
To grow an existing disk:<br />
<br />
# Go into your VM details page and click on ‘Volumes’.<br />
# Click on the volume you wish to expand.<br />
# Click on the ‘Resize Volume’ icon in the top right.<br />
Once the volume has been expanded, you should be able to verify the disk volume has grown with <code>lsblk</code>. There should also be some messages by the kernel when this occurs. However, you will still need to expand any partitions, volumes, and filesystems on your system manually.<br />
<br />
To expand your partition, use the <code>growpart</code> command followed by your disk device and partition number. Eg: <code>/usr/bin/growpart /dev/vda 3</code><br />
<br />
For LVM volume sets, you can expand the volume using the <code>pvresize</code> and <code>lvresize</code> commands:<br />
<br />
* <code>/usr/sbin/pvresize -y -q /dev/vda3</code><br />
* <code>/usr/sbin/lvresize -y -q -r -l +100%FREE /dev/mapper/<volume-name></code><br />
<br />
To expand your filesystem:<br />
<br />
* XFS: <code>/usr/sbin/xfs_growfs <volume></code><br />
* EXT: <code>resize2fs <volume></code><br />
<br />
=== Destroying a VM ===<br />
If you need to delete a VM, click on the red garbage bin icon in the VM instance page. All deletions are irreversible, so please make sure you have a copy of any data you need before proceeding.<br />
[[File:CloudStack Delete VM.png|alt=CloudStack Delete VM|none|thumb|CloudStack Delete VM]]<br />
The VM root volume can be deleted immediately by enabling the 'Expunge' option in the dialog box. If left disabled, the VM root volume will linger for a day before it is deleted by the system. You may wish to expunge a volume if you are running low on space or volume quota.<br />
<br />
<br />
== Virtual machine networking ==<br />
The CloudStack platform allows you to define custom virtual private cloud (VPC) network which can contain any number of guest networks that your virtual machines connect to. Each guest network has its own private network address space and is not directly routable from campus or the internet. For virtual machines that require internet access, the VPC or guest network it is connected to must have a NAT IP address associated. The following diagram shows how a guest network connects to the internet and campus network.<br />
[[File:CloudStack Guest Networking.png|alt=CloudStack Guest Networking|none|thumb|CloudStack Guest Networking]]<br />
In order to expose a virtual machine's services to campus or the internet, the appropriate port forwardings must be set up on the VPC containing the guest network. More on this will be discussed in the next section.<br />
<br />
Having multiple guest networks allows for more advanced network setups but is not required. We recommend using a single flat network for most workloads. <br />
<br />
By default, all CloudStack accounts come with a default VPC and guest network set up with a NAT IP assigned.<br />
<br />
=== IP addresses ===<br />
Due to the design decisions made during the setup of the CloudStack platform, only internal 10.44.12X.X IPs can be assigned to your VPC. These IP addresses are accessible from the university campus network. However, there is a special section of IP addresses that can be accessed from the internet.<br />
{| class="wikitable"<br />
!IP address range<br />
!Accessible from<br />
!Internet IP mapping<br />
|-<br />
|10.44.120.3-128<br />
|Campus, Internet<br />
|10.44.120.X maps to 136.159.140.X (ports 80 and 443 only)<br />
|-<br />
|10.44.120.129-255<br />
|Campus only<br />
|N/A<br />
|-<br />
|10.44.121.0-255<br />
|Campus only<br />
|N/A<br />
|-<br />
|10.44.122.0-255<br />
|Campus only<br />
|N/A<br />
|-<br />
|10.44.123.0-255<br />
|Campus only<br />
|N/A<br />
|}<br />
If you need a service exposed to the internet, please request for a public IP address using our [https://ucalgary.service-now.com/it?id=sc_cat_item&sys_id=e3c1d6e91be48554cca5ecefbd4bcb6c ServiceNow request form]. Additionally, if your service is not port 80 or 443, you must also request for a firewall change request to allow the special port through. <br />
<br />
=== Exposing a network service to campus ===<br />
In order to make a virtual machine be visible to the campus network, you must first set up a port forwarding from a campus IP address to your virtual machine.<br />
<br />
To create a port forwarding, navigate to <code>Network -> VPC -> Select your VPC -> Public IP Addresses</code>. If you do not have any available IP addresses, you will need to click on 'Acquire New IP' and select an available IP address. Click on the IP address you wish to use to create a port forwarding on and then navigate to the 'Port Forwarding' tab. Enter the private port range, the public port range, the protocol, and select the target VM. <br />
<br />
For example, to port forward only HTTP (tcp/80) traffic, you would enter the following:<br />
[[File:CloudStack Port Forwarding.png|alt=CloudStack Port Forwarding|none|thumb|CloudStack Port Forwarding]]Once the port forwarding is created, you should be able to access the service from on campus. If for some reason access to your service does not work, there may be a firewall restriction on IT's network. In such circumstances, please contact us for assistance.<br />
<br />
=== Exposing a network to the internet ===<br />
Exposing a service to the internet is the same as exposing it to campus. However, you must create a port forwarding on an IP address that maps to an internet IP address outlined in the IP address table above. If your account does not have one of these IP addresses available, please request for one on the [https://ucalgary.service-now.com/it?id=sc_cat_item&sys_id=e3c1d6e91be48554cca5ecefbd4bcb6c ServiceNow request form].<br />
<br />
By default, only ports 80 and 443 are allowed through the Internet IP address. For all other ports, please [https://ucalgary.service-now.com/it?id=sc_cat_item&sys_id=47cd16d113153a00b5b4ff82e144b0bf create a firewall rule change request in ServiceNow].<br />
<br />
== Cloud-Init Automation ==<br />
<br />
=== Ubuntu ===<br />
The following Cloud-Init configs apply to Ubuntu VM templates.<br />
<br />
==== Ubuntu desktop ====<br />
Use the following Cloud-Init config with the Ubuntu Server template to set up an Ubuntu desktop environment. The setup step takes a up to 15 minutes to complete and you should see a login screen when the setup finishes. <br />
<br />
Adjust the root and user password as desired. Below, the test user password is set to blank, allowing you to login to Gnome without a password.<br />
{{Highlight|code=#cloud-config<br />
disable_root: false<br />
<br />
users:<br />
- name: user<br />
lock_passwd: false<br />
inactive: false<br />
gecos: Test User<br />
primary_group: user<br />
groups: wheel<br />
passwd: $1$ADUODeAy$eCJ1lPSxhSGmSvrmWxjLC1<br />
<br />
chpasswd:<br />
list: {{!}}<br />
root:password<br />
expire: false<br />
<br />
# Install a graphical desktop<br />
runcmd:<br />
- DEBIAN_FRONTEND=noninteractive apt -y upgrade<br />
- DEBIAN_FRONTEND=noninteractive apt -y install tasksel<br />
- tasksel install gnome-desktop<br />
- systemctl set-default graphical.target<br />
- systemctl isolate graphical.target|style=max-height: 300px;overflow:scroll;}}<br />
<br />
=== Rocky Linux ===<br />
The following Cloud-Init configs apply to Rocky Linux templates.<br />
<br />
==== Rocky Linux Desktop ====<br />
Use the following Cloud-Init config using the Rocky Linux template to set up a Rocky Linux desktop environment. The setup step takes up to 10 minutes to complete and you should see a login screen when the setup finishes. <br />
<br />
Adjust the root and user password as desired. Below, the test user password is set to blank, allowing you to login to Gnome without a password.<br />
{{Highlight|code=#cloud-config<br />
disable_root: false<br />
<br />
users:<br />
- name: user<br />
lock_passwd: false<br />
inactive: false<br />
gecos: Test User<br />
primary_group: user<br />
groups: wheel<br />
passwd: $1$ADUODeAy$eCJ1lPSxhSGmSvrmWxjLC1<br />
<br />
chpasswd:<br />
list: {{!}}<br />
root:password<br />
expire: false<br />
<br />
# Install a graphical desktop<br />
runcmd:<br />
- yum -y install "@Workstation"<br />
- systemctl set-default graphical.target<br />
- systemctl isolate graphical.target}}<br />
<br />
==== Rocky Linux Docker host ====<br />
Use the following Cloud-Init config using the Rocky Linux template to set up a new docker host. This server can then be used to run Docker containers. Also included are:<br />
<br />
# The docker-compose utility to help deploy container stacks more easily<br />
# A helper script to expand the <code>/var</code> and <code>/</code> filesystems on first startup based on the available space in the ROOT volume. <br />
<br />
Use the CloudStack generated password with the '<code>rocky</code>' default user account to log in.{{Highlight|code=#cloud-config<br />
<br />
write_files:<br />
- path: /usr/bin/expand_lvm_root<br />
content: {{!}}<br />
#!/bin/bash<br />
/usr/bin/growpart /dev/vda 3<br />
/usr/sbin/pvresize -y -q /dev/vda3<br />
/usr/sbin/lvresize -y -q -r -l +50%FREE /dev/mapper/*root<br />
/usr/sbin/lvresize -y -q -r -l +100%FREE /dev/mapper/*var/dev/mapper/*root<br />
/usr/sbin/xfs_growfs /<br />
/usr/sbin/xfs_growfs /var<br />
permissions: '0700'<br />
<br />
- path: /usr/bin/setup_docker<br />
content: {{!}}<br />
#!/bin/bash<br />
yum install -y yum-utils<br />
yum-config-manager \<br />
--add-repo \<br />
https://download.docker.com/linux/centos/docker-ce.repo<br />
yum install -y docker-ce docker-ce-cli containerd.io<br />
systemctl start docker<br />
systemctl enable docker<br />
<br />
curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose<br />
chmod +x /usr/local/bin/docker-compose<br />
permissions: '0700'<br />
<br />
- path: /root/docker-compose.yml<br />
permissions: '0700'<br />
content: {{!}}<br />
version: '3.3'<br />
services:<br />
web:<br />
image: php:7.4-apache<br />
restart: always<br />
user: "0:0"<br />
volumes:<br />
- /var/www/html:/var/www/html<br />
ports:<br />
- "80:80"<br />
<br />
- path: /var/www/html/index.php<br />
permissions: '0644'<br />
content: {{!}}<br />
<h1>Hello there!</h1><br />
<p>I see you from <?php echo $_SERVER['REMOTE_ADDR']; ?></p><br />
<<nowiki>pre</nowiki>><?php print_r($_SERVER); ?></pre><br />
<br />
# Ensure VM has the largest / possible<br />
runcmd:<br />
- /usr/bin/expand_lvm_root<br />
- /usr/bin/setup_docker<br />
- cd /root; docker-compose up -d|style=max-height: 300px;overflow:scroll;}}<br />
<br />
==== UC Authentication ====<br />
Use the following Cloud-Init config to allow UC-based authentication. Local accounts with the same username as the IT account may use the IT credential to log in.<br />
{{Highlight|code=#cloud-config<br />
<br />
write_files:<br />
- path: /usr/bin/setup_uc_auth<br />
permissions: '0700'<br />
content: {{!}}<br />
#!/bin/bash<br />
yum install -y sssd sssd-dbus sssd-krb5 krb5-workstation authselect-compat<br />
<br />
cat <<EOF > /etc/sssd/sssd.conf<br />
[sssd]<br />
config_file_version = 2<br />
services = nss, pam, ifp<br />
domains = uc.ucalgary.ca<br />
<br />
[domain/uc.ucalgary.ca]<br />
id_provider = files<br />
debug_level = 5<br />
auth_provider = krb5<br />
chpass_provider = krb5<br />
<br />
krb5_realm = UC.UCALGARY.CA<br />
krb5_server = ITSODCSRV14.UC.UCALGARY.CA:88<br />
krb5_validate = false<br />
EOF<br />
chmod 600 /etc/sssd/sssd.conf<br />
<br />
cat <<EOF > /etc/krb5.conf<br />
[logging]<br />
default = FILE:/var/log/krb5libs.log<br />
kdc = FILE:/var/log/krb5kdc.log<br />
admin_server = FILE:/var/log/kadmind.log<br />
<br />
[libdefaults]<br />
default_realm = UC.UCALGARY.CA<br />
dns_lookup_realm = false<br />
dns_lookup_kdc = false<br />
ticket_lifetime = 24h<br />
renew_lifetime = 7d<br />
forwardable = true<br />
<br />
[realms]<br />
UC.UCALGARY.CA = {<br />
kdc = itsodcsrv14.uc.ucalgary.ca<br />
}<br />
<br />
[domain_realm]<br />
uc.ucalgary.ca = UC.UCALGARY.CA<br />
.uc.ucalgary.ca = UC.UCALGARY.CA<br />
EOF<br />
<br />
mkdir -p /etc/authselect/custom/rcs<br />
cd /etc/authselect/custom/rcs<br />
echo H4sIAMYsbGIAA+0ca3PbuDGf+StQ34dLMpEtybLdUevO6BKl56mTuJGv7Uwmo4FISMKJBHQAaEfNub+9uwBJkXrLshxnjjuTyCKJfWF3sQsuFPhS9CtB79keoQpw2mjYT4DZz1r1uPGs1qhX6yf12vFx/Vm1Vqs1jp+R6j6ZSiHWhipCnikpzarn1t3/TuErj8bhhBzccjOs6Igq41MVHBDen71WUey3mCsWHNx5W4wKpT+qSAGjI3lDQxj8SarB0UDIiB2FcsBFRfuKMfHZY4L2QpYbS2MzZMJwnxouxfnXefQHTaNi9nufhprdpQj6XAyYGisuzHIUuYcWIhlTrW/lRkxMNdO0OH5HdHdeXlDNjAGCuhJQ0IM4GjPFx0Om4PGjDM9n8pULP4wDtpkik78q1Lec/WifcMr8kSyE7fB/a9ssYf8Q2PiPc6/3RmNN/K826mez8f/07LSM/48BR4uD8dGaWOytGbc8BK8buSTu5odtFEyPitGxDH2LYXai9kFjjf8fn4HPT/3/GPy/fnZ6Uvr/Y8BXiP/gTHHOL/KZ0Z2HRpGmD2meszi5mIExjbpM3BxquRuOgIkJICE/kE7qsKQYGwjXRbSrfD2Xxe7EVp/yEOMGsjZWzOLSPASuCHJ83iCxwPtdwyN2XqtXqwvYSpHMMKPjfp/7HHFtyIydrWBnXeeFQkT4fak4ZImulwh134n3POr7Mk50sTUWGMy0RoE2hnmRAJHDgzLtwk1ewffmJq/gXZiJBf9iTSaHZGvTA0ZoGGumdsakY1AxC7q8j9qJeUD+SmqwPhAQi5kC7k8B69M4NOc9GhA7TutzOSLIB4g1EvJWnHPIFhT77HBbE9hNXZBmRNxYi0yTlHvhyexaA9cYvhDkGCMZDTfGMmITLiw7wMGNHLECuq2ZCjnIZnVUyeHZmis90YZFNhDtJJ0Mgl9lrxuNhpDwBVxZi4ioHp1Xq2dni0bNu0k2GPwkz8yn1GBqJDWj1FSWCFUwTDCxG+4DHUF8JUXgzBNNrws2u9sspB65k+4SW99s/RdgyNz4w0OsA/eUY6yr/05rtdn8r35S1n+PAjaSBU1rPGA4pA8pjCaJG6NbsS8zbuWDxmTk6jTMogZKxmOHYDsMdiAgEMxkOKYIcp69DEE60CUaMsLQ3twSRzbQBgnr2XprRtKBiCMOJFMWRaoJXcAxn5rCiG9XcRbK7T3RWOP/pyen+fqvgf5/XC3rv0eBb17eYTIbsJDaGs/+cV53dlGWjfsuG+O6zWh8qP43hYW1UQUQ7aqYHC9CYh4P6fx9eanU+3SGn6xiqBGX7CU5X656mKZ/i6uRv52T+rQa2QV3sWbaaQZdwkjwrZSQxqlBxqYiYrCQ0UHTfd4RoybdPlfadDHiz0+V5mYjI1ivmm4i9W6CuSSW9KW6BfdbwnS5tVFubSxmJvUL0oM4wkRXD2kgb70fcnojt0CKUCB3S5IrYG3EZ8pQqO1sZqkJ+JLig6H5C/CgMUaPmQjgsiA6jFW0247Hg0yg5WNXfVkkXdDR2GzKzmacPLH9pPr3sZ20TTQe3/4W05Abm+0UwzyxyuyiGLorRTgpULr3YgPOdFKrE+dT2y49dqMEg6eRo93YSZaIZfjKfbkcmnJf7jvbl3ssGEtt7Gv4PdJYt/93Um/M1P81iC9l/f8YsKFFWydAP7eeZuvVTX13uaf+6X9kEEQv8991/DJZnwvYswJrLhTYGEmtDSNqWKlNNCa5Jj3LayV55KDpvv+uh/IWc00W3G3q1DOUkqIdatVxQGG1nmJ8Yh6+Gj62W2/etfdLY43/n1VPT3P+XwP/rzaOy/3/R4G27boinU7njS1/XKIwu1P2nIauGHcZpU2MsQgKJy+8813B8zosZD62dBEz5JqMlcStc1egsRyDVMMD4GoyVhAuZJ/wAJk0E4+Kud09wHID95U+BAI4OrmgCYWIY3C46x8DpJJEVNABSwtBg8lbJMGtIU0B1qTiTC8iEjF/SAXXkca6YYgcXr5pXb0i/2Cqx5QEJSnyVjF2cdU6JBcmYwKQkfedDkGcV613kLUYpvoUxDISdzycoMlkIF0yDuPBwOqiR32oaYPktgfM+lIIYNMKAnGSj0PkvN9nCmNUWqY4vaE+3uHuFBcwoZGTg/Ygc3dK9oGzHoPJjoEE7pIaOyO/Iv4x6KjpDY0Z6+bREXyLFTvk8ggHHkF+E3jeT5N0BXhVMBaK+2EQ6CHr6isZOVqKgpgKZRXJw+6dCYRvvO+BMbj5D8hziK/sxj08yU8DIzecZlnai0NyjTY0pDcMlQZKRSGFz7weE6zPQQ2QkbJUVn/InFE5wtgNhSqDJLIPsR4+KxEYgpq4R+1UmGGsPcgeY9AlEXEE82yNiesRrB9MWY2iln92HL9CcSYyJoG0hdItFXamRoyNEzXEQqD1g7bhOQXG6A+5YK/sKJgPzz5a8A1tXQb0ghz9GuPWBNdTT0mWVFQGw50LNAox8PAlL4dJc5Nu/Usb/B+KphEIr9DYwwkBtvG2GTpsCWpHLDep1iOSjRKLrIfEaDLD+Ql1bhROUl98/eH924u///KxdX3x4b1XmQPPa8EMOyFBb6hoiUpDH0MEP9qpmspySK5CRjWozNJ3j3iB9OMIptOJazDzYATsiDiXscPBDwx6P9TQwB75ADGtIGIqF72Bxd0qGFFNnRPmFm2Y4dsEIQn7wrWNZM6+CjyCUP9qXVy2frpskw9XKHnrkrxtt65/+djuLFBCThuFzbFm08OEJAndqXfjHSSM9Q9oAk3SSBvYJsiHZn5s+A2zQ2eiGCIGTYDN4k2vWKXNUMP3pTDKhyKLudFACB/Nh0q7VKSLBOqHK4vEbgMQW28cJkJlr1jm6OQ5tLaYPYoeC1Y3cLaQzj2aCITPITUWUfY0RObxWCqYZXQTnEpczNiXcch9bq09CTFIhAs7eKGNzXI82zecSHAJV4lraia3GNlojhf7zgmeZsE8urQsTfBs8t7qkLyXRNogGjEKSxlMxkzjNKJ67opwNI90i+RF5rGJ+Ck/ufbP9VOSe9h6no3Pbmpe5hoTX6bIkxc0KxFjOIdnwO3FIGRFfHB9Fhe+7Cniq0NUgqUUTPG+qItVQ4L9jQvfEIEx9OfrAdC91pg+BLFCHRcNPA5kgqFlN5vxAkYg8I8sraFpUmNzMHxAxXYhBOsJAp762REz/lHSYpBTg8taEiKvhwzsL9laRgNOuUoDBeoEvPS/eaOe7tsVhQXixF0HeRWNmMEZltNNEc97c9GxEa1z1X598fbiNaY1/764fv0zedO6bv3U6qyObdMY9x4XagjCr0jalAQJmqE9cG1tM7bexAbedAkEJ1K3ihu4C84XJw+BA2PgqRQam9w62Jeof7vQ2jJP21wHNO47G0yWGyO95P2dRbuINK57OH14TQqWECbzhD0uYJWgmXMVmmdA1ReO0EHSTZMJvIDoDArXMjPF4DphtkCQtd1McWQNNVugmXbeTPFMm2q2QJR130zxZH0126BJGnByWNyVdUja/2m9u7rMGavnvST58sitQeibOP1zy4vnkbwVpR+QFJMsDpBixF9OIV2ykJJdaNkmayzxCkvsOo6yJX7KIKRn7TZpXXY+TNXwEnMIO8ya9PM/v/iudjhKWAUf2//85eJj+137/XVnXzTWnv85qc7s/+C3cv/nMeAdHWGfinKJc6F4xDwzq5Jc7ZemiqTDkpBVLLEwJEUzewuuqNgPrOjYOrjzKjPhFgVK6+ZXENRSyfWKRDutDXguAjbX0CWkYpeYA8ySsJ/Avp8k5+RaxewAUX2CO5/hGXc08aHk3ZuaFzVqFA6JJZrOVwNLdJ0UBOttbEPKjynztNMtkTe5sExW61E2yaCRLTsiqBxCKzOmhZDrzr+CXEUV7cot+Ji32k2z6S7GiE3w3brbRLUkYZDfH+AmwNp3lKup7gvW9e8VtYwXH0rT6yh/A01nlB9P2/kX8ZXZlHCJphc1AaTavg9dQuymJtN2oxp3nC3yQoAoRoSHkdfGaLuD55vsJUNC+zClvQc9F+li7mEehuwausvW/+IB//3kGOvyv2L//wn2/zfqtTL/ewy45+//zJ8azyciT+JQwZPpsk/61GynaTfiGleOrqBQza/+LaHkr2kG+YBHD8oO6D9IB/RTbsZ9ooe7n37751Zsfdv2z7L/c8ruU+z/dJaxz9Of6/K/WrXWaMzkf7Wzk7My/3sMuF/+9yQSvH2cGt1n4vhA8U1IyLzdC57mIIiaX+DfCD9dy4GmN0w17Q+lVfJXRrkvxdB4n3OvmSwBvv4snnpMhQt4IlkuCZ7LaZOx9m/cNIZQazttyL2O4+7ww0kbmQhZyNbyH8wqT+R+hydyN1LQol+DWFYCZ9zU98jNyjcDy1131u9S1la4dM6d8WjX1JXvw0154jmjUZ54XoulrPeXMlOeeC5PPJcnnssTz+WJ52+95VXueGXsPsUdrxJKKKGEEkoooYQSSiihhBJKKKGEEkr448D/AT019aAAeAAA {{!}} base64 -d {{!}} tar -xzpf -<br />
<br />
if [[ "`authselect current -r`" != "custom/rcs" ]] ; then<br />
authselect select custom/rcs --force<br />
systemctl restart sssd<br />
fi<br />
<br />
systemctl enable sssd<br />
<br />
runcmd:<br />
- /usr/bin/setup_uc_auth|style=max-height: 300px;overflow:scroll;}}<br />
<br />
==== Rocky Linux NFS server ====<br />
Because there is no ability to share storage among multiple VMs, a local NFS server could be useful if you need to share data between multiple VMs.{{Highlight|code=#cloud-config<br />
<br />
write_files:<br />
- path: /usr/bin/expand_lvm_root<br />
content: {{!}}<br />
#!/bin/bash<br />
/usr/bin/growpart /dev/vda 3<br />
/usr/sbin/pvresize -y -q /dev/vda3<br />
/usr/sbin/lvresize -y -q -r -l +50%FREE /dev/mapper/*root<br />
/usr/sbin/lvresize -y -q -r -l +100%FREE /dev/mapper/*var<br />
/usr/sbin/xfs_growfs /<br />
/usr/sbin/xfs_growfs /var<br />
permissions: '0700'<br />
<br />
- path: /usr/bin/setup_nfs<br />
content: {{!}}<br />
#!/bin/bash<br />
yum install -y nfs-utils<br />
<br />
mkdir /export<br />
if [ -b /dev/vdb ] ; then<br />
mkfs.xfs /dev/vdb<br />
echo "/dev/vdb /export xfs defaults 1 2" >> /etc/fstab<br />
mount -a<br />
fi<br />
<br />
ip a {{!}} grep -w inet {{!}} awk '{print $2}' {{!}} while read subnet ; do<br />
echo "/export $subnet(rw,no_subtree_check,no_root_squash,async)" >> /etc/exports<br />
done<br />
<br />
systemctl start nfs-server<br />
systemctl enable nfs-server<br />
<br />
exportfs -ra<br />
<br />
permissions: '0700'<br />
<br />
runcmd:<br />
- /usr/bin/expand_lvm_root<br />
- /usr/bin/setup_nfs|style=max-height: 300px;overflow:scroll;}}NFS clients connected to the same network as the NFS server can then mount <code>/export</code> using a command similar to: <code>mount -t nfs nfs-server:/export /mnt</code>.<br />
<br />
== Infrastructure tools ==<br />
<br />
=== Generating a CloudStack API ===<br />
You can request for a CloudStack API key to automate infrastructure deployment using Terraform or CloudMonkey. A new API key can be generated by navigating to your profile page (top right) and then clicking on the 'Generate keys' button.<br />
[[File:CloudStack API Key.png|alt=CloudStack API Key|none|thumb|CloudStack API Key]]<br />
<br />
=== CloudMonkey ===<br />
CloudMonkey is a utility that makes it easier to interact with the CloudStack API. This tool may be used to help automate VM actions (such as start/stop/reboot), or infrastructure tasks (such as creating/destroying VMs, networks, or firewall rules). <br />
<br />
To get started with CloudMonkey, refer to the following resources:<br />
<br />
* Download from: <nowiki>https://github.com/apache/cloudstack-cloudmonkey/releases/tag/6.1.0</nowiki><br />
* Documentation at: <nowiki>https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+cloudmonkey+CLI</nowiki><br />
<br />
=== Terraform Integration ===<br />
Terraform allows you to define infrastructure as code and can be used in conjunction with CloudStack to configure your virtual machines and guest networks. Use the official CloudStack provider.<br />
<br />
The following is an example Terraform file for reference. Specify your CloudStack API keys either as a separate <code>vars.tf</code>.<br />
{{Highlight|code=# Configure the CloudStack Provider<br />
terraform {<br />
required_providers {<br />
cloudstack = {<br />
source = "cloudstack/cloudstack"<br />
version = "0.4.0"<br />
}<br />
}<br />
}<br />
<br />
provider "cloudstack" {<br />
api_url = "${var.cloudstack_api_url}"<br />
api_key = "${var.cloudstack_api_key}"<br />
secret_key = "${var.cloudstack_secret_key}"<br />
}<br />
<br />
# Create a new VPC<br />
resource "cloudstack_vpc" "default" {<br />
name = "wedgenet-vpc"<br />
display_text = "wedgenet-vpc"<br />
cidr = "100.64.0.0/20"<br />
vpc_offering = "Default VPC offering"<br />
zone = "zone1"<br />
}<br />
<br />
# Create a new ACL<br />
resource "cloudstack_network_acl" "default" {<br />
name = "vpc-acl"<br />
vpc_id = "${cloudstack_vpc.default.id}"<br />
}<br />
<br />
# One ingress and one egress rule for the ACL<br />
resource "cloudstack_network_acl_rule" "ingress" {<br />
acl_id = "${cloudstack_network_acl.default.id}"<br />
<br />
rule {<br />
action = "allow"<br />
cidr_list = ["0.0.0.0/0"]<br />
protocol = "tcp"<br />
ports = ["22", "80", "443"]<br />
traffic_type = "ingress"<br />
}<br />
}<br />
<br />
resource "cloudstack_network_acl_rule" "egress" {<br />
acl_id = "${cloudstack_network_acl.default.id}"<br />
<br />
rule {<br />
action = "allow"<br />
cidr_list = ["0.0.0.0/0"]<br />
protocol = "all"<br />
traffic_type = "egress"<br />
}<br />
}<br />
<br />
<br />
# Create a new network in the VPC<br />
resource "cloudstack_network" "primary" {<br />
name = "primary"<br />
display_text = "primary"<br />
cidr = "100.64.1.0/24"<br />
network_offering = "DefaultIsolatedNetworkOfferingForVpcNetworks"<br />
acl_id = "${cloudstack_network_acl.default.id}"<br />
vpc_id = "${cloudstack_vpc.default.id}"<br />
zone = "zone1"<br />
}<br />
<br />
# Create a new public IP address for this network<br />
resource "cloudstack_ipaddress" "public_ip" {<br />
vpc_id = "${cloudstack_vpc.default.id}"<br />
network_id = "${cloudstack_network.wosnet1.id}"<br />
}<br />
<br />
# Create VMs. <br />
resource "cloudstack_instance" "vm" {<br />
count = 1<br />
name = "vm${count.index+1}"<br />
zone = "zone1"<br />
service_offering = "rcs.c4"<br />
template = "RockyLinux 8.5"<br />
network_id = "${cloudstack_network.wosnet1.id}"<br />
<br />
# Cloud Init data can be used to configure your VM on first startup if your template supports Cloud Init<br />
user_data = <<EOF<br />
#cloud-config<br />
<br />
# Require specific packages<br />
packages:<br />
- tmux<br />
- git<br />
- tcpdump<br />
<br />
EOF<br />
}<br />
<br />
<br />
|style=max-height: 300px;overflow:scroll;}}<br />
<br />
= Troubleshooting =<br />
<br />
=== Cannot create a volume snapshot ===<br />
Volume snapshots can only be taken on VMs that are powered off.<br />
<br />
=== Cannot create a VM snapshot ===<br />
Disk-only VM snapshots cannot be taken when the VM is running. If you intend to snapshot a running system, you must also snapshot its memory.<br />
<br />
=== VM state is still running after shutdown ===<br />
After running 'shutdown' on a VM, the VM state reported by CloudStack is still running. <br />
<br />
Please try to do a force shutdown from the CloudStack management console. The VM state isn't updated by CloudStack and as a result, the state of a VM isn't properly reflected when power state changes outside of CloudStack (likely a bug?)</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=RCS_Home_Page&diff=1869RCS Home Page2022-06-02T21:09:40Z<p>Darcy: /* General information */</p>
<hr />
<div>Research Computing Services (RCS) is a group within the wider University of Calgary Information Technologies team that plans, manages, and supports high performance computing (HPC) systems in use by researchers throughout the University of Calgary. Our primary focus is to meet the increasing demand for engineering and scientific computation by offering a wide range of specialized services to help researchers solve highly complex real-world problems or run large scale computationally intensive workloads on our high-end HPC resources.<br />
<br />
This RCS Wiki contains technical documentation for use by users of HPC systems operated by RCS<br />
<br />
<!-- <br />
In case cluster status changes:<br />
* set the status to yellow or red <br />
* provide a custom 'title' and 'message'<br />
<br />
{{Cluster Status<br />
|status=green<br />
}}<br />
--><br />
<br />
=== Contact us for support ===<br />
[[File:Map HSC G204Z.png|150px|thumb|right|Find us at G204Z]]<br />
* For general RCS/HPC inquiries, please email: [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca]<br />
* For IT related issues (networking, VPN, email), please email: [mailto:it@ucalgary.ca it@ucalgary.ca]<br />
* For Compute Canada specific questions: [mailto:support@computecanada.ca support@computecanada.ca]<br />
<br />
RCS has an office at the Foothills campus located at [http://ucmapspro.ucalgary.ca/RoomFinder/?Building=HSC&Room=B200D HSC B200D] and can be reached via the IT reception on the main floor at [https://ucmapspro.ucalgary.ca/RoomFinder/?Building=HSC&Room=G204Z G204Z] next to the University bookstore'''. If you would like to have a face-to-face meeting with an analyst, please contact us via email to arrange an appointment beforehand.<br />
<br />
{{Clear}}<br />
<div class="row"><br />
<div class="col-md-6"><br />
<br />
== General information ==<br />
* [[General Cluster Guidelines and Policies]]<br />
* [[How to get an account]]<br />
* [[Data ownership]]<br />
* [[Connecting to RCS HPC Systems]]<br />
* [[External collaborators]]<br />
<br />
* [[CloudStack|Cloud/Virtual Machine Infrastructure (CloudStack)]]<br />
<br />
* [[On-line resources for new Linux and ARC users]]<br />
* [[Acknowledging Research Computing Services Group]]<br />
<br />
== Cluster Guides ==<br />
* [[ ARC Cluster Guide]] - ARC is a general purpose cluster for University of Calgary researchers.<br />
* [[Helix Cluster Guide]] - Helix is a specialized cluster mainly provided for Cumming School of Medicine projects<br />
* [[GLaDOS Cluster Guide]] - GLaDOS is a researcher-owned cluster maintained by Research Computing Services.<br />
* [[TALC Cluster Guide]] - Teaching and Learning Cluster (TALC) is a cluster created by Research Computing Services to support academic courses and workshops.<br />
* [[MARC Cluster Guide]] -- Medical Advanced Research Computing cluster at the University of Calgary created by Research Computing Services in 2020.<br />
<br />
== Other services ==<br />
<br />
* [[Jupyter Notebooks]]<br />
* [[Open OnDemand | Open OnDemand portal]]<br />
<br />
== Software pages ==<br />
* [[Managing software on ARC]]<br />
* [https://hpc.ucalgary.ca/arc/software/conda Using Conda (external link)]<br />
* [[Gaussian on ARC]] -- How to use Gaussian 16 on ARC.<br />
* [[Apache Spark on ARC]]<br />
* [[ARC Software pages]]<br />
* [[Bioinformatics applications]]<br />
</div><br />
<div class="col-md-6"><br />
<br />
== Running courses on HPC resources ==<br />
* [[TALC Cluster|TALC]] - Teaching and Learning Cluster (TALC) is a cluster created by Research Computing Services to support academic courses and workshops.<br />
* [[TALC Terms of Use]] - Terms of use to which TALC account holders must agree to use the cluster.<br />
* [[List of courses on TALC]] - A list of current and historical courses taught using TALC.<br />
<br />
== Training ==<br />
* Our [[HPC Systems]]<br />
* [[HPC Linux topics]] - A list of topics on which RCS technical support staff can provide one-on-one or group training<br />
* [[Courses]]<br />
* [[Linux Introduction]]<br />
* [[What is a scheduler?]]<br />
* [[Running jobs]]<br />
* [[Data storage options for UofC researchers]]<br />
* [[Security and privacy]]<br />
* [[How to transfer data]]<br />
</div><br />
</div><br />
<br />
{{Clear}}<br />
<br />
==What's New==<br />
* [[CHGI Transition]] - Information on the current CHGI Transition<br />
<br />
<br />
__NOTOC__</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=CloudStack&diff=1868CloudStack2022-06-02T20:54:45Z<p>Darcy: /* Differences between RCS HPC and CloudStack */</p>
<hr />
<div>Apache CloudStack is an Infrastructure as a Service (IaaS) offering that allows researchers to quickly deploy virtual machines for research projects. This service is part of Research Computing Services' Digital Research Infrastructure (DRI) and is free for all University of Calgary researchers and principal investigators.<br />
<br />
== Use cases ==<br />
CloudStack allows you to create virtual machines for a wide range of workloads and use cases, including:<br />
<br />
* Running a internal or public facing web site<br />
* Running a database<br />
* Experiment with new software tools<br />
* Test out the latest release of a software package<br />
Please note that CloudStack is offered as a research environment and is supported as such. For workloads that demand high availability and high uptime, this may not be the appropriate choice. Researchers are solely responsible for maintaining any VMs they deploy. The expectation is that the owners of the VMs will be patching and doing other maintenance work on a regular basis. We will be more than happy to provide guidance on this process. <br />
<br />
Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] for details.<br />
<br />
=== Differences between RCS HPC and CloudStack ===<br />
There are some overlaps between the CloudStack offering and our existing High Performance Computing (HPC) cluster environment.<br />
{| class="wikitable"<br />
!<br />
!RCS HPC Cluster<br />
!CloudStack<br />
|-<br />
|CPU intensive workloads<br />
|Yes; 48 CPUs per node, 100's of nodes<br />
|No; 1-8 CPUs per VM<br />
|-<br />
|Memory intensive workloads<br />
|Yes; up to 2TB memory per node<br />
|No; up to 32GB memory per VM<br />
|-<br />
|High storage requirement workloads<br />
|Yes; shared multi-petabyte storage<br />
|No; up to 1TB per account<br />
|-<br />
|Data classification<br />
|Level 1 & 2 (ARC), Level 3 & 4 (MARC)<br />
|Level 1 & 2 only<br />
|-<br />
|Customized software requirements<br />
|Yes; use singularity containers<br />
|Yes<br />
|-<br />
|Custom OS configuration<br />
|No<br />
|Yes<br />
|-<br />
|Persistent software or services<br />
|No; time limited jobs only<br />
|Yes<br />
|-<br />
|Managed environment<br />
|Yes<br />
|No; self managed VMs only<br />
|-<br />
|Research support by analysts<br />
|Yes<br />
|Limited<br />
|}<br />
Not sure if you need a virtual machine or a compute cluster? Is this the "Cloud"? Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] and we will assist you in using this service to support your research goals.<br />
<br />
== Getting access to CloudStack ==<br />
If you are a researcher or principal investigator, please review our [[CloudStack End User Agreement]] and then request a CloudStack account through [https://ucalgary.service-now.com/it?id=sc_cat_item&sys_id=e3c1d6e91be48554cca5ecefbd4bcb6c Service Now].<br />
<br />
Once your account is ready, please refer to the [[CloudStack User Guide]] for details on using CloudStack.<br />
<br />
__NOTOC__</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=CloudStack&diff=1867CloudStack2022-06-02T20:33:39Z<p>Darcy: /* Differences between RCS HPC and CloudStack */</p>
<hr />
<div>Apache CloudStack is an Infrastructure as a Service (IaaS) offering that allows researchers to quickly deploy virtual machines for research projects. This service is part of Research Computing Services' Digital Research Infrastructure (DRI) and is free for all University of Calgary researchers and principal investigators.<br />
<br />
== Use cases ==<br />
CloudStack allows you to create virtual machines for a wide range of workloads and use cases, including:<br />
<br />
* Running a internal or public facing web site<br />
* Running a database<br />
* Experiment with new software tools<br />
* Test out the latest release of a software package<br />
Please note that CloudStack is offered as a research environment and is supported as such. For workloads that demand high availability and high uptime, this may not be the appropriate choice. Researchers are solely responsible for maintaining any VMs they deploy. The expectation is that the owners of the VMs will be patching and doing other maintenance work on a regular basis. We will be more than happy to provide guidance on this process. <br />
<br />
Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] for details.<br />
<br />
=== Differences between RCS HPC and CloudStack ===<br />
There are some overlaps between the CloudStack offering and our existing High Performance Computing (HPC) cluster environment.<br />
{| class="wikitable"<br />
!<br />
!RCS HPC Cluster<br />
!CloudStack<br />
|-<br />
|CPU intensive workloads<br />
|Yes; 48 CPUs per node, 100's of nodes<br />
|No; 1-8 CPUs per VM<br />
|-<br />
|Memory intensive workloads<br />
|Yes; up to 2TB memory per node<br />
|No; up to 16GB memory per VM<br />
|-<br />
|High storage requirement workloads<br />
|Yes; shared multi-petabyte storage<br />
|No; up to 1TB per account<br />
|-<br />
|Data classification<br />
|Level 1 & 2 (ARC), Level 3 & 4 (MARC)<br />
|Level 1 & 2 only<br />
|-<br />
|Customized software requirements<br />
|Yes; use singularity containers<br />
|Yes<br />
|-<br />
|Custom OS configuration<br />
|No<br />
|Yes<br />
|-<br />
|Persistent software or services<br />
|No; time limited jobs only<br />
|Yes<br />
|-<br />
|Managed environment<br />
|Yes<br />
|No; self managed VMs only<br />
|-<br />
|Research support by analysts<br />
|Yes<br />
|Limited<br />
|}<br />
Not sure if you need a virtual machine or a compute cluster? Is this the "Cloud"? Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] and we will assist you in using this service to support your research goals.<br />
<br />
== Getting access to CloudStack ==<br />
If you are a researcher or principal investigator, please review our [[CloudStack End User Agreement]] and then request a CloudStack account through [https://ucalgary.service-now.com/it?id=sc_cat_item&sys_id=e3c1d6e91be48554cca5ecefbd4bcb6c Service Now].<br />
<br />
Once your account is ready, please refer to the [[CloudStack User Guide]] for details on using CloudStack.<br />
<br />
__NOTOC__</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=CloudStack&diff=1866CloudStack2022-06-02T20:32:23Z<p>Darcy: /* Differences between RCS HPC and CloudStack */</p>
<hr />
<div>Apache CloudStack is an Infrastructure as a Service (IaaS) offering that allows researchers to quickly deploy virtual machines for research projects. This service is part of Research Computing Services' Digital Research Infrastructure (DRI) and is free for all University of Calgary researchers and principal investigators.<br />
<br />
== Use cases ==<br />
CloudStack allows you to create virtual machines for a wide range of workloads and use cases, including:<br />
<br />
* Running a internal or public facing web site<br />
* Running a database<br />
* Experiment with new software tools<br />
* Test out the latest release of a software package<br />
Please note that CloudStack is offered as a research environment and is supported as such. For workloads that demand high availability and high uptime, this may not be the appropriate choice. Researchers are solely responsible for maintaining any VMs they deploy. The expectation is that the owners of the VMs will be patching and doing other maintenance work on a regular basis. We will be more than happy to provide guidance on this process. <br />
<br />
Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] for details.<br />
<br />
=== Differences between RCS HPC and CloudStack ===<br />
There are some overlaps between the CloudStack offering and our existing High Performance Computing (HPC) cluster environment.<br />
{| class="wikitable"<br />
!<br />
!RCS HPC Cluster<br />
!CloudStack<br />
|-<br />
|CPU intensive workloads<br />
|Yes; 48 CPUs per node, 100's of nodes<br />
|No; 1-8 CPUs per VM<br />
|-<br />
|Memory intensive workloads<br />
|Yes; up to 2TB memory per node<br />
|No; up to 16GB memory per VM<br />
|-<br />
|High storage requirement workloads<br />
|Yes; shared multi-petabyte storage<br />
|No; up to 1TB per account<br />
|-<br />
|Data classification<br />
|Level 1 & 2 (ARC), Level 3 & 4 (MARC)<br />
|Level 1 & 2 only<br />
|-<br />
|Customized software requirements<br />
|Yes; use singularity containers<br />
|Yes<br />
|-<br />
|Custom OS cofiguration<br />
|No<br />
|Yes<br />
|-<br />
|Persistent software or services<br />
|No; time limited jobs only<br />
|Yes<br />
|-<br />
|Managed environment<br />
|Yes<br />
|No; self managed VMs only<br />
|-<br />
|Research support by analysts<br />
|Yes<br />
|Limited<br />
|}<br />
Not sure if you need a virtual machine or a compute cluster? Is this the "Cloud"? Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] and we will assist you in using this service to support your research goals.<br />
<br />
== Getting access to CloudStack ==<br />
If you are a researcher or principal investigator, please review our [[CloudStack End User Agreement]] and then request a CloudStack account through [https://ucalgary.service-now.com/it?id=sc_cat_item&sys_id=e3c1d6e91be48554cca5ecefbd4bcb6c Service Now].<br />
<br />
Once your account is ready, please refer to the [[CloudStack User Guide]] for details on using CloudStack.<br />
<br />
__NOTOC__</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=CloudStack_User_Guide&diff=1859CloudStack User Guide2022-06-02T19:25:03Z<p>Darcy: /* Keep security in mind */</p>
<hr />
<div>This is a user's guide on using CloudStack provided by Research Computing Services.<br />
<br />
== Introduction==<br />
Apache CloudStack is an Infrastructure as a Service (IaaS) platform that allows users to quickly spin up Linux/Non-Windows based virtual machines. RCS is providing this service to help researchers quickly set up and prototype short-term research related software on premises. CloudStack is not appropriate for workloads that depend on Windows. Services set up on CloudStack virtual machines can be accessed from the campus network and also the internet if required.<br />
<br />
Access to CloudStack can be requested via [https://ucalgary.service-now.com/it?id=sc_cat_item&sys_id=e3c1d6e91be48554cca5ecefbd4bcb6c ServiceNow].<br />
<br />
Please refer to our [[CloudStack End User Agreement]] for acceptable uses and requirements.<br />
<br />
== Using your virtual machine ==<br />
You will be able to run whatever virtual machine you wish (with the exception of Windows). Clearly we cannot provide specific management advice on each and every operating system available. We can provide you with some suggestions on important considerations to be aware of.<br />
<br />
=== Educate yourself ===<br />
All operating systems (OS) have user groups, web sites, wikis, or mailing lists somewhere on the internet. They can be a valuable resource. Most OS providers have on-line documentation that describes using their product. For example Rocky Linux, used by RCS, has a [https://docs.rockylinux.org/ documentation site]. These are excellent resources and can help you understand how to manage your virtual machine.<br />
=== Keep security in mind===<br />
To help keep our network and infrastructure safe from cyber attacks, it is critical that your VMs are properly configured to reduce the number of ways that hackers could exploit it. Here are some common tasks that you can do to help harden your VM:<br />
*Ensure that the only services running on your VM are the ones you must run. Each OS has a way of managing what services are running (sysinit, systemd etc). Please ensure that unnecessary services have been disabled.<br />
<br />
*Disable or delete any unused accounts. Many OSs will have pre-configured accounts, and many applications will have pre-configured accounts. Make sure they are either disabled or not allowed to login.<br />
<br />
*All accounts should have strong [https://it.ucalgary.ca/it-security/passwords-do-i-have-change-them/ passwords].<br />
<br />
*Many OS's have the ability to automatically update themselves. If possible please consider doing this. Updates can also be configured to skip certain software if it will interfere with your research, but please be advised that doing so could place your system at risk.<br />
<br />
*If your VM must be exposed to the internet, consider using Trend Micro Cloud One Workload Security from IT security to help monitor for and block cyber attacks.<br />
<br />
== Accessing CloudStack ==<br />
===Accessing the CloudStack management console===<br />
<br />
The CloudStack management console is a web-based portal that allows you to view and manage your cloud infrastructure including virtual machines, storage, and network. Any modern web browsers including Chrome, Firefox, Edge, and Safari is supported. <br />
<br />
Access the CloudStack management console is possible only from an IT-managed computer or through the IT General VPN when working on unmanaged machines (eg. AirUC) or when working off campus (eg. at home). Please review the IT [https://ucalgary.service-now.com/it?id=kb_article&sys_id=52a169d6dbe5bc506ad32637059619cd knowledge base article on connecting to the General VPN] or contact IT support if you need assistance connecting to the General VPN. <br />
[[File:CloudStack VPN Connection.png|alt=CloudStack VPN Connection|none|thumb|CloudStack VPN Connection]]<br />
=== Login to CloudStack===<br />
<br />
To log in to CloudStack, navigate to https://cloudstack.rcs.ucalgary.ca/. If this site fails to load, please make sure you are either on a IT managed computer or connected to the General VPN.<br />
<br />
Sign in to CloudStack using the Single Sign-On option as shown in the image below. This method will require you to authenticate through our central authentication service using your University of Calgary IT credentials and will require multi-factor authentication. You must have multi-factor authentication set up either via your phone or with the Microsoft Authenticator app.<br />
[[File:CloudStack Login Page.png|alt=CloudStack Login Page|none|thumb|CloudStack Login Page]]<br />
<br />
'''Note:''' Due to a bug with the UI, if the Single Sign-On option is disabled, please refresh the login page and try again. This issue should be addressed in our next update for CloudStack.<br />
<br />
=== CloudStack Dashboard===<br />
<br />
After logging in, you will be presented with your CloudStack management console. The dashboard shows you a general overview of your account's status.<br />
[[File:CloudStack Dashboard.png|alt=CloudStack Dashboard|none|thumb|CloudStack Dashboard]]On the right hand side of the dashboard, you will also see recent activity and events that was done within your CloudStack account.<br />
<br />
If you wish to see your CloudStack account resource quota and allocation, navigate to: <code>Accounts -> Click on your account -> Resources</code>. <br />
[[File:CloudStack Resource Quota.png|alt=CloudStack Resource Quota|none|thumb|CloudStack Resource Quota]]<br />
<br />
== Working with virtual machines==<br />
<br />
CloudStack allows you to control the lifecycle of virtual machines within your cloud account. VMs may be started, stopped, rebooted, or destroyed within your management console.<br />
<br />
===Create a VM===<br />
<br />
To create a new VM, enter the CloudStack management console and navigate to: <code>Compute -> Instances -> Add Instance</code>[[File:CloudStack Instance Summary.png|alt=CloudStack Instance Summary|thumb|CloudStack Instance Summary|493x493px]]<br />
<br />
Virtual Machines require the following details:<br />
<br />
# '''Deployment zone'''. Your account will already be placed in the appropriate zone.<br />
# '''Boot template or ISO'''. You may choose either a pre-created template or boot from a custom CD-ROM ISO file.<br />
# '''Compute offering'''. You may select an appropriate size for your new VM. Resources will be counted against your account's quota.<br />
# '''Data Disk'''. You may choose to add an additional virtual disk to your VM to store your data. Alternatively, if you wish to use a single virtual disk for your VM, you may choose to override the size of your root disk in step 2 and select 'No thanks' in this step.<br />
# '''Networks'''. You may choose one or more networks your VM should connect to. All CloudStack accounts come with a default network already created and ready to be used.<br />
# '''SSH keypairs'''. For templates that support custom SSH key pairs, you may choose to use a custom SSH keypair to be installed as part of the deployment process.<br />
# '''Advanced settings'''. For templates that support custom user-data (Cloud-Init), you may choose to enable the advanced settings and provide your own Cloud-Init user-data payload. More on this in the advanced tasks section below.<br />
# '''Other VM details'''. You may give your new VM a friendly name and make it part of a group. Groups allow you to group related VMs together for better organization. You may change these details at a later time.<br />
<br />
When you are done, review the instance summary on the right hand side and then click on the 'Launch Virtual Machine' button.<br />
<br />
=== Selecting your VM Operating system ===<br />
Many OSs will provide various editions that are tailored to a specific use case. A desktop VM may not be appropriate when you need to run a database server. The OS provider will have guides on how to choose an edition.<br />
<br />
You may choose to install the operating system to your virtual machine using either pre-built templates or from scratch using an ISO image.<br />
<br />
====Install from a virtual machine template====<br />
<br />
We provide a Rocky Linux 8.5 and a Ubuntu Server 22.04 LTS template for your convenience. These templates are pre-built images with the operating system installed and ready for use. Our templates also support further automated setup configured using Cloud-Init configuration data that can be provided when deploying a new VM. Currently, we offer the following templates: <br />
{| class="wikitable"<br />
!Template<br />
!Cloud-Init Support<br />
!Password Support<br />
!Default Username<br />
|-<br />
|Rocky Linux 8.5<br />
|Yes<br />
|Yes<br />
|rocky<br />
|-<br />
|Ubuntu Server 22.04<br />
|Yes<br />
|Yes<br />
|ubuntu<br />
|}<br />
Rocky Linux is an open source Linux distribution that is binary-compatible with Red Hat Enterprise Linux and is what RCS recommends.<br />
<br />
For templates that support passwords, the generated password that appears after a VM is created is applied to the default username.<br />
<br />
Security note: All VM templates are configured with SSH password authentication enabled. You should be able to SSH to your VM from another system connected to the same guest network. Do not expose port 22 unless required and we highly recommend using key based authentication.<br />
<br />
===== Virtual machine credentials =====<br />
VM templates that have password support will have a randomly generated password set when the VM is first created or when a password reset request is made (available only when the VM is powered off). A randomly generated 6 character password will be displayed when a new password is set and appears as a notification in your CloudStack management console. <br />
[[File:CloudStack VM Password.png|alt=CloudStack VM Password|none|thumb|CloudStack VM Password]]<br />
This password is set on the default username for your template. For example, the Rocky Linux VM template will set this password to the ''''rocky'''<nowiki/>' user account. You may become the super user by logging in as the <code>rocky</code> user and then running <code>sudo su</code>.<br />
<br />
Note: If you specify a custom Cloud-Init config that creates additional users or sets account passwords, the displayed password will be overridden and have no effect.<br />
<br />
==== Install from an ISO image====<br />
We provide various ISO images for popular Linux distributions. You may select one of these ISO images instead of using a pre-built template when deploying a new virtual machine. We currently provide:<br />
{| class="wikitable"<br />
!Distribution<br />
!ISO<br />
|-<br />
|Ubuntu 20.04<br />
|ubuntu-20.04.4-desktop-amd64.iso<br />
<br />
ubuntu-20.04.4-live-server-amd64.iso<br />
|-<br />
|Ubuntu 21.10<br />
|ubuntu-21.10-desktop-amd64.iso<br />
<br />
ubuntu-21.10-live-server-amd64.iso<br />
|-<br />
|Ubuntu 22.04<br />
|ubuntu-22.04-live-server-amd64.iso<br />
|-<br />
|Rocky Linux 8.5<br />
|Rocky-8.5-x86_64-minimal.iso<br />
|-<br />
|Fedora 35<br />
|Fedora-Workstation-Live-x86_64-35-1.2.iso<br />
|}<br />
You may install custom ISO file into your CloudStack account either by directly uploading the ISO through the web console or by providing a URL to the ISO file on the internet.<br />
<br />
Please do not install Windows on our CloudStack infrastructure. It is against our user agreement to run Windows based systems in this infrastructure. If you need a Windows VM, please contact us for alternative solutions.<br />
<br />
===== Register a ISO with a URL=====<br />
[[File:CloudStack Download ISO.png|alt=CloudStack Download ISO|thumb|CloudStack Download ISO|190x190px]]<br />
<br />
To add a custom ISO file from the internet, enter the CloudStack management console and navigate to: <code>Images -> ISOs -> Register ISO</code><br />
<br />
You may check the state of the ISO file by clicking on it and verify the state of the file. If the file is successfully downloaded, its ready state should become ‘true’. The ISO file will only appear in the selection list when the file is downloaded successfully.<br />
[[File:CloudStack ISO Ready.png|alt=CloudStack ISO Ready|none|thumb|172x172px|CloudStack ISO Ready]]<br />
<br />
=====Upload a custom ISO=====<br />
<br />
To upload an ISO file, enter the CloudStack management console and navigate to: <code>Images -> ISOs -> Upload ISO from Local (icon)</code><br />
<br />
[[File:CloudStack Upload ISO.png|alt=CloudStack Upload ISO|none|thumb|CloudStack Upload ISO|217x217px]]<br />
<br />
===Connecting to your VM console===<br />
The CloudStack management console has a KVM (keyboard, video, mouse) feature built-in, allowing you to remotely connect to and interact with your virtual machine. To connect to your virtual machine's console, navigate to: <code>Compute -> Instances -> Your Instance -> View console</code>.<br />
[[File:CloudStack View Console.png|alt=CloudStack View Console|none|thumb|CloudStack View Console]]<br />
<br />
=== Expanding a VM disk ===<br />
[[File:CloudStack Expand Volume.png|alt=CloudStack Expand Volume|thumb|CloudStack Expand Volume]]<br />
Virtual machine disks can be expanded after they are created within CloudStack. However, you will need to expand the partitions and filesystems manually.<br />
<br />
To grow an existing disk:<br />
<br />
# Go into your VM details page and click on ‘Volumes’.<br />
# Click on the volume you wish to expand.<br />
# Click on the ‘Resize Volume’ icon in the top right.<br />
Once the volume has been expanded, you should be able to verify the disk volume has grown with <code>lsblk</code>. There should also be some messages by the kernel when this occurs. However, you will still need to expand any partitions, volumes, and filesystems on your system manually.<br />
<br />
To expand your partition, use the <code>growpart</code> command followed by your disk device and partition number. Eg: <code>/usr/bin/growpart /dev/vda 3</code><br />
<br />
For LVM volume sets, you can expand the volume using the <code>pvresize</code> and <code>lvresize</code> commands:<br />
<br />
* <code>/usr/sbin/pvresize -y -q /dev/vda3</code><br />
* <code>/usr/sbin/lvresize -y -q -r -l +100%FREE /dev/mapper/<volume-name></code><br />
<br />
To expand your filesystem:<br />
<br />
* XFS: <code>/usr/sbin/xfs_growfs <volume></code><br />
* EXT: <code>resize2fs <volume></code><br />
<br />
=== Destroying a VM ===<br />
If you need to delete a VM, click on the red garbage bin icon in the VM instance page. All deletions are irreversible, so please make sure you have a copy of any data you need before proceeding.<br />
[[File:CloudStack Delete VM.png|alt=CloudStack Delete VM|none|thumb|CloudStack Delete VM]]<br />
The VM root volume can be deleted immediately by enabling the 'Expunge' option in the dialog box. If left disabled, the VM root volume will linger for a day before it is deleted by the system. You may wish to expunge a volume if you are running low on space or volume quota.<br />
<br />
<br />
== Virtual machine networking ==<br />
The CloudStack platform allows you to define custom virtual private cloud (VPC) network which can contain any number of guest networks that your virtual machines connect to. Each guest network has its own private network address space and is not directly routable from campus or the internet. For virtual machines that require internet access, the VPC or guest network it is connected to must have a NAT IP address associated. The following diagram shows how a guest network connects to the internet and campus network.<br />
[[File:CloudStack Guest Networking.png|alt=CloudStack Guest Networking|none|thumb|CloudStack Guest Networking]]<br />
In order to expose a virtual machine's services to campus or the internet, the appropriate port forwardings must be set up on the VPC containing the guest network. More on this will be discussed in the next section.<br />
<br />
Having multiple guest networks allows for more advanced network setups but is not required. We recommend using a single flat network for most workloads. <br />
<br />
By default, all CloudStack accounts come with a default VPC and guest network set up with a NAT IP assigned.<br />
<br />
=== IP addresses ===<br />
Due to the design decisions made during the setup of the CloudStack platform, only internal 10.44.12X.X IPs can be assigned to your VPC. These IP addresses are accessible from the university campus network. However, there is a special section of IP addresses that can be accessed from the internet.<br />
{| class="wikitable"<br />
!IP address range<br />
!Accessible from<br />
!Internet IP mapping<br />
|-<br />
|10.44.120.3-128<br />
|Campus, Internet<br />
|10.44.120.X maps to 136.159.140.X (ports 80 and 443 only)<br />
|-<br />
|10.44.120.129-255<br />
|Campus only<br />
|N/A<br />
|-<br />
|10.44.121.0-255<br />
|Campus only<br />
|N/A<br />
|-<br />
|10.44.122.0-255<br />
|Campus only<br />
|N/A<br />
|-<br />
|10.44.123.0-255<br />
|Campus only<br />
|N/A<br />
|}<br />
If you need a service exposed to the internet, please request for a public IP address using our [https://ucalgary.service-now.com/it?id=sc_cat_item&sys_id=e3c1d6e91be48554cca5ecefbd4bcb6c ServiceNow request form]. Additionally, if your service is not port 80 or 443, you must also request for a firewall change request to allow the special port through. <br />
<br />
=== Exposing a network service to campus ===<br />
In order to make a virtual machine be visible to the campus network, you must first set up a port forwarding from a campus IP address to your virtual machine.<br />
<br />
To create a port forwarding, navigate to <code>Network -> VPC -> Select your VPC -> Public IP Addresses</code>. If you do not have any available IP addresses, you will need to click on 'Acquire New IP' and select an available IP address. Click on the IP address you wish to use to create a port forwarding on and then navigate to the 'Port Forwarding' tab. Enter the private port range, the public port range, the protocol, and select the target VM. <br />
<br />
For example, to port forward only HTTP (tcp/80) traffic, you would enter the following:<br />
[[File:CloudStack Port Forwarding.png|alt=CloudStack Port Forwarding|none|thumb|CloudStack Port Forwarding]]Once the port forwarding is created, you should be able to access the service from on campus. If for some reason access to your service does not work, there may be a firewall restriction on IT's network. In such circumstances, please contact us for assistance.<br />
<br />
=== Exposing a network to the internet ===<br />
Exposing a service to the internet is the same as exposing it to campus. However, you must create a port forwarding on an IP address that maps to an internet IP address outlined in the IP address table above. If your account does not have one of these IP addresses available, please request for one on the [https://ucalgary.service-now.com/it?id=sc_cat_item&sys_id=e3c1d6e91be48554cca5ecefbd4bcb6c ServiceNow request form].<br />
<br />
By default, only ports 80 and 443 are allowed through the Internet IP address. For all other ports, please [https://ucalgary.service-now.com/it?id=sc_cat_item&sys_id=47cd16d113153a00b5b4ff82e144b0bf create a firewall rule change request in ServiceNow].<br />
<br />
== Cloud-Init Automation ==<br />
<br />
=== Ubuntu ===<br />
The following Cloud-Init configs apply to Ubuntu VM templates.<br />
<br />
==== Ubuntu desktop ====<br />
Use the following Cloud-Init config with the Ubuntu Server template to set up an Ubuntu desktop environment. The setup step takes a up to 15 minutes to complete and you should see a login screen when the setup finishes. <br />
<br />
Adjust the root and user password as desired. Below, the test user password is set to blank, allowing you to login to Gnome without a password.<br />
{{Highlight|code=#cloud-config<br />
disable_root: false<br />
<br />
users:<br />
- name: user<br />
lock_passwd: false<br />
inactive: false<br />
gecos: Test User<br />
primary_group: user<br />
groups: wheel<br />
passwd: $1$ADUODeAy$eCJ1lPSxhSGmSvrmWxjLC1<br />
<br />
chpasswd:<br />
list: {{!}}<br />
root:password<br />
expire: false<br />
<br />
# Install a graphical desktop<br />
runcmd:<br />
- DEBIAN_FRONTEND=noninteractive apt -y upgrade<br />
- DEBIAN_FRONTEND=noninteractive apt -y install tasksel<br />
- tasksel install gnome-desktop<br />
- systemctl set-default graphical.target<br />
- systemctl isolate graphical.target|style=max-height: 300px;overflow:scroll;}}<br />
<br />
=== Rocky Linux ===<br />
The following Cloud-Init configs apply to Rocky Linux templates.<br />
<br />
==== Rocky Linux Desktop ====<br />
Use the following Cloud-Init config using the Rocky Linux template to set up a Rocky Linux desktop environment. The setup step takes up to 10 minutes to complete and you should see a login screen when the setup finishes. <br />
<br />
Adjust the root and user password as desired. Below, the test user password is set to blank, allowing you to login to Gnome without a password.<br />
{{Highlight|code=#cloud-config<br />
disable_root: false<br />
<br />
users:<br />
- name: user<br />
lock_passwd: false<br />
inactive: false<br />
gecos: Test User<br />
primary_group: user<br />
groups: wheel<br />
passwd: $1$ADUODeAy$eCJ1lPSxhSGmSvrmWxjLC1<br />
<br />
chpasswd:<br />
list: {{!}}<br />
root:password<br />
expire: false<br />
<br />
# Install a graphical desktop<br />
runcmd:<br />
- yum -y install "@Workstation"<br />
- systemctl set-default graphical.target<br />
- systemctl isolate graphical.target}}<br />
<br />
==== Rocky Linux Docker host ====<br />
Use the following Cloud-Init config using the Rocky Linux template to set up a new docker host. This server can then be used to run Docker containers. Also included are:<br />
<br />
# The docker-compose utility to help deploy container stacks more easily<br />
# A helper script to expand the <code>/var</code> and <code>/</code> filesystems on first startup based on the available space in the ROOT volume. <br />
<br />
Use the CloudStack generated password with the '<code>rocky</code>' default user account to log in.{{Highlight|code=#cloud-config<br />
<br />
write_files:<br />
- path: /usr/bin/expand_lvm_root<br />
content: {{!}}<br />
#!/bin/bash<br />
/usr/bin/growpart /dev/vda 3<br />
/usr/sbin/pvresize -y -q /dev/vda3<br />
/usr/sbin/lvresize -y -q -r -l +50%FREE /dev/mapper/*root<br />
/usr/sbin/lvresize -y -q -r -l +100%FREE /dev/mapper/*var/dev/mapper/*root<br />
/usr/sbin/xfs_growfs /<br />
/usr/sbin/xfs_growfs /var<br />
permissions: '0700'<br />
<br />
- path: /usr/bin/setup_docker<br />
content: {{!}}<br />
#!/bin/bash<br />
yum install -y yum-utils<br />
yum-config-manager \<br />
--add-repo \<br />
https://download.docker.com/linux/centos/docker-ce.repo<br />
yum install -y docker-ce docker-ce-cli containerd.io<br />
systemctl start docker<br />
systemctl enable docker<br />
<br />
curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose<br />
chmod +x /usr/local/bin/docker-compose<br />
permissions: '0700'<br />
<br />
- path: /root/docker-compose.yml<br />
permissions: '0700'<br />
content: {{!}}<br />
version: '3.3'<br />
services:<br />
web:<br />
image: php:7.4-apache<br />
restart: always<br />
user: "0:0"<br />
volumes:<br />
- /var/www/html:/var/www/html<br />
ports:<br />
- "80:80"<br />
<br />
- path: /var/www/html/index.php<br />
permissions: '0644'<br />
content: {{!}}<br />
<h1>Hello there!</h1><br />
<p>I see you from <?php echo $_SERVER['REMOTE_ADDR']; ?></p><br />
<<nowiki>pre</nowiki>><?php print_r($_SERVER); ?></pre><br />
<br />
# Ensure VM has the largest / possible<br />
runcmd:<br />
- /usr/bin/expand_lvm_root<br />
- /usr/bin/setup_docker<br />
- cd /root; docker-compose up -d|style=max-height: 300px;overflow:scroll;}}<br />
<br />
==== UC Authentication ====<br />
Use the following Cloud-Init config to allow UC-based authentication. Local accounts with the same username as the IT account may use the IT credential to log in.<br />
{{Highlight|code=#cloud-config<br />
<br />
write_files:<br />
- path: /usr/bin/setup_uc_auth<br />
permissions: '0700'<br />
content: {{!}}<br />
#!/bin/bash<br />
yum install -y sssd sssd-dbus sssd-krb5 krb5-workstation authselect-compat<br />
<br />
cat <<EOF > /etc/sssd/sssd.conf<br />
[sssd]<br />
config_file_version = 2<br />
services = nss, pam, ifp<br />
domains = uc.ucalgary.ca<br />
<br />
[domain/uc.ucalgary.ca]<br />
id_provider = files<br />
debug_level = 5<br />
auth_provider = krb5<br />
chpass_provider = krb5<br />
<br />
krb5_realm = UC.UCALGARY.CA<br />
krb5_server = ITSODCSRV14.UC.UCALGARY.CA:88<br />
krb5_validate = false<br />
EOF<br />
chmod 600 /etc/sssd/sssd.conf<br />
<br />
cat <<EOF > /etc/krb5.conf<br />
[logging]<br />
default = FILE:/var/log/krb5libs.log<br />
kdc = FILE:/var/log/krb5kdc.log<br />
admin_server = FILE:/var/log/kadmind.log<br />
<br />
[libdefaults]<br />
default_realm = UC.UCALGARY.CA<br />
dns_lookup_realm = false<br />
dns_lookup_kdc = false<br />
ticket_lifetime = 24h<br />
renew_lifetime = 7d<br />
forwardable = true<br />
<br />
[realms]<br />
UC.UCALGARY.CA = {<br />
kdc = itsodcsrv14.uc.ucalgary.ca<br />
}<br />
<br />
[domain_realm]<br />
uc.ucalgary.ca = UC.UCALGARY.CA<br />
.uc.ucalgary.ca = UC.UCALGARY.CA<br />
EOF<br />
<br />
mkdir -p /etc/authselect/custom/rcs<br />
cd /etc/authselect/custom/rcs<br />
echo H4sIAMYsbGIAA+0ca3PbuDGf+StQ34dLMpEtybLdUevO6BKl56mTuJGv7Uwmo4FISMKJBHQAaEfNub+9uwBJkXrLshxnjjuTyCKJfWF3sQsuFPhS9CtB79keoQpw2mjYT4DZz1r1uPGs1qhX6yf12vFx/Vm1Vqs1jp+R6j6ZSiHWhipCnikpzarn1t3/TuErj8bhhBzccjOs6Igq41MVHBDen71WUey3mCsWHNx5W4wKpT+qSAGjI3lDQxj8SarB0UDIiB2FcsBFRfuKMfHZY4L2QpYbS2MzZMJwnxouxfnXefQHTaNi9nufhprdpQj6XAyYGisuzHIUuYcWIhlTrW/lRkxMNdO0OH5HdHdeXlDNjAGCuhJQ0IM4GjPFx0Om4PGjDM9n8pULP4wDtpkik78q1Lec/WifcMr8kSyE7fB/a9ssYf8Q2PiPc6/3RmNN/K826mez8f/07LSM/48BR4uD8dGaWOytGbc8BK8buSTu5odtFEyPitGxDH2LYXai9kFjjf8fn4HPT/3/GPy/fnZ6Uvr/Y8BXiP/gTHHOL/KZ0Z2HRpGmD2meszi5mIExjbpM3BxquRuOgIkJICE/kE7qsKQYGwjXRbSrfD2Xxe7EVp/yEOMGsjZWzOLSPASuCHJ83iCxwPtdwyN2XqtXqwvYSpHMMKPjfp/7HHFtyIydrWBnXeeFQkT4fak4ZImulwh134n3POr7Mk50sTUWGMy0RoE2hnmRAJHDgzLtwk1ewffmJq/gXZiJBf9iTSaHZGvTA0ZoGGumdsakY1AxC7q8j9qJeUD+SmqwPhAQi5kC7k8B69M4NOc9GhA7TutzOSLIB4g1EvJWnHPIFhT77HBbE9hNXZBmRNxYi0yTlHvhyexaA9cYvhDkGCMZDTfGMmITLiw7wMGNHLECuq2ZCjnIZnVUyeHZmis90YZFNhDtJJ0Mgl9lrxuNhpDwBVxZi4ioHp1Xq2dni0bNu0k2GPwkz8yn1GBqJDWj1FSWCFUwTDCxG+4DHUF8JUXgzBNNrws2u9sspB65k+4SW99s/RdgyNz4w0OsA/eUY6yr/05rtdn8r35S1n+PAjaSBU1rPGA4pA8pjCaJG6NbsS8zbuWDxmTk6jTMogZKxmOHYDsMdiAgEMxkOKYIcp69DEE60CUaMsLQ3twSRzbQBgnr2XprRtKBiCMOJFMWRaoJXcAxn5rCiG9XcRbK7T3RWOP/pyen+fqvgf5/XC3rv0eBb17eYTIbsJDaGs/+cV53dlGWjfsuG+O6zWh8qP43hYW1UQUQ7aqYHC9CYh4P6fx9eanU+3SGn6xiqBGX7CU5X656mKZ/i6uRv52T+rQa2QV3sWbaaQZdwkjwrZSQxqlBxqYiYrCQ0UHTfd4RoybdPlfadDHiz0+V5mYjI1ivmm4i9W6CuSSW9KW6BfdbwnS5tVFubSxmJvUL0oM4wkRXD2kgb70fcnojt0CKUCB3S5IrYG3EZ8pQqO1sZqkJ+JLig6H5C/CgMUaPmQjgsiA6jFW0247Hg0yg5WNXfVkkXdDR2GzKzmacPLH9pPr3sZ20TTQe3/4W05Abm+0UwzyxyuyiGLorRTgpULr3YgPOdFKrE+dT2y49dqMEg6eRo93YSZaIZfjKfbkcmnJf7jvbl3ssGEtt7Gv4PdJYt/93Um/M1P81iC9l/f8YsKFFWydAP7eeZuvVTX13uaf+6X9kEEQv8991/DJZnwvYswJrLhTYGEmtDSNqWKlNNCa5Jj3LayV55KDpvv+uh/IWc00W3G3q1DOUkqIdatVxQGG1nmJ8Yh6+Gj62W2/etfdLY43/n1VPT3P+XwP/rzaOy/3/R4G27boinU7njS1/XKIwu1P2nIauGHcZpU2MsQgKJy+8813B8zosZD62dBEz5JqMlcStc1egsRyDVMMD4GoyVhAuZJ/wAJk0E4+Kud09wHID95U+BAI4OrmgCYWIY3C46x8DpJJEVNABSwtBg8lbJMGtIU0B1qTiTC8iEjF/SAXXkca6YYgcXr5pXb0i/2Cqx5QEJSnyVjF2cdU6JBcmYwKQkfedDkGcV613kLUYpvoUxDISdzycoMlkIF0yDuPBwOqiR32oaYPktgfM+lIIYNMKAnGSj0PkvN9nCmNUWqY4vaE+3uHuFBcwoZGTg/Ygc3dK9oGzHoPJjoEE7pIaOyO/Iv4x6KjpDY0Z6+bREXyLFTvk8ggHHkF+E3jeT5N0BXhVMBaK+2EQ6CHr6isZOVqKgpgKZRXJw+6dCYRvvO+BMbj5D8hziK/sxj08yU8DIzecZlnai0NyjTY0pDcMlQZKRSGFz7weE6zPQQ2QkbJUVn/InFE5wtgNhSqDJLIPsR4+KxEYgpq4R+1UmGGsPcgeY9AlEXEE82yNiesRrB9MWY2iln92HL9CcSYyJoG0hdItFXamRoyNEzXEQqD1g7bhOQXG6A+5YK/sKJgPzz5a8A1tXQb0ghz9GuPWBNdTT0mWVFQGw50LNAox8PAlL4dJc5Nu/Usb/B+KphEIr9DYwwkBtvG2GTpsCWpHLDep1iOSjRKLrIfEaDLD+Ql1bhROUl98/eH924u///KxdX3x4b1XmQPPa8EMOyFBb6hoiUpDH0MEP9qpmspySK5CRjWozNJ3j3iB9OMIptOJazDzYATsiDiXscPBDwx6P9TQwB75ADGtIGIqF72Bxd0qGFFNnRPmFm2Y4dsEIQn7wrWNZM6+CjyCUP9qXVy2frpskw9XKHnrkrxtt65/+djuLFBCThuFzbFm08OEJAndqXfjHSSM9Q9oAk3SSBvYJsiHZn5s+A2zQ2eiGCIGTYDN4k2vWKXNUMP3pTDKhyKLudFACB/Nh0q7VKSLBOqHK4vEbgMQW28cJkJlr1jm6OQ5tLaYPYoeC1Y3cLaQzj2aCITPITUWUfY0RObxWCqYZXQTnEpczNiXcch9bq09CTFIhAs7eKGNzXI82zecSHAJV4lraia3GNlojhf7zgmeZsE8urQsTfBs8t7qkLyXRNogGjEKSxlMxkzjNKJ67opwNI90i+RF5rGJ+Ck/ufbP9VOSe9h6no3Pbmpe5hoTX6bIkxc0KxFjOIdnwO3FIGRFfHB9Fhe+7Cniq0NUgqUUTPG+qItVQ4L9jQvfEIEx9OfrAdC91pg+BLFCHRcNPA5kgqFlN5vxAkYg8I8sraFpUmNzMHxAxXYhBOsJAp762REz/lHSYpBTg8taEiKvhwzsL9laRgNOuUoDBeoEvPS/eaOe7tsVhQXixF0HeRWNmMEZltNNEc97c9GxEa1z1X598fbiNaY1/764fv0zedO6bv3U6qyObdMY9x4XagjCr0jalAQJmqE9cG1tM7bexAbedAkEJ1K3ihu4C84XJw+BA2PgqRQam9w62Jeof7vQ2jJP21wHNO47G0yWGyO95P2dRbuINK57OH14TQqWECbzhD0uYJWgmXMVmmdA1ReO0EHSTZMJvIDoDArXMjPF4DphtkCQtd1McWQNNVugmXbeTPFMm2q2QJR130zxZH0126BJGnByWNyVdUja/2m9u7rMGavnvST58sitQeibOP1zy4vnkbwVpR+QFJMsDpBixF9OIV2ykJJdaNkmayzxCkvsOo6yJX7KIKRn7TZpXXY+TNXwEnMIO8ya9PM/v/iudjhKWAUf2//85eJj+137/XVnXzTWnv85qc7s/+C3cv/nMeAdHWGfinKJc6F4xDwzq5Jc7ZemiqTDkpBVLLEwJEUzewuuqNgPrOjYOrjzKjPhFgVK6+ZXENRSyfWKRDutDXguAjbX0CWkYpeYA8ySsJ/Avp8k5+RaxewAUX2CO5/hGXc08aHk3ZuaFzVqFA6JJZrOVwNLdJ0UBOttbEPKjynztNMtkTe5sExW61E2yaCRLTsiqBxCKzOmhZDrzr+CXEUV7cot+Ji32k2z6S7GiE3w3brbRLUkYZDfH+AmwNp3lKup7gvW9e8VtYwXH0rT6yh/A01nlB9P2/kX8ZXZlHCJphc1AaTavg9dQuymJtN2oxp3nC3yQoAoRoSHkdfGaLuD55vsJUNC+zClvQc9F+li7mEehuwausvW/+IB//3kGOvyv2L//wn2/zfqtTL/ewy45+//zJ8azyciT+JQwZPpsk/61GynaTfiGleOrqBQza/+LaHkr2kG+YBHD8oO6D9IB/RTbsZ9ooe7n37751Zsfdv2z7L/c8ruU+z/dJaxz9Of6/K/WrXWaMzkf7Wzk7My/3sMuF/+9yQSvH2cGt1n4vhA8U1IyLzdC57mIIiaX+DfCD9dy4GmN0w17Q+lVfJXRrkvxdB4n3OvmSwBvv4snnpMhQt4IlkuCZ7LaZOx9m/cNIZQazttyL2O4+7ww0kbmQhZyNbyH8wqT+R+hydyN1LQol+DWFYCZ9zU98jNyjcDy1131u9S1la4dM6d8WjX1JXvw0154jmjUZ54XoulrPeXMlOeeC5PPJcnnssTz+WJ52+95VXueGXsPsUdrxJKKKGEEkoooYQSSiihhBJKKKGEEkr448D/AT019aAAeAAA {{!}} base64 -d {{!}} tar -xzpf -<br />
<br />
if [[ "`authselect current -r`" != "custom/rcs" ]] ; then<br />
authselect select custom/rcs --force<br />
systemctl restart sssd<br />
fi<br />
<br />
systemctl enable sssd<br />
<br />
runcmd:<br />
- /usr/bin/setup_uc_auth|style=max-height: 300px;overflow:scroll;}}<br />
<br />
==== Rocky Linux NFS server ====<br />
Because there is no ability to share storage among multiple VMs, a local NFS server could be useful if you need to share data between multiple VMs.{{Highlight|code=#cloud-config<br />
<br />
write_files:<br />
- path: /usr/bin/expand_lvm_root<br />
content: {{!}}<br />
#!/bin/bash<br />
/usr/bin/growpart /dev/vda 3<br />
/usr/sbin/pvresize -y -q /dev/vda3<br />
/usr/sbin/lvresize -y -q -r -l +50%FREE /dev/mapper/*root<br />
/usr/sbin/lvresize -y -q -r -l +100%FREE /dev/mapper/*var<br />
/usr/sbin/xfs_growfs /<br />
/usr/sbin/xfs_growfs /var<br />
permissions: '0700'<br />
<br />
- path: /usr/bin/setup_nfs<br />
content: {{!}}<br />
#!/bin/bash<br />
yum install -y nfs-utils<br />
<br />
mkdir /export<br />
if [ -b /dev/vdb ] ; then<br />
mkfs.xfs /dev/vdb<br />
echo "/dev/vdb /export xfs defaults 1 2" >> /etc/fstab<br />
mount -a<br />
fi<br />
<br />
ip a {{!}} grep -w inet {{!}} awk '{print $2}' {{!}} while read subnet ; do<br />
echo "/export $subnet(rw,no_subtree_check,no_root_squash,async)" >> /etc/exports<br />
done<br />
<br />
systemctl start nfs-server<br />
systemctl enable nfs-server<br />
<br />
exportfs -ra<br />
<br />
permissions: '0700'<br />
<br />
runcmd:<br />
- /usr/bin/expand_lvm_root<br />
- /usr/bin/setup_nfs|style=max-height: 300px;overflow:scroll;}}NFS clients connected to the same network as the NFS server can then mount <code>/export</code> using a command similar to: <code>mount -t nfs nfs-server:/export /mnt</code>.<br />
<br />
== Infrastructure tools ==<br />
<br />
=== Generating a CloudStack API ===<br />
You can request for a CloudStack API key to automate infrastructure deployment using Terraform or CloudMonkey. A new API key can be generated by navigating to your profile page (top right) and then clicking on the 'Generate keys' button.<br />
[[File:CloudStack API Key.png|alt=CloudStack API Key|none|thumb|CloudStack API Key]]<br />
<br />
=== CloudMonkey ===<br />
CloudMonkey is a utility that makes it easier to interact with the CloudStack API. This tool may be used to help automate VM actions (such as start/stop/reboot), or infrastructure tasks (such as creating/destroying VMs, networks, or firewall rules). <br />
<br />
To get started with CloudMonkey, refer to the following resources:<br />
<br />
* Download from: <nowiki>https://github.com/apache/cloudstack-cloudmonkey/releases/tag/6.1.0</nowiki><br />
* Documentation at: <nowiki>https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+cloudmonkey+CLI</nowiki><br />
<br />
=== Terraform Integration ===<br />
Terraform allows you to define infrastructure as code and can be used in conjunction with CloudStack to configure your virtual machines and guest networks. Use the official CloudStack provider.<br />
<br />
The following is an example Terraform file for reference. Specify your CloudStack API keys either as a separate <code>vars.tf</code>.<br />
{{Highlight|code=# Configure the CloudStack Provider<br />
terraform {<br />
required_providers {<br />
cloudstack = {<br />
source = "cloudstack/cloudstack"<br />
version = "0.4.0"<br />
}<br />
}<br />
}<br />
<br />
provider "cloudstack" {<br />
api_url = "${var.cloudstack_api_url}"<br />
api_key = "${var.cloudstack_api_key}"<br />
secret_key = "${var.cloudstack_secret_key}"<br />
}<br />
<br />
# Create a new VPC<br />
resource "cloudstack_vpc" "default" {<br />
name = "wedgenet-vpc"<br />
display_text = "wedgenet-vpc"<br />
cidr = "100.64.0.0/20"<br />
vpc_offering = "Default VPC offering"<br />
zone = "zone1"<br />
}<br />
<br />
# Create a new ACL<br />
resource "cloudstack_network_acl" "default" {<br />
name = "vpc-acl"<br />
vpc_id = "${cloudstack_vpc.default.id}"<br />
}<br />
<br />
# One ingress and one egress rule for the ACL<br />
resource "cloudstack_network_acl_rule" "ingress" {<br />
acl_id = "${cloudstack_network_acl.default.id}"<br />
<br />
rule {<br />
action = "allow"<br />
cidr_list = ["0.0.0.0/0"]<br />
protocol = "tcp"<br />
ports = ["22", "80", "443"]<br />
traffic_type = "ingress"<br />
}<br />
}<br />
<br />
resource "cloudstack_network_acl_rule" "egress" {<br />
acl_id = "${cloudstack_network_acl.default.id}"<br />
<br />
rule {<br />
action = "allow"<br />
cidr_list = ["0.0.0.0/0"]<br />
protocol = "all"<br />
traffic_type = "egress"<br />
}<br />
}<br />
<br />
<br />
# Create a new network in the VPC<br />
resource "cloudstack_network" "primary" {<br />
name = "primary"<br />
display_text = "primary"<br />
cidr = "100.64.1.0/24"<br />
network_offering = "DefaultIsolatedNetworkOfferingForVpcNetworks"<br />
acl_id = "${cloudstack_network_acl.default.id}"<br />
vpc_id = "${cloudstack_vpc.default.id}"<br />
zone = "zone1"<br />
}<br />
<br />
# Create a new public IP address for this network<br />
resource "cloudstack_ipaddress" "public_ip" {<br />
vpc_id = "${cloudstack_vpc.default.id}"<br />
network_id = "${cloudstack_network.wosnet1.id}"<br />
}<br />
<br />
# Create VMs. <br />
resource "cloudstack_instance" "vm" {<br />
count = 1<br />
name = "vm${count.index+1}"<br />
zone = "zone1"<br />
service_offering = "rcs.c4"<br />
template = "RockyLinux 8.5"<br />
network_id = "${cloudstack_network.wosnet1.id}"<br />
<br />
# Cloud Init data can be used to configure your VM on first startup if your template supports Cloud Init<br />
user_data = <<EOF<br />
#cloud-config<br />
<br />
# Require specific packages<br />
packages:<br />
- tmux<br />
- git<br />
- tcpdump<br />
<br />
EOF<br />
}<br />
<br />
<br />
|style=max-height: 300px;overflow:scroll;}}<br />
<br />
= Troubleshooting =<br />
<br />
=== Cannot create a volume snapshot ===<br />
Volume snapshots can only be taken on VMs that are powered off.<br />
<br />
=== Cannot create a VM snapshot ===<br />
Disk-only VM snapshots cannot be taken when the VM is running. If you intend to snapshot a running system, you must also snapshot its memory.<br />
<br />
=== VM state is still running after shutdown ===<br />
After running 'shutdown' on a VM, the VM state reported by CloudStack is still running. <br />
<br />
Please try to do a force shutdown from the CloudStack management console. The VM state isn't updated by CloudStack and as a result, the state of a VM isn't properly reflected when power state changes outside of CloudStack (likely a bug?)</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=CloudStack_User_Guide&diff=1853CloudStack User Guide2022-06-02T17:25:54Z<p>Darcy: /* Managing your virtual machine */</p>
<hr />
<div>This is a user's guide on using CloudStack provided by Research Computing Services.<br />
<br />
== Introduction==<br />
Apache CloudStack is an Infrastructure as a Service (IaaS) platform that allows users to quickly spin up Linux/Non-Windows based virtual machines. RCS is providing this service to help researchers quickly set up and prototype short-term research related software on premises. CloudStack is not appropriate for workloads that depend on Windows. Services set up on CloudStack virtual machines can be accessed from the campus network and also the internet if required.<br />
<br />
Access to CloudStack can be requested via [https://ucalgary.service-now.com/it?id=sc_cat_item&sys_id=e3c1d6e91be48554cca5ecefbd4bcb6c ServiceNow].<br />
<br />
Please refer to our [[CloudStack End User Agreement]] for acceptable uses and requirements.<br />
<br />
== Using your virtual machine ==<br />
You will be able to run whatever virtual machine you wish (with the exception of Windows). Clearly we cannot provide specific management advice on each and every operating system available. We can provide you with some suggestions on important considerations to be aware of.<br />
<br />
=== Educate yourself ===<br />
All operating systems (OS) have user groups, web sites, wikis, or mailing lists somewhere on the internet. They can be a valuable resource. Most OS providers have on-line documentation that describes using their product. For example Rocky Linux, used by RCS, has a [https://docs.rockylinux.org/ documentation site]. These are excellent resources and can help you understand how to manage your virtual machine.<br />
<br />
=== Configure your VM's OS ===<br />
It is critical to ensure that the only services running on your VM are the ones you must run. Each OS has a way of managing what services are running (sysinit, systemd etc). Please ensure that unnecessary services have been disabled.<br />
<br />
Many OSs will have pre-configured accounts, and many applications will have pre-configured accounts. Make sure they are either disabled or not allowed to login.<br />
<br />
All un-used accounts should be disabled or preferably deleted.<br />
<br />
All accounts should have strong [https://it.ucalgary.ca/it-security/passwords-do-i-have-change-them/ passwords].<br />
<br />
Many OS's have the ability to automatically update themselves. If possible please consider doing this.<br />
<br />
Updates can also be configured to skip certain software if it will interfere with your research, but please be advised that doing so could place your system at risk.<br />
<br />
=== Exposed to the Internet ===<br />
Not everyone is a computer security expert. If your VM must be exposed to the internet, please consider using Trend Micro Cloud One Workload Security from IT security to enhance your security posture.<br />
<br />
== Accessing CloudStack ==<br />
===Accessing the CloudStack management console===<br />
<br />
The CloudStack management console is a web-based portal that allows you to view and manage your cloud infrastructure including virtual machines, storage, and network. Any modern web browsers including Chrome, Firefox, Edge, and Safari is supported. <br />
<br />
Access the CloudStack management console is possible only from an IT-managed computer or through the IT General VPN when working on unmanaged machines (eg. AirUC) or when working off campus (eg. at home). Please review the IT [https://ucalgary.service-now.com/it?id=kb_article&sys_id=52a169d6dbe5bc506ad32637059619cd knowledge base article on connecting to the General VPN] or contact IT support if you need assistance connecting to the General VPN. <br />
[[File:CloudStack VPN Connection.png|alt=CloudStack VPN Connection|none|thumb|CloudStack VPN Connection]]<br />
=== Login to CloudStack===<br />
<br />
To log in to CloudStack, navigate to https://cloudstack.rcs.ucalgary.ca/. If this site fails to load, please make sure you are either on a IT managed computer or connected to the General VPN.<br />
<br />
Sign in to CloudStack using the Single Sign-On option as shown in the image below. This method will require you to authenticate through our central authentication service using your University of Calgary IT credentials and will require multi-factor authentication. You must have multi-factor authentication set up either via your phone or with the Microsoft Authenticator app.<br />
[[File:CloudStack Login Page.png|alt=CloudStack Login Page|none|thumb|CloudStack Login Page]]<br />
<br />
'''Note:''' Due to a bug with the UI, if the Single Sign-On option is disabled, please refresh the login page and try again. This issue should be addressed in our next update for CloudStack.<br />
<br />
=== CloudStack Dashboard===<br />
<br />
After logging in, you will be presented with your CloudStack management console. The dashboard shows you a general overview of your account's status.<br />
[[File:CloudStack Dashboard.png|alt=CloudStack Dashboard|none|thumb|CloudStack Dashboard]]On the right hand side of the dashboard, you will also see recent activity and events that was done within your CloudStack account.<br />
<br />
If you wish to see your CloudStack account resource quota and allocation, navigate to: <code>Accounts -> Click on your account -> Resources</code>. <br />
[[File:CloudStack Resource Quota.png|alt=CloudStack Resource Quota|none|thumb|CloudStack Resource Quota]]<br />
<br />
== Working with virtual machines==<br />
<br />
CloudStack allows you to control the lifecycle of virtual machines within your cloud account. VMs may be started, stopped, rebooted, or destroyed within your management console.<br />
<br />
===Create a VM===<br />
<br />
To create a new VM, enter the CloudStack management console and navigate to: <code>Compute -> Instances -> Add Instance</code>[[File:CloudStack Instance Summary.png|alt=CloudStack Instance Summary|thumb|CloudStack Instance Summary|493x493px]]<br />
<br />
Virtual Machines require the following details:<br />
<br />
# '''Deployment zone'''. Your account will already be placed in the appropriate zone.<br />
# '''Boot template or ISO'''. You may choose either a pre-created template or boot from a custom CD-ROM ISO file.<br />
# '''Compute offering'''. You may select an appropriate size for your new VM. Resources will be counted against your account's quota.<br />
# '''Data Disk'''. You may choose to add an additional virtual disk to your VM to store your data. Alternatively, if you wish to use a single virtual disk for your VM, you may choose to override the size of your root disk in step 2 and select 'No thanks' in this step.<br />
# '''Networks'''. You may choose one or more networks your VM should connect to. All CloudStack accounts come with a default network already created and ready to be used.<br />
# '''SSH keypairs'''. For templates that support custom SSH key pairs, you may choose to use a custom SSH keypair to be installed as part of the deployment process.<br />
# '''Advanced settings'''. For templates that support custom user-data (Cloud-Init), you may choose to enable the advanced settings and provide your own Cloud-Init user-data payload. More on this in the advanced tasks section below.<br />
# '''Other VM details'''. You may give your new VM a friendly name and make it part of a group. Groups allow you to group related VMs together for better organization. You may change these details at a later time.<br />
<br />
When you are done, review the instance summary on the right hand side and then click on the 'Launch Virtual Machine' button.<br />
<br />
=== Selecting your VM Operating system ===<br />
Many OSs will provide various editions that are tailored to a specific use case. A desktop VM may not be appropriate when you need to run a database server. The OS provider will have guides on how to choose an edition.<br />
<br />
You may choose to install the operating system to your virtual machine using either pre-built templates or from scratch using an ISO image.<br />
<br />
====Install from a virtual machine template====<br />
<br />
We provide a Rocky Linux 8.5 and a Ubuntu Server 22.04 LTS template for your convenience. These templates are pre-built images with the operating system installed and ready for use. Our templates also support further automated setup configured using Cloud-Init configuration data that can be provided when deploying a new VM. Currently, we offer the following templates: <br />
{| class="wikitable"<br />
!Template<br />
!Cloud-Init Support<br />
!Password Support<br />
!Default Username<br />
|-<br />
|Rocky Linux 8.5<br />
|Yes<br />
|Yes<br />
|rocky<br />
|-<br />
|Ubuntu Server 22.04<br />
|Yes<br />
|Yes<br />
|ubuntu<br />
|}<br />
Rocky Linux is an open source Linux distribution that is binary-compatible with Red Hat Enterprise Linux and is what RCS recommends.<br />
<br />
For templates that support passwords, the generated password that appears after a VM is created is applied to the default username.<br />
<br />
Security note: All VM templates are configured with SSH password authentication enabled. You should be able to SSH to your VM from another system connected to the same guest network. Do not expose port 22 unless required and we highly recommend using key based authentication.<br />
<br />
===== Virtual machine credentials =====<br />
VM templates that support password will have a randomly generated password set when the VM is first created or when a password reset request is made (available only when the VM is powered off). A randomly generated 6 character password will be displayed when a new password is set and appears as a notification in your CloudStack management console. <br />
[[File:CloudStack VM Password.png|alt=CloudStack VM Password|none|thumb|CloudStack VM Password]]<br />
This password is set on the default username for your template. For example, the Rocky Linux VM template will set this password to the ''''rocky'''<nowiki/>' user account. You may become the super user by logging in as the <code>rocky</code> user and then running <code>sudo su</code>.<br />
<br />
Note: If you specify a custom Cloud-Init config that creates additional users or sets account passwords, the displayed password will be overridden and have no effect.<br />
<br />
==== Install from an ISO image====<br />
We provide various ISO images for popular Linux distributions. You may select one of these ISO images instead of using a pre-built template when deploying a new virtual machine. We currently provide:<br />
{| class="wikitable"<br />
!Distribution<br />
!ISO<br />
|-<br />
|Ubuntu 20.04<br />
|ubuntu-20.04.4-desktop-amd64.iso<br />
<br />
ubuntu-20.04.4-live-server-amd64.iso<br />
|-<br />
|Ubuntu 21.10<br />
|ubuntu-21.10-desktop-amd64.iso<br />
<br />
ubuntu-21.10-live-server-amd64.iso<br />
|-<br />
|Ubuntu 22.04<br />
|ubuntu-22.04-live-server-amd64.iso<br />
|-<br />
|Rocky Linux 8.5<br />
|Rocky-8.5-x86_64-minimal.iso<br />
|-<br />
|Fedora 35<br />
|Fedora-Workstation-Live-x86_64-35-1.2.iso<br />
|}<br />
You may install custom ISO file into your CloudStack account either by directly uploading the ISO through the web console or by providing a URL to the ISO file on the internet.<br />
<br />
Please do not install Windows on our CloudStack infrastructure. It is against our user agreement to run Windows based systems in this infrastructure. If you need a Windows VM, please contact us for alternative solutions.<br />
<br />
===== Register a ISO with a URL=====<br />
[[File:CloudStack Download ISO.png|alt=CloudStack Download ISO|thumb|CloudStack Download ISO|190x190px]]<br />
<br />
To add a custom ISO file from the internet, enter the CloudStack management console and navigate to: <code>Images -> ISOs -> Register ISO</code><br />
<br />
You may check the state of the ISO file by clicking on it and verify the state of the file. If the file is successfully downloaded, its ready state should become ‘true’. The ISO file will only appear in the selection list when the file is downloaded successfully.<br />
[[File:CloudStack ISO Ready.png|alt=CloudStack ISO Ready|none|thumb|172x172px|CloudStack ISO Ready]]<br />
<br />
=====Upload a custom ISO=====<br />
<br />
To upload an ISO file, enter the CloudStack management console and navigate to: <code>Images -> ISOs -> Upload ISO from Local (icon)</code><br />
<br />
[[File:CloudStack Upload ISO.png|alt=CloudStack Upload ISO|none|thumb|CloudStack Upload ISO|217x217px]]<br />
<br />
===Connecting to your VM console===<br />
The CloudStack management console has a KVM (keyboard, video, mouse) feature built-in, allowing you to remotely connect to and interact with your virtual machine. To connect to your virtual machine's console, navigate to: <code>Compute -> Instances -> Your Instance -> View console</code>.<br />
[[File:CloudStack View Console.png|alt=CloudStack View Console|none|thumb|CloudStack View Console]]<br />
<br />
=== Expanding a VM disk ===<br />
[[File:CloudStack Expand Volume.png|alt=CloudStack Expand Volume|thumb|CloudStack Expand Volume]]<br />
Virtual machine disks can be expanded after they are created within CloudStack. However, you will need to expand the partitions and filesystems manually.<br />
<br />
To grow an existing disk:<br />
<br />
# Go into your VM details page and click on ‘Volumes’.<br />
# Click on the volume you wish to expand.<br />
# Click on the ‘Resize Volume’ icon in the top right.<br />
Once the volume has been expanded, you should be able to verify the disk volume has grown with <code>lsblk</code>. There should also be some messages by the kernel when this occurs. However, you will still need to expand any partitions, volumes, and filesystems on your system manually.<br />
<br />
To expand your partition, use the <code>growpart</code> command followed by your disk device and partition number. Eg: <code>/usr/bin/growpart /dev/vda 3</code><br />
<br />
For LVM volume sets, you can expand the volume using the <code>pvresize</code> and <code>lvresize</code> commands:<br />
<br />
* <code>/usr/sbin/pvresize -y -q /dev/vda3</code><br />
* <code>/usr/sbin/lvresize -y -q -r -l +100%FREE /dev/mapper/<volume-name></code><br />
<br />
To expand your filesystem:<br />
<br />
* XFS: <code>/usr/sbin/xfs_growfs <volume></code><br />
* EXT: <code>resize2fs <volume></code><br />
<br />
=== Destroying a VM ===<br />
If you need to delete a VM, click on the red garbage bin icon in the VM instance page. All deletions are irreversible, so please make sure you have a copy of any data you need before proceeding.<br />
[[File:CloudStack Delete VM.png|alt=CloudStack Delete VM|none|thumb|CloudStack Delete VM]]<br />
The VM root volume can be deleted immediately by enabling the 'Expunge' option in the dialog box. If left disabled, the VM root volume will linger for a day before it is deleted by the system. You may wish to expunge a volume if you are running low on space or volume quota.<br />
<br />
<br />
== Virtual machine networking ==<br />
The CloudStack platform allows you to define custom virtual private cloud (VPC) network which can contain any number of guest networks that your virtual machines connect to. Each guest network has its own private network address space and is not directly routable from campus or the internet. For virtual machines that require internet access, the VPC or guest network it is connected to must have a NAT IP address associated. The following diagram shows how a guest network connects to the internet and campus network.<br />
[[File:CloudStack Guest Networking.png|alt=CloudStack Guest Networking|none|thumb|CloudStack Guest Networking]]<br />
In order to expose a virtual machine's services to campus or the internet, the appropriate port forwardings must be set up on the VPC containing the guest network. More on this will be discussed in the next section.<br />
<br />
Having multiple guest networks allows for more advanced network setups but is not required. We recommend using a single flat network for most workloads. <br />
<br />
By default, all CloudStack accounts come with a default VPC and guest network set up with a NAT IP assigned.<br />
<br />
=== IP addresses ===<br />
Due to the design decisions made during the setup of the CloudStack platform, only internal 10.44.12X.X IPs can be assigned to your VPC. These IP addresses are accessible from the university campus network. However, there is a special section of IP addresses that can be accessed from the internet.<br />
{| class="wikitable"<br />
!IP address range<br />
!Accessible from<br />
!Internet IP mapping<br />
|-<br />
|10.44.120.3-128<br />
|Campus, Internet<br />
|10.44.120.X maps to 136.159.140.X (ports 80 and 443 only)<br />
|-<br />
|10.44.120.129-255<br />
|Campus only<br />
|N/A<br />
|-<br />
|10.44.121.0-255<br />
|Campus only<br />
|N/A<br />
|-<br />
|10.44.122.0-255<br />
|Campus only<br />
|N/A<br />
|-<br />
|10.44.123.0-255<br />
|Campus only<br />
|N/A<br />
|}<br />
If you need a service exposed to the internet, please request for a public IP address using our [https://ucalgary.service-now.com/it?id=sc_cat_item&sys_id=e3c1d6e91be48554cca5ecefbd4bcb6c ServiceNow request form]. Additionally, if your service is not port 80 or 443, you must also request for a firewall change request to allow the special port through. <br />
<br />
=== Exposing a network service to campus ===<br />
In order to make a virtual machine be visible to the campus network, you must first set up a port forwarding from a campus IP address to your virtual machine.<br />
<br />
To create a port forwarding, navigate to <code>Network -> VPC -> Select your VPC -> Public IP Addresses</code>. If you do not have any available IP addresses, you will need to click on 'Acquire New IP' and select an available IP address. Click on the IP address you wish to use to create a port forwarding on and then navigate to the 'Port Forwarding' tab. Enter the private port range, the public port range, the protocol, and select the target VM. <br />
<br />
For example, to port forward only HTTP (tcp/80) traffic, you would enter the following:<br />
[[File:CloudStack Port Forwarding.png|alt=CloudStack Port Forwarding|none|thumb|CloudStack Port Forwarding]]Once the port forwarding is created, you should be able to access the service from on campus. If for some reason access to your service does not work, there may be a firewall restriction on IT's network. In such circumstances, please contact us for assistance.<br />
<br />
=== Exposing a network to the internet ===<br />
Exposing a service to the internet is the same as exposing it to campus. However, you must create a port forwarding on an IP address that maps to an internet IP address outlined in the IP address table above. If your account does not have one of these IP addresses available, please request for one on the [https://ucalgary.service-now.com/it?id=sc_cat_item&sys_id=e3c1d6e91be48554cca5ecefbd4bcb6c ServiceNow request form].<br />
<br />
By default, only ports 80 and 443 are allowed through the Internet IP address. For all other ports, please [https://ucalgary.service-now.com/it?id=sc_cat_item&sys_id=47cd16d113153a00b5b4ff82e144b0bf create a firewall rule change request in ServiceNow].<br />
<br />
== Cloud-Init Automation ==<br />
<br />
=== Ubuntu ===<br />
The following Cloud-Init configs apply to Ubuntu VM templates.<br />
<br />
==== Ubuntu desktop ====<br />
Use the following Cloud-Init config with the Ubuntu Server template to set up an Ubuntu desktop environment. The setup step takes a up to 15 minutes to complete and you should see a login screen when the setup finishes. <br />
<br />
Adjust the root and user password as desired. Below, the test user password is set to blank, allowing you to login to Gnome without a password.<br />
{{Highlight|code=#cloud-config<br />
disable_root: false<br />
<br />
users:<br />
- name: user<br />
lock_passwd: false<br />
inactive: false<br />
gecos: Test User<br />
primary_group: user<br />
groups: wheel<br />
passwd: $1$ADUODeAy$eCJ1lPSxhSGmSvrmWxjLC1<br />
<br />
chpasswd:<br />
list: {{!}}<br />
root:password<br />
expire: false<br />
<br />
# Install a graphical desktop<br />
runcmd:<br />
- DEBIAN_FRONTEND=noninteractive apt -y upgrade<br />
- DEBIAN_FRONTEND=noninteractive apt -y install tasksel<br />
- tasksel install gnome-desktop<br />
- systemctl set-default graphical.target<br />
- systemctl isolate graphical.target}}<br />
<br />
=== Rocky Linux ===<br />
The following Cloud-Init configs apply to Rocky Linux templates.<br />
<br />
==== Rocky Linux Desktop ====<br />
Use the following Cloud-Init config using the Rocky Linux template to set up a Rocky Linux desktop environment. The setup step takes up to 10 minutes to complete and you should see a login screen when the setup finishes. <br />
<br />
Adjust the root and user password as desired. Below, the test user password is set to blank, allowing you to login to Gnome without a password.<br />
{{Highlight|code=#cloud-config<br />
disable_root: false<br />
<br />
users:<br />
- name: user<br />
lock_passwd: false<br />
inactive: false<br />
gecos: Test User<br />
primary_group: user<br />
groups: wheel<br />
passwd: $1$ADUODeAy$eCJ1lPSxhSGmSvrmWxjLC1<br />
<br />
chpasswd:<br />
list: {{!}}<br />
root:password<br />
expire: false<br />
<br />
# Install a graphical desktop<br />
runcmd:<br />
- yum -y install "@Workstation"<br />
- systemctl set-default graphical.target<br />
- systemctl isolate graphical.target}}<br />
<br />
==== Rocky Linux Docker host ====<br />
Use the following Cloud-Init config using the Rocky Linux template to set up a new docker host. This server can then be used to run Docker containers. Also included are:<br />
<br />
# The docker-compose utility to help deploy container stacks more easily<br />
# A helper script to expand the <code>/var</code> and <code>/</code> filesystems on first startup based on the available space in the ROOT volume. <br />
<br />
Use the CloudStack generated password with the '<code>rocky</code>' default user account to log in.{{Highlight|code=#cloud-config<br />
<br />
write_files:<br />
- path: /usr/bin/expand_lvm_root<br />
content: {{!}}<br />
#!/bin/bash<br />
/usr/bin/growpart /dev/vda 3<br />
/usr/sbin/pvresize -y -q /dev/vda3<br />
/usr/sbin/lvresize -y -q -r -l +50%FREE /dev/mapper/*root<br />
/usr/sbin/lvresize -y -q -r -l +100%FREE /dev/mapper/*var/dev/mapper/*root<br />
/usr/sbin/xfs_growfs /<br />
/usr/sbin/xfs_growfs /var<br />
permissions: '0700'<br />
<br />
- path: /usr/bin/setup_docker<br />
content: {{!}}<br />
#!/bin/bash<br />
yum install -y yum-utils<br />
yum-config-manager \<br />
--add-repo \<br />
https://download.docker.com/linux/centos/docker-ce.repo<br />
yum install -y docker-ce docker-ce-cli containerd.io<br />
systemctl start docker<br />
systemctl enable docker<br />
<br />
curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose<br />
chmod +x /usr/local/bin/docker-compose<br />
permissions: '0700'<br />
<br />
- path: /root/docker-compose.yml<br />
permissions: '0700'<br />
content: {{!}}<br />
version: '3.3'<br />
services:<br />
web:<br />
image: php:7.4-apache<br />
restart: always<br />
user: "0:0"<br />
volumes:<br />
- /var/www/html:/var/www/html<br />
ports:<br />
- "80:80"<br />
<br />
- path: /var/www/html/index.php<br />
permissions: '0644'<br />
content: {{!}}<br />
<h1>Hello there!</h1><br />
<p>I see you from <?php echo $_SERVER['REMOTE_ADDR']; ?></p><br />
<<nowiki>pre</nowiki>><?php print_r($_SERVER); ?></pre><br />
<br />
# Ensure VM has the largest / possible<br />
runcmd:<br />
- /usr/bin/expand_lvm_root<br />
- /usr/bin/setup_docker<br />
- cd /root; docker-compose up -d}}<br />
<br />
==== UC Authentication ====<br />
Use the following Cloud-Init config to allow UC-based authentication. Local accounts with the same username as the IT account may use the IT credential to log in.<br />
{{Highlight|code=#cloud-config<br />
<br />
write_files:<br />
- path: /usr/bin/setup_uc_auth<br />
permissions: '0700'<br />
content: {{!}}<br />
#!/bin/bash<br />
yum install -y sssd sssd-dbus sssd-krb5 krb5-workstation authselect-compat<br />
<br />
cat <<EOF > /etc/sssd/sssd.conf<br />
[sssd]<br />
config_file_version = 2<br />
services = nss, pam, ifp<br />
domains = uc.ucalgary.ca<br />
<br />
[domain/uc.ucalgary.ca]<br />
id_provider = files<br />
debug_level = 5<br />
auth_provider = krb5<br />
chpass_provider = krb5<br />
<br />
krb5_realm = UC.UCALGARY.CA<br />
krb5_server = ITSODCSRV14.UC.UCALGARY.CA:88<br />
krb5_validate = false<br />
EOF<br />
chmod 600 /etc/sssd/sssd.conf<br />
<br />
cat <<EOF > /etc/krb5.conf<br />
[logging]<br />
default = FILE:/var/log/krb5libs.log<br />
kdc = FILE:/var/log/krb5kdc.log<br />
admin_server = FILE:/var/log/kadmind.log<br />
<br />
[libdefaults]<br />
default_realm = UC.UCALGARY.CA<br />
dns_lookup_realm = false<br />
dns_lookup_kdc = false<br />
ticket_lifetime = 24h<br />
renew_lifetime = 7d<br />
forwardable = true<br />
<br />
[realms]<br />
UC.UCALGARY.CA = {<br />
kdc = itsodcsrv14.uc.ucalgary.ca<br />
}<br />
<br />
[domain_realm]<br />
uc.ucalgary.ca = UC.UCALGARY.CA<br />
.uc.ucalgary.ca = UC.UCALGARY.CA<br />
EOF<br />
<br />
mkdir -p /etc/authselect/custom/rcs<br />
cd /etc/authselect/custom/rcs<br />
echo H4sIAMYsbGIAA+0ca3PbuDGf+StQ34dLMpEtybLdUevO6BKl56mTuJGv7Uwmo4FISMKJBHQAaEfNub+9uwBJkXrLshxnjjuTyCKJfWF3sQsuFPhS9CtB79keoQpw2mjYT4DZz1r1uPGs1qhX6yf12vFx/Vm1Vqs1jp+R6j6ZSiHWhipCnikpzarn1t3/TuErj8bhhBzccjOs6Igq41MVHBDen71WUey3mCsWHNx5W4wKpT+qSAGjI3lDQxj8SarB0UDIiB2FcsBFRfuKMfHZY4L2QpYbS2MzZMJwnxouxfnXefQHTaNi9nufhprdpQj6XAyYGisuzHIUuYcWIhlTrW/lRkxMNdO0OH5HdHdeXlDNjAGCuhJQ0IM4GjPFx0Om4PGjDM9n8pULP4wDtpkik78q1Lec/WifcMr8kSyE7fB/a9ssYf8Q2PiPc6/3RmNN/K826mez8f/07LSM/48BR4uD8dGaWOytGbc8BK8buSTu5odtFEyPitGxDH2LYXai9kFjjf8fn4HPT/3/GPy/fnZ6Uvr/Y8BXiP/gTHHOL/KZ0Z2HRpGmD2meszi5mIExjbpM3BxquRuOgIkJICE/kE7qsKQYGwjXRbSrfD2Xxe7EVp/yEOMGsjZWzOLSPASuCHJ83iCxwPtdwyN2XqtXqwvYSpHMMKPjfp/7HHFtyIydrWBnXeeFQkT4fak4ZImulwh134n3POr7Mk50sTUWGMy0RoE2hnmRAJHDgzLtwk1ewffmJq/gXZiJBf9iTSaHZGvTA0ZoGGumdsakY1AxC7q8j9qJeUD+SmqwPhAQi5kC7k8B69M4NOc9GhA7TutzOSLIB4g1EvJWnHPIFhT77HBbE9hNXZBmRNxYi0yTlHvhyexaA9cYvhDkGCMZDTfGMmITLiw7wMGNHLECuq2ZCjnIZnVUyeHZmis90YZFNhDtJJ0Mgl9lrxuNhpDwBVxZi4ioHp1Xq2dni0bNu0k2GPwkz8yn1GBqJDWj1FSWCFUwTDCxG+4DHUF8JUXgzBNNrws2u9sspB65k+4SW99s/RdgyNz4w0OsA/eUY6yr/05rtdn8r35S1n+PAjaSBU1rPGA4pA8pjCaJG6NbsS8zbuWDxmTk6jTMogZKxmOHYDsMdiAgEMxkOKYIcp69DEE60CUaMsLQ3twSRzbQBgnr2XprRtKBiCMOJFMWRaoJXcAxn5rCiG9XcRbK7T3RWOP/pyen+fqvgf5/XC3rv0eBb17eYTIbsJDaGs/+cV53dlGWjfsuG+O6zWh8qP43hYW1UQUQ7aqYHC9CYh4P6fx9eanU+3SGn6xiqBGX7CU5X656mKZ/i6uRv52T+rQa2QV3sWbaaQZdwkjwrZSQxqlBxqYiYrCQ0UHTfd4RoybdPlfadDHiz0+V5mYjI1ivmm4i9W6CuSSW9KW6BfdbwnS5tVFubSxmJvUL0oM4wkRXD2kgb70fcnojt0CKUCB3S5IrYG3EZ8pQqO1sZqkJ+JLig6H5C/CgMUaPmQjgsiA6jFW0247Hg0yg5WNXfVkkXdDR2GzKzmacPLH9pPr3sZ20TTQe3/4W05Abm+0UwzyxyuyiGLorRTgpULr3YgPOdFKrE+dT2y49dqMEg6eRo93YSZaIZfjKfbkcmnJf7jvbl3ssGEtt7Gv4PdJYt/93Um/M1P81iC9l/f8YsKFFWydAP7eeZuvVTX13uaf+6X9kEEQv8991/DJZnwvYswJrLhTYGEmtDSNqWKlNNCa5Jj3LayV55KDpvv+uh/IWc00W3G3q1DOUkqIdatVxQGG1nmJ8Yh6+Gj62W2/etfdLY43/n1VPT3P+XwP/rzaOy/3/R4G27boinU7njS1/XKIwu1P2nIauGHcZpU2MsQgKJy+8813B8zosZD62dBEz5JqMlcStc1egsRyDVMMD4GoyVhAuZJ/wAJk0E4+Kud09wHID95U+BAI4OrmgCYWIY3C46x8DpJJEVNABSwtBg8lbJMGtIU0B1qTiTC8iEjF/SAXXkca6YYgcXr5pXb0i/2Cqx5QEJSnyVjF2cdU6JBcmYwKQkfedDkGcV613kLUYpvoUxDISdzycoMlkIF0yDuPBwOqiR32oaYPktgfM+lIIYNMKAnGSj0PkvN9nCmNUWqY4vaE+3uHuFBcwoZGTg/Ygc3dK9oGzHoPJjoEE7pIaOyO/Iv4x6KjpDY0Z6+bREXyLFTvk8ggHHkF+E3jeT5N0BXhVMBaK+2EQ6CHr6isZOVqKgpgKZRXJw+6dCYRvvO+BMbj5D8hziK/sxj08yU8DIzecZlnai0NyjTY0pDcMlQZKRSGFz7weE6zPQQ2QkbJUVn/InFE5wtgNhSqDJLIPsR4+KxEYgpq4R+1UmGGsPcgeY9AlEXEE82yNiesRrB9MWY2iln92HL9CcSYyJoG0hdItFXamRoyNEzXEQqD1g7bhOQXG6A+5YK/sKJgPzz5a8A1tXQb0ghz9GuPWBNdTT0mWVFQGw50LNAox8PAlL4dJc5Nu/Usb/B+KphEIr9DYwwkBtvG2GTpsCWpHLDep1iOSjRKLrIfEaDLD+Ql1bhROUl98/eH924u///KxdX3x4b1XmQPPa8EMOyFBb6hoiUpDH0MEP9qpmspySK5CRjWozNJ3j3iB9OMIptOJazDzYATsiDiXscPBDwx6P9TQwB75ADGtIGIqF72Bxd0qGFFNnRPmFm2Y4dsEIQn7wrWNZM6+CjyCUP9qXVy2frpskw9XKHnrkrxtt65/+djuLFBCThuFzbFm08OEJAndqXfjHSSM9Q9oAk3SSBvYJsiHZn5s+A2zQ2eiGCIGTYDN4k2vWKXNUMP3pTDKhyKLudFACB/Nh0q7VKSLBOqHK4vEbgMQW28cJkJlr1jm6OQ5tLaYPYoeC1Y3cLaQzj2aCITPITUWUfY0RObxWCqYZXQTnEpczNiXcch9bq09CTFIhAs7eKGNzXI82zecSHAJV4lraia3GNlojhf7zgmeZsE8urQsTfBs8t7qkLyXRNogGjEKSxlMxkzjNKJ67opwNI90i+RF5rGJ+Ck/ufbP9VOSe9h6no3Pbmpe5hoTX6bIkxc0KxFjOIdnwO3FIGRFfHB9Fhe+7Cniq0NUgqUUTPG+qItVQ4L9jQvfEIEx9OfrAdC91pg+BLFCHRcNPA5kgqFlN5vxAkYg8I8sraFpUmNzMHxAxXYhBOsJAp762REz/lHSYpBTg8taEiKvhwzsL9laRgNOuUoDBeoEvPS/eaOe7tsVhQXixF0HeRWNmMEZltNNEc97c9GxEa1z1X598fbiNaY1/764fv0zedO6bv3U6qyObdMY9x4XagjCr0jalAQJmqE9cG1tM7bexAbedAkEJ1K3ihu4C84XJw+BA2PgqRQam9w62Jeof7vQ2jJP21wHNO47G0yWGyO95P2dRbuINK57OH14TQqWECbzhD0uYJWgmXMVmmdA1ReO0EHSTZMJvIDoDArXMjPF4DphtkCQtd1McWQNNVugmXbeTPFMm2q2QJR130zxZH0126BJGnByWNyVdUja/2m9u7rMGavnvST58sitQeibOP1zy4vnkbwVpR+QFJMsDpBixF9OIV2ykJJdaNkmayzxCkvsOo6yJX7KIKRn7TZpXXY+TNXwEnMIO8ya9PM/v/iudjhKWAUf2//85eJj+137/XVnXzTWnv85qc7s/+C3cv/nMeAdHWGfinKJc6F4xDwzq5Jc7ZemiqTDkpBVLLEwJEUzewuuqNgPrOjYOrjzKjPhFgVK6+ZXENRSyfWKRDutDXguAjbX0CWkYpeYA8ySsJ/Avp8k5+RaxewAUX2CO5/hGXc08aHk3ZuaFzVqFA6JJZrOVwNLdJ0UBOttbEPKjynztNMtkTe5sExW61E2yaCRLTsiqBxCKzOmhZDrzr+CXEUV7cot+Ji32k2z6S7GiE3w3brbRLUkYZDfH+AmwNp3lKup7gvW9e8VtYwXH0rT6yh/A01nlB9P2/kX8ZXZlHCJphc1AaTavg9dQuymJtN2oxp3nC3yQoAoRoSHkdfGaLuD55vsJUNC+zClvQc9F+li7mEehuwausvW/+IB//3kGOvyv2L//wn2/zfqtTL/ewy45+//zJ8azyciT+JQwZPpsk/61GynaTfiGleOrqBQza/+LaHkr2kG+YBHD8oO6D9IB/RTbsZ9ooe7n37751Zsfdv2z7L/c8ruU+z/dJaxz9Of6/K/WrXWaMzkf7Wzk7My/3sMuF/+9yQSvH2cGt1n4vhA8U1IyLzdC57mIIiaX+DfCD9dy4GmN0w17Q+lVfJXRrkvxdB4n3OvmSwBvv4snnpMhQt4IlkuCZ7LaZOx9m/cNIZQazttyL2O4+7ww0kbmQhZyNbyH8wqT+R+hydyN1LQol+DWFYCZ9zU98jNyjcDy1131u9S1la4dM6d8WjX1JXvw0154jmjUZ54XoulrPeXMlOeeC5PPJcnnssTz+WJ52+95VXueGXsPsUdrxJKKKGEEkoooYQSSiihhBJKKKGEEkr448D/AT019aAAeAAA {{!}} base64 -d {{!}} tar -xzpf -<br />
<br />
if [[ "`authselect current -r`" != "custom/rcs" ]] ; then<br />
authselect select custom/rcs --force<br />
systemctl restart sssd<br />
fi<br />
<br />
systemctl enable sssd<br />
<br />
runcmd:<br />
- /usr/bin/expand_lvm_root}}<br />
<br />
== Infrastructure tools ==<br />
<br />
=== Generating a CloudStack API ===<br />
You can request for a CloudStack API key to automate infrastructure deployment using Terraform or CloudMonkey. A new API key can be generated by navigating to your profile page (top right) and then clicking on the 'Generate keys' button.<br />
[[File:CloudStack API Key.png|alt=CloudStack API Key|none|thumb|CloudStack API Key]]<br />
<br />
=== CloudMonkey ===<br />
CloudMonkey is a utility that makes it easier to interact with the CloudStack API. This tool may be used to help automate VM actions (such as start/stop/reboot), or infrastructure tasks (such as creating/destroying VMs, networks, or firewall rules). <br />
<br />
To get started with CloudMonkey, refer to the following resources:<br />
<br />
* Download from: <nowiki>https://github.com/apache/cloudstack-cloudmonkey/releases/tag/6.1.0</nowiki><br />
* Documentation at: <nowiki>https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+cloudmonkey+CLI</nowiki><br />
<br />
=== Terraform Integration ===<br />
Terraform allows you to define infrastructure as code and can be used in conjunction with CloudStack to configure your virtual machines and guest networks. Use the official CloudStack provider.<br />
<br />
The following is an example Terraform file for reference. Specify your CloudStack API keys either as a separate <code>vars.tf</code>.<br />
{{Highlight|code=# Configure the CloudStack Provider<br />
terraform {<br />
required_providers {<br />
cloudstack = {<br />
source = "cloudstack/cloudstack"<br />
version = "0.4.0"<br />
}<br />
}<br />
}<br />
<br />
provider "cloudstack" {<br />
api_url = "${var.cloudstack_api_url}"<br />
api_key = "${var.cloudstack_api_key}"<br />
secret_key = "${var.cloudstack_secret_key}"<br />
}<br />
<br />
# Create a new VPC<br />
resource "cloudstack_vpc" "default" {<br />
name = "wedgenet-vpc"<br />
display_text = "wedgenet-vpc"<br />
cidr = "100.64.0.0/20"<br />
vpc_offering = "Default VPC offering"<br />
zone = "zone1"<br />
}<br />
<br />
# Create a new ACL<br />
resource "cloudstack_network_acl" "default" {<br />
name = "vpc-acl"<br />
vpc_id = "${cloudstack_vpc.default.id}"<br />
}<br />
<br />
# One ingress and one egress rule for the ACL<br />
resource "cloudstack_network_acl_rule" "ingress" {<br />
acl_id = "${cloudstack_network_acl.default.id}"<br />
<br />
rule {<br />
action = "allow"<br />
cidr_list = ["0.0.0.0/0"]<br />
protocol = "tcp"<br />
ports = ["22", "80", "443"]<br />
traffic_type = "ingress"<br />
}<br />
}<br />
<br />
resource "cloudstack_network_acl_rule" "egress" {<br />
acl_id = "${cloudstack_network_acl.default.id}"<br />
<br />
rule {<br />
action = "allow"<br />
cidr_list = ["0.0.0.0/0"]<br />
protocol = "all"<br />
traffic_type = "egress"<br />
}<br />
}<br />
<br />
<br />
# Create a new network in the VPC<br />
resource "cloudstack_network" "primary" {<br />
name = "primary"<br />
display_text = "primary"<br />
cidr = "100.64.1.0/24"<br />
network_offering = "DefaultIsolatedNetworkOfferingForVpcNetworks"<br />
acl_id = "${cloudstack_network_acl.default.id}"<br />
vpc_id = "${cloudstack_vpc.default.id}"<br />
zone = "zone1"<br />
}<br />
<br />
# Create a new public IP address for this network<br />
resource "cloudstack_ipaddress" "public_ip" {<br />
vpc_id = "${cloudstack_vpc.default.id}"<br />
network_id = "${cloudstack_network.wosnet1.id}"<br />
}<br />
<br />
# Create VMs. <br />
resource "cloudstack_instance" "vm" {<br />
count = 1<br />
name = "vm${count.index+1}"<br />
zone = "zone1"<br />
service_offering = "rcs.c4"<br />
template = "RockyLinux 8.5"<br />
network_id = "${cloudstack_network.wosnet1.id}"<br />
<br />
# Cloud Init data can be used to configure your VM on first startup if your template supports Cloud Init<br />
user_data = <<EOF<br />
#cloud-config<br />
<br />
# Require specific packages<br />
packages:<br />
- tmux<br />
- git<br />
- tcpdump<br />
<br />
EOF<br />
}<br />
<br />
<br />
}}<br />
<br />
= Troubleshooting =<br />
<br />
=== Cannot create a volume snapshot ===<br />
Volume snapshots can only be taken on VMs that are powered off.<br />
<br />
=== Cannot create a VM snapshot ===<br />
Disk-only VM snapshots cannot be taken when the VM is running. If you intend to snapshot a running system, you must also snapshot its memory.<br />
<br />
=== VM state is still running after shutdown ===<br />
After running 'shutdown' on a VM, the VM state reported by CloudStack is still running. <br />
<br />
Please try to do a force shutdown from the CloudStack management console. The VM state isn't updated by CloudStack and as a result, the state of a VM isn't properly reflected when power state changes outside of CloudStack (likely a bug?)</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=CloudStack_User_Guide&diff=1852CloudStack User Guide2022-06-02T17:25:17Z<p>Darcy: </p>
<hr />
<div>This is a user's guide on using CloudStack provided by Research Computing Services.<br />
<br />
== Introduction==<br />
Apache CloudStack is an Infrastructure as a Service (IaaS) platform that allows users to quickly spin up Linux/Non-Windows based virtual machines. RCS is providing this service to help researchers quickly set up and prototype short-term research related software on premises. CloudStack is not appropriate for workloads that depend on Windows. Services set up on CloudStack virtual machines can be accessed from the campus network and also the internet if required.<br />
<br />
Access to CloudStack can be requested via [https://ucalgary.service-now.com/it?id=sc_cat_item&sys_id=e3c1d6e91be48554cca5ecefbd4bcb6c ServiceNow].<br />
<br />
Please refer to our [[CloudStack End User Agreement]] for acceptable uses and requirements.<br />
<br />
== Managing your virtual machine ==<br />
You will be able to run whatever virtual machine you wish (with the exception of Windows). Clearly we cannot provide specific management advice on each and every operating system available. We can provide you with some suggestions on important considerations to be aware of.<br />
<br />
=== Educate yourself ===<br />
All operating systems (OS) have user groups, web sites, wikis, or mailing lists somewhere on the internet. They can be a valuable resource. Most OS providers have on-line documentation that describes using their product. For example Rocky Linux, used by RCS, has a [https://docs.rockylinux.org/ documentation site]. These are excellent resources and can help you understand how to manage your virtual machine.<br />
<br />
=== Configure your VM's OS ===<br />
It is critical to ensure that the only services running on your VM are the ones you must run. Each OS has a way of managing what services are running (sysinit, systemd etc). Please ensure that unnecessary services have been disabled.<br />
<br />
Many OSs will have pre-configured accounts, and many applications will have pre-configured accounts. Make sure they are either disabled or not allowed to login.<br />
<br />
All un-used accounts should be disabled or preferably deleted.<br />
<br />
All accounts should have strong [https://it.ucalgary.ca/it-security/passwords-do-i-have-change-them/ passwords].<br />
<br />
Many OS's have the ability to automatically update themselves. If possible please consider doing this.<br />
<br />
Updates can also be configured to skip certain software if it will interfere with your research, but please be advised that doing so could place your system at risk.<br />
<br />
=== Exposed to the Internet ===<br />
Not everyone is a computer security expert. If your VM must be exposed to the internet, please consider using Trend Micro Cloud One Workload Security from IT security to enhance your security posture.<br />
<br />
== Accessing CloudStack ==<br />
===Accessing the CloudStack management console===<br />
<br />
The CloudStack management console is a web-based portal that allows you to view and manage your cloud infrastructure including virtual machines, storage, and network. Any modern web browsers including Chrome, Firefox, Edge, and Safari is supported. <br />
<br />
Access the CloudStack management console is possible only from an IT-managed computer or through the IT General VPN when working on unmanaged machines (eg. AirUC) or when working off campus (eg. at home). Please review the IT [https://ucalgary.service-now.com/it?id=kb_article&sys_id=52a169d6dbe5bc506ad32637059619cd knowledge base article on connecting to the General VPN] or contact IT support if you need assistance connecting to the General VPN. <br />
[[File:CloudStack VPN Connection.png|alt=CloudStack VPN Connection|none|thumb|CloudStack VPN Connection]]<br />
=== Login to CloudStack===<br />
<br />
To log in to CloudStack, navigate to https://cloudstack.rcs.ucalgary.ca/. If this site fails to load, please make sure you are either on a IT managed computer or connected to the General VPN.<br />
<br />
Sign in to CloudStack using the Single Sign-On option as shown in the image below. This method will require you to authenticate through our central authentication service using your University of Calgary IT credentials and will require multi-factor authentication. You must have multi-factor authentication set up either via your phone or with the Microsoft Authenticator app.<br />
[[File:CloudStack Login Page.png|alt=CloudStack Login Page|none|thumb|CloudStack Login Page]]<br />
<br />
'''Note:''' Due to a bug with the UI, if the Single Sign-On option is disabled, please refresh the login page and try again. This issue should be addressed in our next update for CloudStack.<br />
<br />
=== CloudStack Dashboard===<br />
<br />
After logging in, you will be presented with your CloudStack management console. The dashboard shows you a general overview of your account's status.<br />
[[File:CloudStack Dashboard.png|alt=CloudStack Dashboard|none|thumb|CloudStack Dashboard]]On the right hand side of the dashboard, you will also see recent activity and events that was done within your CloudStack account.<br />
<br />
If you wish to see your CloudStack account resource quota and allocation, navigate to: <code>Accounts -> Click on your account -> Resources</code>. <br />
[[File:CloudStack Resource Quota.png|alt=CloudStack Resource Quota|none|thumb|CloudStack Resource Quota]]<br />
<br />
== Working with virtual machines==<br />
<br />
CloudStack allows you to control the lifecycle of virtual machines within your cloud account. VMs may be started, stopped, rebooted, or destroyed within your management console.<br />
<br />
===Create a VM===<br />
<br />
To create a new VM, enter the CloudStack management console and navigate to: <code>Compute -> Instances -> Add Instance</code>[[File:CloudStack Instance Summary.png|alt=CloudStack Instance Summary|thumb|CloudStack Instance Summary|493x493px]]<br />
<br />
Virtual Machines require the following details:<br />
<br />
# '''Deployment zone'''. Your account will already be placed in the appropriate zone.<br />
# '''Boot template or ISO'''. You may choose either a pre-created template or boot from a custom CD-ROM ISO file.<br />
# '''Compute offering'''. You may select an appropriate size for your new VM. Resources will be counted against your account's quota.<br />
# '''Data Disk'''. You may choose to add an additional virtual disk to your VM to store your data. Alternatively, if you wish to use a single virtual disk for your VM, you may choose to override the size of your root disk in step 2 and select 'No thanks' in this step.<br />
# '''Networks'''. You may choose one or more networks your VM should connect to. All CloudStack accounts come with a default network already created and ready to be used.<br />
# '''SSH keypairs'''. For templates that support custom SSH key pairs, you may choose to use a custom SSH keypair to be installed as part of the deployment process.<br />
# '''Advanced settings'''. For templates that support custom user-data (Cloud-Init), you may choose to enable the advanced settings and provide your own Cloud-Init user-data payload. More on this in the advanced tasks section below.<br />
# '''Other VM details'''. You may give your new VM a friendly name and make it part of a group. Groups allow you to group related VMs together for better organization. You may change these details at a later time.<br />
<br />
When you are done, review the instance summary on the right hand side and then click on the 'Launch Virtual Machine' button.<br />
<br />
=== Selecting your VM Operating system ===<br />
Many OSs will provide various editions that are tailored to a specific use case. A desktop VM may not be appropriate when you need to run a database server. The OS provider will have guides on how to choose an edition.<br />
<br />
You may choose to install the operating system to your virtual machine using either pre-built templates or from scratch using an ISO image.<br />
<br />
====Install from a virtual machine template====<br />
<br />
We provide a Rocky Linux 8.5 and a Ubuntu Server 22.04 LTS template for your convenience. These templates are pre-built images with the operating system installed and ready for use. Our templates also support further automated setup configured using Cloud-Init configuration data that can be provided when deploying a new VM. Currently, we offer the following templates: <br />
{| class="wikitable"<br />
!Template<br />
!Cloud-Init Support<br />
!Password Support<br />
!Default Username<br />
|-<br />
|Rocky Linux 8.5<br />
|Yes<br />
|Yes<br />
|rocky<br />
|-<br />
|Ubuntu Server 22.04<br />
|Yes<br />
|Yes<br />
|ubuntu<br />
|}<br />
Rocky Linux is an open source Linux distribution that is binary-compatible with Red Hat Enterprise Linux and is what RCS recommends.<br />
<br />
For templates that support passwords, the generated password that appears after a VM is created is applied to the default username.<br />
<br />
Security note: All VM templates are configured with SSH password authentication enabled. You should be able to SSH to your VM from another system connected to the same guest network. Do not expose port 22 unless required and we highly recommend using key based authentication.<br />
<br />
===== Virtual machine credentials =====<br />
VM templates that support password will have a randomly generated password set when the VM is first created or when a password reset request is made (available only when the VM is powered off). A randomly generated 6 character password will be displayed when a new password is set and appears as a notification in your CloudStack management console. <br />
[[File:CloudStack VM Password.png|alt=CloudStack VM Password|none|thumb|CloudStack VM Password]]<br />
This password is set on the default username for your template. For example, the Rocky Linux VM template will set this password to the ''''rocky'''<nowiki/>' user account. You may become the super user by logging in as the <code>rocky</code> user and then running <code>sudo su</code>.<br />
<br />
Note: If you specify a custom Cloud-Init config that creates additional users or sets account passwords, the displayed password will be overridden and have no effect.<br />
<br />
==== Install from an ISO image====<br />
We provide various ISO images for popular Linux distributions. You may select one of these ISO images instead of using a pre-built template when deploying a new virtual machine. We currently provide:<br />
{| class="wikitable"<br />
!Distribution<br />
!ISO<br />
|-<br />
|Ubuntu 20.04<br />
|ubuntu-20.04.4-desktop-amd64.iso<br />
<br />
ubuntu-20.04.4-live-server-amd64.iso<br />
|-<br />
|Ubuntu 21.10<br />
|ubuntu-21.10-desktop-amd64.iso<br />
<br />
ubuntu-21.10-live-server-amd64.iso<br />
|-<br />
|Ubuntu 22.04<br />
|ubuntu-22.04-live-server-amd64.iso<br />
|-<br />
|Rocky Linux 8.5<br />
|Rocky-8.5-x86_64-minimal.iso<br />
|-<br />
|Fedora 35<br />
|Fedora-Workstation-Live-x86_64-35-1.2.iso<br />
|}<br />
You may install custom ISO file into your CloudStack account either by directly uploading the ISO through the web console or by providing a URL to the ISO file on the internet.<br />
<br />
Please do not install Windows on our CloudStack infrastructure. It is against our user agreement to run Windows based systems in this infrastructure. If you need a Windows VM, please contact us for alternative solutions.<br />
<br />
===== Register a ISO with a URL=====<br />
[[File:CloudStack Download ISO.png|alt=CloudStack Download ISO|thumb|CloudStack Download ISO|190x190px]]<br />
<br />
To add a custom ISO file from the internet, enter the CloudStack management console and navigate to: <code>Images -> ISOs -> Register ISO</code><br />
<br />
You may check the state of the ISO file by clicking on it and verify the state of the file. If the file is successfully downloaded, its ready state should become ‘true’. The ISO file will only appear in the selection list when the file is downloaded successfully.<br />
[[File:CloudStack ISO Ready.png|alt=CloudStack ISO Ready|none|thumb|172x172px|CloudStack ISO Ready]]<br />
<br />
=====Upload a custom ISO=====<br />
<br />
To upload an ISO file, enter the CloudStack management console and navigate to: <code>Images -> ISOs -> Upload ISO from Local (icon)</code><br />
<br />
[[File:CloudStack Upload ISO.png|alt=CloudStack Upload ISO|none|thumb|CloudStack Upload ISO|217x217px]]<br />
<br />
===Connecting to your VM console===<br />
The CloudStack management console has a KVM (keyboard, video, mouse) feature built-in, allowing you to remotely connect to and interact with your virtual machine. To connect to your virtual machine's console, navigate to: <code>Compute -> Instances -> Your Instance -> View console</code>.<br />
[[File:CloudStack View Console.png|alt=CloudStack View Console|none|thumb|CloudStack View Console]]<br />
<br />
=== Expanding a VM disk ===<br />
[[File:CloudStack Expand Volume.png|alt=CloudStack Expand Volume|thumb|CloudStack Expand Volume]]<br />
Virtual machine disks can be expanded after they are created within CloudStack. However, you will need to expand the partitions and filesystems manually.<br />
<br />
To grow an existing disk:<br />
<br />
# Go into your VM details page and click on ‘Volumes’.<br />
# Click on the volume you wish to expand.<br />
# Click on the ‘Resize Volume’ icon in the top right.<br />
Once the volume has been expanded, you should be able to verify the disk volume has grown with <code>lsblk</code>. There should also be some messages by the kernel when this occurs. However, you will still need to expand any partitions, volumes, and filesystems on your system manually.<br />
<br />
To expand your partition, use the <code>growpart</code> command followed by your disk device and partition number. Eg: <code>/usr/bin/growpart /dev/vda 3</code><br />
<br />
For LVM volume sets, you can expand the volume using the <code>pvresize</code> and <code>lvresize</code> commands:<br />
<br />
* <code>/usr/sbin/pvresize -y -q /dev/vda3</code><br />
* <code>/usr/sbin/lvresize -y -q -r -l +100%FREE /dev/mapper/<volume-name></code><br />
<br />
To expand your filesystem:<br />
<br />
* XFS: <code>/usr/sbin/xfs_growfs <volume></code><br />
* EXT: <code>resize2fs <volume></code><br />
<br />
=== Destroying a VM ===<br />
If you need to delete a VM, click on the red garbage bin icon in the VM instance page. All deletions are irreversible, so please make sure you have a copy of any data you need before proceeding.<br />
[[File:CloudStack Delete VM.png|alt=CloudStack Delete VM|none|thumb|CloudStack Delete VM]]<br />
The VM root volume can be deleted immediately by enabling the 'Expunge' option in the dialog box. If left disabled, the VM root volume will linger for a day before it is deleted by the system. You may wish to expunge a volume if you are running low on space or volume quota.<br />
<br />
<br />
== Virtual machine networking ==<br />
The CloudStack platform allows you to define custom virtual private cloud (VPC) network which can contain any number of guest networks that your virtual machines connect to. Each guest network has its own private network address space and is not directly routable from campus or the internet. For virtual machines that require internet access, the VPC or guest network it is connected to must have a NAT IP address associated. The following diagram shows how a guest network connects to the internet and campus network.<br />
[[File:CloudStack Guest Networking.png|alt=CloudStack Guest Networking|none|thumb|CloudStack Guest Networking]]<br />
In order to expose a virtual machine's services to campus or the internet, the appropriate port forwardings must be set up on the VPC containing the guest network. More on this will be discussed in the next section.<br />
<br />
Having multiple guest networks allows for more advanced network setups but is not required. We recommend using a single flat network for most workloads. <br />
<br />
By default, all CloudStack accounts come with a default VPC and guest network set up with a NAT IP assigned.<br />
<br />
=== IP addresses ===<br />
Due to the design decisions made during the setup of the CloudStack platform, only internal 10.44.12X.X IPs can be assigned to your VPC. These IP addresses are accessible from the university campus network. However, there is a special section of IP addresses that can be accessed from the internet.<br />
{| class="wikitable"<br />
!IP address range<br />
!Accessible from<br />
!Internet IP mapping<br />
|-<br />
|10.44.120.3-128<br />
|Campus, Internet<br />
|10.44.120.X maps to 136.159.140.X (ports 80 and 443 only)<br />
|-<br />
|10.44.120.129-255<br />
|Campus only<br />
|N/A<br />
|-<br />
|10.44.121.0-255<br />
|Campus only<br />
|N/A<br />
|-<br />
|10.44.122.0-255<br />
|Campus only<br />
|N/A<br />
|-<br />
|10.44.123.0-255<br />
|Campus only<br />
|N/A<br />
|}<br />
If you need a service exposed to the internet, please request for a public IP address using our [https://ucalgary.service-now.com/it?id=sc_cat_item&sys_id=e3c1d6e91be48554cca5ecefbd4bcb6c ServiceNow request form]. Additionally, if your service is not port 80 or 443, you must also request for a firewall change request to allow the special port through. <br />
<br />
=== Exposing a network service to campus ===<br />
In order to make a virtual machine be visible to the campus network, you must first set up a port forwarding from a campus IP address to your virtual machine.<br />
<br />
To create a port forwarding, navigate to <code>Network -> VPC -> Select your VPC -> Public IP Addresses</code>. If you do not have any available IP addresses, you will need to click on 'Acquire New IP' and select an available IP address. Click on the IP address you wish to use to create a port forwarding on and then navigate to the 'Port Forwarding' tab. Enter the private port range, the public port range, the protocol, and select the target VM. <br />
<br />
For example, to port forward only HTTP (tcp/80) traffic, you would enter the following:<br />
[[File:CloudStack Port Forwarding.png|alt=CloudStack Port Forwarding|none|thumb|CloudStack Port Forwarding]]Once the port forwarding is created, you should be able to access the service from on campus. If for some reason access to your service does not work, there may be a firewall restriction on IT's network. In such circumstances, please contact us for assistance.<br />
<br />
=== Exposing a network to the internet ===<br />
Exposing a service to the internet is the same as exposing it to campus. However, you must create a port forwarding on an IP address that maps to an internet IP address outlined in the IP address table above. If your account does not have one of these IP addresses available, please request for one on the [https://ucalgary.service-now.com/it?id=sc_cat_item&sys_id=e3c1d6e91be48554cca5ecefbd4bcb6c ServiceNow request form].<br />
<br />
By default, only ports 80 and 443 are allowed through the Internet IP address. For all other ports, please [https://ucalgary.service-now.com/it?id=sc_cat_item&sys_id=47cd16d113153a00b5b4ff82e144b0bf create a firewall rule change request in ServiceNow].<br />
<br />
== Cloud-Init Automation ==<br />
<br />
=== Ubuntu ===<br />
The following Cloud-Init configs apply to Ubuntu VM templates.<br />
<br />
==== Ubuntu desktop ====<br />
Use the following Cloud-Init config with the Ubuntu Server template to set up an Ubuntu desktop environment. The setup step takes a up to 15 minutes to complete and you should see a login screen when the setup finishes. <br />
<br />
Adjust the root and user password as desired. Below, the test user password is set to blank, allowing you to login to Gnome without a password.<br />
{{Highlight|code=#cloud-config<br />
disable_root: false<br />
<br />
users:<br />
- name: user<br />
lock_passwd: false<br />
inactive: false<br />
gecos: Test User<br />
primary_group: user<br />
groups: wheel<br />
passwd: $1$ADUODeAy$eCJ1lPSxhSGmSvrmWxjLC1<br />
<br />
chpasswd:<br />
list: {{!}}<br />
root:password<br />
expire: false<br />
<br />
# Install a graphical desktop<br />
runcmd:<br />
- DEBIAN_FRONTEND=noninteractive apt -y upgrade<br />
- DEBIAN_FRONTEND=noninteractive apt -y install tasksel<br />
- tasksel install gnome-desktop<br />
- systemctl set-default graphical.target<br />
- systemctl isolate graphical.target}}<br />
<br />
=== Rocky Linux ===<br />
The following Cloud-Init configs apply to Rocky Linux templates.<br />
<br />
==== Rocky Linux Desktop ====<br />
Use the following Cloud-Init config using the Rocky Linux template to set up a Rocky Linux desktop environment. The setup step takes up to 10 minutes to complete and you should see a login screen when the setup finishes. <br />
<br />
Adjust the root and user password as desired. Below, the test user password is set to blank, allowing you to login to Gnome without a password.<br />
{{Highlight|code=#cloud-config<br />
disable_root: false<br />
<br />
users:<br />
- name: user<br />
lock_passwd: false<br />
inactive: false<br />
gecos: Test User<br />
primary_group: user<br />
groups: wheel<br />
passwd: $1$ADUODeAy$eCJ1lPSxhSGmSvrmWxjLC1<br />
<br />
chpasswd:<br />
list: {{!}}<br />
root:password<br />
expire: false<br />
<br />
# Install a graphical desktop<br />
runcmd:<br />
- yum -y install "@Workstation"<br />
- systemctl set-default graphical.target<br />
- systemctl isolate graphical.target}}<br />
<br />
==== Rocky Linux Docker host ====<br />
Use the following Cloud-Init config using the Rocky Linux template to set up a new docker host. This server can then be used to run Docker containers. Also included are:<br />
<br />
# The docker-compose utility to help deploy container stacks more easily<br />
# A helper script to expand the <code>/var</code> and <code>/</code> filesystems on first startup based on the available space in the ROOT volume. <br />
<br />
Use the CloudStack generated password with the '<code>rocky</code>' default user account to log in.{{Highlight|code=#cloud-config<br />
<br />
write_files:<br />
- path: /usr/bin/expand_lvm_root<br />
content: {{!}}<br />
#!/bin/bash<br />
/usr/bin/growpart /dev/vda 3<br />
/usr/sbin/pvresize -y -q /dev/vda3<br />
/usr/sbin/lvresize -y -q -r -l +50%FREE /dev/mapper/*root<br />
/usr/sbin/lvresize -y -q -r -l +100%FREE /dev/mapper/*var/dev/mapper/*root<br />
/usr/sbin/xfs_growfs /<br />
/usr/sbin/xfs_growfs /var<br />
permissions: '0700'<br />
<br />
- path: /usr/bin/setup_docker<br />
content: {{!}}<br />
#!/bin/bash<br />
yum install -y yum-utils<br />
yum-config-manager \<br />
--add-repo \<br />
https://download.docker.com/linux/centos/docker-ce.repo<br />
yum install -y docker-ce docker-ce-cli containerd.io<br />
systemctl start docker<br />
systemctl enable docker<br />
<br />
curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose<br />
chmod +x /usr/local/bin/docker-compose<br />
permissions: '0700'<br />
<br />
- path: /root/docker-compose.yml<br />
permissions: '0700'<br />
content: {{!}}<br />
version: '3.3'<br />
services:<br />
web:<br />
image: php:7.4-apache<br />
restart: always<br />
user: "0:0"<br />
volumes:<br />
- /var/www/html:/var/www/html<br />
ports:<br />
- "80:80"<br />
<br />
- path: /var/www/html/index.php<br />
permissions: '0644'<br />
content: {{!}}<br />
<h1>Hello there!</h1><br />
<p>I see you from <?php echo $_SERVER['REMOTE_ADDR']; ?></p><br />
<<nowiki>pre</nowiki>><?php print_r($_SERVER); ?></pre><br />
<br />
# Ensure VM has the largest / possible<br />
runcmd:<br />
- /usr/bin/expand_lvm_root<br />
- /usr/bin/setup_docker<br />
- cd /root; docker-compose up -d}}<br />
<br />
==== UC Authentication ====<br />
Use the following Cloud-Init config to allow UC-based authentication. Local accounts with the same username as the IT account may use the IT credential to log in.<br />
{{Highlight|code=#cloud-config<br />
<br />
write_files:<br />
- path: /usr/bin/setup_uc_auth<br />
permissions: '0700'<br />
content: {{!}}<br />
#!/bin/bash<br />
yum install -y sssd sssd-dbus sssd-krb5 krb5-workstation authselect-compat<br />
<br />
cat <<EOF > /etc/sssd/sssd.conf<br />
[sssd]<br />
config_file_version = 2<br />
services = nss, pam, ifp<br />
domains = uc.ucalgary.ca<br />
<br />
[domain/uc.ucalgary.ca]<br />
id_provider = files<br />
debug_level = 5<br />
auth_provider = krb5<br />
chpass_provider = krb5<br />
<br />
krb5_realm = UC.UCALGARY.CA<br />
krb5_server = ITSODCSRV14.UC.UCALGARY.CA:88<br />
krb5_validate = false<br />
EOF<br />
chmod 600 /etc/sssd/sssd.conf<br />
<br />
cat <<EOF > /etc/krb5.conf<br />
[logging]<br />
default = FILE:/var/log/krb5libs.log<br />
kdc = FILE:/var/log/krb5kdc.log<br />
admin_server = FILE:/var/log/kadmind.log<br />
<br />
[libdefaults]<br />
default_realm = UC.UCALGARY.CA<br />
dns_lookup_realm = false<br />
dns_lookup_kdc = false<br />
ticket_lifetime = 24h<br />
renew_lifetime = 7d<br />
forwardable = true<br />
<br />
[realms]<br />
UC.UCALGARY.CA = {<br />
kdc = itsodcsrv14.uc.ucalgary.ca<br />
}<br />
<br />
[domain_realm]<br />
uc.ucalgary.ca = UC.UCALGARY.CA<br />
.uc.ucalgary.ca = UC.UCALGARY.CA<br />
EOF<br />
<br />
mkdir -p /etc/authselect/custom/rcs<br />
cd /etc/authselect/custom/rcs<br />
echo H4sIAMYsbGIAA+0ca3PbuDGf+StQ34dLMpEtybLdUevO6BKl56mTuJGv7Uwmo4FISMKJBHQAaEfNub+9uwBJkXrLshxnjjuTyCKJfWF3sQsuFPhS9CtB79keoQpw2mjYT4DZz1r1uPGs1qhX6yf12vFx/Vm1Vqs1jp+R6j6ZSiHWhipCnikpzarn1t3/TuErj8bhhBzccjOs6Igq41MVHBDen71WUey3mCsWHNx5W4wKpT+qSAGjI3lDQxj8SarB0UDIiB2FcsBFRfuKMfHZY4L2QpYbS2MzZMJwnxouxfnXefQHTaNi9nufhprdpQj6XAyYGisuzHIUuYcWIhlTrW/lRkxMNdO0OH5HdHdeXlDNjAGCuhJQ0IM4GjPFx0Om4PGjDM9n8pULP4wDtpkik78q1Lec/WifcMr8kSyE7fB/a9ssYf8Q2PiPc6/3RmNN/K826mez8f/07LSM/48BR4uD8dGaWOytGbc8BK8buSTu5odtFEyPitGxDH2LYXai9kFjjf8fn4HPT/3/GPy/fnZ6Uvr/Y8BXiP/gTHHOL/KZ0Z2HRpGmD2meszi5mIExjbpM3BxquRuOgIkJICE/kE7qsKQYGwjXRbSrfD2Xxe7EVp/yEOMGsjZWzOLSPASuCHJ83iCxwPtdwyN2XqtXqwvYSpHMMKPjfp/7HHFtyIydrWBnXeeFQkT4fak4ZImulwh134n3POr7Mk50sTUWGMy0RoE2hnmRAJHDgzLtwk1ewffmJq/gXZiJBf9iTSaHZGvTA0ZoGGumdsakY1AxC7q8j9qJeUD+SmqwPhAQi5kC7k8B69M4NOc9GhA7TutzOSLIB4g1EvJWnHPIFhT77HBbE9hNXZBmRNxYi0yTlHvhyexaA9cYvhDkGCMZDTfGMmITLiw7wMGNHLECuq2ZCjnIZnVUyeHZmis90YZFNhDtJJ0Mgl9lrxuNhpDwBVxZi4ioHp1Xq2dni0bNu0k2GPwkz8yn1GBqJDWj1FSWCFUwTDCxG+4DHUF8JUXgzBNNrws2u9sspB65k+4SW99s/RdgyNz4w0OsA/eUY6yr/05rtdn8r35S1n+PAjaSBU1rPGA4pA8pjCaJG6NbsS8zbuWDxmTk6jTMogZKxmOHYDsMdiAgEMxkOKYIcp69DEE60CUaMsLQ3twSRzbQBgnr2XprRtKBiCMOJFMWRaoJXcAxn5rCiG9XcRbK7T3RWOP/pyen+fqvgf5/XC3rv0eBb17eYTIbsJDaGs/+cV53dlGWjfsuG+O6zWh8qP43hYW1UQUQ7aqYHC9CYh4P6fx9eanU+3SGn6xiqBGX7CU5X656mKZ/i6uRv52T+rQa2QV3sWbaaQZdwkjwrZSQxqlBxqYiYrCQ0UHTfd4RoybdPlfadDHiz0+V5mYjI1ivmm4i9W6CuSSW9KW6BfdbwnS5tVFubSxmJvUL0oM4wkRXD2kgb70fcnojt0CKUCB3S5IrYG3EZ8pQqO1sZqkJ+JLig6H5C/CgMUaPmQjgsiA6jFW0247Hg0yg5WNXfVkkXdDR2GzKzmacPLH9pPr3sZ20TTQe3/4W05Abm+0UwzyxyuyiGLorRTgpULr3YgPOdFKrE+dT2y49dqMEg6eRo93YSZaIZfjKfbkcmnJf7jvbl3ssGEtt7Gv4PdJYt/93Um/M1P81iC9l/f8YsKFFWydAP7eeZuvVTX13uaf+6X9kEEQv8991/DJZnwvYswJrLhTYGEmtDSNqWKlNNCa5Jj3LayV55KDpvv+uh/IWc00W3G3q1DOUkqIdatVxQGG1nmJ8Yh6+Gj62W2/etfdLY43/n1VPT3P+XwP/rzaOy/3/R4G27boinU7njS1/XKIwu1P2nIauGHcZpU2MsQgKJy+8813B8zosZD62dBEz5JqMlcStc1egsRyDVMMD4GoyVhAuZJ/wAJk0E4+Kud09wHID95U+BAI4OrmgCYWIY3C46x8DpJJEVNABSwtBg8lbJMGtIU0B1qTiTC8iEjF/SAXXkca6YYgcXr5pXb0i/2Cqx5QEJSnyVjF2cdU6JBcmYwKQkfedDkGcV613kLUYpvoUxDISdzycoMlkIF0yDuPBwOqiR32oaYPktgfM+lIIYNMKAnGSj0PkvN9nCmNUWqY4vaE+3uHuFBcwoZGTg/Ygc3dK9oGzHoPJjoEE7pIaOyO/Iv4x6KjpDY0Z6+bREXyLFTvk8ggHHkF+E3jeT5N0BXhVMBaK+2EQ6CHr6isZOVqKgpgKZRXJw+6dCYRvvO+BMbj5D8hziK/sxj08yU8DIzecZlnai0NyjTY0pDcMlQZKRSGFz7weE6zPQQ2QkbJUVn/InFE5wtgNhSqDJLIPsR4+KxEYgpq4R+1UmGGsPcgeY9AlEXEE82yNiesRrB9MWY2iln92HL9CcSYyJoG0hdItFXamRoyNEzXEQqD1g7bhOQXG6A+5YK/sKJgPzz5a8A1tXQb0ghz9GuPWBNdTT0mWVFQGw50LNAox8PAlL4dJc5Nu/Usb/B+KphEIr9DYwwkBtvG2GTpsCWpHLDep1iOSjRKLrIfEaDLD+Ql1bhROUl98/eH924u///KxdX3x4b1XmQPPa8EMOyFBb6hoiUpDH0MEP9qpmspySK5CRjWozNJ3j3iB9OMIptOJazDzYATsiDiXscPBDwx6P9TQwB75ADGtIGIqF72Bxd0qGFFNnRPmFm2Y4dsEIQn7wrWNZM6+CjyCUP9qXVy2frpskw9XKHnrkrxtt65/+djuLFBCThuFzbFm08OEJAndqXfjHSSM9Q9oAk3SSBvYJsiHZn5s+A2zQ2eiGCIGTYDN4k2vWKXNUMP3pTDKhyKLudFACB/Nh0q7VKSLBOqHK4vEbgMQW28cJkJlr1jm6OQ5tLaYPYoeC1Y3cLaQzj2aCITPITUWUfY0RObxWCqYZXQTnEpczNiXcch9bq09CTFIhAs7eKGNzXI82zecSHAJV4lraia3GNlojhf7zgmeZsE8urQsTfBs8t7qkLyXRNogGjEKSxlMxkzjNKJ67opwNI90i+RF5rGJ+Ck/ufbP9VOSe9h6no3Pbmpe5hoTX6bIkxc0KxFjOIdnwO3FIGRFfHB9Fhe+7Cniq0NUgqUUTPG+qItVQ4L9jQvfEIEx9OfrAdC91pg+BLFCHRcNPA5kgqFlN5vxAkYg8I8sraFpUmNzMHxAxXYhBOsJAp762REz/lHSYpBTg8taEiKvhwzsL9laRgNOuUoDBeoEvPS/eaOe7tsVhQXixF0HeRWNmMEZltNNEc97c9GxEa1z1X598fbiNaY1/764fv0zedO6bv3U6qyObdMY9x4XagjCr0jalAQJmqE9cG1tM7bexAbedAkEJ1K3ihu4C84XJw+BA2PgqRQam9w62Jeof7vQ2jJP21wHNO47G0yWGyO95P2dRbuINK57OH14TQqWECbzhD0uYJWgmXMVmmdA1ReO0EHSTZMJvIDoDArXMjPF4DphtkCQtd1McWQNNVugmXbeTPFMm2q2QJR130zxZH0126BJGnByWNyVdUja/2m9u7rMGavnvST58sitQeibOP1zy4vnkbwVpR+QFJMsDpBixF9OIV2ykJJdaNkmayzxCkvsOo6yJX7KIKRn7TZpXXY+TNXwEnMIO8ya9PM/v/iudjhKWAUf2//85eJj+137/XVnXzTWnv85qc7s/+C3cv/nMeAdHWGfinKJc6F4xDwzq5Jc7ZemiqTDkpBVLLEwJEUzewuuqNgPrOjYOrjzKjPhFgVK6+ZXENRSyfWKRDutDXguAjbX0CWkYpeYA8ySsJ/Avp8k5+RaxewAUX2CO5/hGXc08aHk3ZuaFzVqFA6JJZrOVwNLdJ0UBOttbEPKjynztNMtkTe5sExW61E2yaCRLTsiqBxCKzOmhZDrzr+CXEUV7cot+Ji32k2z6S7GiE3w3brbRLUkYZDfH+AmwNp3lKup7gvW9e8VtYwXH0rT6yh/A01nlB9P2/kX8ZXZlHCJphc1AaTavg9dQuymJtN2oxp3nC3yQoAoRoSHkdfGaLuD55vsJUNC+zClvQc9F+li7mEehuwausvW/+IB//3kGOvyv2L//wn2/zfqtTL/ewy45+//zJ8azyciT+JQwZPpsk/61GynaTfiGleOrqBQza/+LaHkr2kG+YBHD8oO6D9IB/RTbsZ9ooe7n37751Zsfdv2z7L/c8ruU+z/dJaxz9Of6/K/WrXWaMzkf7Wzk7My/3sMuF/+9yQSvH2cGt1n4vhA8U1IyLzdC57mIIiaX+DfCD9dy4GmN0w17Q+lVfJXRrkvxdB4n3OvmSwBvv4snnpMhQt4IlkuCZ7LaZOx9m/cNIZQazttyL2O4+7ww0kbmQhZyNbyH8wqT+R+hydyN1LQol+DWFYCZ9zU98jNyjcDy1131u9S1la4dM6d8WjX1JXvw0154jmjUZ54XoulrPeXMlOeeC5PPJcnnssTz+WJ52+95VXueGXsPsUdrxJKKKGEEkoooYQSSiihhBJKKKGEEkr448D/AT019aAAeAAA {{!}} base64 -d {{!}} tar -xzpf -<br />
<br />
if [[ "`authselect current -r`" != "custom/rcs" ]] ; then<br />
authselect select custom/rcs --force<br />
systemctl restart sssd<br />
fi<br />
<br />
systemctl enable sssd<br />
<br />
runcmd:<br />
- /usr/bin/expand_lvm_root}}<br />
<br />
== Infrastructure tools ==<br />
<br />
=== Generating a CloudStack API ===<br />
You can request for a CloudStack API key to automate infrastructure deployment using Terraform or CloudMonkey. A new API key can be generated by navigating to your profile page (top right) and then clicking on the 'Generate keys' button.<br />
[[File:CloudStack API Key.png|alt=CloudStack API Key|none|thumb|CloudStack API Key]]<br />
<br />
=== CloudMonkey ===<br />
CloudMonkey is a utility that makes it easier to interact with the CloudStack API. This tool may be used to help automate VM actions (such as start/stop/reboot), or infrastructure tasks (such as creating/destroying VMs, networks, or firewall rules). <br />
<br />
To get started with CloudMonkey, refer to the following resources:<br />
<br />
* Download from: <nowiki>https://github.com/apache/cloudstack-cloudmonkey/releases/tag/6.1.0</nowiki><br />
* Documentation at: <nowiki>https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+cloudmonkey+CLI</nowiki><br />
<br />
=== Terraform Integration ===<br />
Terraform allows you to define infrastructure as code and can be used in conjunction with CloudStack to configure your virtual machines and guest networks. Use the official CloudStack provider.<br />
<br />
The following is an example Terraform file for reference. Specify your CloudStack API keys either as a separate <code>vars.tf</code>.<br />
{{Highlight|code=# Configure the CloudStack Provider<br />
terraform {<br />
required_providers {<br />
cloudstack = {<br />
source = "cloudstack/cloudstack"<br />
version = "0.4.0"<br />
}<br />
}<br />
}<br />
<br />
provider "cloudstack" {<br />
api_url = "${var.cloudstack_api_url}"<br />
api_key = "${var.cloudstack_api_key}"<br />
secret_key = "${var.cloudstack_secret_key}"<br />
}<br />
<br />
# Create a new VPC<br />
resource "cloudstack_vpc" "default" {<br />
name = "wedgenet-vpc"<br />
display_text = "wedgenet-vpc"<br />
cidr = "100.64.0.0/20"<br />
vpc_offering = "Default VPC offering"<br />
zone = "zone1"<br />
}<br />
<br />
# Create a new ACL<br />
resource "cloudstack_network_acl" "default" {<br />
name = "vpc-acl"<br />
vpc_id = "${cloudstack_vpc.default.id}"<br />
}<br />
<br />
# One ingress and one egress rule for the ACL<br />
resource "cloudstack_network_acl_rule" "ingress" {<br />
acl_id = "${cloudstack_network_acl.default.id}"<br />
<br />
rule {<br />
action = "allow"<br />
cidr_list = ["0.0.0.0/0"]<br />
protocol = "tcp"<br />
ports = ["22", "80", "443"]<br />
traffic_type = "ingress"<br />
}<br />
}<br />
<br />
resource "cloudstack_network_acl_rule" "egress" {<br />
acl_id = "${cloudstack_network_acl.default.id}"<br />
<br />
rule {<br />
action = "allow"<br />
cidr_list = ["0.0.0.0/0"]<br />
protocol = "all"<br />
traffic_type = "egress"<br />
}<br />
}<br />
<br />
<br />
# Create a new network in the VPC<br />
resource "cloudstack_network" "primary" {<br />
name = "primary"<br />
display_text = "primary"<br />
cidr = "100.64.1.0/24"<br />
network_offering = "DefaultIsolatedNetworkOfferingForVpcNetworks"<br />
acl_id = "${cloudstack_network_acl.default.id}"<br />
vpc_id = "${cloudstack_vpc.default.id}"<br />
zone = "zone1"<br />
}<br />
<br />
# Create a new public IP address for this network<br />
resource "cloudstack_ipaddress" "public_ip" {<br />
vpc_id = "${cloudstack_vpc.default.id}"<br />
network_id = "${cloudstack_network.wosnet1.id}"<br />
}<br />
<br />
# Create VMs. <br />
resource "cloudstack_instance" "vm" {<br />
count = 1<br />
name = "vm${count.index+1}"<br />
zone = "zone1"<br />
service_offering = "rcs.c4"<br />
template = "RockyLinux 8.5"<br />
network_id = "${cloudstack_network.wosnet1.id}"<br />
<br />
# Cloud Init data can be used to configure your VM on first startup if your template supports Cloud Init<br />
user_data = <<EOF<br />
#cloud-config<br />
<br />
# Require specific packages<br />
packages:<br />
- tmux<br />
- git<br />
- tcpdump<br />
<br />
EOF<br />
}<br />
<br />
<br />
}}<br />
<br />
= Troubleshooting =<br />
<br />
=== Cannot create a volume snapshot ===<br />
Volume snapshots can only be taken on VMs that are powered off.<br />
<br />
=== Cannot create a VM snapshot ===<br />
Disk-only VM snapshots cannot be taken when the VM is running. If you intend to snapshot a running system, you must also snapshot its memory.<br />
<br />
=== VM state is still running after shutdown ===<br />
After running 'shutdown' on a VM, the VM state reported by CloudStack is still running. <br />
<br />
Please try to do a force shutdown from the CloudStack management console. The VM state isn't updated by CloudStack and as a result, the state of a VM isn't properly reflected when power state changes outside of CloudStack (likely a bug?)</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=CloudStack_User_Guide&diff=1849CloudStack User Guide2022-06-02T16:56:52Z<p>Darcy: /* Exposed to the Internet */</p>
<hr />
<div>This is a user's guide on using CloudStack provided by Research Computing Services.<br />
<br />
== Introduction==<br />
Apache CloudStack is an Infrastructure as a Service (IaaS) platform that allows users to quickly spin up Linux/Non-Windows based virtual machines. RCS is providing this service to help researchers quickly set up and prototype short-term research related software on premises. CloudStack is not appropriate for workloads that depend on Windows. Services set up on CloudStack virtual machines can be accessed from the campus network and also the internet if required.<br />
<br />
Access to CloudStack can be requested via [https://ucalgary.service-now.com/it?id=sc_cat_item&sys_id=e3c1d6e91be48554cca5ecefbd4bcb6c ServiceNow].<br />
<br />
Please refer to our [[CloudStack End User Agreement]] for acceptable uses and requirements.<br />
<br />
===Accessing the CloudStack management console===<br />
<br />
The CloudStack management console is a web-based portal that allows you to view and manage your cloud infrastructure including virtual machines, storage, and network. Any modern web browsers including Chrome, Firefox, Edge, and Safari is supported. <br />
<br />
Access the CloudStack management console is possible only from an IT-managed computer or through the IT General VPN when working on unmanaged machines (eg. AirUC) or when working off campus (eg. at home). Please review the IT [https://ucalgary.service-now.com/it?id=kb_article&sys_id=52a169d6dbe5bc506ad32637059619cd knowledge base article on connecting to the General VPN] or contact IT support if you need assistance connecting to the General VPN. <br />
[[File:CloudStack VPN Connection.png|alt=CloudStack VPN Connection|none|thumb|CloudStack VPN Connection]]<br />
=== Login to CloudStack===<br />
<br />
To log in to CloudStack, navigate to https://cloudstack.rcs.ucalgary.ca/. If this site fails to load, please make sure you are either on a IT managed computer or connected to the General VPN.<br />
<br />
Sign in to CloudStack using the Single Sign-On option as shown in the image below. This method will require you to authenticate through our central authentication service using your University of Calgary IT credentials and will require multi-factor authentication. You must have multi-factor authentication set up either via your phone or with the Microsoft Authenticator app.<br />
[[File:CloudStack Login Page.png|alt=CloudStack Login Page|none|thumb|CloudStack Login Page]]<br />
<br />
'''Note:''' Due to a bug with the UI, if the Single Sign-On option is disabled, please refresh the login page and try again. This issue should be addressed in our next update for CloudStack.<br />
<br />
=== CloudStack Dashboard===<br />
<br />
After logging in, you will be presented with your CloudStack management console. The dashboard shows you a general overview of your account's status.<br />
[[File:CloudStack Dashboard.png|alt=CloudStack Dashboard|none|thumb|CloudStack Dashboard]]On the right hand side of the dashboard, you will also see recent activity and events that was done within your CloudStack account.<br />
<br />
If you wish to see your CloudStack account resource quota and allocation, navigate to: <code>Accounts -> Click on your account -> Resources</code>. <br />
[[File:CloudStack Resource Quota.png|alt=CloudStack Resource Quota|none|thumb|CloudStack Resource Quota]]<br />
<br />
== Working with virtual machines==<br />
<br />
CloudStack allows you to control the lifecycle of virtual machines within your cloud account. VMs may be started, stopped, rebooted, or destroyed within your management console.<br />
<br />
===Create a VM===<br />
<br />
To create a new VM, enter the CloudStack management console and navigate to: <code>Compute -> Instances -> Add Instance</code>[[File:CloudStack Instance Summary.png|alt=CloudStack Instance Summary|thumb|CloudStack Instance Summary|493x493px]]<br />
<br />
Virtual Machines require the following details:<br />
<br />
# '''Deployment zone'''. Your account will already be placed in the appropriate zone.<br />
# '''Boot template or ISO'''. You may choose either a pre-created template or boot from a custom CD-ROM ISO file.<br />
# '''Compute offering'''. You may select an appropriate size for your new VM. Resources will be counted against your account's quota.<br />
# '''Data Disk'''. You may choose to add an additional virtual disk to your VM to store your data. Alternatively, if you wish to use a single virtual disk for your VM, you may choose to override the size of your root disk in step 2 and select 'No thanks' in this step.<br />
# '''Networks'''. You may choose one or more networks your VM should connect to. All CloudStack accounts come with a default network already created and ready to be used.<br />
# '''SSH keypairs'''. For templates that support custom SSH key pairs, you may choose to use a custom SSH keypair to be installed as part of the deployment process.<br />
# '''Advanced settings'''. For templates that support custom user-data (Cloud-Init), you may choose to enable the advanced settings and provide your own Cloud-Init user-data payload. More on this in the advanced tasks section below.<br />
# '''Other VM details'''. You may give your new VM a friendly name and make it part of a group. Groups allow you to group related VMs together for better organization. You may change these details at a later time.<br />
<br />
When you are done, review the instance summary on the right hand side and then click on the 'Launch Virtual Machine' button.<br />
<br />
====Choosing a virtual machine template====<br />
<br />
We provide a Rocky Linux 8.5 and a Ubuntu Server 22.04 LTS template for your convenience. These templates are pre-built images with the operating system installed and ready for use. Our templates also support further automated setup configured using Cloud-Init configuration data that can be provided when deploying a new VM. Currently, we offer the following templates: <br />
{| class="wikitable"<br />
!Template<br />
!Cloud-Init Support<br />
!Password Support<br />
!Default Username<br />
|-<br />
|Rocky Linux 8.5<br />
|Yes<br />
|Yes<br />
|rocky<br />
|-<br />
|Ubuntu Server 22.04<br />
|Yes<br />
|Yes<br />
|ubuntu<br />
|}<br />
Rocky Linux is an open source Linux distribution that is binary-compatible with Red Hat Enterprise Linux and is what RCS recommends.<br />
<br />
For templates that support passwords, the generated password that appears after a VM is created is applied to the default username.<br />
<br />
Security note: All VM templates are configured with SSH password authentication enabled. You should be able to SSH to your VM from another system connected to the same guest network. Do not expose port 22 unless required and we highly recommend using key based authentication.<br />
<br />
==== Virtual machine credentials ====<br />
VM templates that support password will have a randomly generated password set when the VM is first created or when a password reset request is made (available only when the VM is powered off). A randomly generated 6 character password will be displayed when a new password is set and appears as a notification in your CloudStack management console. <br />
[[File:CloudStack VM Password.png|alt=CloudStack VM Password|none|thumb|CloudStack VM Password]]<br />
This password is set on the default username for your template. For example, the Rocky Linux VM template will set this password to the ''''rocky'''<nowiki/>' user account. You may become the super user by logging in as the <code>rocky</code> user and then running <code>sudo su</code>.<br />
<br />
Note: If you specify a custom Cloud-Init config that creates additional users or sets account passwords, the displayed password will be overridden and have no effect.<br />
<br />
=== Creating a custom template===<br />
<br />
Alternatively, you may decide to install a custom OS such as a different Linux distribution or other UNIX based operating systems and create a template from that. To create a custom template:<br />
<br />
# Create a new virtual machine and select your custom ISO media. If you wish to upload your own ISO, see the 'register ISO' section below.<br />
# Start the virtual machine and proceed through the OS setup process<br />
# Once the system has been set up, prepare the VM to be templated by removing any host-specific files such as SSH host keys, static network configuration settings, temporary files and caches.<br />
# Power off the virtual machine<br />
# Navigate to the virtual machine page and click on the 'create template' button<br />
[[File:CloudStack Instance Controls.png|alt=CloudStack Instance Controls|none|thumb|CloudStack Instance Controls]]<br />
<br />
===Registering a custom ISO===<br />
<br />
You may install custom ISO file into your CloudStack account either by directly uploading the ISO through the web console or by providing a URL to the ISO file on the internet.<br />
<br />
Please do not install Windows on our CloudStack infrastructure. If you need a Windows VM, please contact us as we have alternative solutions.<br />
<br />
==== Download a ISO from the internet====<br />
[[File:CloudStack Download ISO.png|alt=CloudStack Download ISO|thumb|CloudStack Download ISO|190x190px]]<br />
<br />
To add a custom ISO file from the internet, enter the CloudStack management console and navigate to: <code>Images -> ISOs -> Register ISO</code><br />
<br />
You may check the state of the ISO file by clicking on it and verify the state of the file. If the file is successfully downloaded, its ready state should become ‘true’. The ISO file will only appear in the selection list when the file is downloaded successfully.<br />
[[File:CloudStack ISO Ready.png|alt=CloudStack ISO Ready|none|thumb|172x172px|CloudStack ISO Ready]]<br />
<br />
====Upload a custom ISO====<br />
<br />
To upload an ISO file, enter the CloudStack management console and navigate to: <code>Images -> ISOs -> Upload ISO from Local (icon)</code><br />
<br />
[[File:CloudStack Upload ISO.png|alt=CloudStack Upload ISO|none|thumb|CloudStack Upload ISO|217x217px]]<br />
<br />
===Connecting to your VM console===<br />
The CloudStack management console has a KVM (keyboard, video, mouse) feature built-in, allowing you to remotely connect to and interact with your virtual machine. To connect to your virtual machine's console, navigate to: <code>Compute -> Instances -> Your Instance -> View console</code>.<br />
[[File:CloudStack View Console.png|alt=CloudStack View Console|none|thumb|CloudStack View Console]]<br />
<br />
=== Expanding a VM disk ===<br />
[[File:CloudStack Expand Volume.png|alt=CloudStack Expand Volume|thumb|CloudStack Expand Volume]]<br />
Virtual machine disks can be expanded after they are created within CloudStack. However, you will need to expand the partitions and filesystems manually.<br />
<br />
To grow an existing disk:<br />
<br />
# Go into your VM details page and click on ‘Volumes’.<br />
# Click on the volume you wish to expand.<br />
# Click on the ‘Resize Volume’ icon in the top right.<br />
Once the volume has been expanded, you should be able to verify the disk volume has grown with <code>lsblk</code>. There should also be some messages by the kernel when this occurs. However, you will still need to expand any partitions, volumes, and filesystems on your system manually.<br />
<br />
To expand your partition, use the <code>growpart</code> command followed by your disk device and partition number. Eg: <code>/usr/bin/growpart /dev/vda 3</code><br />
<br />
For LVM volume sets, you can expand the volume using the <code>pvresize</code> and <code>lvresize</code> commands:<br />
<br />
* <code>/usr/sbin/pvresize -y -q /dev/vda3</code><br />
* <code>/usr/sbin/lvresize -y -q -r -l +100%FREE /dev/mapper/<volume-name></code><br />
<br />
To expand your filesystem:<br />
<br />
* XFS: <code>/usr/sbin/xfs_growfs <volume></code><br />
* EXT: <code>resize2fs <volume></code><br />
<br />
=== Destroying a VM ===<br />
If you need to delete a VM, click on the red garbage bin icon in the VM instance page. All deletions are irreversible, so please make sure you have a copy of any data you need before proceeding.<br />
[[File:CloudStack Delete VM.png|alt=CloudStack Delete VM|none|thumb|CloudStack Delete VM]]<br />
The VM root volume can be deleted immediately by enabling the 'Expunge' option in the dialog box. If left disabled, the VM root volume will linger for a day before it is deleted by the system. You may wish to expunge a volume if you are running low on space or volume quota.<br />
<br />
== Managing your virtual machine ==<br />
You will be able to run whatever virtual machine you wish (with the exception of Windows). Clearly we cannot provide specific management advice on each and every operating system available. We can provide you with some suggestions on important considerations to be aware of.<br />
<br />
=== Educate yourself ===<br />
All operating systems (OS) have user groups, web sites, wikis, or mailing lists somewhere on the internet. They can be a valuable resource. Most OS providers have on-line documentation that describes using their product. For example Rocky Linux, used by RCS, has a [https://docs.rockylinux.org/ documentation site]. These are excellent resources and can help you understand how to manage your virtual machine.<br />
<br />
=== Choose an appropriate OS edition ===<br />
Many OSs will provide various editions that are tailored to a specific use case. A desktop VM may not be appropriate when you need to run a database server. The OS provider will have guides on how to choose an edition.<br />
<br />
=== Configure your VM's OS ===<br />
It is critical to ensure that the only services running on your VM are the ones you must run. Each OS has a way of managing what services are running (sysinit, systemd etc). Please ensure that unnecessary services have been disabled.<br />
<br />
Many OSs will have pre-configured accounts, and many applications will have pre-configured accounts. Make sure they are either disabled or not allowed to login.<br />
<br />
All un-used accounts should be disabled or preferably deleted.<br />
<br />
All accounts should have strong [https://it.ucalgary.ca/it-security/passwords-do-i-have-change-them/ passwords].<br />
<br />
Many OS's have the ability to automatically update themselves. If possible please consider doing this.<br />
<br />
Updates can also be configured to skip certain software if it will interfere with your research, but please be advised that doing so could place your system at risk.<br />
<br />
=== Exposed to the Internet ===<br />
Not everyone is a computer security expert. If your VM must be exposed to the internet, please consider using Trend Micro Cloud One Workload Security from IT security to enhance your security posture.<br />
<br />
== Virtual machine networking ==<br />
The CloudStack platform allows you to define custom virtual private cloud (VPC) network which can contain any number of guest networks that your virtual machines connect to. Each guest network has its own private network address space and is not directly routable from campus or the internet. For virtual machines that require internet access, the VPC or guest network it is connected to must have a NAT IP address associated. The following diagram shows how a guest network connects to the internet and campus network.<br />
[[File:CloudStack Guest Networking.png|alt=CloudStack Guest Networking|none|thumb|CloudStack Guest Networking]]<br />
In order to expose a virtual machine's services to campus or the internet, the appropriate port forwardings must be set up on the VPC containing the guest network. More on this will be discussed in the next section.<br />
<br />
Having multiple guest networks allows for more advanced network setups but is not required. We recommend using a single flat network for most workloads. <br />
<br />
By default, all CloudStack accounts come with a default VPC and guest network set up with a NAT IP assigned.<br />
<br />
=== IP addresses ===<br />
Due to the design decisions made during the setup of the CloudStack platform, only internal 10.44.12X.X IPs can be assigned to your VPC. These IP addresses are accessible from the university campus network. However, there is a special section of IP addresses that can be accessed from the internet.<br />
{| class="wikitable"<br />
!IP address range<br />
!Accessible from<br />
!Internet IP mapping<br />
|-<br />
|10.44.120.3-128<br />
|Campus, Internet<br />
|10.44.120.X maps to 136.159.140.X (ports 80 and 443 only)<br />
|-<br />
|10.44.120.129-255<br />
|Campus only<br />
|N/A<br />
|-<br />
|10.44.121.0-255<br />
|Campus only<br />
|N/A<br />
|-<br />
|10.44.122.0-255<br />
|Campus only<br />
|N/A<br />
|-<br />
|10.44.123.0-255<br />
|Campus only<br />
|N/A<br />
|}<br />
If you need a service exposed to the internet, please request for a public IP address using our [https://ucalgary.service-now.com/it?id=sc_cat_item&sys_id=e3c1d6e91be48554cca5ecefbd4bcb6c ServiceNow request form]. Additionally, if your service is not port 80 or 443, you must also request for a firewall change request to allow the special port through. <br />
<br />
=== Exposing a network service to campus ===<br />
In order to make a virtual machine be visible to the campus network, you must first set up a port forwarding from a campus IP address to your virtual machine.<br />
<br />
To create a port forwarding, navigate to <code>Network -> VPC -> Select your VPC -> Public IP Addresses</code>. If you do not have any available IP addresses, you will need to click on 'Acquire New IP' and select an available IP address. Click on the IP address you wish to use to create a port forwarding on and then navigate to the 'Port Forwarding' tab. Enter the private port range, the public port range, the protocol, and select the target VM. <br />
<br />
For example, to port forward only HTTP (tcp/80) traffic, you would enter the following:<br />
[[File:CloudStack Port Forwarding.png|alt=CloudStack Port Forwarding|none|thumb|CloudStack Port Forwarding]]Once the port forwarding is created, you should be able to access the service from on campus. If for some reason access to your service does not work, there may be a firewall restriction on IT's network. In such circumstances, please contact us for assistance.<br />
<br />
=== Exposing a network to the internet ===<br />
Exposing a service to the internet is the same as exposing it to campus. However, you must create a port forwarding on an IP address that maps to an internet IP address outlined in the IP address table above. If your account does not have one of these IP addresses available, please request for one on the [https://ucalgary.service-now.com/it?id=sc_cat_item&sys_id=e3c1d6e91be48554cca5ecefbd4bcb6c ServiceNow request form].<br />
<br />
By default, only ports 80 and 443 are allowed through the Internet IP address. For all other ports, please [https://ucalgary.service-now.com/it?id=sc_cat_item&sys_id=47cd16d113153a00b5b4ff82e144b0bf create a firewall rule change request in ServiceNow].<br />
<br />
== Cloud-Init Automation ==<br />
<br />
=== Ubuntu ===<br />
The following Cloud-Init configs apply to Ubuntu VM templates.<br />
<br />
==== Ubuntu desktop ====<br />
Use the following Cloud-Init config with the Ubuntu Server template to set up an Ubuntu desktop environment. The setup step takes a up to 15 minutes to complete and you should see a login screen when the setup finishes. <br />
<br />
Adjust the root and user password as desired. Below, the test user password is set to blank, allowing you to login to Gnome without a password.<br />
{{Highlight|code=#cloud-config<br />
disable_root: false<br />
<br />
users:<br />
- name: user<br />
lock_passwd: false<br />
inactive: false<br />
gecos: Test User<br />
primary_group: user<br />
groups: wheel<br />
passwd: $1$ADUODeAy$eCJ1lPSxhSGmSvrmWxjLC1<br />
<br />
chpasswd:<br />
list: {{!}}<br />
root:password<br />
expire: false<br />
<br />
# Install a graphical desktop<br />
runcmd:<br />
- DEBIAN_FRONTEND=noninteractive apt -y upgrade<br />
- DEBIAN_FRONTEND=noninteractive apt -y install tasksel<br />
- tasksel install gnome-desktop<br />
- systemctl set-default graphical.target<br />
- systemctl isolate graphical.target}}<br />
<br />
=== Rocky Linux ===<br />
The following Cloud-Init configs apply to Rocky Linux templates.<br />
<br />
==== Rocky Linux Desktop ====<br />
Use the following Cloud-Init config using the Rocky Linux template to set up a Rocky Linux desktop environment. The setup step takes up to 10 minutes to complete and you should see a login screen when the setup finishes. <br />
<br />
Adjust the root and user password as desired. Below, the test user password is set to blank, allowing you to login to Gnome without a password.<br />
{{Highlight|code=#cloud-config<br />
disable_root: false<br />
<br />
users:<br />
- name: user<br />
lock_passwd: false<br />
inactive: false<br />
gecos: Test User<br />
primary_group: user<br />
groups: wheel<br />
passwd: $1$ADUODeAy$eCJ1lPSxhSGmSvrmWxjLC1<br />
<br />
chpasswd:<br />
list: {{!}}<br />
root:password<br />
expire: false<br />
<br />
# Install a graphical desktop<br />
runcmd:<br />
- yum -y install "@Workstation"<br />
- systemctl set-default graphical.target<br />
- systemctl isolate graphical.target}}<br />
<br />
==== Rocky Linux Docker host ====<br />
Use the following Cloud-Init config using the Rocky Linux template to set up a new docker host. This server can then be used to run Docker containers. Also included are:<br />
<br />
# The docker-compose utility to help deploy container stacks more easily<br />
# A helper script to expand the <code>/var</code> and <code>/</code> filesystems on first startup based on the available space in the ROOT volume. <br />
<br />
Use the CloudStack generated password with the '<code>rocky</code>' default user account to log in.{{Highlight|code=#cloud-config<br />
<br />
write_files:<br />
- path: /usr/bin/expand_lvm_root<br />
content: {{!}}<br />
#!/bin/bash<br />
/usr/bin/growpart /dev/vda 3<br />
/usr/sbin/pvresize -y -q /dev/vda3<br />
/usr/sbin/lvresize -y -q -r -l +50%FREE /dev/mapper/*root<br />
/usr/sbin/lvresize -y -q -r -l +100%FREE /dev/mapper/*var/dev/mapper/*root<br />
/usr/sbin/xfs_growfs /<br />
/usr/sbin/xfs_growfs /var<br />
permissions: '0700'<br />
<br />
- path: /usr/bin/setup_docker<br />
content: {{!}}<br />
#!/bin/bash<br />
yum install -y yum-utils<br />
yum-config-manager \<br />
--add-repo \<br />
https://download.docker.com/linux/centos/docker-ce.repo<br />
yum install -y docker-ce docker-ce-cli containerd.io<br />
systemctl start docker<br />
systemctl enable docker<br />
<br />
curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose<br />
chmod +x /usr/local/bin/docker-compose<br />
permissions: '0700'<br />
<br />
- path: /root/docker-compose.yml<br />
permissions: '0700'<br />
content: {{!}}<br />
version: '3.3'<br />
services:<br />
web:<br />
image: php:7.4-apache<br />
restart: always<br />
user: "0:0"<br />
volumes:<br />
- /var/www/html:/var/www/html<br />
ports:<br />
- "80:80"<br />
<br />
- path: /var/www/html/index.php<br />
permissions: '0644'<br />
content: {{!}}<br />
<h1>Hello there!</h1><br />
<p>I see you from <?php echo $_SERVER['REMOTE_ADDR']; ?></p><br />
<<nowiki>pre</nowiki>><?php print_r($_SERVER); ?></pre><br />
<br />
# Ensure VM has the largest / possible<br />
runcmd:<br />
- /usr/bin/expand_lvm_root<br />
- /usr/bin/setup_docker<br />
- cd /root; docker-compose up -d}}<br />
<br />
==== UC Authentication ====<br />
Use the following Cloud-Init config to allow UC-based authentication. Local accounts with the same username as the IT account may use the IT credential to log in.<br />
{{Highlight|code=#cloud-config<br />
<br />
write_files:<br />
- path: /usr/bin/setup_uc_auth<br />
permissions: '0700'<br />
content: {{!}}<br />
#!/bin/bash<br />
yum install -y sssd sssd-dbus sssd-krb5 krb5-workstation authselect-compat<br />
<br />
cat <<EOF > /etc/sssd/sssd.conf<br />
[sssd]<br />
config_file_version = 2<br />
services = nss, pam, ifp<br />
domains = uc.ucalgary.ca<br />
<br />
[domain/uc.ucalgary.ca]<br />
id_provider = files<br />
debug_level = 5<br />
auth_provider = krb5<br />
chpass_provider = krb5<br />
<br />
krb5_realm = UC.UCALGARY.CA<br />
krb5_server = ITSODCSRV14.UC.UCALGARY.CA:88<br />
krb5_validate = false<br />
EOF<br />
chmod 600 /etc/sssd/sssd.conf<br />
<br />
cat <<EOF > /etc/krb5.conf<br />
[logging]<br />
default = FILE:/var/log/krb5libs.log<br />
kdc = FILE:/var/log/krb5kdc.log<br />
admin_server = FILE:/var/log/kadmind.log<br />
<br />
[libdefaults]<br />
default_realm = UC.UCALGARY.CA<br />
dns_lookup_realm = false<br />
dns_lookup_kdc = false<br />
ticket_lifetime = 24h<br />
renew_lifetime = 7d<br />
forwardable = true<br />
<br />
[realms]<br />
UC.UCALGARY.CA = {<br />
kdc = itsodcsrv14.uc.ucalgary.ca<br />
}<br />
<br />
[domain_realm]<br />
uc.ucalgary.ca = UC.UCALGARY.CA<br />
.uc.ucalgary.ca = UC.UCALGARY.CA<br />
EOF<br />
<br />
mkdir -p /etc/authselect/custom/rcs<br />
cd /etc/authselect/custom/rcs<br />
echo H4sIAMYsbGIAA+0ca3PbuDGf+StQ34dLMpEtybLdUevO6BKl56mTuJGv7Uwmo4FISMKJBHQAaEfNub+9uwBJkXrLshxnjjuTyCKJfWF3sQsuFPhS9CtB79keoQpw2mjYT4DZz1r1uPGs1qhX6yf12vFx/Vm1Vqs1jp+R6j6ZSiHWhipCnikpzarn1t3/TuErj8bhhBzccjOs6Igq41MVHBDen71WUey3mCsWHNx5W4wKpT+qSAGjI3lDQxj8SarB0UDIiB2FcsBFRfuKMfHZY4L2QpYbS2MzZMJwnxouxfnXefQHTaNi9nufhprdpQj6XAyYGisuzHIUuYcWIhlTrW/lRkxMNdO0OH5HdHdeXlDNjAGCuhJQ0IM4GjPFx0Om4PGjDM9n8pULP4wDtpkik78q1Lec/WifcMr8kSyE7fB/a9ssYf8Q2PiPc6/3RmNN/K826mez8f/07LSM/48BR4uD8dGaWOytGbc8BK8buSTu5odtFEyPitGxDH2LYXai9kFjjf8fn4HPT/3/GPy/fnZ6Uvr/Y8BXiP/gTHHOL/KZ0Z2HRpGmD2meszi5mIExjbpM3BxquRuOgIkJICE/kE7qsKQYGwjXRbSrfD2Xxe7EVp/yEOMGsjZWzOLSPASuCHJ83iCxwPtdwyN2XqtXqwvYSpHMMKPjfp/7HHFtyIydrWBnXeeFQkT4fak4ZImulwh134n3POr7Mk50sTUWGMy0RoE2hnmRAJHDgzLtwk1ewffmJq/gXZiJBf9iTSaHZGvTA0ZoGGumdsakY1AxC7q8j9qJeUD+SmqwPhAQi5kC7k8B69M4NOc9GhA7TutzOSLIB4g1EvJWnHPIFhT77HBbE9hNXZBmRNxYi0yTlHvhyexaA9cYvhDkGCMZDTfGMmITLiw7wMGNHLECuq2ZCjnIZnVUyeHZmis90YZFNhDtJJ0Mgl9lrxuNhpDwBVxZi4ioHp1Xq2dni0bNu0k2GPwkz8yn1GBqJDWj1FSWCFUwTDCxG+4DHUF8JUXgzBNNrws2u9sspB65k+4SW99s/RdgyNz4w0OsA/eUY6yr/05rtdn8r35S1n+PAjaSBU1rPGA4pA8pjCaJG6NbsS8zbuWDxmTk6jTMogZKxmOHYDsMdiAgEMxkOKYIcp69DEE60CUaMsLQ3twSRzbQBgnr2XprRtKBiCMOJFMWRaoJXcAxn5rCiG9XcRbK7T3RWOP/pyen+fqvgf5/XC3rv0eBb17eYTIbsJDaGs/+cV53dlGWjfsuG+O6zWh8qP43hYW1UQUQ7aqYHC9CYh4P6fx9eanU+3SGn6xiqBGX7CU5X656mKZ/i6uRv52T+rQa2QV3sWbaaQZdwkjwrZSQxqlBxqYiYrCQ0UHTfd4RoybdPlfadDHiz0+V5mYjI1ivmm4i9W6CuSSW9KW6BfdbwnS5tVFubSxmJvUL0oM4wkRXD2kgb70fcnojt0CKUCB3S5IrYG3EZ8pQqO1sZqkJ+JLig6H5C/CgMUaPmQjgsiA6jFW0247Hg0yg5WNXfVkkXdDR2GzKzmacPLH9pPr3sZ20TTQe3/4W05Abm+0UwzyxyuyiGLorRTgpULr3YgPOdFKrE+dT2y49dqMEg6eRo93YSZaIZfjKfbkcmnJf7jvbl3ssGEtt7Gv4PdJYt/93Um/M1P81iC9l/f8YsKFFWydAP7eeZuvVTX13uaf+6X9kEEQv8991/DJZnwvYswJrLhTYGEmtDSNqWKlNNCa5Jj3LayV55KDpvv+uh/IWc00W3G3q1DOUkqIdatVxQGG1nmJ8Yh6+Gj62W2/etfdLY43/n1VPT3P+XwP/rzaOy/3/R4G27boinU7njS1/XKIwu1P2nIauGHcZpU2MsQgKJy+8813B8zosZD62dBEz5JqMlcStc1egsRyDVMMD4GoyVhAuZJ/wAJk0E4+Kud09wHID95U+BAI4OrmgCYWIY3C46x8DpJJEVNABSwtBg8lbJMGtIU0B1qTiTC8iEjF/SAXXkca6YYgcXr5pXb0i/2Cqx5QEJSnyVjF2cdU6JBcmYwKQkfedDkGcV613kLUYpvoUxDISdzycoMlkIF0yDuPBwOqiR32oaYPktgfM+lIIYNMKAnGSj0PkvN9nCmNUWqY4vaE+3uHuFBcwoZGTg/Ygc3dK9oGzHoPJjoEE7pIaOyO/Iv4x6KjpDY0Z6+bREXyLFTvk8ggHHkF+E3jeT5N0BXhVMBaK+2EQ6CHr6isZOVqKgpgKZRXJw+6dCYRvvO+BMbj5D8hziK/sxj08yU8DIzecZlnai0NyjTY0pDcMlQZKRSGFz7weE6zPQQ2QkbJUVn/InFE5wtgNhSqDJLIPsR4+KxEYgpq4R+1UmGGsPcgeY9AlEXEE82yNiesRrB9MWY2iln92HL9CcSYyJoG0hdItFXamRoyNEzXEQqD1g7bhOQXG6A+5YK/sKJgPzz5a8A1tXQb0ghz9GuPWBNdTT0mWVFQGw50LNAox8PAlL4dJc5Nu/Usb/B+KphEIr9DYwwkBtvG2GTpsCWpHLDep1iOSjRKLrIfEaDLD+Ql1bhROUl98/eH924u///KxdX3x4b1XmQPPa8EMOyFBb6hoiUpDH0MEP9qpmspySK5CRjWozNJ3j3iB9OMIptOJazDzYATsiDiXscPBDwx6P9TQwB75ADGtIGIqF72Bxd0qGFFNnRPmFm2Y4dsEIQn7wrWNZM6+CjyCUP9qXVy2frpskw9XKHnrkrxtt65/+djuLFBCThuFzbFm08OEJAndqXfjHSSM9Q9oAk3SSBvYJsiHZn5s+A2zQ2eiGCIGTYDN4k2vWKXNUMP3pTDKhyKLudFACB/Nh0q7VKSLBOqHK4vEbgMQW28cJkJlr1jm6OQ5tLaYPYoeC1Y3cLaQzj2aCITPITUWUfY0RObxWCqYZXQTnEpczNiXcch9bq09CTFIhAs7eKGNzXI82zecSHAJV4lraia3GNlojhf7zgmeZsE8urQsTfBs8t7qkLyXRNogGjEKSxlMxkzjNKJ67opwNI90i+RF5rGJ+Ck/ufbP9VOSe9h6no3Pbmpe5hoTX6bIkxc0KxFjOIdnwO3FIGRFfHB9Fhe+7Cniq0NUgqUUTPG+qItVQ4L9jQvfEIEx9OfrAdC91pg+BLFCHRcNPA5kgqFlN5vxAkYg8I8sraFpUmNzMHxAxXYhBOsJAp762REz/lHSYpBTg8taEiKvhwzsL9laRgNOuUoDBeoEvPS/eaOe7tsVhQXixF0HeRWNmMEZltNNEc97c9GxEa1z1X598fbiNaY1/764fv0zedO6bv3U6qyObdMY9x4XagjCr0jalAQJmqE9cG1tM7bexAbedAkEJ1K3ihu4C84XJw+BA2PgqRQam9w62Jeof7vQ2jJP21wHNO47G0yWGyO95P2dRbuINK57OH14TQqWECbzhD0uYJWgmXMVmmdA1ReO0EHSTZMJvIDoDArXMjPF4DphtkCQtd1McWQNNVugmXbeTPFMm2q2QJR130zxZH0126BJGnByWNyVdUja/2m9u7rMGavnvST58sitQeibOP1zy4vnkbwVpR+QFJMsDpBixF9OIV2ykJJdaNkmayzxCkvsOo6yJX7KIKRn7TZpXXY+TNXwEnMIO8ya9PM/v/iudjhKWAUf2//85eJj+137/XVnXzTWnv85qc7s/+C3cv/nMeAdHWGfinKJc6F4xDwzq5Jc7ZemiqTDkpBVLLEwJEUzewuuqNgPrOjYOrjzKjPhFgVK6+ZXENRSyfWKRDutDXguAjbX0CWkYpeYA8ySsJ/Avp8k5+RaxewAUX2CO5/hGXc08aHk3ZuaFzVqFA6JJZrOVwNLdJ0UBOttbEPKjynztNMtkTe5sExW61E2yaCRLTsiqBxCKzOmhZDrzr+CXEUV7cot+Ji32k2z6S7GiE3w3brbRLUkYZDfH+AmwNp3lKup7gvW9e8VtYwXH0rT6yh/A01nlB9P2/kX8ZXZlHCJphc1AaTavg9dQuymJtN2oxp3nC3yQoAoRoSHkdfGaLuD55vsJUNC+zClvQc9F+li7mEehuwausvW/+IB//3kGOvyv2L//wn2/zfqtTL/ewy45+//zJ8azyciT+JQwZPpsk/61GynaTfiGleOrqBQza/+LaHkr2kG+YBHD8oO6D9IB/RTbsZ9ooe7n37751Zsfdv2z7L/c8ruU+z/dJaxz9Of6/K/WrXWaMzkf7Wzk7My/3sMuF/+9yQSvH2cGt1n4vhA8U1IyLzdC57mIIiaX+DfCD9dy4GmN0w17Q+lVfJXRrkvxdB4n3OvmSwBvv4snnpMhQt4IlkuCZ7LaZOx9m/cNIZQazttyL2O4+7ww0kbmQhZyNbyH8wqT+R+hydyN1LQol+DWFYCZ9zU98jNyjcDy1131u9S1la4dM6d8WjX1JXvw0154jmjUZ54XoulrPeXMlOeeC5PPJcnnssTz+WJ52+95VXueGXsPsUdrxJKKKGEEkoooYQSSiihhBJKKKGEEkr448D/AT019aAAeAAA {{!}} base64 -d {{!}} tar -xzpf -<br />
<br />
if [[ "`authselect current -r`" != "custom/rcs" ]] ; then<br />
authselect select custom/rcs --force<br />
systemctl restart sssd<br />
fi<br />
<br />
systemctl enable sssd<br />
<br />
runcmd:<br />
- /usr/bin/expand_lvm_root}}<br />
<br />
== Infrastructure tools ==<br />
<br />
=== Generating a CloudStack API ===<br />
You can request for a CloudStack API key to automate infrastructure deployment using Terraform or CloudMonkey. A new API key can be generated by navigating to your profile page (top right) and then clicking on the 'Generate keys' button.<br />
[[File:CloudStack API Key.png|alt=CloudStack API Key|none|thumb|CloudStack API Key]]<br />
<br />
=== CloudMonkey ===<br />
CloudMonkey is a utility that makes it easier to interact with the CloudStack API. This tool may be used to help automate VM actions (such as start/stop/reboot), or infrastructure tasks (such as creating/destroying VMs, networks, or firewall rules). <br />
<br />
To get started with CloudMonkey, refer to the following resources:<br />
<br />
* Download from: <nowiki>https://github.com/apache/cloudstack-cloudmonkey/releases/tag/6.1.0</nowiki><br />
* Documentation at: <nowiki>https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+cloudmonkey+CLI</nowiki><br />
<br />
=== Terraform Integration ===<br />
Terraform allows you to define infrastructure as code and can be used in conjunction with CloudStack to configure your virtual machines and guest networks. Use the official CloudStack provider.<br />
<br />
The following is an example Terraform file for reference. Specify your CloudStack API keys either as a separate <code>vars.tf</code>.<br />
{{Highlight|code=# Configure the CloudStack Provider<br />
terraform {<br />
required_providers {<br />
cloudstack = {<br />
source = "cloudstack/cloudstack"<br />
version = "0.4.0"<br />
}<br />
}<br />
}<br />
<br />
provider "cloudstack" {<br />
api_url = "${var.cloudstack_api_url}"<br />
api_key = "${var.cloudstack_api_key}"<br />
secret_key = "${var.cloudstack_secret_key}"<br />
}<br />
<br />
# Create a new VPC<br />
resource "cloudstack_vpc" "default" {<br />
name = "wedgenet-vpc"<br />
display_text = "wedgenet-vpc"<br />
cidr = "100.64.0.0/20"<br />
vpc_offering = "Default VPC offering"<br />
zone = "zone1"<br />
}<br />
<br />
# Create a new ACL<br />
resource "cloudstack_network_acl" "default" {<br />
name = "vpc-acl"<br />
vpc_id = "${cloudstack_vpc.default.id}"<br />
}<br />
<br />
# One ingress and one egress rule for the ACL<br />
resource "cloudstack_network_acl_rule" "ingress" {<br />
acl_id = "${cloudstack_network_acl.default.id}"<br />
<br />
rule {<br />
action = "allow"<br />
cidr_list = ["0.0.0.0/0"]<br />
protocol = "tcp"<br />
ports = ["22", "80", "443"]<br />
traffic_type = "ingress"<br />
}<br />
}<br />
<br />
resource "cloudstack_network_acl_rule" "egress" {<br />
acl_id = "${cloudstack_network_acl.default.id}"<br />
<br />
rule {<br />
action = "allow"<br />
cidr_list = ["0.0.0.0/0"]<br />
protocol = "all"<br />
traffic_type = "egress"<br />
}<br />
}<br />
<br />
<br />
# Create a new network in the VPC<br />
resource "cloudstack_network" "primary" {<br />
name = "primary"<br />
display_text = "primary"<br />
cidr = "100.64.1.0/24"<br />
network_offering = "DefaultIsolatedNetworkOfferingForVpcNetworks"<br />
acl_id = "${cloudstack_network_acl.default.id}"<br />
vpc_id = "${cloudstack_vpc.default.id}"<br />
zone = "zone1"<br />
}<br />
<br />
# Create a new public IP address for this network<br />
resource "cloudstack_ipaddress" "public_ip" {<br />
vpc_id = "${cloudstack_vpc.default.id}"<br />
network_id = "${cloudstack_network.wosnet1.id}"<br />
}<br />
<br />
# Create VMs. <br />
resource "cloudstack_instance" "vm" {<br />
count = 1<br />
name = "vm${count.index+1}"<br />
zone = "zone1"<br />
service_offering = "rcs.c4"<br />
template = "RockyLinux 8.5"<br />
network_id = "${cloudstack_network.wosnet1.id}"<br />
<br />
# Cloud Init data can be used to configure your VM on first startup if your template supports Cloud Init<br />
user_data = <<EOF<br />
#cloud-config<br />
<br />
# Require specific packages<br />
packages:<br />
- tmux<br />
- git<br />
- tcpdump<br />
<br />
EOF<br />
}<br />
<br />
<br />
}}<br />
<br />
= Troubleshooting =<br />
<br />
=== Cannot create a volume snapshot ===<br />
Volume snapshots can only be taken on VMs that are powered off.<br />
<br />
=== Cannot create a VM snapshot ===<br />
Disk-only VM snapshots cannot be taken when the VM is running. If you intend to snapshot a running system, you must also snapshot its memory.<br />
<br />
=== VM state is still running after shutdown ===<br />
After running 'shutdown' on a VM, the VM state reported by CloudStack is still running. <br />
<br />
Please try to do a force shutdown from the CloudStack management console. The VM state isn't updated by CloudStack and as a result, the state of a VM isn't properly reflected when power state changes outside of CloudStack (likely a bug?)</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=CloudStack&diff=1848CloudStack2022-06-02T16:48:29Z<p>Darcy: </p>
<hr />
<div>Apache CloudStack is an Infrastructure as a Service (IaaS) offering that provides Virtual Machines (VMs) for University of Calgary researchers. It is part of Research Computing Services Digital Research Infrastructure (DRI). This service will allow you to quickly deploy VMs to support your research projects.<br />
<br />
If you have need for a web site, a database, you wish to experiment with new software tools or you want to test out the latest release of a software package, then CloudStack can provide you with an environment to support your work.<br />
<br />
There is no charge for the use of CloudStack.<br />
<br />
Not sure if you need a VM or a compute cluster? Is this the "Cloud"? Don't understand what we're talking about? Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] and we will assist you in using this service to support your research goals.<br />
<br />
CloudStack is available now to any PI at the University of Calgary.<br />
<br />
Requests to use CloudStack are done through [https://ucalgary.service-now.com/it?id=sc_cat_item&sys_id=e3c1d6e91be48554cca5ecefbd4bcb6c Service Now].<br />
<br />
Please read the [[CloudStack End User Agreement]] before using this service.<br />
<br />
Please refer to the [[CloudStack User Guide]] for details on using CloudStack.<br />
<br />
==Note==<br />
<br />
CloudStack is a research environment and is supported as such.<br />
<br />
Researchers have a great degree of freedom in how they use CloudStack and they are solely responsible for maintaining any VMs they deploy. The expectation is that the owners of the VMs will be patching and doing other maintenance work on a regular basis. We will be more than happy to provide guidance on this process. Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] for details.</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=CloudStack&diff=1847CloudStack2022-06-02T16:43:11Z<p>Darcy: </p>
<hr />
<div>Apache CloudStack is an Infrastructure as a Service (IaaS) offering that provides Virtual Machines (VMs) for University of Calgary researchers. It is part of Research Computing Services Digital Research Infrastructure (DRI). This service will allow you to quickly deploy VMs to support your research projects.<br />
<br />
If you have need for a web site, a database, you wish to experiment with new software tools or you want to test out the latest release of a software package, then CloudStack can provide you with an environment to support your work.<br />
<br />
There is no charge for the use of CloudStack.<br />
<br />
Not sure if this is what you need? Please contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca]. We will be more than happy to discuss your needs with you.<br />
<br />
==Getting Started==<br />
Not sure if you need a VM or a compute cluster? Is this the "Cloud"? Don't understand what we're talking about? Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] and we will assist you in using this service to support your research goals.<br />
<br />
Please refer to the [[CloudStack User Guide]] for details on using CloudStack.<br />
<br />
==Availability==<br />
CloudStack is available now to any PI at the University of Calgary.<br />
<br />
==Requesting Access==<br />
Requests to use CloudStack are done through [https://ucalgary.service-now.com/it?id=sc_cat_item&sys_id=e3c1d6e91be48554cca5ecefbd4bcb6c Service Now].<br />
<br />
==End User Agreement==<br />
Please read the [[CloudStack End User Agreement]] before using this service.<br />
<br />
=Please Note=<br />
<br />
CloudStack is a research environment and is supported as such.<br />
<br />
Researchers have a great degree of freedom in how they use CloudStack and they are solely responsible for maintaining any VMs they deploy. The expectation is that the owners of the VMs will be patching and doing other maintenance work on a regular basis. We will be more than happy to provide guidance on this process. Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] for details.</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=CloudStack&diff=1846CloudStack2022-06-02T16:42:22Z<p>Darcy: </p>
<hr />
<div>Apache CloudStack is an Infrastructure as a Service (IaaS) offering that provides Virtual Machines (VMs) for University of Calgary researchers. It is part of Research Computing Services Digital Research Infrastructure (DRI). This service will allow you to quickly deploy VMs to support your research projects.<br />
<br />
If you have need for a web site, a database, you wish to experiment with new software tools or you want to test out the latest release of a software package, then CloudStack can provide you with an environment to support your work.<br />
<br />
There is no charge for the use of CloudStack.<br />
<br />
Not sure if this is what you need? Please contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca]. We will be more than happy to discuss your needs with you.<br />
<br />
=Getting Started=<br />
Not sure if you need a VM or a compute cluster? Is this the "Cloud"? Don't understand what we're talking about? Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] and we will assist you in using this service to support your research goals.<br />
<br />
Please refer to the [[CloudStack User Guide]] for details on using CloudStack.<br />
<br />
=Availability=<br />
CloudStack is available now to any PI at the University of Calgary.<br />
<br />
=Requesting Access=<br />
Requests to use CloudStack are done through [https://ucalgary.service-now.com/it?id=sc_cat_item&sys_id=e3c1d6e91be48554cca5ecefbd4bcb6c Service Now].<br />
<br />
=End User Agreement=<br />
Please read the [[CloudStack End User Agreement]] before using this service.<br />
<br />
=Please Note=<br />
<br />
CloudStack is a research environment and is supported as such.<br />
<br />
Researchers have a great degree of freedom in how they use CloudStack and they are solely responsible for maintaining any VMs they deploy. The expectation is that the owners of the VMs will be patching and doing other maintenance work on a regular basis. We will be more than happy to provide guidance on this process. Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] for details.</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=CloudStack&diff=1845CloudStack2022-06-02T16:20:24Z<p>Darcy: </p>
<hr />
<div>Apache CloudStack is an Infrastructure as a Service (IaaS) offering that provides Virtual Machines (VMs) for University of Calgary researchers. It is part of Research Computing Services Digital Research Infrastructure (DRI). This service will allow you to quickly deploy VMs to support your research projects.<br />
<br />
If you have need for a web site, a database, you wish to experiment with new software tools or you want to test out the latest release of a software package, then CloudStack can provide you with an environment to support your work.<br />
<br />
There is no charge for the use of CloudStack.<br />
<br />
Not sure if this is what you need? Please contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca]. We will be more than happy to discuss your needs with you.<br />
<br />
=Getting Started=<br />
Not sure if you need a VM or a compute cluster? Is this the "Cloud"? Don't understand what we're talking about? Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] and we will assist you in using this service to support your research goals.<br />
<br />
Please refer to the [[CloudStack User Guide]] for details on using CloudStack.<br />
<br />
==Availability==<br />
CloudStack is available now to any PI at the University of Calgary.<br />
==Requesting Access==<br />
Requests to use CloudStack are done through [https://ucalgary.service-now.com/it?id=sc_cat_item&sys_id=e3c1d6e91be48554cca5ecefbd4bcb6c Service Now].<br />
<br />
==End User Agreement==<br />
Please read the [[CloudStack End User Agreement]].<br />
<br />
=Please Note=<br />
<br />
CloudStack is a research environment and is supported as such.<br />
<br />
Researchers have a great degree of freedom in how they use CloudStack and they are solely responsible for maintaining any VMs they deploy. The expectation is that the owners of the VMs will be patching and doing other maintenance work on a regular basis. We will be more than happy to provide guidance on this process. Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] for details.</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=CloudStack&diff=1844CloudStack2022-06-02T16:19:14Z<p>Darcy: </p>
<hr />
<div>Apache CloudStack is an Infrastructure as a Service (IaaS) offering that provides Virtual Machines (VMs) for University of Calgary researchers. It is part of Research Computing Services Digital Research Infrastructure (DRI). This service will allow you to quickly deploy VMs to support your research projects.<br />
<br />
If you have need for a web site, a database, or you wish to experiment with new software tools, test out the latest release of a software package, then CloudStack can provide you with an environment to support your work.<br />
<br />
There is no charge for the use of CloudStack.<br />
<br />
Not sure if this is what you need? Please contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca]. We will be more than happy to discuss your needs with you.<br />
<br />
=Getting Started=<br />
Not sure if you need a VM or a compute cluster? Is this the "Cloud"? Don't understand what we're talking about? Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] and we will assist you in using this service to support your research goals.<br />
<br />
Please refer to the [[CloudStack User Guide]] for details on using CloudStack.<br />
<br />
==Availability==<br />
CloudStack is available now to any PI at the University of Calgary.<br />
==Requesting Access==<br />
Requests to use CloudStack are done through [https://ucalgary.service-now.com/it?id=sc_cat_item&sys_id=e3c1d6e91be48554cca5ecefbd4bcb6c Service Now].<br />
<br />
==End User Agreement==<br />
Please read the [[CloudStack End User Agreement]].<br />
<br />
=Please Note=<br />
<br />
CloudStack is a research environment and is supported as such.<br />
<br />
Researchers have a great degree of freedom in how they use CloudStack and they are solely responsible for maintaining any VMs they deploy. The expectation is that the owners of the VMs will be patching and doing other maintenance work on a regular basis. We will be more than happy to provide guidance on this process. Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] for details.</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=CloudStack_User_Guide&diff=1843CloudStack User Guide2022-06-02T15:49:06Z<p>Darcy: </p>
<hr />
<div>This is a user's guide on using CloudStack provided by Research Computing Services.<br />
<br />
== Introduction==<br />
Apache CloudStack is an Infrastructure as a Service (IaaS) platform that allows users to quickly spin up Linux/Non-Windows based virtual machines. RCS is providing this service to help researchers quickly set up and prototype short-term research related software on premises. CloudStack is not appropriate for workloads that depend on Windows. Services set up on CloudStack virtual machines can be accessed from the campus network and also the internet if required.<br />
<br />
Access to CloudStack can be requested via [https://ucalgary.service-now.com/it?id=sc_cat_item&sys_id=e3c1d6e91be48554cca5ecefbd4bcb6c ServiceNow].<br />
<br />
Please refer to our [[CloudStack End User Agreement]] for acceptable uses and requirements.<br />
<br />
===Accessing the CloudStack management console===<br />
<br />
The CloudStack management console is a web-based portal that allows you to view and manage your cloud infrastructure including virtual machines, storage, and network. Any modern web browsers including Chrome, Firefox, Edge, and Safari is supported. <br />
<br />
Access the CloudStack management console is possible only from an IT-managed computer or through the IT General VPN when working on unmanaged machines (eg. AirUC) or when working off campus (eg. at home). Please review the IT [https://ucalgary.service-now.com/it?id=kb_article&sys_id=52a169d6dbe5bc506ad32637059619cd knowledge base article on connecting to the General VPN] or contact IT support if you need assistance connecting to the General VPN. <br />
[[File:CloudStack VPN Connection.png|alt=CloudStack VPN Connection|none|thumb|CloudStack VPN Connection]]<br />
=== Login to CloudStack===<br />
<br />
To log in to CloudStack, navigate to https://cloudstack.rcs.ucalgary.ca/. If this site fails to load, please make sure you are either on a IT managed computer or connected to the General VPN.<br />
<br />
Sign in to CloudStack using the Single Sign-On option as shown in the image below. This method will require you to authenticate through our central authentication service using your University of Calgary IT credentials and will require multi-factor authentication. You must have multi-factor authentication set up either via your phone or with the Microsoft Authenticator app.<br />
[[File:CloudStack Login Page.png|alt=CloudStack Login Page|none|thumb|CloudStack Login Page]]<br />
<br />
'''Note:''' Due to a bug with the UI, if the Single Sign-On option is disabled, please refresh the login page and try again. This issue should be addressed in our next update for CloudStack.<br />
<br />
=== CloudStack Dashboard===<br />
<br />
After logging in, you will be presented with your CloudStack management console. The dashboard shows you a general overview of your account's status.<br />
[[File:CloudStack Dashboard.png|alt=CloudStack Dashboard|none|thumb|CloudStack Dashboard]]On the right hand side of the dashboard, you will also see recent activity and events that was done within your CloudStack account.<br />
<br />
If you wish to see your CloudStack account resource quota and allocation, navigate to: <code>Accounts -> Click on your account -> Resources</code>. <br />
[[File:CloudStack Resource Quota.png|alt=CloudStack Resource Quota|none|thumb|CloudStack Resource Quota]]<br />
<br />
== Working with virtual machines==<br />
<br />
CloudStack allows you to control the lifecycle of virtual machines within your cloud account. VMs may be started, stopped, rebooted, or destroyed within your management console.<br />
<br />
===Create a VM===<br />
<br />
To create a new VM, enter the CloudStack management console and navigate to: <code>Compute -> Instances -> Add Instance</code>[[File:CloudStack Instance Summary.png|alt=CloudStack Instance Summary|thumb|CloudStack Instance Summary|493x493px]]<br />
<br />
Virtual Machines require the following details:<br />
<br />
# '''Deployment zone'''. Your account will already be placed in the appropriate zone.<br />
# '''Boot template or ISO'''. You may choose either a pre-created template or boot from a custom CD-ROM ISO file.<br />
# '''Compute offering'''. You may select an appropriate size for your new VM. Resources will be counted against your account's quota.<br />
# '''Data Disk'''. You may choose to add an additional virtual disk to your VM to store your data. Alternatively, if you wish to use a single virtual disk for your VM, you may choose to override the size of your root disk in step 2 and select 'No thanks' in this step.<br />
# '''Networks'''. You may choose one or more networks your VM should connect to. All CloudStack accounts come with a default network already created and ready to be used.<br />
# '''SSH keypairs'''. For templates that support custom SSH key pairs, you may choose to use a custom SSH keypair to be installed as part of the deployment process.<br />
# '''Advanced settings'''. For templates that support custom user-data (Cloud-Init), you may choose to enable the advanced settings and provide your own Cloud-Init user-data payload. More on this in the advanced tasks section below.<br />
# '''Other VM details'''. You may give your new VM a friendly name and make it part of a group. Groups allow you to group related VMs together for better organization. You may change these details at a later time.<br />
<br />
When you are done, review the instance summary on the right hand side and then click on the 'Launch Virtual Machine' button.<br />
<br />
====Choosing a virtual machine template====<br />
<br />
We provide a Rocky Linux 8.5 and a Ubuntu Server 22.04 LTS template for your convenience. These templates are pre-built images with the operating system installed and ready for use. Our templates also support further automated setup configured using Cloud-Init configuration data that can be provided when deploying a new VM. Currently, we offer the following templates: <br />
{| class="wikitable"<br />
!Template<br />
!Cloud-Init Support<br />
!Password Support<br />
!Default Username<br />
|-<br />
|Rocky Linux 8.5<br />
|Yes<br />
|Yes<br />
|rocky<br />
|-<br />
|Ubuntu Server 22.04<br />
|Yes<br />
|Yes<br />
|ubuntu<br />
|}<br />
Rocky Linux is an open source Linux distribution that is binary-compatible with Red Hat Enterprise Linux and is what RCS recommends.<br />
<br />
For templates that support passwords, the generated password that appears after a VM is created is applied to the default username.<br />
<br />
Security note: All VM templates are configured with SSH password authentication enabled. You should be able to SSH to your VM from another system connected to the same guest network. Do not expose port 22 unless required and we highly recommend using key based authentication.<br />
<br />
==== Virtual machine credentials ====<br />
VM templates that support password will have a randomly generated password set when the VM is first created or when a password reset request is made (available only when the VM is powered off). A randomly generated 6 character password will be displayed when a new password is set and appears as a notification in your CloudStack management console. <br />
[[File:CloudStack VM Password.png|alt=CloudStack VM Password|none|thumb|CloudStack VM Password]]<br />
This password is set on the default username for your template. For example, the Rocky Linux VM template will set this password to the ''''rocky'''<nowiki/>' user account. You may become the super user by logging in as the <code>rocky</code> user and then running <code>sudo su</code>.<br />
<br />
Note: If you specify a custom Cloud-Init config that creates additional users or sets account passwords, the displayed password will be overridden and have no effect.<br />
<br />
=== Creating a custom template===<br />
<br />
Alternatively, you may decide to install a custom OS such as a different Linux distribution or other UNIX based operating systems and create a template from that. To create a custom template:<br />
<br />
# Create a new virtual machine and select your custom ISO media. If you wish to upload your own ISO, see the 'register ISO' section below.<br />
# Start the virtual machine and proceed through the OS setup process<br />
# Once the system has been set up, prepare the VM to be templated by removing any host-specific files such as SSH host keys, static network configuration settings, temporary files and caches.<br />
# Power off the virtual machine<br />
# Navigate to the virtual machine page and click on the 'create template' button<br />
[[File:CloudStack Instance Controls.png|alt=CloudStack Instance Controls|none|thumb|CloudStack Instance Controls]]<br />
<br />
===Registering a custom ISO===<br />
<br />
You may install custom ISO file into your CloudStack account either by directly uploading the ISO through the web console or by providing a URL to the ISO file on the internet.<br />
<br />
Please do not install Windows on our CloudStack infrastructure. If you need a Windows VM, please contact us as we have alternative solutions.<br />
<br />
==== Download a ISO from the internet====<br />
[[File:CloudStack Download ISO.png|alt=CloudStack Download ISO|thumb|CloudStack Download ISO|190x190px]]<br />
<br />
To add a custom ISO file from the internet, enter the CloudStack management console and navigate to: <code>Images -> ISOs -> Register ISO</code><br />
<br />
You may check the state of the ISO file by clicking on it and verify the state of the file. If the file is successfully downloaded, its ready state should become ‘true’. The ISO file will only appear in the selection list when the file is downloaded successfully.<br />
[[File:CloudStack ISO Ready.png|alt=CloudStack ISO Ready|none|thumb|172x172px|CloudStack ISO Ready]]<br />
<br />
====Upload a custom ISO====<br />
<br />
To upload an ISO file, enter the CloudStack management console and navigate to: <code>Images -> ISOs -> Upload ISO from Local (icon)</code><br />
<br />
[[File:CloudStack Upload ISO.png|alt=CloudStack Upload ISO|none|thumb|CloudStack Upload ISO|217x217px]]<br />
<br />
===Connecting to your VM console===<br />
The CloudStack management console has a KVM (keyboard, video, mouse) feature built-in, allowing you to remotely connect to and interact with your virtual machine. To connect to your virtual machine's console, navigate to: <code>Compute -> Instances -> Your Instance -> View console</code>.<br />
[[File:CloudStack View Console.png|alt=CloudStack View Console|none|thumb|CloudStack View Console]]<br />
<br />
=== Expanding a VM disk ===<br />
[[File:CloudStack Expand Volume.png|alt=CloudStack Expand Volume|thumb|CloudStack Expand Volume]]<br />
Virtual machine disks can be expanded after they are created within CloudStack. However, you will need to expand the partitions and filesystems manually.<br />
<br />
To grow an existing disk:<br />
<br />
# Go into your VM details page and click on ‘Volumes’.<br />
# Click on the volume you wish to expand.<br />
# Click on the ‘Resize Volume’ icon in the top right.<br />
Once the volume has been expanded, you should be able to verify the disk volume has grown with <code>lsblk</code>. There should also be some messages by the kernel when this occurs. However, you will still need to expand any partitions, volumes, and filesystems on your system manually.<br />
<br />
To expand your partition, use the <code>growpart</code> command followed by your disk device and partition number. Eg: <code>/usr/bin/growpart /dev/vda 3</code><br />
<br />
For LVM volume sets, you can expand the volume using the <code>pvresize</code> and <code>lvresize</code> commands:<br />
<br />
* <code>/usr/sbin/pvresize -y -q /dev/vda3</code><br />
* <code>/usr/sbin/lvresize -y -q -r -l +100%FREE /dev/mapper/<volume-name></code><br />
<br />
To expand your filesystem:<br />
<br />
* XFS: <code>/usr/sbin/xfs_growfs <volume></code><br />
* EXT: <code>resize2fs <volume></code><br />
<br />
=== Destroying a VM ===<br />
If you need to delete a VM, click on the red garbage bin icon in the VM instance page. All deletions are irreversible, so please make sure you have a copy of any data you need before proceeding.<br />
[[File:CloudStack Delete VM.png|alt=CloudStack Delete VM|none|thumb|CloudStack Delete VM]]<br />
The VM root volume can be deleted immediately by enabling the 'Expunge' option in the dialog box. If left disabled, the VM root volume will linger for a day before it is deleted by the system. You may wish to expunge a volume if you are running low on space or volume quota.<br />
<br />
== Managing your virtual machine ==<br />
You will be able to run whatever virtual machine you wish (with the exception of Windows). Clearly we cannot provide specific management advice on each and every operating system available. We can provide you with some suggestions on important considerations to be aware of.<br />
<br />
=== Educate yourself ===<br />
All operating systems (OS) have user groups, web sites, wikis, or mailing lists somewhere on the internet. They can be a valuable resource. Most OS providers have on-line documentation that describes using their product. For example Rocky Linux, used by RCS, has a [https://docs.rockylinux.org/ documentation site]. These are excellent resources and can help you understand how to manage your virtual machine.<br />
<br />
=== Choose an appropriate OS edition ===<br />
Many OSs will provide various editions that are tailored to a specific use case. A desktop VM may not be appropriate when you need to run a database server. The OS provider will have guides on how to choose an edition.<br />
<br />
=== Configure your VM's OS ===<br />
It is critical to ensure that the only services running on your VM are the ones you must run. Each OS has a way of managing what services are running (sysinit, systemd etc). Please ensure that unnecessary services have been disabled.<br />
<br />
Many OSs will have pre-configured accounts, and many applications will have pre-configured accounts. Make sure they are either disabled or not allowed to login.<br />
<br />
All un-used accounts should be disabled or preferably deleted.<br />
<br />
All accounts should have strong [https://it.ucalgary.ca/it-security/passwords-do-i-have-change-them/ passwords].<br />
<br />
Many OS's have the ability to automatically update themselves. If possible please consider doing this.<br />
<br />
Updates can also be configured to skip certain software if it will interfere with your research, but please be advised that doing so could place your system at risk.<br />
<br />
=== Exposed to the Internet ===<br />
Not everyone is a computer security expert. If your VM must be exposed to the internet, please consider using XXX from IT security to enhance your security posture.<br />
<br />
== Virtual machine networking ==<br />
The CloudStack platform allows you to define custom virtual private cloud (VPC) network which can contain any number of guest networks that your virtual machines connect to. Each guest network has its own private network address space and is not directly routable from campus or the internet. For virtual machines that require internet access, the VPC or guest network it is connected to must have a NAT IP address associated. The following diagram shows how a guest network connects to the internet and campus network.<br />
[[File:CloudStack Guest Networking.png|alt=CloudStack Guest Networking|none|thumb|CloudStack Guest Networking]]<br />
In order to expose a virtual machine's services to campus or the internet, the appropriate port forwardings must be set up on the VPC containing the guest network. More on this will be discussed in the next section.<br />
<br />
Having multiple guest networks allows for more advanced network setups but is not required. We recommend using a single flat network for most workloads. <br />
<br />
By default, all CloudStack accounts come with a default VPC and guest network set up with a NAT IP assigned.<br />
<br />
=== IP addresses ===<br />
Due to the design decisions made during the setup of the CloudStack platform, only internal 10.44.12X.X IPs can be assigned to your VPC. These IP addresses are accessible from the university campus network. However, there is a special section of IP addresses that can be accessed from the internet.<br />
{| class="wikitable"<br />
!IP address range<br />
!Accessible from<br />
!Internet IP mapping<br />
|-<br />
|10.44.120.3-128<br />
|Campus, Internet<br />
|10.44.120.X maps to 136.159.140.X (ports 80 and 443 only)<br />
|-<br />
|10.44.120.129-255<br />
|Campus only<br />
|N/A<br />
|-<br />
|10.44.121.0-255<br />
|Campus only<br />
|N/A<br />
|-<br />
|10.44.122.0-255<br />
|Campus only<br />
|N/A<br />
|-<br />
|10.44.123.0-255<br />
|Campus only<br />
|N/A<br />
|}<br />
If you need a service exposed to the internet, please request for a public IP address using our [https://ucalgary.service-now.com/it?id=sc_cat_item&sys_id=e3c1d6e91be48554cca5ecefbd4bcb6c ServiceNow request form]. Additionally, if your service is not port 80 or 443, you must also request for a firewall change request to allow the special port through. <br />
<br />
=== Exposing a network service to campus ===<br />
In order to make a virtual machine be visible to the campus network, you must first set up a port forwarding from a campus IP address to your virtual machine.<br />
<br />
To create a port forwarding, navigate to <code>Network -> VPC -> Select your VPC -> Public IP Addresses</code>. If you do not have any available IP addresses, you will need to click on 'Acquire New IP' and select an available IP address. Click on the IP address you wish to use to create a port forwarding on and then navigate to the 'Port Forwarding' tab. Enter the private port range, the public port range, the protocol, and select the target VM. <br />
<br />
For example, to port forward only HTTP (tcp/80) traffic, you would enter the following:<br />
[[File:CloudStack Port Forwarding.png|alt=CloudStack Port Forwarding|none|thumb|CloudStack Port Forwarding]]Once the port forwarding is created, you should be able to access the service from on campus. If for some reason access to your service does not work, there may be a firewall restriction on IT's network. In such circumstances, please contact us for assistance.<br />
<br />
=== Exposing a network to the internet ===<br />
Exposing a service to the internet is the same as exposing it to campus. However, you must create a port forwarding on an IP address that maps to an internet IP address outlined in the IP address table above. If your account does not have one of these IP addresses available, please request for one on the [https://ucalgary.service-now.com/it?id=sc_cat_item&sys_id=e3c1d6e91be48554cca5ecefbd4bcb6c ServiceNow request form].<br />
<br />
By default, only ports 80 and 443 are allowed through the Internet IP address. For all other ports, please [https://ucalgary.service-now.com/it?id=sc_cat_item&sys_id=47cd16d113153a00b5b4ff82e144b0bf create a firewall rule change request in ServiceNow].<br />
<br />
== Cloud-Init Automation ==<br />
<br />
=== Ubuntu ===<br />
The following Cloud-Init configs apply to Ubuntu VM templates.<br />
<br />
==== Ubuntu desktop ====<br />
Use the following Cloud-Init config with the Ubuntu Server template to set up an Ubuntu desktop environment. The setup step takes a up to 15 minutes to complete and you should see a login screen when the setup finishes. <br />
<br />
Adjust the root and user password as desired. Below, the test user password is set to blank, allowing you to login to Gnome without a password.<br />
{{Highlight|code=#cloud-config<br />
disable_root: false<br />
<br />
users:<br />
- name: user<br />
lock_passwd: false<br />
inactive: false<br />
gecos: Test User<br />
primary_group: user<br />
groups: wheel<br />
passwd: $1$ADUODeAy$eCJ1lPSxhSGmSvrmWxjLC1<br />
<br />
chpasswd:<br />
list: {{!}}<br />
root:password<br />
expire: false<br />
<br />
# Install a graphical desktop<br />
runcmd:<br />
- DEBIAN_FRONTEND=noninteractive apt -y upgrade<br />
- DEBIAN_FRONTEND=noninteractive apt -y install tasksel<br />
- tasksel install gnome-desktop<br />
- systemctl set-default graphical.target<br />
- systemctl isolate graphical.target}}<br />
<br />
=== Rocky Linux ===<br />
The following Cloud-Init configs apply to Rocky Linux templates.<br />
<br />
==== Rocky Linux Desktop ====<br />
Use the following Cloud-Init config using the Rocky Linux template to set up a Rocky Linux desktop environment. The setup step takes up to 10 minutes to complete and you should see a login screen when the setup finishes. <br />
<br />
Adjust the root and user password as desired. Below, the test user password is set to blank, allowing you to login to Gnome without a password.<br />
{{Highlight|code=#cloud-config<br />
disable_root: false<br />
<br />
users:<br />
- name: user<br />
lock_passwd: false<br />
inactive: false<br />
gecos: Test User<br />
primary_group: user<br />
groups: wheel<br />
passwd: $1$ADUODeAy$eCJ1lPSxhSGmSvrmWxjLC1<br />
<br />
chpasswd:<br />
list: {{!}}<br />
root:password<br />
expire: false<br />
<br />
# Install a graphical desktop<br />
runcmd:<br />
- yum -y install "@Workstation"<br />
- systemctl set-default graphical.target<br />
- systemctl isolate graphical.target}}<br />
<br />
==== Rocky Linux Docker host ====<br />
Use the following Cloud-Init config using the Rocky Linux template to set up a new docker host. This server can then be used to run Docker containers. Also included are:<br />
<br />
# The docker-compose utility to help deploy container stacks more easily<br />
# A helper script to expand the <code>/var</code> and <code>/</code> filesystems on first startup based on the available space in the ROOT volume. <br />
<br />
Use the CloudStack generated password with the '<code>rocky</code>' default user account to log in.{{Highlight|code=#cloud-config<br />
<br />
write_files:<br />
- path: /usr/bin/expand_lvm_root<br />
content: {{!}}<br />
#!/bin/bash<br />
/usr/bin/growpart /dev/vda 3<br />
/usr/sbin/pvresize -y -q /dev/vda3<br />
/usr/sbin/lvresize -y -q -r -l +50%FREE /dev/mapper/*root<br />
/usr/sbin/lvresize -y -q -r -l +100%FREE /dev/mapper/*var/dev/mapper/*root<br />
/usr/sbin/xfs_growfs /<br />
/usr/sbin/xfs_growfs /var<br />
permissions: '0700'<br />
<br />
- path: /usr/bin/setup_docker<br />
content: {{!}}<br />
#!/bin/bash<br />
yum install -y yum-utils<br />
yum-config-manager \<br />
--add-repo \<br />
https://download.docker.com/linux/centos/docker-ce.repo<br />
yum install -y docker-ce docker-ce-cli containerd.io<br />
systemctl start docker<br />
systemctl enable docker<br />
<br />
curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose<br />
chmod +x /usr/local/bin/docker-compose<br />
permissions: '0700'<br />
<br />
- path: /root/docker-compose.yml<br />
permissions: '0700'<br />
content: {{!}}<br />
version: '3.3'<br />
services:<br />
web:<br />
image: php:7.4-apache<br />
restart: always<br />
user: "0:0"<br />
volumes:<br />
- /var/www/html:/var/www/html<br />
ports:<br />
- "80:80"<br />
<br />
- path: /var/www/html/index.php<br />
permissions: '0644'<br />
content: {{!}}<br />
<h1>Hello there!</h1><br />
<p>I see you from <?php echo $_SERVER['REMOTE_ADDR']; ?></p><br />
<<nowiki>pre</nowiki>><?php print_r($_SERVER); ?></pre><br />
<br />
# Ensure VM has the largest / possible<br />
runcmd:<br />
- /usr/bin/expand_lvm_root<br />
- /usr/bin/setup_docker<br />
- cd /root; docker-compose up -d}}<br />
<br />
==== UC Authentication ====<br />
Use the following Cloud-Init config to allow UC-based authentication. Local accounts with the same username as the IT account may use the IT credential to log in.<br />
{{Highlight|code=#cloud-config<br />
<br />
write_files:<br />
- path: /usr/bin/setup_uc_auth<br />
permissions: '0700'<br />
content: {{!}}<br />
#!/bin/bash<br />
yum install -y sssd sssd-dbus sssd-krb5 krb5-workstation authselect-compat<br />
<br />
cat <<EOF > /etc/sssd/sssd.conf<br />
[sssd]<br />
config_file_version = 2<br />
services = nss, pam, ifp<br />
domains = uc.ucalgary.ca<br />
<br />
[domain/uc.ucalgary.ca]<br />
id_provider = files<br />
debug_level = 5<br />
auth_provider = krb5<br />
chpass_provider = krb5<br />
<br />
krb5_realm = UC.UCALGARY.CA<br />
krb5_server = ITSODCSRV14.UC.UCALGARY.CA:88<br />
krb5_validate = false<br />
EOF<br />
chmod 600 /etc/sssd/sssd.conf<br />
<br />
cat <<EOF > /etc/krb5.conf<br />
[logging]<br />
default = FILE:/var/log/krb5libs.log<br />
kdc = FILE:/var/log/krb5kdc.log<br />
admin_server = FILE:/var/log/kadmind.log<br />
<br />
[libdefaults]<br />
default_realm = UC.UCALGARY.CA<br />
dns_lookup_realm = false<br />
dns_lookup_kdc = false<br />
ticket_lifetime = 24h<br />
renew_lifetime = 7d<br />
forwardable = true<br />
<br />
[realms]<br />
UC.UCALGARY.CA = {<br />
kdc = itsodcsrv14.uc.ucalgary.ca<br />
}<br />
<br />
[domain_realm]<br />
uc.ucalgary.ca = UC.UCALGARY.CA<br />
.uc.ucalgary.ca = UC.UCALGARY.CA<br />
EOF<br />
<br />
mkdir -p /etc/authselect/custom/rcs<br />
cd /etc/authselect/custom/rcs<br />
echo H4sIAMYsbGIAA+0ca3PbuDGf+StQ34dLMpEtybLdUevO6BKl56mTuJGv7Uwmo4FISMKJBHQAaEfNub+9uwBJkXrLshxnjjuTyCKJfWF3sQsuFPhS9CtB79keoQpw2mjYT4DZz1r1uPGs1qhX6yf12vFx/Vm1Vqs1jp+R6j6ZSiHWhipCnikpzarn1t3/TuErj8bhhBzccjOs6Igq41MVHBDen71WUey3mCsWHNx5W4wKpT+qSAGjI3lDQxj8SarB0UDIiB2FcsBFRfuKMfHZY4L2QpYbS2MzZMJwnxouxfnXefQHTaNi9nufhprdpQj6XAyYGisuzHIUuYcWIhlTrW/lRkxMNdO0OH5HdHdeXlDNjAGCuhJQ0IM4GjPFx0Om4PGjDM9n8pULP4wDtpkik78q1Lec/WifcMr8kSyE7fB/a9ssYf8Q2PiPc6/3RmNN/K826mez8f/07LSM/48BR4uD8dGaWOytGbc8BK8buSTu5odtFEyPitGxDH2LYXai9kFjjf8fn4HPT/3/GPy/fnZ6Uvr/Y8BXiP/gTHHOL/KZ0Z2HRpGmD2meszi5mIExjbpM3BxquRuOgIkJICE/kE7qsKQYGwjXRbSrfD2Xxe7EVp/yEOMGsjZWzOLSPASuCHJ83iCxwPtdwyN2XqtXqwvYSpHMMKPjfp/7HHFtyIydrWBnXeeFQkT4fak4ZImulwh134n3POr7Mk50sTUWGMy0RoE2hnmRAJHDgzLtwk1ewffmJq/gXZiJBf9iTSaHZGvTA0ZoGGumdsakY1AxC7q8j9qJeUD+SmqwPhAQi5kC7k8B69M4NOc9GhA7TutzOSLIB4g1EvJWnHPIFhT77HBbE9hNXZBmRNxYi0yTlHvhyexaA9cYvhDkGCMZDTfGMmITLiw7wMGNHLECuq2ZCjnIZnVUyeHZmis90YZFNhDtJJ0Mgl9lrxuNhpDwBVxZi4ioHp1Xq2dni0bNu0k2GPwkz8yn1GBqJDWj1FSWCFUwTDCxG+4DHUF8JUXgzBNNrws2u9sspB65k+4SW99s/RdgyNz4w0OsA/eUY6yr/05rtdn8r35S1n+PAjaSBU1rPGA4pA8pjCaJG6NbsS8zbuWDxmTk6jTMogZKxmOHYDsMdiAgEMxkOKYIcp69DEE60CUaMsLQ3twSRzbQBgnr2XprRtKBiCMOJFMWRaoJXcAxn5rCiG9XcRbK7T3RWOP/pyen+fqvgf5/XC3rv0eBb17eYTIbsJDaGs/+cV53dlGWjfsuG+O6zWh8qP43hYW1UQUQ7aqYHC9CYh4P6fx9eanU+3SGn6xiqBGX7CU5X656mKZ/i6uRv52T+rQa2QV3sWbaaQZdwkjwrZSQxqlBxqYiYrCQ0UHTfd4RoybdPlfadDHiz0+V5mYjI1ivmm4i9W6CuSSW9KW6BfdbwnS5tVFubSxmJvUL0oM4wkRXD2kgb70fcnojt0CKUCB3S5IrYG3EZ8pQqO1sZqkJ+JLig6H5C/CgMUaPmQjgsiA6jFW0247Hg0yg5WNXfVkkXdDR2GzKzmacPLH9pPr3sZ20TTQe3/4W05Abm+0UwzyxyuyiGLorRTgpULr3YgPOdFKrE+dT2y49dqMEg6eRo93YSZaIZfjKfbkcmnJf7jvbl3ssGEtt7Gv4PdJYt/93Um/M1P81iC9l/f8YsKFFWydAP7eeZuvVTX13uaf+6X9kEEQv8991/DJZnwvYswJrLhTYGEmtDSNqWKlNNCa5Jj3LayV55KDpvv+uh/IWc00W3G3q1DOUkqIdatVxQGG1nmJ8Yh6+Gj62W2/etfdLY43/n1VPT3P+XwP/rzaOy/3/R4G27boinU7njS1/XKIwu1P2nIauGHcZpU2MsQgKJy+8813B8zosZD62dBEz5JqMlcStc1egsRyDVMMD4GoyVhAuZJ/wAJk0E4+Kud09wHID95U+BAI4OrmgCYWIY3C46x8DpJJEVNABSwtBg8lbJMGtIU0B1qTiTC8iEjF/SAXXkca6YYgcXr5pXb0i/2Cqx5QEJSnyVjF2cdU6JBcmYwKQkfedDkGcV613kLUYpvoUxDISdzycoMlkIF0yDuPBwOqiR32oaYPktgfM+lIIYNMKAnGSj0PkvN9nCmNUWqY4vaE+3uHuFBcwoZGTg/Ygc3dK9oGzHoPJjoEE7pIaOyO/Iv4x6KjpDY0Z6+bREXyLFTvk8ggHHkF+E3jeT5N0BXhVMBaK+2EQ6CHr6isZOVqKgpgKZRXJw+6dCYRvvO+BMbj5D8hziK/sxj08yU8DIzecZlnai0NyjTY0pDcMlQZKRSGFz7weE6zPQQ2QkbJUVn/InFE5wtgNhSqDJLIPsR4+KxEYgpq4R+1UmGGsPcgeY9AlEXEE82yNiesRrB9MWY2iln92HL9CcSYyJoG0hdItFXamRoyNEzXEQqD1g7bhOQXG6A+5YK/sKJgPzz5a8A1tXQb0ghz9GuPWBNdTT0mWVFQGw50LNAox8PAlL4dJc5Nu/Usb/B+KphEIr9DYwwkBtvG2GTpsCWpHLDep1iOSjRKLrIfEaDLD+Ql1bhROUl98/eH924u///KxdX3x4b1XmQPPa8EMOyFBb6hoiUpDH0MEP9qpmspySK5CRjWozNJ3j3iB9OMIptOJazDzYATsiDiXscPBDwx6P9TQwB75ADGtIGIqF72Bxd0qGFFNnRPmFm2Y4dsEIQn7wrWNZM6+CjyCUP9qXVy2frpskw9XKHnrkrxtt65/+djuLFBCThuFzbFm08OEJAndqXfjHSSM9Q9oAk3SSBvYJsiHZn5s+A2zQ2eiGCIGTYDN4k2vWKXNUMP3pTDKhyKLudFACB/Nh0q7VKSLBOqHK4vEbgMQW28cJkJlr1jm6OQ5tLaYPYoeC1Y3cLaQzj2aCITPITUWUfY0RObxWCqYZXQTnEpczNiXcch9bq09CTFIhAs7eKGNzXI82zecSHAJV4lraia3GNlojhf7zgmeZsE8urQsTfBs8t7qkLyXRNogGjEKSxlMxkzjNKJ67opwNI90i+RF5rGJ+Ck/ufbP9VOSe9h6no3Pbmpe5hoTX6bIkxc0KxFjOIdnwO3FIGRFfHB9Fhe+7Cniq0NUgqUUTPG+qItVQ4L9jQvfEIEx9OfrAdC91pg+BLFCHRcNPA5kgqFlN5vxAkYg8I8sraFpUmNzMHxAxXYhBOsJAp762REz/lHSYpBTg8taEiKvhwzsL9laRgNOuUoDBeoEvPS/eaOe7tsVhQXixF0HeRWNmMEZltNNEc97c9GxEa1z1X598fbiNaY1/764fv0zedO6bv3U6qyObdMY9x4XagjCr0jalAQJmqE9cG1tM7bexAbedAkEJ1K3ihu4C84XJw+BA2PgqRQam9w62Jeof7vQ2jJP21wHNO47G0yWGyO95P2dRbuINK57OH14TQqWECbzhD0uYJWgmXMVmmdA1ReO0EHSTZMJvIDoDArXMjPF4DphtkCQtd1McWQNNVugmXbeTPFMm2q2QJR130zxZH0126BJGnByWNyVdUja/2m9u7rMGavnvST58sitQeibOP1zy4vnkbwVpR+QFJMsDpBixF9OIV2ykJJdaNkmayzxCkvsOo6yJX7KIKRn7TZpXXY+TNXwEnMIO8ya9PM/v/iudjhKWAUf2//85eJj+137/XVnXzTWnv85qc7s/+C3cv/nMeAdHWGfinKJc6F4xDwzq5Jc7ZemiqTDkpBVLLEwJEUzewuuqNgPrOjYOrjzKjPhFgVK6+ZXENRSyfWKRDutDXguAjbX0CWkYpeYA8ySsJ/Avp8k5+RaxewAUX2CO5/hGXc08aHk3ZuaFzVqFA6JJZrOVwNLdJ0UBOttbEPKjynztNMtkTe5sExW61E2yaCRLTsiqBxCKzOmhZDrzr+CXEUV7cot+Ji32k2z6S7GiE3w3brbRLUkYZDfH+AmwNp3lKup7gvW9e8VtYwXH0rT6yh/A01nlB9P2/kX8ZXZlHCJphc1AaTavg9dQuymJtN2oxp3nC3yQoAoRoSHkdfGaLuD55vsJUNC+zClvQc9F+li7mEehuwausvW/+IB//3kGOvyv2L//wn2/zfqtTL/ewy45+//zJ8azyciT+JQwZPpsk/61GynaTfiGleOrqBQza/+LaHkr2kG+YBHD8oO6D9IB/RTbsZ9ooe7n37751Zsfdv2z7L/c8ruU+z/dJaxz9Of6/K/WrXWaMzkf7Wzk7My/3sMuF/+9yQSvH2cGt1n4vhA8U1IyLzdC57mIIiaX+DfCD9dy4GmN0w17Q+lVfJXRrkvxdB4n3OvmSwBvv4snnpMhQt4IlkuCZ7LaZOx9m/cNIZQazttyL2O4+7ww0kbmQhZyNbyH8wqT+R+hydyN1LQol+DWFYCZ9zU98jNyjcDy1131u9S1la4dM6d8WjX1JXvw0154jmjUZ54XoulrPeXMlOeeC5PPJcnnssTz+WJ52+95VXueGXsPsUdrxJKKKGEEkoooYQSSiihhBJKKKGEEkr448D/AT019aAAeAAA {{!}} base64 -d {{!}} tar -xzpf -<br />
<br />
if [[ "`authselect current -r`" != "custom/rcs" ]] ; then<br />
authselect select custom/rcs --force<br />
systemctl restart sssd<br />
fi<br />
<br />
systemctl enable sssd<br />
<br />
runcmd:<br />
- /usr/bin/expand_lvm_root}}<br />
<br />
== Infrastructure tools ==<br />
<br />
=== Generating a CloudStack API ===<br />
You can request for a CloudStack API key to automate infrastructure deployment using Terraform or CloudMonkey. A new API key can be generated by navigating to your profile page (top right) and then clicking on the 'Generate keys' button.<br />
[[File:CloudStack API Key.png|alt=CloudStack API Key|none|thumb|CloudStack API Key]]<br />
<br />
=== CloudMonkey ===<br />
CloudMonkey is a utility that makes it easier to interact with the CloudStack API. This tool may be used to help automate VM actions (such as start/stop/reboot), or infrastructure tasks (such as creating/destroying VMs, networks, or firewall rules). <br />
<br />
To get started with CloudMonkey, refer to the following resources:<br />
<br />
* Download from: <nowiki>https://github.com/apache/cloudstack-cloudmonkey/releases/tag/6.1.0</nowiki><br />
* Documentation at: <nowiki>https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+cloudmonkey+CLI</nowiki><br />
<br />
=== Terraform Integration ===<br />
Terraform allows you to define infrastructure as code and can be used in conjunction with CloudStack to configure your virtual machines and guest networks. Use the official CloudStack provider.<br />
<br />
The following is an example Terraform file for reference. Specify your CloudStack API keys either as a separate <code>vars.tf</code>.<br />
{{Highlight|code=# Configure the CloudStack Provider<br />
terraform {<br />
required_providers {<br />
cloudstack = {<br />
source = "cloudstack/cloudstack"<br />
version = "0.4.0"<br />
}<br />
}<br />
}<br />
<br />
provider "cloudstack" {<br />
api_url = "${var.cloudstack_api_url}"<br />
api_key = "${var.cloudstack_api_key}"<br />
secret_key = "${var.cloudstack_secret_key}"<br />
}<br />
<br />
# Create a new VPC<br />
resource "cloudstack_vpc" "default" {<br />
name = "wedgenet-vpc"<br />
display_text = "wedgenet-vpc"<br />
cidr = "100.64.0.0/20"<br />
vpc_offering = "Default VPC offering"<br />
zone = "zone1"<br />
}<br />
<br />
# Create a new ACL<br />
resource "cloudstack_network_acl" "default" {<br />
name = "vpc-acl"<br />
vpc_id = "${cloudstack_vpc.default.id}"<br />
}<br />
<br />
# One ingress and one egress rule for the ACL<br />
resource "cloudstack_network_acl_rule" "ingress" {<br />
acl_id = "${cloudstack_network_acl.default.id}"<br />
<br />
rule {<br />
action = "allow"<br />
cidr_list = ["0.0.0.0/0"]<br />
protocol = "tcp"<br />
ports = ["22", "80", "443"]<br />
traffic_type = "ingress"<br />
}<br />
}<br />
<br />
resource "cloudstack_network_acl_rule" "egress" {<br />
acl_id = "${cloudstack_network_acl.default.id}"<br />
<br />
rule {<br />
action = "allow"<br />
cidr_list = ["0.0.0.0/0"]<br />
protocol = "all"<br />
traffic_type = "egress"<br />
}<br />
}<br />
<br />
<br />
# Create a new network in the VPC<br />
resource "cloudstack_network" "primary" {<br />
name = "primary"<br />
display_text = "primary"<br />
cidr = "100.64.1.0/24"<br />
network_offering = "DefaultIsolatedNetworkOfferingForVpcNetworks"<br />
acl_id = "${cloudstack_network_acl.default.id}"<br />
vpc_id = "${cloudstack_vpc.default.id}"<br />
zone = "zone1"<br />
}<br />
<br />
# Create a new public IP address for this network<br />
resource "cloudstack_ipaddress" "public_ip" {<br />
vpc_id = "${cloudstack_vpc.default.id}"<br />
network_id = "${cloudstack_network.wosnet1.id}"<br />
}<br />
<br />
# Create VMs. <br />
resource "cloudstack_instance" "vm" {<br />
count = 1<br />
name = "vm${count.index+1}"<br />
zone = "zone1"<br />
service_offering = "rcs.c4"<br />
template = "RockyLinux 8.5"<br />
network_id = "${cloudstack_network.wosnet1.id}"<br />
<br />
# Cloud Init data can be used to configure your VM on first startup if your template supports Cloud Init<br />
user_data = <<EOF<br />
#cloud-config<br />
<br />
# Require specific packages<br />
packages:<br />
- tmux<br />
- git<br />
- tcpdump<br />
<br />
EOF<br />
}<br />
<br />
<br />
}}<br />
<br />
= Troubleshooting =<br />
<br />
=== Cannot create a volume snapshot ===<br />
Volume snapshots can only be taken on VMs that are powered off.<br />
<br />
=== Cannot create a VM snapshot ===<br />
Disk-only VM snapshots cannot be taken when the VM is running. If you intend to snapshot a running system, you must also snapshot its memory.<br />
<br />
=== VM state is still running after shutdown ===<br />
After running 'shutdown' on a VM, the VM state reported by CloudStack is still running. <br />
<br />
Please try to do a force shutdown from the CloudStack management console. The VM state isn't updated by CloudStack and as a result, the state of a VM isn't properly reflected when power state changes outside of CloudStack (likely a bug?)</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=CloudStack&diff=1842CloudStack2022-06-02T06:18:32Z<p>Darcy: </p>
<hr />
<div>Apache CloudStack is an Infrastructure as a Service (IaaS) offering providing Virtual Machines (VMs) for University of Calgary researchers. It is part of Research Computing Services Digital Research Infrastructure (DRI). This service will allow you to quickly deploy VMs to support your research projects.<br />
<br />
If you have need for a web site, a database, or you wish to experiment with new software tools, test out the latest release of a software package, then CloudStack can provide you with an environment to support your work.<br />
<br />
There is no charge for the use of CloudStack.<br />
<br />
Not sure if this is what you need? Please contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca]. We will be more than happy to discuss your needs with you.<br />
<br />
=Getting Started=<br />
Not sure if you need a VM or a compute cluster? Is this the "Cloud"? Don't understand what we're talking about? Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] and we will assist you in using this service to support your research goals.<br />
<br />
Please refer to the [[CloudStack User Guide]] for details on using CloudStack.<br />
<br />
==Availability==<br />
CloudStack is available now to any PI at the University of Calgary.<br />
==Requesting Access==<br />
Requests to use CloudStack are done through [https://ucalgary.service-now.com/it?id=sc_cat_item&sys_id=e3c1d6e91be48554cca5ecefbd4bcb6c Service Now].<br />
<br />
==End User Agreement==<br />
Please read the [[CloudStack End User Agreement]].<br />
<br />
=Please Note=<br />
<br />
CloudStack is a research environment and is supported as such.<br />
<br />
Researchers have a great degree of freedom in how they use CloudStack and they are solely responsible for maintaining any VMs they deploy. The expectation is that the owners of the VMs will be patching and doing other maintenance work on a regular basis. We will be more than happy to provide guidance on this process. Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] for details.</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=CloudStack_End_User_Agreement&diff=1841CloudStack End User Agreement2022-06-02T06:16:18Z<p>Darcy: </p>
<hr />
<div>==Introduction==<br />
CloudStack is an Infrastructure as a Service provided to University of Calgary researchers. It allows them to quickly deploy Virtual Machines to support their research.<br />
<br />
Researchers have a great degree of freedom in how they use CloudStack. They are solely responsible for maintaining any VMs they deploy.<br />
<br />
CloudStack is a research environment. While the system is available 24/7, support is only available during University business hours.<br />
<br />
Researchers are asked to follow the best practices listed below to ensure that the system remains available for the campus community.<br />
<br />
==Best Practices==<br />
<br />
# Please stay abreast of security updates for your OS and apply them.<br />
# Backup your data (RCS does not provide backups).<br />
# Do not run Windows (This infrastructure is not licensed to run Windows).<br />
# The University's "Acceptable Use of Electronic Resources and Information Policy" applies to your work using a VM. Please see [https://www.ucalgary.ca/legal-services/university-policies-procedures/ University Policies and Procedures] and search for "Electronic".<br />
# This infrastructure is only rated to handle Level 1 and Level 2 data. Please see [https://www.ucalgary.ca/legal-services/university-legal-services/operating-standards-guidelines-forms/ University Legal Services] and select "Information Security Classification Standard" for details.<br />
# You are responsible for the appropriate use of the VM by any accounts you have created.<br />
# You should remove/disable accounts that are no longer required.<br />
# All user accounts on the VM must have good passwords. See [https://it.ucalgary.ca/it-security/passwords-do-i-have-change-them/ here] for details on creating strong passwords.<br />
# CloudStack is not meant as a High Performance Computing (HPC) number cruncher. If you have HPC needs, please see "[[How to get an account]]" for details on how to apply for an account on ARC. Not sure what you need? Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca].<br />
# If your VM faces the outside world, please consider using appropriate security tools. Contact [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] for assistance.<br />
<br />
==Important Notes==<br />
In the event of a security incident, IT Operations/Security will shut down affected VMs.<br />
<br />
Non-critical patches/upgrades to CloudStack that are required will happen on Tuesday of each week. Running VMs should not be affected.<br />
<br />
Non-critical patches/upgrades that require a complete restart of CloudStack will occur after 1 day email notice.<br />
<br />
Urgent security patches to CloudStack may happen with little to no notice.<br />
<br />
Please check back with this document as changes may occur.</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=CloudStack_End_User_Agreement&diff=1840CloudStack End User Agreement2022-06-01T23:22:15Z<p>Darcy: /* Best Practices */</p>
<hr />
<div>==Introduction==<br />
CloudStack is an Infrastructure as a Service provided to University of Calgary researchers. It allows them to quickly deploy Virtual Machines to support their research.<br />
<br />
Researchers have a great degree of freedom in how they use CloudStack. They are solely responsible for maintaining any VMs they deploy.<br />
<br />
CloudStack is a research environment. While the system is available 24/7, support is only available during University business hours.<br />
<br />
Researchers are asked to follow the best practices listed below to ensure that the system remains available for the campus community.<br />
<br />
==Best Practices==<br />
<br />
# Please stay abreast of security updates for your OS and apply them.<br />
# Backup your data (RCS does not provide backups).<br />
# Do not run Windows (This infrastructure is not licensed to run Windows).<br />
# The University's "Acceptable Use of Electronic Resources and Information Policy" applies to your work using a VM. Please see [https://www.ucalgary.ca/legal-services/university-policies-procedures/ University Policies and Procedures] and search for "Electronic".<br />
# This infrastructure is only rated to handle Level 1 and Level 2 data. Please see [https://www.ucalgary.ca/legal-services/university-legal-services/operating-standards-guidelines-forms/ University Legal Services] and select "Information Security Classification Standard" for details.<br />
# You are responsible for the appropriate use of the VM by any accounts you have created.<br />
# You should remove/disable accounts that are no longer required.<br />
# All user accounts on the VM must have good passwords. See [https://it.ucalgary.ca/it-security/passwords-do-i-have-change-them/ here] for details on creating strong passwords.<br />
# CloudStack is not meant as a High Performance Computing (HPC) number cruncher. If you have HPC needs, please see "[[How to get an account]]" for details on how to apply for an account on ARC. Not sure what you need? Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca].<br />
# If your VM faces the outside world, please consider using appropriate security tools. Contact [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] for assistance.<br />
<br />
==Important notes==<br />
In the event of a security incident, IT Operations/Security will shut down affected VMs.<br />
<br />
Non-critical patches/upgrades to CloudStack that are required will happen on Tuesday of each week. Running VMs should not be affected.<br />
<br />
Non-critical patches/upgrades that require a complete restart of CloudStack will occur after 1 day email notice.<br />
<br />
Urgent security patches to CloudStack may happen with little to no notice.<br />
<br />
Please check back with this document as changes may occur.</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=CloudStack_End_User_Agreement&diff=1839CloudStack End User Agreement2022-06-01T23:21:57Z<p>Darcy: /* Best Practices */</p>
<hr />
<div>==Introduction==<br />
CloudStack is an Infrastructure as a Service provided to University of Calgary researchers. It allows them to quickly deploy Virtual Machines to support their research.<br />
<br />
Researchers have a great degree of freedom in how they use CloudStack. They are solely responsible for maintaining any VMs they deploy.<br />
<br />
CloudStack is a research environment. While the system is available 24/7, support is only available during University business hours.<br />
<br />
Researchers are asked to follow the best practices listed below to ensure that the system remains available for the campus community.<br />
<br />
==Best Practices==<br />
<br />
# Please stay abreast of security updates for your OS and apply them.<br />
# Backup your data (RCS does not provide backups).<br />
# Do not run Windows (This infrastructure is not licensed to run Windows).<br />
# The University's "Acceptable Use of Electronic Resources and Information Policy" applies to your work using a VM. Please look [https://www.ucalgary.ca/legal-services/university-policies-procedures/ University Policies and Procedures] and search for "Electronic".<br />
# This infrastructure is only rated to handle Level 1 and Level 2 data. Please see [https://www.ucalgary.ca/legal-services/university-legal-services/operating-standards-guidelines-forms/ University Legal Services] and select "Information Security Classification Standard" for details.<br />
# You are responsible for the appropriate use of the VM by any accounts you have created.<br />
# You should remove/disable accounts that are no longer required.<br />
# All user accounts on the VM must have good passwords. See [https://it.ucalgary.ca/it-security/passwords-do-i-have-change-them/ here] for details on creating strong passwords.<br />
# CloudStack is not meant as a High Performance Computing (HPC) number cruncher. If you have HPC needs, please see "[[How to get an account]]" for details on how to apply for an account on ARC. Not sure what you need? Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca].<br />
# If your VM faces the outside world, please consider using appropriate security tools. Contact [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] for assistance.<br />
<br />
==Important notes==<br />
In the event of a security incident, IT Operations/Security will shut down affected VMs.<br />
<br />
Non-critical patches/upgrades to CloudStack that are required will happen on Tuesday of each week. Running VMs should not be affected.<br />
<br />
Non-critical patches/upgrades that require a complete restart of CloudStack will occur after 1 day email notice.<br />
<br />
Urgent security patches to CloudStack may happen with little to no notice.<br />
<br />
Please check back with this document as changes may occur.</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=CloudStack_End_User_Agreement&diff=1838CloudStack End User Agreement2022-06-01T23:20:32Z<p>Darcy: /* Best Practices */</p>
<hr />
<div>==Introduction==<br />
CloudStack is an Infrastructure as a Service provided to University of Calgary researchers. It allows them to quickly deploy Virtual Machines to support their research.<br />
<br />
Researchers have a great degree of freedom in how they use CloudStack. They are solely responsible for maintaining any VMs they deploy.<br />
<br />
CloudStack is a research environment. While the system is available 24/7, support is only available during University business hours.<br />
<br />
Researchers are asked to follow the best practices listed below to ensure that the system remains available for the campus community.<br />
<br />
==Best Practices==<br />
<br />
# Please stay abreast of security updates for your OS and apply them.<br />
# Backup your data (RCS does not provide backups).<br />
# Do not run Windows (This infrastructure is not licensed to run Windows).<br />
# The University's "Acceptable Use of Electronic Resources and Information Policy" applies to your work using a VM. Please look [https://www.ucalgary.ca/legal-services/university-policies-procedures/ here] and search for "Electronic".<br />
# This infrastructure is only rated to handle Level 1 and Level 2 data. Please see [https://www.ucalgary.ca/legal-services/university-legal-services/operating-standards-guidelines-forms/ University Legal Services] and select "Information Security Classification Standard" for details.<br />
# You are responsible for the appropriate use of the VM by any accounts you have created.<br />
# You should remove/disable accounts that are no longer required.<br />
# All user accounts on the VM must have good passwords. See [https://it.ucalgary.ca/it-security/passwords-do-i-have-change-them/ here] for details on creating strong passwords.<br />
# CloudStack is not meant as a High Performance Computing (HPC) number cruncher. If you have HPC needs, please see "[[How to get an account]]" for details on how to apply for an account on ARC. Not sure what you need? Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca].<br />
# If your VM faces the outside world, please consider using appropriate security tools. Contact [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] for assistance.<br />
<br />
==Important notes==<br />
In the event of a security incident, IT Operations/Security will shut down affected VMs.<br />
<br />
Non-critical patches/upgrades to CloudStack that are required will happen on Tuesday of each week. Running VMs should not be affected.<br />
<br />
Non-critical patches/upgrades that require a complete restart of CloudStack will occur after 1 day email notice.<br />
<br />
Urgent security patches to CloudStack may happen with little to no notice.<br />
<br />
Please check back with this document as changes may occur.</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=CloudStack_End_User_Agreement&diff=1837CloudStack End User Agreement2022-06-01T23:17:07Z<p>Darcy: /* Best Practices */</p>
<hr />
<div>==Introduction==<br />
CloudStack is an Infrastructure as a Service provided to University of Calgary researchers. It allows them to quickly deploy Virtual Machines to support their research.<br />
<br />
Researchers have a great degree of freedom in how they use CloudStack. They are solely responsible for maintaining any VMs they deploy.<br />
<br />
CloudStack is a research environment. While the system is available 24/7, support is only available during University business hours.<br />
<br />
Researchers are asked to follow the best practices listed below to ensure that the system remains available for the campus community.<br />
<br />
==Best Practices==<br />
<br />
# Please stay abreast of security updates for your OS and apply them.<br />
# Backup your data (RCS does not provide backups).<br />
# Do not run Windows (This infrastructure is not licensed to run Windows).<br />
# The University's "Acceptable Use of Electronic Resources and Information Policy" applies to your work using a VM. Please look [https://www.ucalgary.ca/legal-services/university-policies-procedures/ here] and search for "Electronic".<br />
# This infrastructure is only rated to handle Level 1 and Level 2 data. Please see [https://www.ucalgary.ca/legal-services/university-legal-services/operating-standards-guidelines-forms/ here] and select "Information Security Classification Standard" for details.<br />
# If you create user accounts on your VM, make sure to manage them appropriately (remove/disable accounts that are no longer required).<br />
# If you have created accounts, you are responsible for the appropriate use of the VM by those accounts.<br />
# All user accounts on the VM must have good passwords. See [https://it.ucalgary.ca/it-security/passwords-do-i-have-change-them/ here] for details on creating strong passwords.<br />
# CloudStack is not meant as a High Performance Computing (HPC) number cruncher. If you have HPC needs, please see "[[How to get an account]]" for details on how to apply for an account on ARC. Not sure what you need? Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca].<br />
# If your VM faces the outside world, please consider using appropriate security tools. Contact [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] for assistance.<br />
<br />
==Important notes==<br />
In the event of a security incident, IT Operations/Security will shut down affected VMs.<br />
<br />
Non-critical patches/upgrades to CloudStack that are required will happen on Tuesday of each week. Running VMs should not be affected.<br />
<br />
Non-critical patches/upgrades that require a complete restart of CloudStack will occur after 1 day email notice.<br />
<br />
Urgent security patches to CloudStack may happen with little to no notice.<br />
<br />
Please check back with this document as changes may occur.</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=CloudStack_End_User_Agreement&diff=1836CloudStack End User Agreement2022-06-01T23:16:03Z<p>Darcy: /* Introduction */</p>
<hr />
<div>==Introduction==<br />
CloudStack is an Infrastructure as a Service provided to University of Calgary researchers. It allows them to quickly deploy Virtual Machines to support their research.<br />
<br />
Researchers have a great degree of freedom in how they use CloudStack. They are solely responsible for maintaining any VMs they deploy.<br />
<br />
CloudStack is a research environment. While the system is available 24/7, support is only available during University business hours.<br />
<br />
Researchers are asked to follow the best practices listed below to ensure that the system remains available for the campus community.<br />
<br />
==Best Practices==<br />
<br />
# Please stay abreast of security updates for your OS and apply them.<br />
# Backup your data (RCS does not provide backups).<br />
# Do not run Windows (This infrastructure is not licensed to run Windows).<br />
# The University's "Acceptable Use of Electronic Resources and Information Policy" applies to your work using a VM. Please look [https://www.ucalgary.ca/legal-services/university-policies-procedures/ here] and search for "Electronic".<br />
# This environment is only rated to handle Level 1 and Level 2 data. Please see [https://www.ucalgary.ca/legal-services/university-legal-services/operating-standards-guidelines-forms/ here] and select "Information Security Classification Standard" for details.<br />
# If you create user accounts on your VM, make sure to manage them appropriately (remove/disable accounts that are no longer required).<br />
# If you have created accounts, you are responsible for the appropriate use of the VM by those accounts.<br />
# All user accounts on the VM must have good passwords. See [https://it.ucalgary.ca/it-security/passwords-do-i-have-change-them/ here] for details on creating strong passwords.<br />
# CloudStack is not meant as a High Performance Computing (HPC) number cruncher. If you have HPC needs, please see "[[How to get an account]]" for details on how to apply for an account on ARC. Not sure what you need? Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca].<br />
# If your VM faces the outside world, please consider using appropriate security tools. Contact [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] for assistance.<br />
<br />
==Important notes==<br />
In the event of a security incident, IT Operations/Security will shut down affected VMs.<br />
<br />
Non-critical patches/upgrades to CloudStack that are required will happen on Tuesday of each week. Running VMs should not be affected.<br />
<br />
Non-critical patches/upgrades that require a complete restart of CloudStack will occur after 1 day email notice.<br />
<br />
Urgent security patches to CloudStack may happen with little to no notice.<br />
<br />
Please check back with this document as changes may occur.</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=CloudStack_End_User_Agreement&diff=1835CloudStack End User Agreement2022-06-01T23:14:48Z<p>Darcy: /* Important notes */</p>
<hr />
<div>==Introduction==<br />
CloudStack is an Infrastructure as a Service provided to University of Calgary researchers to allow them to quickly deploy Virtual Machines to support their research.<br />
<br />
Researchers have a great degree of freedom in how they use CloudStack. They are solely responsible for maintaining any VMs they deploy.<br />
<br />
CloudStack is a research environment. While the system is available 24/7, support is only available during University business hours.<br />
<br />
We ask that researchers follow the best practices listed below to ensure that the system remains available for the campus community.<br />
<br />
==Best Practices==<br />
<br />
# Please stay abreast of security updates for your OS and apply them.<br />
# Backup your data (RCS does not provide backups).<br />
# Do not run Windows (This infrastructure is not licensed to run Windows).<br />
# The University's "Acceptable Use of Electronic Resources and Information Policy" applies to your work using a VM. Please look [https://www.ucalgary.ca/legal-services/university-policies-procedures/ here] and search for "Electronic".<br />
# This environment is only rated to handle Level 1 and Level 2 data. Please see [https://www.ucalgary.ca/legal-services/university-legal-services/operating-standards-guidelines-forms/ here] and select "Information Security Classification Standard" for details.<br />
# If you create user accounts on your VM, make sure to manage them appropriately (remove/disable accounts that are no longer required).<br />
# If you have created accounts, you are responsible for the appropriate use of the VM by those accounts.<br />
# All user accounts on the VM must have good passwords. See [https://it.ucalgary.ca/it-security/passwords-do-i-have-change-them/ here] for details on creating strong passwords.<br />
# CloudStack is not meant as a High Performance Computing (HPC) number cruncher. If you have HPC needs, please see "[[How to get an account]]" for details on how to apply for an account on ARC. Not sure what you need? Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca].<br />
# If your VM faces the outside world, please consider using appropriate security tools. Contact [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] for assistance.<br />
<br />
==Important notes==<br />
In the event of a security incident, IT Operations/Security will shut down affected VMs.<br />
<br />
Non-critical patches/upgrades to CloudStack that are required will happen on Tuesday of each week. Running VMs should not be affected.<br />
<br />
Non-critical patches/upgrades that require a complete restart of CloudStack will occur after 1 day email notice.<br />
<br />
Urgent security patches to CloudStack may happen with little to no notice.<br />
<br />
Please check back with this document as changes may occur.</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=CloudStack_End_User_Agreement&diff=1834CloudStack End User Agreement2022-06-01T23:13:44Z<p>Darcy: /* Best Practices */</p>
<hr />
<div>==Introduction==<br />
CloudStack is an Infrastructure as a Service provided to University of Calgary researchers to allow them to quickly deploy Virtual Machines to support their research.<br />
<br />
Researchers have a great degree of freedom in how they use CloudStack. They are solely responsible for maintaining any VMs they deploy.<br />
<br />
CloudStack is a research environment. While the system is available 24/7, support is only available during University business hours.<br />
<br />
We ask that researchers follow the best practices listed below to ensure that the system remains available for the campus community.<br />
<br />
==Best Practices==<br />
<br />
# Please stay abreast of security updates for your OS and apply them.<br />
# Backup your data (RCS does not provide backups).<br />
# Do not run Windows (This infrastructure is not licensed to run Windows).<br />
# The University's "Acceptable Use of Electronic Resources and Information Policy" applies to your work using a VM. Please look [https://www.ucalgary.ca/legal-services/university-policies-procedures/ here] and search for "Electronic".<br />
# This environment is only rated to handle Level 1 and Level 2 data. Please see [https://www.ucalgary.ca/legal-services/university-legal-services/operating-standards-guidelines-forms/ here] and select "Information Security Classification Standard" for details.<br />
# If you create user accounts on your VM, make sure to manage them appropriately (remove/disable accounts that are no longer required).<br />
# If you have created accounts, you are responsible for the appropriate use of the VM by those accounts.<br />
# All user accounts on the VM must have good passwords. See [https://it.ucalgary.ca/it-security/passwords-do-i-have-change-them/ here] for details on creating strong passwords.<br />
# CloudStack is not meant as a High Performance Computing (HPC) number cruncher. If you have HPC needs, please see "[[How to get an account]]" for details on how to apply for an account on ARC. Not sure what you need? Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca].<br />
# If your VM faces the outside world, please consider using appropriate security tools. Contact [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] for assistance.<br />
<br />
==Important notes==<br />
In the event of a security incident, IT Operations/Security will shut down affected VMs.<br />
<br />
Non-critical patches/upgrades to CloudStack that are required will happen on Tuesday of each week. Running VMs should not not be affected.<br />
<br />
Non-critical patches/upgrades that require a complete restart of CloudStack will occur after 1 day email notice.<br />
<br />
Urgent security patches to CloudStack may happen with little to no notice.<br />
<br />
Please check back with this document as changes may occur.</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=CloudStack_End_User_Agreement&diff=1833CloudStack End User Agreement2022-06-01T23:09:35Z<p>Darcy: /* Introduction */</p>
<hr />
<div>==Introduction==<br />
CloudStack is an Infrastructure as a Service provided to University of Calgary researchers to allow them to quickly deploy Virtual Machines to support their research.<br />
<br />
Researchers have a great degree of freedom in how they use CloudStack. They are solely responsible for maintaining any VMs they deploy.<br />
<br />
CloudStack is a research environment. While the system is available 24/7, support is only available during University business hours.<br />
<br />
We ask that researchers follow the best practices listed below to ensure that the system remains available for the campus community.<br />
<br />
==Best Practices==<br />
<br />
# Please stay abreast of security updates for your OS and apply them.<br />
# Backup your data (RCS does not provide backups).<br />
# Do not run Windows (Our infrastructure is not licensed to run Windows).<br />
# The University's "Acceptable Use of Electronic Resources and Information Policy" applies to your work using a VM. Please look [https://www.ucalgary.ca/legal-services/university-policies-procedures/ here] and search for "Electronic".<br />
# This environment is only rated to handle Level 1 and Level 2 data. Please see [https://www.ucalgary.ca/legal-services/university-legal-services/operating-standards-guidelines-forms/ here] and select "Information Security Classification Standard" for details.<br />
# If you create user accounts on your VM, make sure to manage them appropriately (remove/disable accounts that are no longer required).<br />
# If you have created accounts, you are responsible for the appropriate use of the VM by those accounts.<br />
# All user accounts on the VM must have good passwords. See [https://it.ucalgary.ca/it-security/passwords-do-i-have-change-them/ here] for details on creating strong passwords.<br />
# CloudStack is not meant as a High Performance Computing (HPC) number cruncher. If you have HPC needs, please see "[[How to get an account]]" for details on how to apply for an account on ARC. Not sure what you need? Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca].<br />
# If your VM faces the outside world, please consider using appropriate security tools. Contact [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] for assistance.<br />
<br />
==Important notes==<br />
In the event of a security incident, IT Operations/Security will shut down affected VMs.<br />
<br />
Non-critical patches/upgrades to CloudStack that are required will happen on Tuesday of each week. Running VMs should not not be affected.<br />
<br />
Non-critical patches/upgrades that require a complete restart of CloudStack will occur after 1 day email notice.<br />
<br />
Urgent security patches to CloudStack may happen with little to no notice.<br />
<br />
Please check back with this document as changes may occur.</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=CloudStack_End_User_Agreement&diff=1832CloudStack End User Agreement2022-06-01T23:08:42Z<p>Darcy: /* Best Practices */</p>
<hr />
<div>==Introduction==<br />
CloudStack is an Infrastructure as a Service provided to University of Calgary researchers to allow them to quickly deploy Virtual Machines to support their research.<br />
<br />
Researchers have a great degree of freedom in how they use CloudStack. As such, they are solely responsible for maintaining any VMs they deploy.<br />
<br />
CloudStack is a research environment. While the system is available 24/7, support is only available during University business hours.<br />
<br />
We ask that researchers follow the best practices listed below to ensure that the system remains available for the campus community.<br />
<br />
==Best Practices==<br />
<br />
# Please stay abreast of security updates for your OS and apply them.<br />
# Backup your data (RCS does not provide backups).<br />
# Do not run Windows (Our infrastructure is not licensed to run Windows).<br />
# The University's "Acceptable Use of Electronic Resources and Information Policy" applies to your work using a VM. Please look [https://www.ucalgary.ca/legal-services/university-policies-procedures/ here] and search for "Electronic".<br />
# This environment is only rated to handle Level 1 and Level 2 data. Please see [https://www.ucalgary.ca/legal-services/university-legal-services/operating-standards-guidelines-forms/ here] and select "Information Security Classification Standard" for details.<br />
# If you create user accounts on your VM, make sure to manage them appropriately (remove/disable accounts that are no longer required).<br />
# If you have created accounts, you are responsible for the appropriate use of the VM by those accounts.<br />
# All user accounts on the VM must have good passwords. See [https://it.ucalgary.ca/it-security/passwords-do-i-have-change-them/ here] for details on creating strong passwords.<br />
# CloudStack is not meant as a High Performance Computing (HPC) number cruncher. If you have HPC needs, please see "[[How to get an account]]" for details on how to apply for an account on ARC. Not sure what you need? Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca].<br />
# If your VM faces the outside world, please consider using appropriate security tools. Contact [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] for assistance.<br />
<br />
==Important notes==<br />
In the event of a security incident, IT Operations/Security will shut down affected VMs.<br />
<br />
Non-critical patches/upgrades to CloudStack that are required will happen on Tuesday of each week. Running VMs should not not be affected.<br />
<br />
Non-critical patches/upgrades that require a complete restart of CloudStack will occur after 1 day email notice.<br />
<br />
Urgent security patches to CloudStack may happen with little to no notice.<br />
<br />
Please check back with this document as changes may occur.</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=CloudStack_End_User_Agreement&diff=1831CloudStack End User Agreement2022-06-01T23:06:59Z<p>Darcy: /* Important notes */</p>
<hr />
<div>==Introduction==<br />
CloudStack is an Infrastructure as a Service provided to University of Calgary researchers to allow them to quickly deploy Virtual Machines to support their research.<br />
<br />
Researchers have a great degree of freedom in how they use CloudStack. As such, they are solely responsible for maintaining any VMs they deploy.<br />
<br />
CloudStack is a research environment. While the system is available 24/7, support is only available during University business hours.<br />
<br />
We ask that researchers follow the best practices listed below to ensure that the system remains available for the campus community.<br />
<br />
==Best Practices==<br />
<br />
# Please stay abreast of security updates for your OS and apply them.<br />
# Backup your data (RCS does not provide backups).<br />
# Do not run Windows (Our infrastructure is not licensed to run Windows).<br />
# Remember that the University's "Acceptable Use of Electronic Resources and Information Policy" applies to your work using a VM. Please look [https://www.ucalgary.ca/legal-services/university-policies-procedures/ here] and search for "Electronic".<br />
# This environment is only rated to handle Level 1 and Level 2 data. Please see [https://www.ucalgary.ca/legal-services/university-legal-services/operating-standards-guidelines-forms/ here] and select "Information Security Classification Standard" for details.<br />
# If you create user accounts on your VM, make sure to manage them appropriately (remove/disable accounts that are no longer required).<br />
# If you have created accounts, remember that you are responsible for the appropriate use of the VM by those accounts.<br />
# All user accounts on the VM must have good passwords. See [https://it.ucalgary.ca/it-security/passwords-do-i-have-change-them/ here] for details on creating strong passwords.<br />
# CloudStack is not meant as a High Performance Computing (HPC) number cruncher. If you have HPC needs, please see "[[How to get an account]]" for details on how to apply for an account on ARC. Not sure what you need? Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca].<br />
# If your VM faces the outside world, please consider using appropriate security tools. Contact [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] for assistance.<br />
<br />
==Important notes==<br />
In the event of a security incident, IT Operations/Security will shut down affected VMs.<br />
<br />
Non-critical patches/upgrades to CloudStack that are required will happen on Tuesday of each week. Running VMs should not not be affected.<br />
<br />
Non-critical patches/upgrades that require a complete restart of CloudStack will occur after 1 day email notice.<br />
<br />
Urgent security patches to CloudStack may happen with little to no notice.<br />
<br />
Please check back with this document as changes may occur.</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=CloudStack_End_User_Agreement&diff=1830CloudStack End User Agreement2022-06-01T21:48:03Z<p>Darcy: /* Important notes */</p>
<hr />
<div>==Introduction==<br />
CloudStack is an Infrastructure as a Service provided to University of Calgary researchers to allow them to quickly deploy Virtual Machines to support their research.<br />
<br />
Researchers have a great degree of freedom in how they use CloudStack. As such, they are solely responsible for maintaining any VMs they deploy.<br />
<br />
CloudStack is a research environment. While the system is available 24/7, support is only available during University business hours.<br />
<br />
We ask that researchers follow the best practices listed below to ensure that the system remains available for the campus community.<br />
<br />
==Best Practices==<br />
<br />
# Please stay abreast of security updates for your OS and apply them.<br />
# Backup your data (RCS does not provide backups).<br />
# Do not run Windows (Our infrastructure is not licensed to run Windows).<br />
# Remember that the University's "Acceptable Use of Electronic Resources and Information Policy" applies to your work using a VM. Please look [https://www.ucalgary.ca/legal-services/university-policies-procedures/ here] and search for "Electronic".<br />
# This environment is only rated to handle Level 1 and Level 2 data. Please see [https://www.ucalgary.ca/legal-services/university-legal-services/operating-standards-guidelines-forms/ here] and select "Information Security Classification Standard" for details.<br />
# If you create user accounts on your VM, make sure to manage them appropriately (remove/disable accounts that are no longer required).<br />
# If you have created accounts, remember that you are responsible for the appropriate use of the VM by those accounts.<br />
# All user accounts on the VM must have good passwords. See [https://it.ucalgary.ca/it-security/passwords-do-i-have-change-them/ here] for details on creating strong passwords.<br />
# CloudStack is not meant as a High Performance Computing (HPC) number cruncher. If you have HPC needs, please see "[[How to get an account]]" for details on how to apply for an account on ARC. Not sure what you need? Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca].<br />
# If your VM faces the outside world, please consider using appropriate security tools. Contact [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] for assistance.<br />
<br />
==Important notes==<br />
Please note that in the event of a security incident IT Operations/Security will shut down affected VMs.<br />
<br />
Any non-critical patches/upgrades to CloudStack that are required will happen on Tuesday of each week. Running VMs should not not be affected.<br />
<br />
Any non-critical patches/upgrades that require a complete restart of CloudStack will occur after 1 day email notice.<br />
<br />
Urgent security patches to CloudStack may happen with little to no notice.<br />
<br />
Please check back with this document as changes may occur.</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=CloudStack_End_User_Agreement&diff=1829CloudStack End User Agreement2022-06-01T20:16:40Z<p>Darcy: /* Best Practices */</p>
<hr />
<div>==Introduction==<br />
CloudStack is an Infrastructure as a Service provided to University of Calgary researchers to allow them to quickly deploy Virtual Machines to support their research.<br />
<br />
Researchers have a great degree of freedom in how they use CloudStack. As such, they are solely responsible for maintaining any VMs they deploy.<br />
<br />
CloudStack is a research environment. While the system is available 24/7, support is only available during University business hours.<br />
<br />
We ask that researchers follow the best practices listed below to ensure that the system remains available for the campus community.<br />
<br />
==Best Practices==<br />
<br />
# Please stay abreast of security updates for your OS and apply them.<br />
# Backup your data (RCS does not provide backups).<br />
# Do not run Windows (Our infrastructure is not licensed to run Windows).<br />
# Remember that the University's "Acceptable Use of Electronic Resources and Information Policy" applies to your work using a VM. Please look [https://www.ucalgary.ca/legal-services/university-policies-procedures/ here] and search for "Electronic".<br />
# This environment is only rated to handle Level 1 and Level 2 data. Please see [https://www.ucalgary.ca/legal-services/university-legal-services/operating-standards-guidelines-forms/ here] and select "Information Security Classification Standard" for details.<br />
# If you create user accounts on your VM, make sure to manage them appropriately (remove/disable accounts that are no longer required).<br />
# If you have created accounts, remember that you are responsible for the appropriate use of the VM by those accounts.<br />
# All user accounts on the VM must have good passwords. See [https://it.ucalgary.ca/it-security/passwords-do-i-have-change-them/ here] for details on creating strong passwords.<br />
# CloudStack is not meant as a High Performance Computing (HPC) number cruncher. If you have HPC needs, please see "[[How to get an account]]" for details on how to apply for an account on ARC. Not sure what you need? Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca].<br />
# If your VM faces the outside world, please consider using appropriate security tools. Contact [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] for assistance.<br />
<br />
==Important notes==<br />
Please note that in the event of a security incident IT Operations/Security will shutdown affected VMs.<br />
<br />
Any non-critical patches/upgrades to CloudStack that are required will happen on Tuesday of each week. Running VMs should not not be affected.<br />
<br />
Any non-critical patches/upgrades that require a complete restart of CloudStack will occur after 1 day email notice.<br />
<br />
Urgent security patches to CloudStack may happen with little to no notice.<br />
<br />
Please check back with this document as changes may occur.</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=CloudStack_End_User_Agreement&diff=1828CloudStack End User Agreement2022-06-01T19:55:06Z<p>Darcy: /* Best Practices */</p>
<hr />
<div>==Introduction==<br />
CloudStack is an Infrastructure as a Service provided to University of Calgary researchers to allow them to quickly deploy Virtual Machines to support their research.<br />
<br />
Researchers have a great degree of freedom in how they use CloudStack. As such, they are solely responsible for maintaining any VMs they deploy.<br />
<br />
CloudStack is a research environment. While the system is available 24/7, support is only available during University business hours.<br />
<br />
We ask that researchers follow the best practices listed below to ensure that the system remains available for the campus community.<br />
<br />
==Best Practices==<br />
<br />
# Please stay abreast of security updates for your OS and apply them.<br />
# Backup your data (RCS does not provide backups).<br />
# Do not run Windows (Our infrastructure is not licensed to run Windows).<br />
# Remember that the University's "Acceptable Use of Electronic Resources and Information Policy" applies to your work using a VM. Please look [https://www.ucalgary.ca/legal-services/university-policies-procedures/ here] and search for "Electronic".<br />
# This environment is only rated to handle Level 1 and Level 2 data. Please see [https://www.ucalgary.ca/legal-services/university-legal-services/operating-standards-guidelines-forms/ here] and select "Information Security Classification Standard".<br />
# If you create user accounts on your VM, make sure to manage them appropriately (remove/disable accounts that are no longer required).<br />
# If you have created accounts, remember that you are responsible for the appropriate use of the VM by those accounts.<br />
# All user accounts on the VM must have good passwords. See [https://it.ucalgary.ca/it-security/passwords-do-i-have-change-them/ here] for details on creating strong passwords.<br />
# CloudStack is not meant as a High Performance Computing (HPC) number cruncher. If you have HPC needs, please see "[[How to get an account]]" for details on how to apply for an account on ARC. Not sure what you need? Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca].<br />
# If your VM faces the outside world, please consider using appropriate security tools. Contact [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] for assistance.<br />
<br />
==Important notes==<br />
Please note that in the event of a security incident IT Operations/Security will shutdown affected VMs.<br />
<br />
Any non-critical patches/upgrades to CloudStack that are required will happen on Tuesday of each week. Running VMs should not not be affected.<br />
<br />
Any non-critical patches/upgrades that require a complete restart of CloudStack will occur after 1 day email notice.<br />
<br />
Urgent security patches to CloudStack may happen with little to no notice.<br />
<br />
Please check back with this document as changes may occur.</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=CloudStack_End_User_Agreement&diff=1827CloudStack End User Agreement2022-06-01T19:54:37Z<p>Darcy: /* Best Practices */</p>
<hr />
<div>==Introduction==<br />
CloudStack is an Infrastructure as a Service provided to University of Calgary researchers to allow them to quickly deploy Virtual Machines to support their research.<br />
<br />
Researchers have a great degree of freedom in how they use CloudStack. As such, they are solely responsible for maintaining any VMs they deploy.<br />
<br />
CloudStack is a research environment. While the system is available 24/7, support is only available during University business hours.<br />
<br />
We ask that researchers follow the best practices listed below to ensure that the system remains available for the campus community.<br />
<br />
==Best Practices==<br />
<br />
# Please stay abreast of security updates for your OS and apply them.<br />
# Backup your data (RCS does not provide backups).<br />
# Do not run Windows (Our infrastructure is not licensed to run Windows).<br />
# Remember that the University's "Acceptable Use of Electronic Resources and Information Policy" applies to your work using a VM. Please look [https://www.ucalgary.ca/legal-services/university-policies-procedures/ here] and search for "Electronic".<br />
# This environment is only rated to handle Level 1 and Level 2 data. Please see [https://www.ucalgary.ca/legal-services/university-legal-services/operating-standards-guidelines-forms/ here] and select "Information Security Classification Standard".<br />
# If you create user accounts on your VM, make sure to manage them appropriately (remove/disable accounts that are no longer required).<br />
# If you have created accounts, remember that you are responsible for the appropriate use of the VM by those accounts.<br />
# All user accounts on the VM must have good passwords. See [https://it.ucalgary.ca/it-security/passwords-do-i-have-change-them/ here] for details on creating strong passwords.<br />
# CloudStack is not meant as a High Performance Computing (HPC) number cruncher. If you have HPC needs, please see "[[How to get an account]]" for details on how to apply for an account on [[ARC Cluster Guide] ARC]. Not sure what you need? Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca].<br />
# If your VM faces the outside world, please consider using appropriate security tools. Contact [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] for assistance.<br />
<br />
==Important notes==<br />
Please note that in the event of a security incident IT Operations/Security will shutdown affected VMs.<br />
<br />
Any non-critical patches/upgrades to CloudStack that are required will happen on Tuesday of each week. Running VMs should not not be affected.<br />
<br />
Any non-critical patches/upgrades that require a complete restart of CloudStack will occur after 1 day email notice.<br />
<br />
Urgent security patches to CloudStack may happen with little to no notice.<br />
<br />
Please check back with this document as changes may occur.</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=CloudStack_End_User_Agreement&diff=1826CloudStack End User Agreement2022-06-01T19:53:42Z<p>Darcy: /* Best Practices */</p>
<hr />
<div>==Introduction==<br />
CloudStack is an Infrastructure as a Service provided to University of Calgary researchers to allow them to quickly deploy Virtual Machines to support their research.<br />
<br />
Researchers have a great degree of freedom in how they use CloudStack. As such, they are solely responsible for maintaining any VMs they deploy.<br />
<br />
CloudStack is a research environment. While the system is available 24/7, support is only available during University business hours.<br />
<br />
We ask that researchers follow the best practices listed below to ensure that the system remains available for the campus community.<br />
<br />
==Best Practices==<br />
<br />
# Please stay abreast of security updates for your OS and apply them.<br />
# Backup your data (RCS does not provide backups).<br />
# Do not run Windows (Our infrastructure is not licensed to run Windows).<br />
# Remember that the University's "Acceptable Use of Electronic Resources and Information Policy" applies to your work using a VM. Please look [https://www.ucalgary.ca/legal-services/university-policies-procedures/ here] and search for "Electronic".<br />
# This environment is only rated to handle Level 1 and Level 2 data. Please see [https://www.ucalgary.ca/legal-services/university-legal-services/operating-standards-guidelines-forms/ here] and select "Information Security Classification Standard".<br />
# If you create user accounts on your VM, make sure to manage them appropriately (remove/disable accounts that are no longer required).<br />
# If you have created accounts, remember that you are responsible for the appropriate use of the VM by those accounts.<br />
# All user accounts on the VM must have good passwords. See [https://it.ucalgary.ca/it-security/passwords-do-i-have-change-them/ here] for details on creating strong passwords.<br />
# CloudStack is not meant as a High Performance Computing (HPC) number cruncher. If you have HPC needs, please see "[[How to get an account]]" for details on how to apply for an account on [ARC_Cluster_Guide ARC]. Not sure what you need? Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca].<br />
# If your VM faces the outside world, please consider using appropriate security tools. Contact [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] for assistance.<br />
<br />
==Important notes==<br />
Please note that in the event of a security incident IT Operations/Security will shutdown affected VMs.<br />
<br />
Any non-critical patches/upgrades to CloudStack that are required will happen on Tuesday of each week. Running VMs should not not be affected.<br />
<br />
Any non-critical patches/upgrades that require a complete restart of CloudStack will occur after 1 day email notice.<br />
<br />
Urgent security patches to CloudStack may happen with little to no notice.<br />
<br />
Please check back with this document as changes may occur.</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=CloudStack_End_User_Agreement&diff=1825CloudStack End User Agreement2022-06-01T19:51:11Z<p>Darcy: /* Best Practices */</p>
<hr />
<div>==Introduction==<br />
CloudStack is an Infrastructure as a Service provided to University of Calgary researchers to allow them to quickly deploy Virtual Machines to support their research.<br />
<br />
Researchers have a great degree of freedom in how they use CloudStack. As such, they are solely responsible for maintaining any VMs they deploy.<br />
<br />
CloudStack is a research environment. While the system is available 24/7, support is only available during University business hours.<br />
<br />
We ask that researchers follow the best practices listed below to ensure that the system remains available for the campus community.<br />
<br />
==Best Practices==<br />
<br />
# Please stay abreast of security updates for your OS and apply them.<br />
# Backup your data (RCS does not provide backups).<br />
# Do not run Windows (Our infrastructure is not licensed to run Windows).<br />
# Remember that the University's "Acceptable Use of Electronic Resources and Information Policy" applies to your work using a VM. Please look [https://www.ucalgary.ca/legal-services/university-policies-procedures/ here] and search for "Electronic".<br />
# This environment is only rated to handle Level 1 and Level 2 data. Please see [https://www.ucalgary.ca/legal-services/university-legal-services/operating-standards-guidelines-forms/ here] and select "Information Security Classification Standard".<br />
# If you create user accounts on your VM, make sure to manage them appropriately (remove/disable accounts that are no longer required).<br />
# If you have created accounts, remember that you are responsible for the appropriate use of the VM by those accounts.<br />
# All user accounts on the VM must have good passwords. See [https://it.ucalgary.ca/it-security/passwords-do-i-have-change-them/ here] for details on creating strong passwords.<br />
# CloudStack is not meant as a High Performance Computing (HPC) number cruncher. If you have HPC needs, please see "[[How to get an account]]" for details on how to apply for an account. Not sure what you need? Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca].<br />
# If your VM faces the outside world, please consider using appropriate security tools. Contact [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] for assistance.<br />
<br />
==Important notes==<br />
Please note that in the event of a security incident IT Operations/Security will shutdown affected VMs.<br />
<br />
Any non-critical patches/upgrades to CloudStack that are required will happen on Tuesday of each week. Running VMs should not not be affected.<br />
<br />
Any non-critical patches/upgrades that require a complete restart of CloudStack will occur after 1 day email notice.<br />
<br />
Urgent security patches to CloudStack may happen with little to no notice.<br />
<br />
Please check back with this document as changes may occur.</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=CloudStack_End_User_Agreement&diff=1817CloudStack End User Agreement2022-05-31T20:28:20Z<p>Darcy: </p>
<hr />
<div>==Introduction==<br />
CloudStack is an Infrastructure as a Service provided to University of Calgary researchers to allow them to quickly deploy Virtual Machines to support their research.<br />
<br />
Researchers have a great degree of freedom in how they use CloudStack. As such, they are solely responsible for maintaining any VMs they deploy.<br />
<br />
CloudStack is a research environment. While the system is available 24/7, support is only available during University business hours.<br />
<br />
We ask that researchers follow the best practices listed below to ensure that the system remains available for the campus community.<br />
<br />
==Best Practices==<br />
<br />
# Please stay abreast of security updates for your OS and apply them.<br />
# Backup your data (RCS does not provide backups).<br />
# Do not run Windows (Our infrastructure is not licensed to run Windows).<br />
# Remember that the University's "Acceptable Use of Electronic Resources and Information Policy" applies to your work using a VM. Please look [https://www.ucalgary.ca/legal-services/university-policies-procedures/ here] and search for "Electronic".<br />
# This environment is only rated to handle Level 1 and Level 2 data. Please see [https://www.ucalgary.ca/legal-services/university-legal-services/operating-standards-guidelines-forms/ here] and select "Information Security Classification Standard".<br />
# If you create user accounts on your VM, make sure to manage them appropriately (remove/disable accounts that are no longer required).<br />
# If you have created accounts, remember that you are responsible for the appropriate use of the VM by those accounts.<br />
# All user accounts on the VM must have good passwords See [https://it.ucalgary.ca/it-security/passwords-do-i-have-change-them/ here] for details on creating strong passwords.<br />
# CloudStack is not meant as a High Performance Computing (HPC) number cruncher. If you have HPC needs, please see "[[How to get an account]]" for details on how to apply for an account. Not sure what you need? Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca].<br />
# If your VM faces the outside world, please consider using appropriate security tools. Contact [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] for assistance.<br />
<br />
==Important notes==<br />
Please note that in the event of a security incident IT Operations/Security will shutdown affected VMs.<br />
<br />
Any non-critical patches/upgrades to CloudStack that are required will happen on Tuesday of each week. Running VMs should not not be affected.<br />
<br />
Any non-critical patches/upgrades that require a complete restart of CloudStack will occur after 1 day email notice.<br />
<br />
Urgent security patches to CloudStack may happen with little to no notice.<br />
<br />
Please check back with this document as changes may occur.</div>Darcyhttps://rcs.ucalgary.ca/index.php?title=CloudStack_End_User_Agreement&diff=1816CloudStack End User Agreement2022-05-31T20:06:41Z<p>Darcy: </p>
<hr />
<div>==Introduction==<br />
CloudStack is an Infrastructure as a Service provided to University of Calgary researchers to allow them to quickly deploy Virtual Machines to support their research.<br />
<br />
Researchers have a great degree of freedom in how they use CloudStack. As such, they are solely responsible for maintaining any VMs they deploy.<br />
<br />
CloudStack is a research environment. While the system is available 24/7, support is only available during University business hours.<br />
<br />
We ask that researchers follow the best practices listed below to ensure that the system remains available for the campus community.<br />
<br />
==Best Practices==<br />
<br />
# Please stay abreast of security updates for your OS and apply them.<br />
# Backup your data (RCS does not provide backups).<br />
# Do not run Windows (Our infrastructure is not licensed to run Windows).<br />
# Remember that the University's "Acceptable Use of Electronic Resources and Information Policy" applies to your work using a VM. Please look [https://www.ucalgary.ca/legal-services/university-policies-procedures/ here] and search for "Electronic".<br />
# This environment is only rated to handle Level 1 and Level 2 data. Please see [https://www.ucalgary.ca/legal-services/university-legal-services/operating-standards-guidelines-forms/ here] and select "Information Security Classification Standard".<br />
# If you create user accounts on your VM, make sure to manage them appropriately (remove/disable accounts that are no longer required).<br />
# If you have created accounts, remember that you are responsible for the appropriate use of the VM by those accounts.<br />
# All user accounts on the VM must have good passwords See [https://it.ucalgary.ca/it-security/passwords-do-i-have-change-them/ here] for details on creating strong passwords.<br />
# CloudStack is not meant as a High Performance Computing (HPC) number cruncher. If you have HPC needs, please see "[[How to get an account]]" for details on how to apply for an account. Not sure what you need? Contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca].<br />
# If your VM faces the outside world, please consider using appropriate security tools. Contact [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca] for assistance.<br />
<br />
==Important notes==<br />
Please note that in the event of a security incident IT Security will shutdown affected VMs.<br />
<br />
Any non-critical patches/upgrades to CloudStack that are required will happen on Tuesday of each week. Running VMs should not not be affected.<br />
<br />
Any non-critical patches/upgrades that require a complete restart of CloudStack will occur after 1 day email warning.<br />
<br />
Urgent security patches to CloudStack may happen with little to no notice.<br />
<br />
Please check back with this document as changes may occur.</div>Darcy