RCS Home Page: Difference between revisions

From RCSWiki
Jump to navigation Jump to search
No edit summary
 
(147 intermediate revisions by 8 users not shown)
Line 1: Line 1:
<strong>Welcome to the RCS documentation home page.</strong>
Research Computing Services (RCS) is a group within the wider University of Calgary Information Technologies team that plans, manages, and supports high performance computing (HPC) systems in use by researchers throughout the University of Calgary.  Our primary focus is to meet the increasing demand for engineering and scientific computation by offering a wide range of specialized services to help researchers solve highly complex real-world problems or run large scale computationally intensive workloads on our high-end HPC resources.


Many researchers coming from a Microsoft Windows background, intending to use a remote HPC system for the first time, request some kind of Linux training.
This RCS Wiki contains technical documentation for use by users of HPC systems operated by RCS


The WestGrid [http://www.westgrid.ca/support/quickstart/new_users QuickStart Guide for New Users] gives an overview of the range of topics that are typically relevant for someone getting started on such a system, with links to details that are specific to the WestGrid environment. Many of the topics covered there are relevant for most remote Linux systems, so, you might like to take a look at it.
<!--
In case cluster status changes:
    *  set the status to yellow or red
    *  provide a custom 'title' and 'message'


To focus discussions with those who might be interested in obtaining more pesonalized HPC Linux training from UC IT HPC support staff, an expanded list of possible topics is shown below.
{{Cluster Status
|status=green
}}
-->
=== Contact us for support ===


If you are starting with WestGrid or the local HPC resources, an offer is usually made to meet with you for an hour or two to make sure your own computer is set up and to go over the main things you need to know to start using the remote computational resources.
* For general RCS/HPC inquiries, please email: [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca]
* For IT related issues (networking, VPN, email), please email: [mailto:it@ucalgary.ca it@ucalgary.ca]
* For Compute Canada specific questions: [mailto:support@tech.alliancecan.ca support@tech.alliancecan.ca]


If you are interested in meeting individually or as a group with UC HPC staff to learn more about using remote Linux systems, please contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca].
{{Clear}}
<div class="row">
<div class="col-md-6">


The following topics may be of interest.
== General information ==
* [[General Cluster Guidelines and Policies]]
* [[How to get an account]]
* [[Data ownership]]
* [[Connecting to RCS HPC Systems]]
* [[External collaborators]]


== Know the Hardware ==
* [[CloudStack|Cloud/Virtual Machine Infrastructure (CloudStack)]]
* What are the general characteristics of the hardware
 
* Linux commands for identifying the hardware and system
* [[On-line resources for new Linux and ARC users]]
== Know Your Administrators and Help Resources ==
* [[Acknowledging Research Computing Services Group]]
* Who is looking after the system?
 
* Local contact
== Cluster Guides ==
* IT support contact
* [[ ARC Cluster Guide]] - ARC is a general purpose cluster for University of Calgary researchers.
* Mailing lists
* [[GLaDOS Cluster Guide]] - GLaDOS is a researcher-owned cluster maintained by Research Computing Services.
* Local network support
* [[TALC Cluster Guide]] - Teaching and Learning Cluster (TALC) is a cluster created by Research Computing Services to support academic courses and workshops.
* Where to go for help
* [[MARC Cluster Guide]] -- Medical Advanced Research Computing cluster at the University of Calgary created by Research Computing Services in 2020.
== Setting up Your Computer ==
 
* Who administers your local computer?
== Other services ==
* Is your local machine shared or private?
 
* Security (anti-virus software etc.) on your own computer
* [[Jupyter Notebooks]]
* Data backup on your computer
* [[Open OnDemand | Open OnDemand portal]]
* Ssh client (SSH Secure Shell or PuTTY for Windows, built-in ssh for Linux or Mac)
 
* File transfer client (WinSCP or SSH Secure Shell for Windows, built-in scp or sftp for Linux or Mac, Fetch or Fugu for Mac, etc.)
== Software pages ==
* X Window display server (xming for Windows, built-in for Linux and Mac)
* [[Managing software on ARC]]
* VNC
* [[Gaussian on ARC]] -- How to use Gaussian 16 on ARC.
== Connecting and Logging In to a Remote System ==
* [[Apache Spark on ARC]]
* Name of login server
* [[ARC Software pages]]
* Ssh clients
* [[Bioinformatics applications]]
* X Window tunnelling
</div>
* Changing passwords (passwd)
<div class="col-md-6">
* Public key authentication
 
* Security policies
== Running courses on HPC resources ==
* VNC
* [[TALC Cluster|TALC]] - Teaching and Learning Cluster (TALC) is a cluster created by Research Computing Services to support academic courses and workshops.
== Setting up Your Remote Environment ==
* [[TALC Terms of Use]] - Terms of use to which TALC account holders must agree to use the cluster.
* Startup files (.bashrc, .bash_profile, ...)
* [[List of courses on TALC]] - A list of current and historical courses taught using TALC.
* Command prompt string
 
* set -o emacs
== Training ==
* Aliases (ls vs ls -la)
* Our [[HPC Systems]]
* Default umask
* [[HPC Linux topics]] - A list of topics on which RCS technical support staff can provide one-on-one or group training
* Does backspace work? (ctrl-H, ctrl-?, backspace, delete,...)
* [[Courses]]
VNC
* [[Linux Introduction]]
== Transferring Files ==
* [[What is a scheduler?]]
* scp, sftp, WinSCP, SSH Secure Shell
* [[Running jobs]]
* line ending issues
* [[Data storage options for UofC researchers]]
* File names (case sensitive, don't use spaces, some special characters)
* [[Security and privacy]]
* Wireless limitations
* [[How to transfer data]]
* Data transfer speeds
 
== Editing Files ==
* [[UofC Services]]
* UNIX character-based editors (emacs, vi)
 
* Graphical editor (nedit)
* [[Book online training sessions]]
* Editing on your own workstation
 
== Working Interactively on a Linux System ==
 
* UNIX shell
* [[How-Tos | More How-Tos]]
* Typical UNIX command line, arguments, continuation character
</div>
* I/O redirection, standard output and error streams
</div>
* man command
 
* Command history (arrow keys, "set -o emacs", history command)
{{Clear}}
* Processes, ps
 
== The Toolbox of UNIX Commands ==
__NOTOC__
* Converting line endings: dos2unix, unix2dos, file
* Finding stuff: find, locate, grep
* Looking at text files: more, less, cat
* Manipulating text files: sort, uniq, wc, awk, grep, sed
== Shell Scripting ==
* Bash scripts
* Perl
* Python
== Data Management ==
* Local storage policies (quotas, time limits, home directories vs. scratch)
* Backup policy and procedures
* df, du, gzip, tar
== Batch Processing ==
* Interactive vs. Batch
* Batch environment (TORQUE, SLURM, SGE, ...)
* Batch job scripts
* Commands for submitting, monitoring, deleting jobs
* File systems and where to submit your jobs
* Ssh to compute nodes for monitoring processes,  memory use, parallel efficiency
== Programming ==
* Languages
* Compilers
* Make
* Debugging
== Software ==
* Locating installed software
* Who is responsible for software installation?
* Typical UNIX software installation procedures
== Data Visualization ==
* Visualization for debugging
* Where to do visualization (local or remote)
* Batch-oriented visualization
* UNIX programs for data visualization
== Special Topics ==
* How to run software specific to your discipline

Latest revision as of 15:30, 5 November 2024

Research Computing Services (RCS) is a group within the wider University of Calgary Information Technologies team that plans, manages, and supports high performance computing (HPC) systems in use by researchers throughout the University of Calgary. Our primary focus is to meet the increasing demand for engineering and scientific computation by offering a wide range of specialized services to help researchers solve highly complex real-world problems or run large scale computationally intensive workloads on our high-end HPC resources.

This RCS Wiki contains technical documentation for use by users of HPC systems operated by RCS

Contact us for support

General information

Cluster Guides

  • ARC Cluster Guide - ARC is a general purpose cluster for University of Calgary researchers.
  • GLaDOS Cluster Guide - GLaDOS is a researcher-owned cluster maintained by Research Computing Services.
  • TALC Cluster Guide - Teaching and Learning Cluster (TALC) is a cluster created by Research Computing Services to support academic courses and workshops.
  • MARC Cluster Guide -- Medical Advanced Research Computing cluster at the University of Calgary created by Research Computing Services in 2020.

Other services

Software pages

Running courses on HPC resources

  • TALC - Teaching and Learning Cluster (TALC) is a cluster created by Research Computing Services to support academic courses and workshops.
  • TALC Terms of Use - Terms of use to which TALC account holders must agree to use the cluster.
  • List of courses on TALC - A list of current and historical courses taught using TALC.

Training



Retrieved from "https://rcs.ucalgary.ca/index.php?title=RCS_Home_Page&oldid=3621"