RCS Home Page: Difference between revisions

From RCSWiki
Jump to navigation Jump to search
No edit summary
(137 intermediate revisions by 7 users not shown)
Line 1: Line 1:
<strong>Welcome to the RCS documentation home page.</strong>
Research Computing Services (RCS) is a group within the wider University of Calgary Information Technologies team that plans, manages, and supports high performance computing (HPC) systems in use by researchers throughout the University of Calgary.  Our primary focus is to meet the increasing demand for engineering and scientific computation by offering a wide range of specialized services to help researchers solve highly complex real-world problems or run large scale computationally intensive workloads on our high-end HPC resources.


Many researchers coming from a Microsoft Windows background, intending to use a remote HPC system for the first time, request some kind of Linux training.
This RCS Wiki contains technical documentation for use by users of HPC systems operated by RCS


The WestGrid [http://www.westgrid.ca/support/quickstart/new_users QuickStart Guide for New Users] gives an overview of the range of topics that are typically relevant for someone getting started on such a system, with links to details that are specific to the WestGrid environment. Many of the topics covered there are relevant for most remote Linux systems, so, you might like to take a look at it.
<!--
In case cluster status changes:
    *  set the status to yellow or red
    *  provide a custom 'title' and 'message'


To focus discussions with those who might be interested in obtaining more pesonalized HPC Linux training from UC IT HPC support staff, an expanded list of possible topics is shown below.  
{{Cluster Status
|status=green
}}
-->
=== Contact us for support ===
[[File:Map HSC G204Z.png|150px|thumb|right|Find us at G204Z]]
* For general RCS/HPC inquiries, please email: [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca]
* For IT related issues (networking, VPN, email), please email: [mailto:it@ucalgary.ca it@ucalgary.ca]
* For Compute Canada specific questions: [mailto:support@computecanada.ca support@computecanada.ca]


If you are starting with WestGrid or the local HPC resources, an offer is usually made to meet with you for an hour or two to make sure your own computer is set up and to go over the main things you need to know to start using the remote computational resources.
{{Clear}}
<div class="row">
<div class="col-md-6">


If you are interested in meeting individually or as a group with UC HPC staff to learn more about using remote Linux systems, please contact us at [mailto:support@hpc.ucalgary.ca support@hpc.ucalgary.ca].
== General information ==
* [[General Cluster Guidelines and Policies]]
* [[How to get an account]]
* [[Data ownership]]
* [[Connecting to RCS HPC Systems]]
* [[External collaborators]]


The following topics may be of interest.
* [[CloudStack|Cloud/Virtual Machine Infrastructure (CloudStack)]]


== [[Know the Hardware]] ==
* [[On-line resources for new Linux and ARC users]]
* What are the general characteristics of the hardware
* [[Acknowledging Research Computing Services Group]]
* Linux commands for identifying the hardware and system
 
== Know Your Administrators and Help Resources ==
== Cluster Guides ==
* Who is looking after the system?
* [[ ARC Cluster Guide]] - ARC is a general purpose cluster for University of Calgary researchers.
* Local contact
* [[Helix Cluster Guide]] - Helix is a specialized cluster mainly provided for Cumming School of Medicine projects
* IT support contact
* [[GLaDOS Cluster Guide]] - GLaDOS is a researcher-owned cluster maintained by Research Computing Services.
* Mailing lists
* [[TALC Cluster Guide]] - Teaching and Learning Cluster (TALC) is a cluster created by Research Computing Services to support academic courses and workshops.
* Local network support
* [[MARC Cluster Guide]] -- Medical Advanced Research Computing cluster at the University of Calgary created by Research Computing Services in 2020.
* Where to go for help
 
== Setting up Your Computer ==
== Other services ==
* Who administers your local computer?
 
* Is your local machine shared or private?
* [[Jupyter Notebooks]]
* Security (anti-virus software etc.) on your own computer
* [[Open OnDemand | Open OnDemand portal]]
* Data backup on your computer
 
* Ssh client (SSH Secure Shell or PuTTY for Windows, built-in ssh for Linux or Mac)
== Software pages ==
* File transfer client (WinSCP or SSH Secure Shell for Windows, built-in scp or sftp for Linux or Mac, Fetch or Fugu for Mac, etc.)
* [[Managing software on ARC]]
* X Window display server (xming for Windows, built-in for Linux and Mac)
* [https://hpc.ucalgary.ca/arc/software/conda Using Conda (external link)]
* VNC
* [[Gaussian on ARC]] -- How to use Gaussian 16 on ARC.
== Connecting and Logging In to a Remote System ==
* [[Apache Spark on ARC]]
* Name of login server
* [[ARC Software pages]]
* Ssh clients
* [[Bioinformatics applications]]
* X Window tunnelling
</div>
* Changing passwords (passwd)
<div class="col-md-6">
* Public key authentication
 
* Security policies
== Running courses on HPC resources ==
* VNC
* [[TALC Cluster|TALC]] - Teaching and Learning Cluster (TALC) is a cluster created by Research Computing Services to support academic courses and workshops.
== Setting up Your Remote Environment ==
* [[TALC Terms of Use]] - Terms of use to which TALC account holders must agree to use the cluster.
* Startup files (.bashrc, .bash_profile, ...)
* [[List of courses on TALC]] - A list of current and historical courses taught using TALC.
* Command prompt string
 
* set -o emacs
== Training ==
* Aliases (ls vs ls -la)
* Our [[HPC Systems]]
* Default umask
* [[HPC Linux topics]] - A list of topics on which RCS technical support staff can provide one-on-one or group training
* Does backspace work? (ctrl-H, ctrl-?, backspace, delete,...)
* [[Courses]]
VNC
* [[Linux Introduction]]
== Transferring Files ==
* [[What is a scheduler?]]
* scp, sftp, WinSCP, SSH Secure Shell
* [[Running jobs]]
* line ending issues
* [[Data storage options for UofC researchers]]
* File names (case sensitive, don't use spaces, some special characters)
* [[Security and privacy]]
* Wireless limitations
* [[How to transfer data]]
* Data transfer speeds
 
== Editing Files ==
* [[UofC Services]]
* UNIX character-based editors (emacs, vi)
 
* Graphical editor (nedit)
* [[Book online training sessions]]
* Editing on your own workstation
 
== Working Interactively on a Linux System ==
 
* UNIX shell
* [[How-Tos | More How-Tos]]
* Typical UNIX command line, arguments, continuation character
</div>
* I/O redirection, standard output and error streams
</div>
* man command
 
* Command history (arrow keys, "set -o emacs", history command)
{{Clear}}
* Processes, ps
 
== The Toolbox of UNIX Commands ==
==What's New==
* Converting line endings: dos2unix, unix2dos, file
* [[CHGI Transition]] - Information on the current CHGI Transition
* Finding stuff: find, locate, grep
 
* Looking at text files: more, less, cat
 
* Manipulating text files: sort, uniq, wc, awk, grep, sed
__NOTOC__
== Shell Scripting ==
* Bash scripts
* Perl
* Python
== Data Management ==
* Local storage policies (quotas, time limits, home directories vs. scratch)
* Backup policy and procedures
* df, du, gzip, tar
== Batch Processing ==
* Interactive vs. Batch
* Batch environment (TORQUE, SLURM, SGE, ...)
* Batch job scripts
* Commands for submitting, monitoring, deleting jobs
* File systems and where to submit your jobs
* Ssh to compute nodes for monitoring processes,  memory use, parallel efficiency
== Programming ==
* Languages
* Compilers
* Make
* Debugging
== Software ==
* Locating installed software
* Who is responsible for software installation?
* Typical UNIX software installation procedures
== Data Visualization ==
* Visualization for debugging
* Where to do visualization (local or remote)
* Batch-oriented visualization
* UNIX programs for data visualization
== Special Topics ==
* How to run software specific to your discipline

Revision as of 22:22, 6 October 2022

Research Computing Services (RCS) is a group within the wider University of Calgary Information Technologies team that plans, manages, and supports high performance computing (HPC) systems in use by researchers throughout the University of Calgary. Our primary focus is to meet the increasing demand for engineering and scientific computation by offering a wide range of specialized services to help researchers solve highly complex real-world problems or run large scale computationally intensive workloads on our high-end HPC resources.

This RCS Wiki contains technical documentation for use by users of HPC systems operated by RCS

Contact us for support

Find us at G204Z

General information

Cluster Guides

  • ARC Cluster Guide - ARC is a general purpose cluster for University of Calgary researchers.
  • Helix Cluster Guide - Helix is a specialized cluster mainly provided for Cumming School of Medicine projects
  • GLaDOS Cluster Guide - GLaDOS is a researcher-owned cluster maintained by Research Computing Services.
  • TALC Cluster Guide - Teaching and Learning Cluster (TALC) is a cluster created by Research Computing Services to support academic courses and workshops.
  • MARC Cluster Guide -- Medical Advanced Research Computing cluster at the University of Calgary created by Research Computing Services in 2020.

Other services

Software pages

Running courses on HPC resources

  • TALC - Teaching and Learning Cluster (TALC) is a cluster created by Research Computing Services to support academic courses and workshops.
  • TALC Terms of Use - Terms of use to which TALC account holders must agree to use the cluster.
  • List of courses on TALC - A list of current and historical courses taught using TALC.

Training


What's New


Retrieved from "https://rcs.ucalgary.ca/index.php?title=RCS_Home_Page&oldid=2122"