HPC Linux topics

Welcome to the RCS documentation home page.

Many researchers coming from a Microsoft Windows background, intending to use a remote HPC system for the first time, request some kind of Linux training.

The WestGrid QuickStart Guide for New Users gives an overview of the range of topics that are typically relevant for someone getting started on such a system, with links to details that are specific to the WestGrid environment. Many of the topics covered there are relevant for most remote Linux systems, so, you might like to take a look at it.

To focus discussions with those who might be interested in obtaining more pesonalized HPC Linux training from UC IT HPC support staff, an expanded list of possible topics is shown below. If you are curious, each of these topics can be visited to learn more about them.

If you are starting with WestGrid or the local HPC resources, an offer is usually made to meet with you for an hour or two to make sure your own computer is set up and to go over the main things you need to know to start using the remote computational resources.

If you are interested in meeting individually or as a group with UC HPC staff to learn more about using remote Linux systems, please contact us at support@hpc.ucalgary.ca.

The following topics may be of interest.

Know the Hardware

• What are the general characteristics of the hardware
• Linux commands for identifying the hardware and system

Know Your Administrators and Help Resources

• Who is looking after the system?
• Local contact
• IT support contact
• Mailing lists
• Local network support
• Where to go for help

Setting up Your Computer

• Who administers your local computer?
• Is your local machine shared or private?
• Security (anti-virus software etc.) on your own computer
• Data backup on your computer
• Ssh client (SSH Secure Shell or PuTTY for Windows, built-in ssh for Linux or Mac)
• File transfer client (WinSCP or SSH Secure Shell for Windows, built-in scp or sftp for Linux or Mac, Fetch or Fugu for Mac, etc.)
• X Window display server (xming for Windows, built-in for Linux and Mac)
• VNC

Connecting and Logging In to a Remote System

• Name of login server
• Ssh clients
• X Window tunnelling
• Changing passwords (passwd)
• Public key authentication
• Security policies
• VNC

Setting up Your Remote Environment

• Startup files (.bashrc, .bash_profile, ...)
• Command prompt string
• set -o emacs
• Aliases (ls vs ls -la)
• Default umask
• Does backspace work? (ctrl-H, ctrl-?, backspace, delete,...)

VNC

Transferring Files

• scp, sftp, WinSCP, SSH Secure Shell
• line ending issues
• File names (case sensitive, don't use spaces, some special characters)
• Wireless limitations
• Data transfer speeds

Editing Files

• UNIX character-based editors (emacs, vi)
• Graphical editor (nedit)
• Editing on your own workstation

Working Interactively on a Linux System

• UNIX shell
• Typical UNIX command line, arguments, continuation character
• I/O redirection, standard output and error streams
• man command
• Command history (arrow keys, "set -o emacs", history command)
• Processes, ps

The Toolbox of UNIX Commands

• Converting line endings: dos2unix, unix2dos, file
• Finding stuff: find, locate, grep
• Looking at text files: more, less, cat
• Manipulating text files: sort, uniq, wc, awk, grep, sed

Shell Scripting

• Bash scripts
• Perl
• Python

Data Management

• Local storage policies (quotas, time limits, home directories vs. scratch)
• Backup policy and procedures
• df, du, gzip, tar

Batch Processing

• Interactive vs. Batch
• Batch environment (TORQUE, SLURM, SGE, ...)
• Batch job scripts
• Commands for submitting, monitoring, deleting jobs
• File systems and where to submit your jobs
• Ssh to compute nodes for monitoring processes, memory use, parallel efficiency

Programming

• Languages
• Compilers
• Make
• Debugging

Software

• Locating installed software
• Who is responsible for software installation?
• Typical UNIX software installation procedures

Data Visualization

• Visualization for debugging
• Where to do visualization (local or remote)
• Batch-oriented visualization
• UNIX programs for data visualization

Special Topics

• How to run software specific to your discipline