HPC Linux topics
Many researchers, particularly those coming from a Microsoft Windows background, intending to use a remote HPC system for the first time, request some kind of Linux training.
The WestGrid QuickStart Guide for New Users gives an overview of the range of topics that are typically relevant for someone getting started on such a system, with links to details that are specific to the WestGrid environment. Many of the topics covered there are relevant for most remote Linux systems, so, you might like to take a look at it.
To focus discussions with those who might be interested in obtaining more pesonalized HPC Linux training from University of Calgary Information Technologies Research Computing Services support staff, an expanded list of possible topics is shown below.
If you are starting with local HPC resources or the national Compute Canada facilities, an offer is usually made to meet with you for an hour or two to make sure your own computer is set up and to go over the main things you need to know to start using the remote computational resources.
If you are interested in meeting individually or as a group with UC HPC staff to learn more about using remote Linux systems, please contact us at firstname.lastname@example.org.
The following topics may be of interest.
- What are the general characteristics of the hardware
- Linux commands for identifying the hardware and system
Know Your Administrators and Help Resources
- Who is looking after the system?
- Local contact
- IT support contact
- Mailing lists
- Local network support
- Where to go for help
Setting up Your Computer
- Who administers your local computer?
- Is your local machine shared or private?
- Security (anti-virus software etc.) on your own computer
- Data backup on your computer
- Ssh client (SSH Secure Shell or PuTTY for Windows, built-in ssh for Linux or Mac)
- File transfer client (WinSCP or SSH Secure Shell for Windows, built-in scp or sftp for Linux or Mac, Fetch or Fugu for Mac, etc.)
- X Window display server (xming for Windows, built-in for Linux and Mac)
Connecting and Logging In to a Remote System
- Name of login server
- Ssh clients
- X Window tunnelling
- Changing passwords (passwd)
- Public key authentication
- Security policies
Setting up Your Remote Environment
- Startup files (.bashrc, .bash_profile, ...)
- Command prompt string
- set -o emacs
- Aliases (ls vs ls -la)
- Default umask
- Does backspace work? (ctrl-H, ctrl-?, backspace, delete,...)
- scp, sftp, WinSCP, SSH Secure Shell
- line ending issues
- File names (case sensitive, don't use spaces, some special characters)
- Wireless limitations
- Data transfer speeds
- UNIX character-based editors (emacs, vi)
- Graphical editor (nedit)
- Editing on your own workstation
Working Interactively on a Linux System
- UNIX shell
- Typical UNIX command line, arguments, continuation character
- I/O redirection, standard output and error streams
- man command
- Command history (arrow keys, "set -o emacs", history command)
- Processes, ps
The Toolbox of UNIX Commands
- Converting line endings: dos2unix, unix2dos, file
- Finding stuff: find, locate, grep
- Looking at text files: more, less, cat
- Manipulating text files: sort, uniq, wc, awk, grep, sed
- Bash scripts
- Local storage policies (quotas, time limits, home directories vs. scratch)
- Backup policy and procedures
- df, du, gzip, tar
- Interactive vs. Batch
- Batch environment (TORQUE, SLURM, SGE, ...)
- Batch job scripts
- Commands for submitting, monitoring, deleting jobs
- File systems and where to submit your jobs
- Ssh to compute nodes for monitoring processes, memory use, parallel efficiency
- Locating installed software
- Who is responsible for software installation?
- Typical UNIX software installation procedures
- Visualization for debugging
- Where to do visualization (local or remote)
- Batch-oriented visualization
- UNIX programs for data visualization
- How to run software specific to your discipline