Sample Job Scripts - Revision history

Ian.percel: revised commands

2021-02-09T22:49:40Z

revised commands

← Older revision		Revision as of 22:49, 9 February 2021
Line 1:		Line 1:
	This page catalogues some typical job scripts for different methods of parallelizing code. These examples are not exhaustive and more examples can be found on individual software pages. However, these are typical of some of the most important styles of computation on an HPC system: Serial Computing, Shared Memory Parallelism, Distributed Memory Parallelism, Job-Level Parallelism and GPU Accelerated Computing.		This page catalogues some typical job scripts for different methods of parallelizing code. These examples are not exhaustive and more examples can be found on individual software pages. However, these are typical of some of the most important styles of computation on an HPC system: Serial Computing, Shared Memory Parallelism, Distributed Memory Parallelism, Job-Level Parallelism and GPU Accelerated Computing. These examples exclude software environment setup and focus on resource requests for different types of parallelism.

	== Serial Computing ==		== Serial Computing ==
Line 69:		Line 69:
	#SBATCH --array=1-10		#SBATCH --array=1-10

	./recon-all $SLURM_ARRAY_TASK_ID		$TOP_DIR=/some/path
			$RESULTS_DIR=/some/other/path

			recon-all -all -i $TOP_DIR/$SLURM_ARRAY_TASK_ID/input.nii -subject $SLURM_ARRAY_TASK_ID -sd $RESULTS_DIR
	</syntaxhighlight>		</syntaxhighlight>

Line 77:		Line 80:
	<syntaxhighlight lang="bash">		<syntaxhighlight lang="bash">
	#!/bin/bash		#!/bin/bash
	#SBATCH --time=12-0:0:0		#SBATCH --time=12:0:0
	#SBATCH --nodes=1		#SBATCH --nodes=1
	#SBATCH --ntasks=1		#SBATCH --ntasks=1

Ian.percel at 06:19, 3 February 2021

2021-02-03T06:19:34Z

← Older revision		Revision as of 06:19, 3 February 2021
Line 54:		Line 54:
	</syntaxhighlight>		</syntaxhighlight>

	== Job-Level Parallelism		== Job-Level Parallelism ==

	<!--Also known as a ''task array'', an array job is a way to submit a whole set of jobs with one command. The individual jobs in the array are distinguished by an environment variable, <code>$SLURM_ARRAY_TASK_ID</code>, which is set to a different value for each instance of the job. The following example will create 10 tasks, with values of <code>$SLURM_ARRAY_TASK_ID</code> ranging from 1 to 10:-->		<!--Also known as a ''task array'', an array job is a way to submit a whole set of jobs with one command. The individual jobs in the array are distinguished by an environment variable, <code>$SLURM_ARRAY_TASK_ID</code>, which is set to a different value for each instance of the job. The following example will create 10 tasks, with values of <code>$SLURM_ARRAY_TASK_ID</code> ranging from 1 to 10:-->

Ian.percel: Update of basic slurm job script examples for different types of parallelism

2021-02-03T06:18:57Z

Update of basic slurm job script examples for different types of parallelism

New page

This page catalogues some typical job scripts for different methods of parallelizing code. These examples are not exhaustive and more examples can be found on individual software pages. However, these are typical of some of the most important styles of computation on an HPC system: Serial Computing, Shared Memory Parallelism, Distributed Memory Parallelism, Job-Level Parallelism and GPU Accelerated Computing.

== Serial Computing ==

This script launches one process with one CPU core to run for 5 minutes with 1G of memory.

'''serial_job.slurm''':
<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --time=0-0:5
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=1
#SBATCH --mem=1G

~/anaconda3/bin/python -c "print(3+5)"
</syntaxhighlight>

== Shared Memory Parallelism ==

This script starts one process with four CPU cores.
In order for this to work, the OpenMP code arrayUpdate.c must be compiled with OpenMP support, e.g.
<code>gcc -fopenmp arrayUpdate.c</code>

'''sharedmemory_job.slurm''':
<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --time=0-1:5:0
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=4
#SBATCH --mem-per-cpu=1000M

export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK
./arrayUpdate
</syntaxhighlight>

== Distributed Memory Parallelism ==

This example script launches four MPI processes, each with 1000 MB of memory.

'''distributedMemory_job.slurm''':
<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --time=1:00:00
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=4 # number of MPI processes should match total ntasks
#SBATCH --cpus-per-task=1
#SBATCH --mem-per-cpu=1000M # memory allocated for each cpu

mpiexec ./matrixProduct_program
</syntaxhighlight>

== Job-Level Parallelism




'''jobParallel_job.slurm''':
<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --time=1-0:0:0
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=1
#SBATCH --mem=1G
#SBATCH --array=1-10

./recon-all $SLURM_ARRAY_TASK_ID
</syntaxhighlight>

== GPU Accelerated Computing ==
'''gpu_job.slurm''':
<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --time=12-0:0:0
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=1
#SBATCH --mem=50000M
#SBATCH --gres=gpu:1

python tensorflow_example.py
</syntaxhighlight>