General Cluster Guidelines and Policies - Revision history

Lleung: Added navbox

2023-09-21T20:41:47Z

Added navbox

← Older revision		Revision as of 20:41, 21 September 2023
Line 44:		Line 44:

	'''Please do not run CPU-only computations on the GPU partitions.'''		'''Please do not run CPU-only computations on the GPU partitions.'''

			[[Category:Administration]]
			{{Navbox Administration}}

Dmitri: /* GPU Partitions */

2023-06-04T05:05:44Z

GPU Partitions

← Older revision		Revision as of 05:05, 4 June 2023
Line 44:		Line 44:

	'''Please do not run CPU-only computations on the GPU partitions.'''		'''Please do not run CPU-only computations on the GPU partitions.'''

	~~__NOTOC__~~

Dmitri: /* GPU Partitions */

2023-06-04T05:05:29Z

GPU Partitions

← Older revision		Revision as of 05:05, 4 June 2023
Line 43:		Line 43:
	The GPU partitions, such as '''gpu-v100''' and '''gpu-a100''', are strictly for computations that utilize GPUs.		The GPU partitions, such as '''gpu-v100''' and '''gpu-a100''', are strictly for computations that utilize GPUs.

	'''Please do not run CPU-only computations on the GPU ~~partition~~.'''		'''Please do not run CPU-only computations on the GPU partitions.'''

	__NOTOC__		__NOTOC__

Dmitri: /* gpu-v100 Partition */

2023-06-04T05:05:07Z

gpu-v100 Partition

← Older revision		Revision as of 05:05, 4 June 2023
Line 40:		Line 40:
	'''Please avoid running low memory computations on the bigmem partition.'''		'''Please avoid running low memory computations on the bigmem partition.'''

	=== ~~gpu-v100 Partition~~ ===		=== GPU Partitions ===
	The gpu-v100 ~~partition is~~ strictly for computations that utilize GPUs.		The GPU partitions, such as '''gpu-v100''' and '''gpu-a100''', are strictly for computations that utilize GPUs.

	'''Please do not run CPU-only computations on the ~~gpu-v100~~ partition.'''		'''Please do not run CPU-only computations on the GPU partition.'''

	__NOTOC__		__NOTOC__

Dmitri: /* General Rules */

2021-08-17T05:26:37Z

General Rules

← Older revision		Revision as of 05:26, 17 August 2021
Line 1:		Line 1:
	== General Rules ==		== General Rules ==
	* '''Never''' run anything related to your research on the '''Login Node'''.		* '''Never''' run anything related to your research on the '''Login Node'''.

	* You have to make sure that the '''resources you request''' for the job are '''used''' by the job. <br />When resources are requested from '''SLURM''' by the job script, the resources are reserved for the job and will not be allocated for other users. Jobs that do not make complete use of the allocated resource reduces the overall cluster efficiency. It is essential that the user makes resource requests that fit with the requirements of their jobs so that resources are properly used.		* You have to make sure that the '''resources you request''' for the job are '''used''' by the job. <br />When resources are requested from '''SLURM''' by the job script, the resources are reserved for the job and will not be allocated for other users. Jobs that do not make complete use of the allocated resource reduces the overall cluster efficiency. It is essential that the user makes resource requests that fit with the requirements of their jobs so that resources are properly used.

			* Users are expected to '''monitor their UofC e-mail addresses''' associated with their accounts, so they can receive communication from system administrators and analysts about the state of the systems and their accounts.

	== Guidelines ==		== Guidelines ==

Dmitri: /* Short jobs */

2020-09-04T19:13:34Z

Short jobs

← Older revision		Revision as of 19:13, 4 September 2020
Line 19:		Line 19:
	Scheduling a node for a new job takes time and if jobs are too short, then scheduling time becomes similar or longer than the actual run time of the job, which is very '''inefficient'''.		Scheduling a node for a new job takes time and if jobs are too short, then scheduling time becomes similar or longer than the actual run time of the job, which is very '''inefficient'''.

	If you are expecting to run a large number of very short jobs, that are from seconds to several minutes long, please '''pack several''' of those computations into ~~one~~ longer jobs, ~~that is~~ 2-3 hours long. For example, if you have 200000 jobs that ran for 30 seconds, consider ~~run~~ 1000 of these computations inside ~~one~~ job, ~~with~~ will result in 200 medium long jobs.		If you are expecting to run a large number of very short jobs, that are from seconds to several minutes long, please '''pack several''' of those computations into longer jobs, 2-3 hours long. For example, if you have 200000 jobs that ran for 30 seconds, consider running 1000 of these computations inside a single job, which will result in 200 medium long jobs instead of 200000 very short ones.

	=== Data Transfer Node ===		=== Data Transfer Node ===

Dmitri: /* Interactive Jobs */

2020-09-04T19:04:50Z

Interactive Jobs

← Older revision		Revision as of 19:04, 4 September 2020
Line 25:		Line 25:

	=== Interactive Jobs ===		=== Interactive Jobs ===
	Interactive jobs can be started using the '''[[Running_jobs#Interactive_jobs\|salloc command]]''' and are limited to a maximum of 5 hours.		Interactive jobs can be started using the '''[[Running_jobs#Interactive_jobs\|salloc command]]''' and are limited to a '''maximum of 5 hours'''.

	The reason for the time restriction on interactive jobs are:		The reason for the time restriction on interactive jobs are:

Dmitri: /* Data Transfer Node */

2020-09-04T19:04:31Z

Data Transfer Node

← Older revision		Revision as of 19:04, 4 September 2020
Line 22:		Line 22:

	=== Data Transfer Node ===		=== Data Transfer Node ===
	If the cluster has a Data Transfer Node (DTN), please use it rather than the login node to transfer files to/from the cluster.		If the cluster has a '''Data Transfer Node''' (DTN), please use it rather than the login node to transfer files to/from the cluster.

	=== Interactive Jobs ===		=== Interactive Jobs ===

Dmitri: /* Data Transfer Node */

2020-09-04T19:04:22Z

Data Transfer Node

← Older revision		Revision as of 19:04, 4 September 2020
Line 22:		Line 22:

	=== Data Transfer Node ===		=== Data Transfer Node ===
	If the cluster has a Data Transfer Node (DTN), please use it rather than the login node to transfer files to/from the cluster.		If the cluster has a Data Transfer Node (DTN), please use it rather than the login node to transfer files to/from the cluster.

	=== Interactive Jobs ===		=== Interactive Jobs ===

Dmitri: /* Login Node */

2020-09-04T19:04:07Z

← Older revision		Revision as of 19:04, 4 September 2020
Line 8:		Line 8:
	=== Login Node ===		=== Login Node ===
	The login node should be used only for:		The login node should be used only for:
	* Data management, that is file management, compression / decompression, and, possibly, data transfer.		* '''Data management''', that is file management, compression / decompression, and, possibly, data transfer.
	* Job management: job script creation / submission / monitoring.		* '''Job management''': job script creation / submission / monitoring.
	* Software development: Source editing / compilation.		* Software '''development''': Source editing / compilation.
	* Short data analysis computations that take 100% of 1 CPU for up to 15 minutes.		* Short data analysis computations that take 100% of 1 CPU for up to '''15 minutes'''.

	Everything else should be run on compute nodes either via the '''[[Running_jobs#Use_sbatch_to_submit_jobs\|sbatch command]]''' or in an interactive job via the '''[[Running_jobs#Interactive_jobs\|salloc command]]'''. These restrictions are in place to ensure that the login node remains available for other users and is not unnecessarily overburdened.		Everything else should be run on compute nodes either via the '''[[Running_jobs#Use_sbatch_to_submit_jobs\|sbatch command]]''' or in an interactive job via the '''[[Running_jobs#Interactive_jobs\|salloc command]]'''. These restrictions are in place to ensure that the login node remains available for other users and is not unnecessarily overburdened.