How to find current limits on ARC - Revision history

Lleung: introduction

2023-09-21T17:19:56Z

introduction

← Older revision		Revision as of 17:19, 21 September 2023
Line 1:		Line 1:
			ARC has many partitions each with different resource limits. It is important to understand these limits when planning a new computation and before submitting a job. For resources that are more scarce such as GPU and bigmem nodes, understanding the resource limits and making your jobs as efficient as possible will help increase the throughput of jobs on the cluster.

	= Resource allocation limits =		= Resource allocation limits =

	~~Checking the current limits on ARC is recommended before planning a new series of computations,~~		The current resource limits can be shown by the <code>arc.limits</code> command:
	~~especially if GPUs are required for the computations.~~

	The current limits can be shown by the <code>arc.limits</code> command:
	<pre>		<pre>
	$ arc.limits		$ arc.limits
Line 57:		Line 56:
	== Backfill partitions ==		== Backfill partitions ==

	Some of the partitions on ARC are not for general use. They may be dedicated to a specific task or may belong to a specific group.		Some of the partitions on ARC are not for general use. They may be dedicated to a specific task or may belong to a specific group. However, some of the nodes from such partitions may also be assigned to additional auxiliary partitions, which names end with the <code>-bfXX</code> suffix, such as <code>cpu2019-bf05</code>, <code>cpu2021-bf24</code>, etc., as they are shown in the [[How to find available partitions on ARC \|output of the <code>arc.hardware</code> command]].
	However, some of the nodes from such partitions may also be assigned to additional auxiliary partitions, which names end with the <code>-bfXX</code> suffix,
	such as <code>cpu2019-bf05</code>, <code>cpu2021-bf24</code>, etc.,
	as they are shown in the [[How to find available partitions on ARC \| output of the <code>arc.hardware</code> command]].

	Currently, the limits for such partitions are set via two auxiliary partitions, <code>backfill05</code> and <code>backfill24</code> instead.		Currently, the limits for such partitions are set via two auxiliary partitions, <code>backfill05</code> and <code>backfill24</code> instead.


	When examining the output of the <code>arc.limits</code> command, and you want to check the limits for any <code>xxxxxxx-bf05</code> partition,		When examining the output of the <code>arc.limits</code> command, and you want to check the limits for any <code>xxxxxxx-bf05</code> partition,
	for example,		for example,
	you have to check the limits of the <code>backfill05</code> instead, to find the current limits for the partition.		you have to check the limits of the <code>backfill05</code> instead, to find the current limits for the partition.


	Correspondingly, for any <code>xxxxxxx-bf24</code> partition, please use the <code>backfill24</code> limits.		Correspondingly, for any <code>xxxxxxx-bf24</code> partition, please use the <code>backfill24</code> limits.
Line 76:		Line 70:
	== gpu-v100 partition limits ==		== gpu-v100 partition limits ==

	'''For example''', the <code>gpu-v100</code> partition is limited to 4000 jobs per user and the maximum run time is limited to 24 hours.		'''For example''', the <code>gpu-v100</code> partition is limited to 4000 jobs per user and the maximum run time is limited to 24 hours. The job must request at least 1 GPU, but not more than 4 GPUs. The total number of GPUs working for a single user's jobs is limited to 8. If the resource request of a submitted job is over the limit, the job will be rejected.
	The job must request at least 1 GPU, but not more than 4 GPUs.
	The total number of GPUs working for a single user's jobs is limited to 8.
	If the resource request of a submitted job is over the limit, the job will be rejected.

	[[Category:Guides]]		[[Category:Guides]]

Lleung: Added arc category and navbox

2023-09-20T22:32:04Z

Added arc category and navbox

← Older revision		Revision as of 22:32, 20 September 2023
Line 83:		Line 83:
	[[Category:Guides]]		[[Category:Guides]]
	[[Category:How-Tos]]		[[Category:How-Tos]]
			[[Category:ARC]]
			{{Navbox ARC}}

Lleung: Added categories

2023-09-20T21:56:11Z

Added categories

← Older revision		Revision as of 21:56, 20 September 2023
Line 80:		Line 80:
	The total number of GPUs working for a single user's jobs is limited to 8.		The total number of GPUs working for a single user's jobs is limited to 8.
	If the resource request of a submitted job is over the limit, the job will be rejected.		If the resource request of a submitted job is over the limit, the job will be rejected.

			[[Category:Guides]]
			[[Category:How-Tos]]

Dmitri: /* Backfill partitions */

2023-03-23T17:07:56Z

Backfill partitions

← Older revision		Revision as of 17:07, 23 March 2023
Line 63:		Line 63:

	Currently, the limits for such partitions are set via two auxiliary partitions, <code>backfill05</code> and <code>backfill24</code> instead.		Currently, the limits for such partitions are set via two auxiliary partitions, <code>backfill05</code> and <code>backfill24</code> instead.


	When examining the output of the <code>arc.limits</code> command, and you want to check the limits for any <code>xxxxxxx-bf05</code> partition,		When examining the output of the <code>arc.limits</code> command, and you want to check the limits for any <code>xxxxxxx-bf05</code> partition,

Dmitri: /* Backfill partitions */

2023-03-23T17:07:36Z

Backfill partitions

← Older revision		Revision as of 17:07, 23 March 2023
Line 69:		Line 69:


	Correspondingly, for any <code>xxxxxxx-bf24</code partition, please use the <code>backfill24</code> limits.		Correspondingly, for any <code>xxxxxxx-bf24</code> partition, please use the <code>backfill24</code> limits.

	= Examples =		= Examples =

Dmitri: /* Backfill partitions */

2023-03-23T17:07:24Z

Backfill partitions

← Older revision		Revision as of 17:07, 23 March 2023
Line 64:		Line 64:
	Currently, the limits for such partitions are set via two auxiliary partitions, <code>backfill05</code> and <code>backfill24</code> instead.		Currently, the limits for such partitions are set via two auxiliary partitions, <code>backfill05</code> and <code>backfill24</code> instead.

	When examining the output of the <code>arc.limits</code> command, and you want to check the limits for any <code>*-bf05<code> partition,		When examining the output of the <code>arc.limits</code> command, and you want to check the limits for any <code>xxxxxxx-bf05</code> partition,
	for example,		for example,
	you have to check the limits of the <code>backfill05</code> instead, to find the current limits for the partition.		you have to check the limits of the <code>backfill05</code> instead, to find the current limits for the partition.


	Correspondingly, for any <code>*-bf24</code partition, please use the <code>backfill24</code> limits.		Correspondingly, for any <code>xxxxxxx-bf24</code partition, please use the <code>backfill24</code> limits.

	= Examples =		= Examples =

Dmitri: /* Backfill partitions */

2023-03-23T17:06:37Z

Backfill partitions

← Older revision		Revision as of 17:06, 23 March 2023
Line 59:		Line 59:
	Some of the partitions on ARC are not for general use. They may be dedicated to a specific task or may belong to a specific group.		Some of the partitions on ARC are not for general use. They may be dedicated to a specific task or may belong to a specific group.
	However, some of the nodes from such partitions may also be assigned to additional auxiliary partitions, which names end with the <code>-bfXX</code> suffix,		However, some of the nodes from such partitions may also be assigned to additional auxiliary partitions, which names end with the <code>-bfXX</code> suffix,
	such as <code>cpu2019-bf05</code>, <code>cpu2021-bf24</code>, etc., as they are shown in the [[How to find available partitions on ARC \| output of the <code>arc.hardware</code> command]].		such as <code>cpu2019-bf05</code>, <code>cpu2021-bf24</code>, etc.,
			as they are shown in the [[How to find available partitions on ARC \| output of the <code>arc.hardware</code> command]].

	Currently, the limits for such partitions are set via two auxiliary partitions, <code>backfill05</code> and <code>backfill24</code> instead.		Currently, the limits for such partitions are set via two auxiliary partitions, <code>backfill05</code> and <code>backfill24</code> instead.


	When examining the output of the <code>arc.limits</code> command, and you want to check the limits for any <code>*-bf05<code> partition,		When examining the output of the <code>arc.limits</code> command, and you want to check the limits for any <code>*-bf05<code> partition,

Dmitri: /* Backfill partitions */

2023-03-23T17:05:50Z

Backfill partitions

← Older revision		Revision as of 17:05, 23 March 2023
Line 62:		Line 62:

	Currently, the limits for such partitions are set via two auxiliary partitions, <code>backfill05</code> and <code>backfill24</code> instead.		Currently, the limits for such partitions are set via two auxiliary partitions, <code>backfill05</code> and <code>backfill24</code> instead.


	When examining the output of the <code>arc.limits</code> command, and you want to check the limits for any <code>*-bf05<code> partition,		When examining the output of the <code>arc.limits</code> command, and you want to check the limits for any <code>*-bf05<code> partition,
	for example,		for example,
	you have to check the limits of the <code>backfill05</code> instead, to find the current limits for the partition.		you have to check the limits of the <code>backfill05</code> instead, to find the current limits for the partition.


	Correspondingly, for any <code>*-bf24</code partition, please use the <code>backfill24</code> limits.		Correspondingly, for any <code>*-bf24</code partition, please use the <code>backfill24</code> limits.

Dmitri: /* Backfill partitions */

2023-03-23T17:05:25Z

Backfill partitions

← Older revision		Revision as of 17:05, 23 March 2023
Line 63:		Line 63:
	Currently, the limits for such partitions are set via two auxiliary partitions, <code>backfill05</code> and <code>backfill24</code> instead.		Currently, the limits for such partitions are set via two auxiliary partitions, <code>backfill05</code> and <code>backfill24</code> instead.

	When examining the output of the <code>arc.limits</code> command, and you want to check the limits for any <code>~~xxxxxxx~~-bf05<code> partition,		When examining the output of the <code>arc.limits</code> command, and you want to check the limits for any <code>*-bf05<code> partition,
	for example,		for example,
	you have to check the limits of the <code>backfill05</code> instead, to find the current limits for the partition.		you have to check the limits of the <code>backfill05</code> instead, to find the current limits for the partition.

	Correspondingly, for any <code>~~xxxxxxxx~~-bf24</code partition, please use the <code>backfill24</code> limits.		Correspondingly, for any <code>*-bf24</code partition, please use the <code>backfill24</code> limits.

	= Examples =		= Examples =

Dmitri: /* Backfill partitions */

2023-03-23T17:04:54Z

Backfill partitions

← Older revision		Revision as of 17:04, 23 March 2023
Line 66:		Line 66:
	for example,		for example,
	you have to check the limits of the <code>backfill05</code> instead, to find the current limits for the partition.		you have to check the limits of the <code>backfill05</code> instead, to find the current limits for the partition.

	Correspondingly, for any <code>xxxxxxxx-bf24</code partition, please use the <code>backfill24</code> limits.		Correspondingly, for any <code>xxxxxxxx-bf24</code partition, please use the <code>backfill24</code> limits.