How to run an interactive job on ARC: Difference between revisions
| Line 12: | Line 12: | ||
(See our [[General Cluster Guidelines and Policies#Login Node | guidelines and policies]]). | (See our [[General Cluster Guidelines and Policies#Login Node | guidelines and policies]]). | ||
=== Interactive work === | |||
Sometimes, there is a need to run some research work related commands that are CPU heavy, | Sometimes, there is a need to run some research work related commands that are CPU heavy, | ||
but which can be '''difficult or tedious''' to run as '''normal non-interactive batch jobs'''. | but which can be '''difficult or tedious''' to run as '''normal non-interactive batch jobs'''. | ||
| Line 28: | Line 28: | ||
The resources you are using are allocated to your only and there is nobody else who can use them. | The resources you are using are allocated to your only and there is nobody else who can use them. | ||
=== Downsides === | |||
However, one has to remember that there are some '''negatives''' of using '''interactive jobs''': | However, one has to remember that there are some '''negatives''' of using '''interactive jobs''': | ||
* '''resource utilization''' is typically '''quite low''' when compared with non-interactive jobs. | * '''resource utilization''' is typically '''quite low''' when compared with non-interactive jobs. | ||
| Line 38: | Line 38: | ||
* Typically, you can only interact with '''one job at a time'''. | * Typically, you can only interact with '''one job at a time'''. | ||
: In contrast, you can run multiple '''non-interactive''' jobs when running normal '''batch''' jobs. | : In contrast, you can run multiple '''non-interactive''' jobs when running normal '''batch''' jobs. | ||
* If the resources you want for your interactive work are not available, you may be waiting for a long time before your interactive job starts. | |||
Because of these reasons, on ARC, interactive jobs have to be limited to up to '''5 hours of run time'''. | |||
Please remember that interactive jobs is not the main intended way of using the ARC cluster. | |||
Revision as of 17:48, 16 August 2023
Background
Running jobs is the main and the only way to run computations on ARC.
Normally, a user creates a job script and submit it to SLURM on ARC using the sbatch command to run a computation.
The login node, ARC's gateway, is only intended to do management work, such as
- data management,
- software management, and
- job management.
The login node could also be used to software development, if you write scripts, or if you work on some new code for your research. The login node, however, cannot be used to run anything that is CPU heavy and relates to research, that is not management. (See our guidelines and policies).
Interactive work
Sometimes, there is a need to run some research work related commands that are CPU heavy, but which can be difficult or tedious to run as normal non-interactive batch jobs. Possibly, you may want to do some interactive work using the command line, transform your data, collect results of a large set of computations, try or test new methods or software. For this kind of work, the best approach is to request an interactive job from ARC's SLURM.
To request an interactive job on ARC via SLURM,
you have to use the salloc command and to specify resources that your interactive work will need,
similar to the resource request in a normal job script.
After that, SLURM will try to allocate resources for your interactive job,
and then it will transfer your command line session from ARC's login node to a new command line session on the allocated compute node.
In this interactive job you can run any command on the command line without interrupting any other user's work.
The resources you are using are allocated to your only and there is nobody else who can use them.
Downsides
However, one has to remember that there are some negatives of using interactive jobs:
- resource utilization is typically quite low when compared with non-interactive jobs.
- The resources are not used when you are reading the outputs, type new commands, or think about what to do next. This is inefficient.
- There is no amount of work that have to be done, so that the resources can be freed for other users.
- The only way an interactive job can end is if it runs out of time, or you manually end it.
- Typically, you can only interact with one job at a time.
- In contrast, you can run multiple non-interactive jobs when running normal batch jobs.
- If the resources you want for your interactive work are not available, you may be waiting for a long time before your interactive job starts.
Because of these reasons, on ARC, interactive jobs have to be limited to up to 5 hours of run time.
Please remember that interactive jobs is not the main intended way of using the ARC cluster.