Difference between revisions of "How to check GPU utilization"

From RCSWiki
Jump to navigation Jump to search
(Created page with "= General = = Links = How-Tos")
 
Line 1: Line 1:
 
= General =
 
= General =
 +
 +
If you have a job that is running on a GPU node and that is expected to use a GPU on that node,
 +
you can check the GPU use by your code by running the following command on ARC's login node:
 +
$ srun -s --jobid 12345678 --pty nvidia-smi
 +
 +
The number here is the job ID of the running job.
 +
 +
The output should look similar to the following:
 +
<pre>
 +
Mon Aug 22 09:27:38 2022     
 +
+-----------------------------------------------------------------------------+
 +
| NVIDIA-SMI 470.57.02    Driver Version: 470.57.02    CUDA Version: 11.4    |
 +
|-------------------------------+----------------------+----------------------+
 +
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
 +
| Fan  Temp  Perf  Pwr:Usage/Cap|        Memory-Usage | GPU-Util  Compute M. |
 +
|                              |                      |              MIG M. |
 +
|===============================+======================+======================|
 +
|  0  Tesla V100-PCIE...  Off  | 00000000:3B:00.0 Off |                    0 |
 +
| N/A  33C    P0    36W / 250W |    848MiB / 16160MiB |    30%      Default |
 +
|                              |                      |                  N/A |
 +
+-------------------------------+----------------------+----------------------+
 +
                                                                             
 +
+-----------------------------------------------------------------------------+
 +
| Processes:                                                                  |
 +
|  GPU  GI  CI        PID  Type  Process name                  GPU Memory |
 +
|        ID  ID                                                  Usage      |
 +
|=============================================================================|
 +
|    0  N/A  N/A  2232533      C  .../Programs/OpenDBA/openDBA      338MiB |
 +
+-----------------------------------------------------------------------------+
 +
</pre>
 +
In this case there was 1 GPU allocated and its usage was '''30%'''.
 +
The code <code>openDBA</code> also uses '''338 MB''' of the GPU memory.
  
 
= Links =
 
= Links =
  
 
[[How-Tos]]
 
[[How-Tos]]

Revision as of 15:33, 22 August 2022

General

If you have a job that is running on a GPU node and that is expected to use a GPU on that node, you can check the GPU use by your code by running the following command on ARC's login node:

$ srun -s --jobid 12345678 --pty nvidia-smi

The number here is the job ID of the running job.

The output should look similar to the following:

Mon Aug 22 09:27:38 2022       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.57.02    Driver Version: 470.57.02    CUDA Version: 11.4     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla V100-PCIE...  Off  | 00000000:3B:00.0 Off |                    0 |
| N/A   33C    P0    36W / 250W |    848MiB / 16160MiB |     30%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A   2232533      C   .../Programs/OpenDBA/openDBA      338MiB |
+-----------------------------------------------------------------------------+

In this case there was 1 GPU allocated and its usage was 30%. The code openDBA also uses 338 MB of the GPU memory.

Links

How-Tos