CESM: Difference between revisions

From RCSWiki
Jump to navigation Jump to search
Line 108: Line 108:
Please note, that this information may change as the cluster is constantly changes based on new hardware being added and  
Please note, that this information may change as the cluster is constantly changes based on new hardware being added and  
old hardware being removed.
old hardware being removed.
The '''default''' machine type is '''arc40''' and this type's '''default queue''' is '''cpu2019-bf05'''.
These are the machine type and the queue  which will be used if they are not specified at the '''create_newcase''' step.


== Creating and running a case ==
== Creating and running a case ==

Revision as of 17:51, 9 August 2022

General

  • National Center for Atmospheric Research (NCAR):
https://ncar.ucar.edu/

The Community Earth System Model is a fully-coupled global climate model developed in collaboration with colleagues in the research community. CESM provides state-of-the-art computer simulations of Earth's past, present, and future climate states.


CESM2 is built on the CIME framework. The majority of the CESM2 User’s Guide is contained in the CIME documentation.


The Common Infrastructure for Modeling the Earth (CIME - pronounced “SEAM”) provides a Case Control System for configuring, compiling and executing Earth system models, data and stub model components, a driver and associated tools and libraries.

CESM on ARC

Currently, there are two versions of CESM are installed on ARC, but only one works and supported. The supported version is 2.1.3.

CESM is installed and setup to be used via environmental modules, using the module command.

$ module avail cesm
------------------- /global/software/Modules/4.6.0/modulefiles -------------------
cesm/2.1.1  cesm/2.1.3 

To activate it please load its module:

$ module load cesm/2.1.3

Loading cesm/2.1.3
  Loading requirement: gcc/9.4.0 cmake/3.17.3 git/2.25.0 svn/1.10.6 openmpi/4.1.1-gnu lib/openblas/0.3.13-gnu

This installation of CESM comes with its own dedicated installs of Python and Perl. To verify that the software has been properly activated you can check the locations of some of the commands provided by the install:

$ which python
alias python='python3'
	/global/software/cesm/python/3.10.4/bin/python3

$ which perl
/global/software/cesm/perl/5.34.1/bin/perl

$ which create_newcase
/global/software/cesm/cesm-2.1.3/cime/scripts/create_newcase

If you have any other software modules loaded on ARC, they may interfere with CESM. Please avoid loading too many modules at the same time.

There is a shared data directory, pointed by the DIN_LOC_ROOT environmental variable for CESM data sets. Sharing this storage directory should reduce the amount of data that needs to be downloaded as well as save storage space in users home directories.

Using CESM on ARC

Machines and Queues

The ARC cluster uses the SLURM scheduling system to manage and control jobs. SLURM assumes that their is one main queue for jobs that need to be executed, and that the cluster consists of several partitions. The partitions are collections of compute nodes, that are grouped based on some common properly. On ARC most partitions are grouped based on hardware similarity, scheduling limits, and ownership. CESM has its own model of a compute cluster, that is based on multiple queues and machine types.

In practice, on ARC CESM is setup to use arc40, arc48, and arc52 machine types, compute nodes on which have 40, 48, and 52 CPU cores per node, respectively. However, these machine types can be used in several SLURM partitions. In this case, these partitions do contain machines of the same kind, but the run time limits are different. CESM model treats these SLURM partitions as queues. To create a new case with CESM, therefore, the machine type as well as the target queue have to be indicated.


Queues of the arc40 machine types:

Queue         #Nodes         #CPUs    Max#nodes    MaxRuntime           Comment
name           total         /node        /user         hours
----------------------------------------------------------------------------
cpu2019          40             40            6           168
cpu2019-bf05     87             40           20             5           default 

Queues of the arc48 machine types:

Queue         #Nodes         #CPUs    Max#nodes    MaxRuntime           Comment
name           total         /node        /user         hours
----------------------------------------------------------------------------
cpu2021          34             48           12           168           default
cpu2021-bf24      7             48            4            24  

Queues of the arc52 machine types:

Queue         #Nodes         #CPUs    Max#nodes    MaxRuntime           Comment
name           total         /node        /user         hours
----------------------------------------------------------------------------
cpu2022          52             52           10           168           default
cpu2022-bf24     16             52            4            24  

Please note, that this information may change as the cluster is constantly changes based on new hardware being added and old hardware being removed.

Creating and running a case

Links

ARC Software pages