Updated 2023-03-31
Run NCO on the Cluster¶
Overview¶
- netCDF Operators is a suite of programs designed to facilitate manipulation and analysis of self-describing data stored in netCDF format.
- This guide will cover how to run NCO on the Cluster.
- You can find NCO home page here.
- You can find a useful slideshow about using NCO here.
Summary¶
- NCO includes many different programs, so the exact commands you execute depend on what operation you are trying to perform on your netCDF file.
- For this guide, we will show you how to run an NCO operation on the Cluster which you can substitute for any other operation that fits your needs.
Walkthrough: Run [Software] on the Cluster¶
- This walkthrough will cover how to output longitude and latitude data from a netCDF file.
Input file
: sresa1b_ncar_ccsm3-example.ncSBATCH
Script can be found here- You can transfer the files to your account on the cluster to follow along. The file transfer guide may be helpful.
Part 1: The SBATCH Script¶
#!/bin/bash
#SBATCH -JncoTest
#SBATCH -A [Account]
#SBATCH -N1 --ntasks-per-node=2
#SBATCH --mem-per-cpu=2G
#SBATCH -t3
#SBATCH -qinferno
#SBATCH -oReport-%j.out
cd $SBATCH
module load gcc/10.3.0
module load mvapich2/2.3.6
module load nco/5.0.1
ncks -v lat,lon sresa1b_ncar_ccsm3-example.nc
- The
#SBATCH
directives are standard, requesting just 3 minutes of walltime and 1 node with 2 cores. More on#SBATCH
directives can be found in the Using Slurm on Phoenix Guide $SBATCH
is a variable that represents the directory you submit the SBATCH script from. Make sure the files you want to use are in the same directory you put the SBATCH script.- Output Files will also show up in this dir as well
module load nco/5.0.1
loads the 4.6.0 version of NCO. To see what NCO versions are available, runmodule avail NCO
, and load the one you want. The other module are dependencies that must be loaded before NCO is loaded.ncks -v lat,lon sresa1b_ncar_ccsm3-example.nc
is used to output the lat and lon variables fromsresa1b_ncar_ccsm3-example.nc
.
Part 2: Submit Job and Check Status¶
- Make sure you're in the dir that contains the
SBATCH
Script as well asNCO
file. - Submit as normal, with
sbatch <script name>
. In this casesbatch nco.sbatch
- Check job status with
squeue --job <jobID>
, replacing with the jobid returned after running sbatch - You can delete the job with
scancel <jobID>
, replacing with the jobid returned after running sbatch
Part 3: Collecting Results¶
SBATCH* Report-<jobID>.out
should look like this
* After the result files are produced, you can move the files off the cluster, refer to the file transfer guide for help.
* Congratulations! You successfully ran NCO on the cluster.