Run NCO on the Cluster¶
- netCDF Operators is a suite of programs designed to facilitate manipulation and analysis of self-describing data stored in netCDF format.
- This guide will cover how to run NCO on the Cluster.
- You can find NCO home page here.
- You can find a useful slideshow about using NCO here.
- NCO includes many different programs, so the exact commands you execute depend on what operation you are trying to perform on your netCDF file.
- For this guide, we will show you how to run an NCO operation on the Cluster which you can substitute for any other operation that fits your needs.
Walkthrough: Run [Software] on the Cluster¶
- This walkthrough will cover how to output longitude and latitude data from a netCDF file.
- The example netCDF file can be downloaded here
- PBS Script can be found here
- You can transfer the files to your account on the cluster to follow along. The file transfer guide may be helpful.
Part 1: The PBS Script¶
#PBS -N ncoTest #PBS -A [Account] #PBS -l nodes=1:ppn=2 #PBS -l pmem=2gb #PBS -l walltime=3:00 #PBS -q inferno #PBS -j oe #PBS -o ncoTest.out cd $PBS_O_WORKDIR module load intel/15.0 module load hdf5/1.8.14 module load netcdf/4.3.3 module load nco/4.6.0 ncks -v lat,lon sresa1b_ncar_ccsm3-example.nc
#PBSdirectives are standard, requesting just 3 minutes of walltime and 1 node with 2 cores. More on
#PBSdirectives can be found in the PBS guide
$PBS_O_WORKDIRis a variable that represents the directory you submit the PBS script from. Make sure the files you want to use are in the same directory you put the PBS script.
- Output Files will also show up in this dir as well
module load nco/4.6.0loads the 4.6.0 version of NCO. To see what NCO versions are available, run
module avail NCO, and load the one you want. The other module are dependencies that must be loaded before NCO is loaded.
ncks -v lat,lon sresa1b_ncar_ccsm3-example.ncis used to output the lat and lon variables from
Part 2: Submit Job and Check Status¶
- Make sure you're in the dir that contains the
PBSScript as well as the
- Submit as normal, with
qsub <pbs script name>. In this case
- Check job status with
qstat -t 22182721, replacing the number with the job id returned after running qsub
- You can delete the job with
qdel 22182721, again replacing the number with the jobid returned after running qsub
Part 3: Collecting Results¶
- In the directory where you submitted the
PBSscript, you should see a
ncoTest.outfile which contains the lat and lon ouput from
cator open the file in a text editor to take a look.
ncoTest.outshould look like this
- After the result files are produced, you can move the files off the cluster, refer to the file transfer guide for help.
- Congratulations! You successfully ran NCO on the cluster.