Updated 2021-05-17
Run openMP Programs on the Cluster¶
Overview¶
- openMP is a useful tool to write parallel programs in C, C++, and fortran
- Workflow is similar to running other programs on the cluster:
- First write PBS Script, which defines resources and compiles C program with openMP
- Submit to scheduler to run on compute nodes
- Gather results
- Example C program for this guide can be found here. The program simply prints out "Hello World" from every thread that is running
- Example PBS script, which will be explained, can be found here
- Move the files to your account on the cluster, or write them on the cluster to follow along
Walkthrough: Run an Example openMP Script¶
Part 1: The PBS Script & Loading the Compiler¶
#PBS -N openMP_example_script
#PBS -l nodes=2:ppn=4
#PBS -l pmem=2gb
#PBS -l walltime=00:30
#PBS -q inferno
#PBS -j oe
#PBS -o openMP_example.out
#PBS -m abe
#PBS -M shollister7@gatech.edu
cd ~/test_directory
echo "Started on `/bin/hostname`"
module load gcc/4.9.0
export OMP_NUM_THREADS=8
gcc -fopenmp openMP_example.c -o openMP_example
./openMP_example
- The
#PBS
lines are directives, covered in depth in the PBS guide - The first step is to tell the cluster to enter the directory where the parallel C program is located (in this case ~/test_directory, a dir I have made in my home directory). Should be whatever directory you put the C script to be run in.
- Make sure any dependencies are in that same directory you told the cluster to enter
- The second line prints the name of the node that job is running on to the output file
- Then, the preffered version of the c compiler is loaded. Many versions of gcc and intel compilers are available on the cluster. To find available versions of a software, run
module avail
module avail <software name> #ex: module avail gcc would show available verisons of gcc
- to load a software, include the
module load <module name>
line in the pbs script. You must load the software's module before you run the software in the pbs script. - Next, you MUST set OMP_NUM_THREADS to the number of threads you requested in the beginning (nodes x ppn), otherwise you might slow down the system. In this case, it would be 4 x 2 = 8.
Warning
You must set OMP_NUM_THREADS to the number of threads specified in the top of the PBS script (nodes x ppn)
- Then, the C program is compiled with openMP
- Finally, the program is run, and the output is automatically recorded in the out file (which will show up in the directory you submitted the script from)
Part 2: Submit the PBS Script and Collect Results¶
- make sure your're in the folder where the openMP PBS Script is located, and run *
qsub <scriptName.pbs> #ex: qsub openMP.pbs for this guide
- if successful, this will print something like
2180446.shared-sched-pace.gatech.edu
- the number in the beginning is the job id, useful for checking predicted wait time in queue or job status
- After a couple seconds, find estimated wait time in queue with
showstart <jobID>
- check job status with
qstat <jobID> or qstat -u someuser3 -n
- for more ways to check status, how to cancel job, and more useful commands, checkout the command cheatsheet
- Any files created by the script will show up in the folder where the script was ran (unless otherwise programmed)
- the output file will be found by typing
ls
and looking for the output file you named in the PBS script, in this caseopenMP_example.out
- To see the contents of the out file, run
cat <output file name> #ex: cat openMP_example.out
- output for example openMP script should print out "hello world" 8 times, one for every thread:
- To move output files off the cluster, see file transfer guide
- Congratulations! you have succesfully run a parallel C script using openMP on the cluster