Updated 2023-03-31

Use TBB on the Cluster

Summary

  • Use module spider intel-tbb to see all the versions of intel compilers and TBB available on the cluster
  • Load the version of the intel-tb you want in your SBATCH script with module load intel/<version number>. module load intel loads the default version.
  • In your SBATCH script, after the lines for loading the correct modules, compile your C++ program with g++
  • Then below that line, execute the C++ program
  • Submit the job with sbatch <filename>.sbatch and collect results

Walkthrough: Use TBB on the Cluster

  • This walkthrough will use a simple C++ program that imports a header file from the TBB library and prints a string
  • Example C program: intel-tbb.cpp
  • SBATCH Script: intel-tbb.sbatch
  • To follow along, manually copy these files into the same directory in your account on the cluster, or file transfer them to your account (again put them in the same folder).

Part 1: The SBATCH Script

#!/bin/bash
#SBATCH -JtbbTest
#SBATCH -A [Account]
#SBATCH -N1 --ntasks-per-node=2
#SBATCH -t1
#SBATCH -qinferno
#SBATCH -oReport-%j.out

cd $SLURM_SUBMIT_DIR
module load intel/20.0.4
module load intel-tbb/2020.3
icc -o example tbb_example.cpp
./example
  • The #SBATCH directives are standard, requesting just 1 minute of walltime and 1 node with 2 cores. More on #SBATCH directives can be found in the Using Slurm on Phoenix Guide
  • $SLURM_SUBMIT_DIR is simply a variable that represents the directory you submit the SBATCH script from. Make sure the .cpp file you want to run is in the same directory you put the SBATCH script. Otherwise, the C++ program wont be found when the job is run.

Important

Output Files, such as the resulting executable, will also show up in the same directory as the SBATCH script

  • The module load lines load gcc (compiler) and gsl.
  • The icc line compiles the C++ program into an executable called example .
  • ./example runs the program

Part 2: Submit Job and Check Status

  • Be sure to change to the directory that contains the SBATCH Script as well as the .cpp file
  • Submit as normal, with qsub <sbatch scriptname.sbatch>. In this case sbatch intel-tbb.sbatch
  • Check job status with squeue --job <jobID>, replacing with the jobid returned after running sbatch
  • You can delete the job with scancel <jobID> , replacing with the jobid returned after running sbatch

Part 3: Collecting Results

  • In the directory where you submitted the SBATCH script, you should see a couple of newly generated files, including example and Report-<jobID>.out
  • example is the executable file created
  • Report-<jobID>.out contains the results of the job
  • Open Report-<jobID>.out with vim Report-<jobID>.out (you can use any text editor).
  • The output should contain the result of the computation:
---------------------------------------
Begin Slurm Prolog: Dec-31-2022 11:19:18
Job ID:    334649
User ID:   svangala3
Account:   phx-pace-staff
Job name:  tbbTest
Partition: cpu-small
QOS:       inferno
---------------------------------------
Simple test importing tbb
---------------------------------------
Begin Slurm Epilog: Dec-31-2022 11:19:25
Job ID:        334649
Array Job ID:  _4294967294
User ID:       svangala3
Account:       phx-pace-staff
Job name:      tbbTest
Resources:     cpu=2,mem=2G,node=1
Rsrc Used:     cput=00:00:14,vmem=9204K,walltime=00:00:07,mem=0,energy_used=0
Partition:     cpu-small
QOS:           inferno
Nodes:         atl1-1-02-007-9-1
---------------------------------------
  • After the result files are produced, you can move the out file as well as any other files off the cluster. Refer to the file transfer guide for help.
  • Congratulations! You successfully used TBB on the cluster.