Updated 2021-05-17
Run Cap’n Proto C++ on the Cluster¶
Summary¶
- Cap'n Proto C++ is a fast data interchange format and RPC system
- Use
module avail capnproto
to see all the available versions on the cluster. Currently, on RHEL6, onlycapnproto-c++/0.5.3
is available - To load Cap'n Proto in your
PBS
script:- Load its dependent module first with
module load gcc/4.9.0
- Load Cap'n Proto with
module load capnproto-c++/0.5.3
- Load its dependent module first with
- To run Cap'n Proto:
- In your
PBS
script, put all lines executing Cap'n Proto after themodule load
line. - Example:
capnp decode myschema.capnp MyType < message.bin > message.txt
would go in thePBS
script after the lines that load the correct modules for Cap'n Proto
- In your
Example PBS Script¶
#PBS -N capnprotoTest
#PBS -A [Account]
#PBS -l nodes=2:ppn=4
#PBS -l walltime=20:00
#PBS -q inferno
#PBS -j oe
#PBS -o capnprotoResult.out
cd $PBS_O_WORKDIR
module load gcc/4.9.0
module load capnproto-c++/0.5.3
capnp decode myschema.capnp MyType < message.bin > message.txt
- The
#PBS
directives are standard, requesting 20 min of walltime and 2 nodes with 4 cores per node. More on#PBS
directives can be found in the PBS guide
Note
If using $PBS_O_WORKDIR
, the .capnp
file, as well as any other files required for the job, must be stored in the same folder as the PBS
script
$PBS_O_WORKDIR
is simply a variable that represents the directory you submit the PBS script from. Make sure the.capnp
files, and any other files you need are in the same directory you put thePBS
script in. This line tells the cluster to enter this directory where you have stored thePBS
script, and look for all the files for the job. If you use$PBS_O_WORKDIR
, you need to have all your files in the same folder as yourPBS
script otherwise the cluster won't be able to find the files it needs.- The
module load
lines load Cap'n Proto and its dependent module (gcc) - The
capnp decode
line is just a general example showing how Cap'n Proto might be used, and is taken from the Cap'n Proto Documentation. Check out the documentation for much more info on Cap'n Proto's functionality. - The point of the
capnp decode
line is to show that commands executing the program must be included after:- Entering the correct folder with all the files and
PBS
script, in this case achieved withcd $PBS_O_WORKDIR
- Cap'n Proto is loaded with the
module load
lines
- Entering the correct folder with all the files and
Submit Job and Check Status¶
- Make sure you're in the directory that contains the
PBS
script, the.capnp
file, and any other files you need. - Submit with
qsub <pbs script name>
. In this caseqsub capnproto.pbs
or whatever you called thePBS
script. You can name thePBS
scripts whatever you want, just keep the.pbs
at the end - Check job status with
qstat -u username3 -n
, replacing "username3" with your gt username - You can delete the job with
qdel 22182721
, replacing the number with the jobid returned after running qsub - Depending on the resources requested and queue the job is run on, it may take varying amounts of time for the job to start. To estimate the time until the job executes, run
showstart 22182721
, replacing the number with the jobid returned after running qsub. More helpful commands can be found in this guide
Collecting Results¶
- All files created will be in the same folder where your
PBS
script is (same directory you ranqsub
from) - The
.out
file will be found here as well. It contains the results of the job, as well as diagnostics and a report of resources used during the job. If the job fails or doesn't produce the result your were hoping for, the.out
file is a great debugging tool. - You can transfer the resulting files off the cluster using scp or a file transfer service