Updated 2023-03-27

MPI for Python

License

MPI for Python on PACE uses the Georgia Tech license, for which an annual access fee is required per user.

Overview

This document describes the MPI for Python package. MPI for Python provides Python bindings for the Message Passing Interface (MPI) standard, allowing Python applications to exploit multiple processors on workstations, clusters and supercomputers.

Walkthrough: Run an Example MPI4py Script

Running MPI4py Interactivley

Allocating Resources

  • In order to run Keras interactivley we can use the salloc command to specify the account, partitions, time, and queue

  • Here is an example of an salloc command you can use: salloc -A [Account] -N 1 -n 8 -t 15 -q embers

  • This will allocate the proper resources to run MPI4py

Using Hello World Example

  • The following example will show running MPI4py interactively using a hello world example

  • Here is what the helloworld.py script should look like:

#!/usr/bin/env python
"""
Parallel Hello World
"""

from mpi4py import MPI
import sys

size = MPI.COMM_WORLD.Get_size()
rank = MPI.COMM_WORLD.Get_rank()
name = MPI.Get_processor_name()

sys.stdout.write(
    "Hello, World! I am process %d of %d on %s.\n"
    % (rank, size, name))

  • We can do the following in order to run this script with MPI4py:

  • Load module: module load py-mpi4py/3.1.2

  • Export the mvapich2 threading:

export MV2_USE_ALIGNED_ALLOC=1

export MV2_USE_THREAD_WARNING=0

  • Run Script: srun python helloworld.py

  • Your output should look something like this: 

Hello, World! I am process 0 of 8 on atl1-1-02-006-15-2.pace.gatech.edu.
Hello, World! I am process 2 of 8 on atl1-1-02-006-15-2.pace.gatech.edu.
Hello, World! I am process 4 of 8 on atl1-1-02-006-15-2.pace.gatech.edu.
Hello, World! I am process 6 of 8 on atl1-1-02-006-15-2.pace.gatech.edu.
Hello, World! I am process 5 of 8 on atl1-1-02-006-15-2.pace.gatech.edu.
Hello, World! I am process 1 of 8 on atl1-1-02-006-15-2.pace.gatech.edu.
Hello, World! I am process 7 of 8 on atl1-1-02-006-15-2.pace.gatech.edu.
Hello, World! I am process 3 of 8 on atl1-1-02-006-15-2.pace.gatech.edu.

Running MPI4py in Batch Mode

  • We can also test this in a normal batch mode. Here is an example batch script:
#!/bin/bash
#SBATCH -J mpi4py_test
#SBATCH -A [Account]
#SBATCH -N 1 -n 8
#SBATCH -t 15
#SBATCH -q embers
#SBATCH -o Report-%j.out
#SBATCH -e Report-%j.err

cd $SLURM_SUBMIT_DIR

module load mpi4py

srun python helloworld.py

Submitting job and collecting results

  • Make sure your're in the folder where the SBATCH Script is located, and run
sbatch <scriptName.sbatch>  #ex: sbatch ex_mpi4py.sbatch

  • If successful, this will print something like Submitted batch job 1280063

  • The number in the beginning is the job id, useful for checking predicted wait time in queue or job status

  • After a couple seconds, find estimated wait time in queue and job status with squeue -u ACCOUNTNAME

  • Any files created by the script will show up in the folder where the script was ran (unless otherwise programmed)

  • The output file will be found by typing ls and looking for the output file you named in the SBATCH script, in this case something like Report-1280063.out
  • To see the contents of the out file, run vi <output file name> #ex: vi mpipython-test.out
  • This is what you can expect as an output of running mpi4py interactively with helloworld.py:
---------------------------------------
Begin Slurm Prolog: Mar-21-2023 01:18:16
Job ID:    1331893
User ID:   gburdell3
Account:   [Account]
Job name:  mpipython-test
Partition: cpu-small
QOS:       embers
---------------------------------------
Hello, World! I am process 0 of 8 on atl1-1-02-008-25-1.pace.gatech.edu.
Hello, World! I am process 3 of 8 on atl1-1-02-008-25-1.pace.gatech.edu.
Hello, World! I am process 4 of 8 on atl1-1-02-008-25-1.pace.gatech.edu.
Hello, World! I am process 1 of 8 on atl1-1-02-008-25-1.pace.gatech.edu.
Hello, World! I am process 6 of 8 on atl1-1-02-008-25-1.pace.gatech.edu.
Hello, World! I am process 2 of 8 on atl1-1-02-008-25-1.pace.gatech.edu.
Hello, World! I am process 5 of 8 on atl1-1-02-008-25-1.pace.gatech.edu.
Hello, World! I am process 7 of 8 on atl1-1-02-008-25-1.pace.gatech.edu.
---------------------------------------
Begin Slurm Epilog: Mar-21-2023 01:18:18
Job ID:        1331893
Array Job ID:  _4294967294
User ID:       gburdell3
Account:       [Account]
  • Congratulations! you have succesfully run a parallel python script using MPI4py on the cluster