Updated 2019-10-16

Transition from RHEL6 to RHEL7: General Information

Overview

PACE will be transitioning over to RHEL7, which will include newer libraries and keep the clusters up to date. This guide will cover what you have to look out for when making the transition

Impact on User-Compiled / Installed Software

If you have compiled or installed your own software on RHEL6, it will need to be recompiled and installed on RHEL7. RHEL7 will have more and newer libraries installed so the process should be easier. Personal module files will have to be adjusted as well.

Storage on RHEL7

  • Storage will remain the same. This means the file system, with home, data and scratch will not change
  • All user files will not be changed in any way. Nothing will be deleted and your directories will remain the same
  • Storage and file transfer guides on RHEL6 apply to RHEL7

PBS Scripts on RHEL7

Job scripts you have created on RHEL6 will remain the same for RHEL7, except for loading of new modules and new queues. use testflight and testflight-gpu for queues, as they are the RHEL7 test queues available to you. If you do not have access to testflight queues, please send us a request at pace-support@oit.gatech.edu. The modules (software) you load in your job scripts will likely have different versions (newer / updated) or be named differently Many modules/software available on RHEL6 are not yet installed on RHEL7.

  • Solution : While logged in to the RHEL7 headnode, use module avail <software name> to make sure the modules you are loading in your PBS script exist and are the right version on RHEL7.

Requesting GPU's on RHEL7 testflight-gpu queue

In the #PBS script, you can request gpus as follows:

#PBS -l nodes=1:ppn=1:gpus=1

Note that you must request nodes and processors per node. Currently, testflight-gpu is the RHEL7 test queue with gpus.

Job Submissions on GPU Nodes RHEL7

All GPU interactive and batch jobs will need to be submitted using the scheduler You will no longer be able to ssh directly into a gpu node unless you have a running job on the node that was submitted using a scheduler.

Default modules set up on RHEL7

On RHEL7, supporting .pacemodules on home directory will be stopped. As new version of LMOD has a better way to support default modules. Here is the example that you can follow:

suppose you want to put gcc/8.2.0 in the default module:

1. `module load gcc/8.2.0
2. `module save # it creates a file ~/.lmod.d/default`

Saving to the default collection will save the modules for future logins.

For more detailed usage, please check Lmod's Documentation.