Updated 2021-08-27

Introduction to PACE software repository

Application software on PACE clusters is categorized into three categories, also known as three tiers. The tier information shall determine who installs the software (PACE team or user), where to install the software (PACE repository or user directory) and who support it (PACE team or user). The tiers are ranked by actual usage in the past 12 months and are adjusted every 6 months. The description of each tier is as below:

  • The first tier (T1) contains the highest-ranking software which covers more than 75% of PACE users. PACE team install them into PACE repository for users to use. PACE team support them with documentation on PACE website or module help.

  • The second tier (T2) contains software that have less impact on the users than the first tier, but they are still used by a considerable number of users and groups. PACE team install them into PACE repository based on user request. PACE team provide the best effort support. 

                    - T1 and T2 covers 98% of the users on PACE.

  • The third tier (T3) also known as community software contains software only used by a few groups or experimental software. It is installed by the Georgia Tech research community instead of PACE staff. They are particularly useful for research groups needing to share specialized software that requires specific installation expertise available within the research group. Communitysoftware that may not be needed by the majority of PACE users. Researchers may need to consider software needs that are research-oriented and exploratory.

User self-installed software in their own directory does not belong to the above 3 categories.

Current software stack release date is Jan 2020

Software stack:

Upgrade cycle will also align with the operating system upgrades. - Infrastructure software such as MPI and other middleware libraries (e.g., HDF) are updated every two years (24-mo cycle, biennial). - Release target is Summer (maintenance window) using official stable release from 3-6 mo prior (i.e., 6-mo lag) to allow time for stack building, verification, and performance validation. - Infrastructure component may be updated to repair a security vulnerability or if dictated for operations. - We shall maintain Intel and GCC stacks. - The GCC mainline flavor is currently gcc/8 - The Intel version will be aligned with a compatible GCC. - We shall build MPI-based middleware libraries with MVAPICH2. - All infrastructure software is included in the T1 software catagory.

End-point application updates:

  • T1 end-point applications are updated bi-annually, if available, and if compatible with existing infrastructure stack.
  • PACE-built applications will use current stack to build each release.
  • Open-source software must include a codified release identifier (e.g., a git tag) that uniquely defines the updated version.

End-point middleware updates:

  • T1 middleware updates (e.g., HDF, compiler) may be updated annually using the current stack.
  • Note, we will deploy annual updates of compilers but will not provide a full stack. User can select a newer version themselves.

Deprecation:

Development stack:

  • We shall provide a stable development environment for 4 years but a major refresh every 2 years. Users are encouraged to progress on the 2 year cycle but can continue with existing builds for 4 years (baring a hardware deprecation).
  • The current stack (S) shall be actively supported for 24 months.
  • The previous stack (S-1) shall be maintained for an additional 24-mo (i.e., 48-mo lifespan).
  • S-2 and older shall be retained but made unreadable to users.
  • A warning is issued when loading the soon-to-be deprecated module with 6mo remaining.
  • Newsletters and email lists advertise updates.
  • Monitor usage to identify potential problems with 12mo notice.

End-point T1 applications:

  • We shall support the current release (R) and previous (R-1) release.
  • We shall
  • maintain the R-2 release but issue brief warning with 3 mos remaining,
  • hide the R-3 release but allow direct loading; issue verbose warning,
  • make R-4 and older unreadable.

T2 software:

  • We maintain the current release (R) and previous (R-1) release.
  • We selectively update T2 applications based on usage statistics. If unused in refresh cycle, we do not update.

Module deprecation:

  • We shall review annually the usage statistics of end-point T1 and T2 software applications and adjust status.
  • Unused T1 applications shall be demoted to T2.
  • T2 packages that are not used within the deprecation cycle, they will become hidden and, eventually, unreadable. This takes 12-24 months of inactivity.
  • T2 packages can be promoted to T1 annually.

Software installation requests:

  • For instructional software, the instructors should install them. However, we can provide the best effort support in the installation process.

  • New software requests are accepted regularly.

  • The appropriateness of the software is subject to a security analysis, licensing restrictions, compatibility with existing system, and necessity for research. Software will only be installed if it is determined that it is necessary for research and meets the security and licensing requirements.

  • Evaluation of and decision for each new software request may take days to weeks, and the installation may take more than several weeks depending on the complexity.

  • All software installation requests are evaluated according to the following criteria:

    • Should benefit at least 5 PACE users
    • Supported by the RHEL version being used by PACE (Linux support is not sufficient)
    • Does not require emulation (dosbox, wine, etc)
    • Doesn't require deployment of database servers (clients querying external DBs are OK)
    • Doesn't require any system-level changes (e.g. changes to /etc, /var, etc)
    • Must be actively supported and publicly available (i.e., no experimental research codes, modified versions or abandoned projects).
    • Contains a verification test
    • No sequential GUI applications that would be a better fit for local workstations (e.g. openGL software that will not work on all nodes)
    • Should support a standard installation framework such as makefiles, autoconf/automake, cmake, setup.py, conda etc. PACE may refuse compilations using non-standard installation procedures requiring manual changes to several configuration files.
  • Software that requires PACE accepting license terms on behalf of users cannot be included in the global repository, even if the software is free for academic use. Exceptions can be made per written approval from the developer/company.
  • Upgrades to existing software will be subject to a review process to determine if the new version is critical for continuation of research.
  • Compiler optimizations that are specific to certain architectures cannot be applied to the global repository if they cause crashes on the remaining architectures that PACE supports.

The PACE community software user agreement

  The PACE community software is installed by the Georgia Tech research 
  community instead of PACE staff. They are particularly useful for 
  research groups needing to share specialized software that requires 
  specific installation expertise available within the research group 
  (e.g., internally developed software, cutting-edge/experimental 
  research software, and departmental specific software). By sharing 
  their installation in PACE's global repository, they not only help 
  their own group but other groups at GT, who may also benefit from that 
  software.

PACE provides the infrastructure for users/research groups to install, test, document, and share their software. PACE is not responsible for the content, quality and any other issue related to use of the software; however, PACE staff shall review the software and documents and reserve the right to modify or remove any of the software or its components at its discretion. The user/research group who installs the software are responsible for the long-term maintenance, updates and support.

Requirement for community software installations:

  • Community software repository can only be accessed by users, namely “Community Contributors”, who are approved by the PACE team. Users need to contact PACE before they become community contributors, which may require training and evaluation of skill sets as needed.   

  • Community Contributors must ensure that the software is functioning as intended. 

  • Community Contributors are responsible for updating and documenting the software. 

  • Community Contributors shall not use the storage allocated or software installation for other purposes. 

  • After the PI agrees to an extended collaboration support (ECS) engagement, PACE staff can provide guidance with users to install open source software projects and Git-based repositories applied to PACE. Where needed, PACE staff can also recommend and build containers for users. In addition, PACE staff can work with users needing assistance with BYOL (Bring Your Own License) software.

PACE reserves the right to remove unmaintained community software installations or installations that violate these requirements.