- Software is a critical component of the clusters. The primary software stack will be based on RedHat Enterprise Linux and other widely available open source tools. Available software and versions can be listed on systems using the "module spider" command. All software on PACE-administered clusters must be installed and used in compliance with Georgia Tech's Computer and Network Usage and Data Access Policies. Software installation policies can be found in below sections.
- There are regularly scheduled maintenance windows scheduled four times annually. The degree of impact depends on the type of maintenance being performed. All efforts are made to limit downtime during prescribed maintenance periods. All users will be notified via email well in advance of upcoming scheduled maintenance events.
- Jobs must be run using provided workload schedulers. We currently use Moab/Torque. This allows users to gain experience with standard HPC environments which are used in other places, including national labs and the XSEDE.
Headnodes (Login Nodes)¶
- Headnodes (a.k.a login nodes) are shared by multiple users and must be used only for submitting jobs and other light weight activities. It is against policy to run any operation on the headnodes that will impact their responsiveness. These include user processes that consume 90% or more of a CPU, 50% of the available memory, and/or large data operations or transfers (including SFTP connections) that have visible adverse impact on the responsiveness of the headnodes. Processes violating this policy will be killed by the PACE team after 30min of run. PACE team will show best effort to contact the users before killing processes, but reserves the right to kill any processes in violation without notification.
- Scratch policy is available in ServiceNow: PACE Scratch Storage Deleter Policy
- Scratch space is cleaned once a month.
- Files that are 60 days or older will be included for deletion.
- Automated notifications are sent by sweeper script, every time it runs. Deleter script does not send notifications.
- First sweeper will run on Tuesday to avoid holidays. Normally, it will be the first Tuesday of the month, except when that day is a holiday. In that case, the first sweeper will be run on the next non-holiday business day. The first notification message is sent.
- Users can request extensions after the first notification, up until one day before the delete runs. All the user's files will be excluded from the current and next deletion cycles.
- Second sweeper runs 7 days after. The second notification message is sent.
- Deleter runs 14 days after the first sweeper. No notification is sent to the users.
- When a user submits more than 2 consecutive exception requests, PACE will provide assistance and guidance to use the storage resources efficiently.
Project and Home Directory Storage¶
- Both home directories and project directories are subject to quotas. Quotas limit the amount of data that can be stored.
- Project storage is subject to quotas based on requests from a PI. Authorized requests must be sent to firstname.lastname@example.org.
- Home and project space are backed up daily. Please contact email@example.com to restore deleted files.
New User Policies¶
- PACE clusters are available to all GT Faculty, staff, students, and external collaborators who have been granted access by an Authorized Requester, namely a member of the faculty (PI) or someone designated by the PI. Authorized requests must be sent to firstname.lastname@example.org.
- PACE clusters are a campus resource subject to the Georgia Tech Cyber Security Policies.
Continued Access to PACE After GT Affiliation Status Changes¶
- Users can continue to access PACE after their status have changed (e.g. graduation, new research group, end of collaboration, etc) if approved by their PI and they have an active GT account that allows them to connect to VPN. Users who no longer have access to VPN can be provided with continued access to PACE after the following steps:
- Find a sponsor with active GT credentials (generally, the PI), who can open them a guest account from passport
- The PI should send email@example.com a request, including the old and new (guest) usernames. If they need existing data migrated to the new account, this should be requested specifically in the ticket.
- PACE will then create a new account using the guest credentials and migrate the files if requested. Ideally, the user will not see any differences in charge accounts or files, other than the changed username.
- The GT sponsor hosting the guest account(s) assumes all responsibility for the users and their conformance to the GT and PACE usage policies. These accounts can be extended as needed by the GT sponsor from passport.