Updated 2021-05-17
An in-depth introduction to local installations of software¶
Overview¶
Many times, it will be necessary to install local versions of software on HPC clusters in which users do not have evaluated privileges to install the software globally. In other cases, the particular version of the software is unique to your research needs (i.e. a local modification or experimental version, which means it is not appropriate for other users of the system.
General considerations¶
Whenever an installation of a piece of software is needed, there are several things to keep in mind:
- How do I obtain the software?
- What are the terms of the license?
- Which toolchain is preferred? Are there any known restrictions, particular versions of compiler and MPI-ecosystem that problematic?
- What dependencies are needed?
- Are those pieces of software available on the system?
- If not, what is involved in obtaining the needed dependencies?
- If there are multiple possbily sets of dependencies, which should I choose?
In some of the preceding questions, there are no right answers.
Obtaining source code or packages¶
Source code in most cases will be available via various internet.
Git Repositories¶
Git repositories are a popular way to share code, especially code that is open to contributions from external collaborators.
Downloading of software archive (e.g. tar.gz
, zip
)¶
Some software is distributed as archives. In most cases, these archives can be downloaded directly to the system via wget
or curl
commands. However, it may be necessary to download the software via normal means to a local machine and then transfer the archive to the remote system.
Downloading / Installing a Tarfile¶
- Start by using
wget
followed by the link to a tarfile (also referred to as a Gzipped source tarball) to download it. - Extract the tarfile with
tar zxvf
followed by the full name of the tarfile (including the extension) cd
into the directory generated by the step above.- Using
ls
to check, if there is not already a configure file, you can generate one withautoconf
. - Run the ./configure file with a prefix to the directory you'd like to install it to:
./configure --prefix=/some/other/directory
. You can usepwd
to find the current directory you are in if that is your desired location. This step may produce a large amount of output. - Use
make
to start the build process. This will also produce a large amount of output. - Optional: you can run
make test
to make sure everything works. Note that this will take a long time, so if you believe that the software you are downloading is reliable, you can skip this. - Finally, use
make install
to finish up, generating a lot more output. - Once this is complete you have successfully installed the software.
Downloading / Installing a Zip file¶
- Start by using
wget
followed by the link to a zip file to download it. - Extract the zip with
unzip
followed by the full name of the tarfile (including the extension) cd
into the directory generated by the step above.- Using
ls
to check, if there is not already a configure file, you can generate one withautoconf
. - Run the ./configure file with a prefix to the directory you'd like to install it to:
./configure --prefix=/some/other/directory
. You can usepwd
to find the current directory you are in if that is your desired location. This step may produce a large amount of output. - Use
make
to start the build process. This will also produce a large amount of output. - Optional: you can run
make test
to make sure everything works. Note that this will take a long time, so if you believe that the software you are downloading is reliable, you can skip this. - Finally, use
make install
to finish up, generating a lot more output. - Once this is complete you have successfully installed the software.
Using the Installed Software¶
- If you want to begin using the software right away, you must specify the full path to the software. To find the full path you can navigate to the directory where the executable is and run
pwd
. The full path is the output frompwd
followed by/ExecutableName
(replace ExecutableName with the name of your executable file). - If you want to set the executable to run with a simpler command, you can set up an alias by using
alias <Short Keyword>=<Full Path Name>
.
Note
An alias will only last for the span of the session. You will have to create the alias again if you log in again to the cluster.
Specific Examples¶
LAMMPS¶
The Large-scale Atomic/Molecular Massively Parallel Simulator is a widely-used