Updated 2021-05-17

An in-depth introduction to local installations of software

Overview

Many times, it will be necessary to install local versions of software on HPC clusters in which users do not have evaluated privileges to install the software globally. In other cases, the particular version of the software is unique to your research needs (i.e. a local modification or experimental version, which means it is not appropriate for other users of the system.

General considerations

Whenever an installation of a piece of software is needed, there are several things to keep in mind:

  • How do I obtain the software?
  • What are the terms of the license?
  • Which toolchain is preferred? Are there any known restrictions, particular versions of compiler and MPI-ecosystem that problematic?
  • What dependencies are needed?
  • Are those pieces of software available on the system?
  • If not, what is involved in obtaining the needed dependencies?
  • If there are multiple possbily sets of dependencies, which should I choose?

In some of the preceding questions, there are no right answers.

Obtaining source code or packages

Source code in most cases will be available via various internet.

Git Repositories

Git repositories are a popular way to share code, especially code that is open to contributions from external collaborators.

Downloading of software archive (e.g. tar.gz, zip)

Some software is distributed as archives. In most cases, these archives can be downloaded directly to the system via wget or curl commands. However, it may be necessary to download the software via normal means to a local machine and then transfer the archive to the remote system.

Downloading / Installing a Tarfile
  • Start by using wget followed by the link to a tarfile (also referred to as a Gzipped source tarball) to download it.
  • Extract the tarfile with tar zxvf followed by the full name of the tarfile (including the extension)
  • cd into the directory generated by the step above.
  • Using ls to check, if there is not already a configure file, you can generate one with autoconf.
  • Run the ./configure file with a prefix to the directory you'd like to install it to: ./configure --prefix=/some/other/directory. You can use pwd to find the current directory you are in if that is your desired location. This step may produce a large amount of output.
  • Use make to start the build process. This will also produce a large amount of output.
  • Optional: you can run make test to make sure everything works. Note that this will take a long time, so if you believe that the software you are downloading is reliable, you can skip this.
  • Finally, use make install to finish up, generating a lot more output.
  • Once this is complete you have successfully installed the software.
Downloading / Installing a Zip file
  • Start by using wget followed by the link to a zip file to download it.
  • Extract the zip with unzip followed by the full name of the tarfile (including the extension)
  • cd into the directory generated by the step above.
  • Using ls to check, if there is not already a configure file, you can generate one with autoconf.
  • Run the ./configure file with a prefix to the directory you'd like to install it to: ./configure --prefix=/some/other/directory. You can use pwd to find the current directory you are in if that is your desired location. This step may produce a large amount of output.
  • Use make to start the build process. This will also produce a large amount of output.
  • Optional: you can run make test to make sure everything works. Note that this will take a long time, so if you believe that the software you are downloading is reliable, you can skip this.
  • Finally, use make install to finish up, generating a lot more output.
  • Once this is complete you have successfully installed the software.
Using the Installed Software
  • If you want to begin using the software right away, you must specify the full path to the software. To find the full path you can navigate to the directory where the executable is and run pwd. The full path is the output from pwd followed by /ExecutableName (replace ExecutableName with the name of your executable file).
  • If you want to set the executable to run with a simpler command, you can set up an alias by using alias <Short Keyword>=<Full Path Name>.

Note

An alias will only last for the span of the session. You will have to create the alias again if you log in again to the cluster.

Specific Examples

LAMMPS

The Large-scale Atomic/Molecular Massively Parallel Simulator is a widely-used