Initial Setup

In High-Performance computing (HPC) there are several compiler and message passing interface systems for users to choose. Selecting a combination of compiler and message passing interface systems for a particular computational job is an involved process that depends upon the intricacies of the calculation and the specifications of the HPC system. This initial setup tutorial will guide the user in setting up a new account to use the most common combination of compiler and message passing interface system: GNU Compiler Collection, and Open MPI.

Startup Scripts

When logging into the cluster, the bash shell starts, and a series of dot files are executed. To setup aliases, functions, or to run specific commands every time a new shell starts, the .bashrc dot file, located in the home directory ( ${HOME}/.bashrc, or, equivalently, ~/.bashrc ), is used.

The .bashrc File

This file will be used in subsequent sections of this article to setup an environment suitable for computational calculations on the cluster. To see the contents of your .bashrc file, use the following command:

user $cat ${HOME}/.bashrc

Your file should look similar to this:

FILE ${HOME}/.bashrc

# .bashrc

# Test for an interactive shell.  There is no need to set anything              
# past this point for scp and rcp, and it's important to refrain from           
# outputting anything in those cases.                                           
if [[ $- != *i* ]] ; then                                                       
        # Shell is non-interactive.  Be done now!                               
        return                                                                  
fi                                                                              

# Source global definitions
if [ -f /etc/bashrc ]; then
	. /etc/bashrc
fi

# User specific aliases and functions

If you'd like to make changes to your .bashrc file, use the nano editor:

user $nano -w ${HOME}/.bashrc

To save any change you make, use Ctrl+O, then hit Enter to accept the file name. Use Ctrl+X to exit nano.

Important
To avoid problems copying files to and from the cluster, make sure that the test for an interactive shell is included at the top of the ${HOME}/.bashrc file.

Set Defaults

Most of the scripts and programs are located in the /home/bgo/shared path and it's sub-directories. For the curious, to see what is currently in /home/bgo/shared/bin, use the ls command. You should see something like this:

user $ls /home/bgo/shared/bin

lmp-config  pw.x-config  qe-config  qpeek  selector-compiler  selector-mpi

The selector-compiler and selector-mpi scripts will be used for setting up a new account.

GNU Compiler Collection

To setup the GNU Compiler Collection for the currently running environment:

user $source /home/bgo/shared/bin/selector-compiler -g

Setup GNU compiler and libraries.

Open MPI

To setup Open MPI that has been compiled with the GNU compiler, along with the default version of all packages, for the currently running environment:

user $source /home/bgo/shared/bin/selector-mpi -i gnu-openmpi

Implementation: gnu-openmpi
Package: lammps/default (20161117)
Package: qe/default (5.0.2)

For details on selecting an MPI implementation, see Selector-mpi.

Note
The settings so far affect only the currently running shell and will be lost when logging back in at a later date. To make changes persistent across logins, see the next section.

Make Changes Persistent and Global

Before a job can be successfully submitted to the cluster it is required to have:

Persistent settings across logins
Same settings on all nodes of cluster

To make the compiler and message passing system settings persistent across logins and all nodes we need to edit the .bashrc file in the home directory. The directions below will be for using the nano editor, but you are free to use whatever text editor you are most comfortable with.

Use the following to start editing the file:

user $nano -w ${HOME}/.bashrc

If this is your first time editing the .bashrc file, it should look just like the one shown in the .bashrc section.

We need to add the two commands we typed earlier (here and here) to the bottom of this file. It should look like this:

FILE ${HOME}/.bashrc(after editing)

# .bashrc

# Test for an interactive shell.  There is no need to set anything              
# past this point for scp and rcp, and it's important to refrain from           
# outputting anything in those cases.                                           
if [[ $- != *i* ]] ; then                                                       
        # Shell is non-interactive.  Be done now!                               
        return                                                                  
fi                                                                              

# Source global definitions
if [ -f /etc/bashrc ]; then
	. /etc/bashrc
fi

# User specific aliases and functions
source /home/bgo/shared/bin/selector-compiler -g -q
source /home/bgo/shared/bin/selector-mpi -i gnu-openmpi -q

You'll notice that the two commands are slightly different than what we used before; they both have a -q added. The -q tells the commands to be 'quiet' and not show any output. You can omit the -q from each command if you'd like to be reminded of what compiler and message passing system is set during each login.

To save the changes, use Ctrl+O, then hit Enter to accept the file name. Use Ctrl+X to exit nano.

Now every time you login, the GNU compiler and Open MPI compiled with the GNU compiler will be the default.