Initial Setup (flory): Difference between revisions

From HPC Wiki
Jump to:navigation Jump to:search
No edit summary
(added flory message)
 
(13 intermediate revisions by the same user not shown)
Line 1: Line 1:
{{Important|This page is specific to the '''Flory''' cluster}}
In [//www.wikipedia.org/wiki/High-performance_computing High-Performance computing] (HPC) there are several [//www.wikipedia.org/wiki/Compiler compiler] and [//www.wikipedia.org/wiki/Message_passing_interface message passing interface] systems for users to choose. Selecting a combination of compiler and message passing interface systems for a particular computational job is an involved process that depends upon the intricacies of the calculation and the specifications of the HPC system. This initial setup tutorial will guide the user in setting up a new account to use the most common combination of compiler and message passing interface system: [//gcc.gnu.org GNU Compiler Collection], and [//www.open-mpi.org Open MPI].


In [//www.wikipedia.org/wiki/High-performance_computing High-Performance computing] (HPC) there are several [//www.wikipedia.org/wiki/Compiler compiler] and [//www.wikipedia.org/wiki/Message_passing message passing] systems for users to choose. Selecting a combination of compiler and message passing systems for a particular computational job is an involved process that depends upon the intricacies of the calculation and the specifications of the HPC system. This initial setup tutorial will guide the user in setting up a new account to use the most common combination of compiler and message passing system: [//gcc.gnu.org GNU Compiler Collection], and [//www.open-mpi.org Open MPI].
__TOC__
 
== Startup Scripts ==
 
When logging into the cluster, the [//wikipedia.org/wiki/Bash_(Unix_shell) bash] shell starts, and a series of [//wikipedia.org/wiki/Hidden_file_and_hidden_directory#Unix_and_Unix-like_environments dot files] are executed. To setup aliases, functions, or to run specific commands every time a new shell starts, the {{C|.bashrc}} dot file, located in the [//www.wikipedia.org/wiki/Home_directory home directory] ( {{Path|${HOME}/.bashrc}}, or, equivalently, {{Path|~/.bashrc}} ), is used.
 
=== The {{C|.bashrc}} File===
 
This file will be used in subsequent sections of this article to setup an environment suitable for computational calculations on the cluster. To see the contents of your .bashrc file, use the following command:
{{Cmd|cat ${HOME}/.bashrc}}
 
Your file should look similar to this:
 
{{FileBox|filename=${HOME}/.bashrc|1=
# .bashrc
 
# Test for an interactive shell. There is no need to set anything             
# past this point for scp and rcp, and it's important to refrain from         
# outputting anything in those cases.                                         
if [[ $- != *i* ]] ; then                                                     
        # Shell is non-interactive. Be done now!                             
        return                                                                 
fi                                                                             


__TOC__
# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi
 
# User specific aliases and functions
}}
 
If you'd like to make changes to your .bashrc file, use the {{C|nano}} editor:
{{Cmd|nano -w ${HOME}/.bashrc}}
 
To save any change you make, use {{Key|Ctrl}}+{{Key|O}}, then hit {{Key|Enter}} to accept the file name. Use {{Key|Ctrl}}+{{Key|X}} to exit {{C|nano}}.


{{Important|To avoid problems copying files to and from the cluster, make sure that the test for an interactive shell is included at the top of the {{Path|${HOME}/.bashrc}} file. }}


{{WIP|author=Bgo}}


== Set Defaults ==
== Set Defaults ==


Most of the scripts and programs are located in the {{Path|{{HPC-Base-Dir}}}} path and it's sub-directories. For the curious, to see what is currently in {{Path|{{HPC-Base-Dir}}/bin}}, use the {{C|ls}} command. You should see something like this:
Most of the scripts and programs are located in the {{Path|{{HPC-Base-Dir|flory}}}} path and it's sub-directories. For the curious, to see what is currently in {{Path|{{HPC-Base-Dir|flory}}/bin}}, use the {{C|ls}} command. You should see something like this:


{{Cmd|ls {{HPC-Base-Dir}}/bin|output=<pre>lmp-config  pw.x-config  selector-compiler  selector-mpi</pre>}}
{{Cmd|ls {{HPC-Base-Dir|flory}}/bin|output=<pre>
lmp-config  pw.x-config qe-config  qpeek selector-compiler  selector-mpi
</pre>}}


The {{Code|selector-compiler}} and {{Code|selector-mpi}} scripts will be used for setting up a new account.
The {{C|selector-compiler}} and {{C|selector-mpi}} scripts will be used for setting up a new account.


=== GNU Compiler Collection ===
=== GNU Compiler Collection ===


To setup the GNU Compiler Collection for the currently running environment:
To setup the GNU Compiler Collection for the currently running environment:
{{Cmd|source {{HPC-Base-Dir}}/bin/selector-compiler -g|output=<pre>Setup GNU compiler and libraries.</pre>}}
{{Cmd|source {{HPC-Base-Dir|flory}}/bin/selector-compiler -g|output=<pre>Setup GNU compiler and libraries.</pre>}}


=== Open MPI ===
=== Open MPI ===


To setup Open MPI that has been compiled with the GNU compiler for the currently running environment:
To setup Open MPI that has been compiled with the GNU compiler, along with the default version of all packages, for the currently running environment:
{{Cmd|source {{HPC-Base-Dir}}/bin/selector-mpi -o|output=<pre>Set MPI to Open MPI with GNU compiler.</pre>}}
{{Cmd|source {{HPC-Base-Dir|flory}}/bin/selector-mpi -i gnu-openmpi|output=<pre>Implementation: gnu-openmpi
Package: lammps/default (20161117)
Package: qe/default (5.0.2)</pre>}}


{{Note|The settings so far affect only the currently running shell and will be lost when logging back in at a later date. To [[#Make Changes Persistent|make changes persistent]] across logins, see the next section.}}
For details on selecting an MPI implementation, see [[Selector-mpi]].


== Make Changes Persistent ==
{{Note|The settings so far affect only the currently running shell and will be lost when logging back in at a later date. To [[#Make Changes Persistent and Global|make changes persistent]] across logins, see the next section.}}


To make the compiler and message passing system settings persistent across logins we need to edit the {{C|.bashrc}} file in the [//www.wikipedia.org/wiki/Home_directory home directory]. The directions below will be for using the {{Code|nano}} editor, but you are free to use whatever text editor you are most comfortable with.  
== Make Changes Persistent and Global ==
 
Before a job can be successfully submitted to the cluster it is required to have:
* Persistent settings across logins
* Same settings on all nodes of cluster
 
To make the compiler and message passing system settings persistent across logins and all nodes we need to edit the {{C|.bashrc}} file in the [//www.wikipedia.org/wiki/Home_directory home directory]. The directions below will be for using the {{C|nano}} editor, but you are free to use whatever text editor you are most comfortable with.  


Use the following to start editing the file:
Use the following to start editing the file:
{{Cmd|nano -w ${HOME}/.bashrc}}
{{Cmd|nano -w ${HOME}/.bashrc}}


If this is your first time editing the {{C|.bashrc}} file, it should look something like this:
If this is your first time editing the {{C|.bashrc}} file, it should look just like the one shown in the [[#The .bashrc File|.bashrc section]].
 
{{GenericCmd|output=<pre>
# .bashrc
 
# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi
 
# User specific aliases and functions
</pre>}}


We need to add the two commands we typed earlier ([[#GNU Compiler Collection|here]] and [[#Open MPI|here]]) to the bottom of this file. It should look like this:
We need to add the two commands we typed earlier ([[#GNU Compiler Collection|here]] and [[#Open MPI|here]]) to the bottom of this file. It should look like this:


{{GenericCmd|output=<pre>
{{FileBox|title=(after editing)|filename=${HOME}/.bashrc|1=
# .bashrc
# .bashrc
# Test for an interactive shell.  There is no need to set anything             
# past this point for scp and rcp, and it's important to refrain from         
# outputting anything in those cases.                                         
if [[ $- != *i* ]] ; then                                                     
        # Shell is non-interactive.  Be done now!                             
        return                                                                 
fi                                                                             


# Source global definitions
# Source global definitions
Line 59: Line 101:
# User specific aliases and functions
# User specific aliases and functions
source /home/bgo/shared/bin/selector-compiler -g -q
source /home/bgo/shared/bin/selector-compiler -g -q
source /home/bgo/shared/bin/selector-mpi -o -q
source /home/bgo/shared/bin/selector-mpi -i gnu-openmpi -q
</pre>}}
}}


You'll notice that the two commands are slightly different that what we used before; they both have a {{C|-q}} added. The {{C|-q}} tells the commands to be 'quiet' and not show any output. You can omit the {{C|-q}} from each command if you'd like to be reminded of what compiler and message passing system is set during each login.
You'll notice that the two commands are slightly different than what we used before; they both have a {{C|-q}} added. The {{C|-q}} tells the commands to be 'quiet' and not show any output. You can omit the {{C|-q}} from each command if you'd like to be reminded of what compiler and message passing system is set during each login.


To save the changes, use {{Key|Ctrl}}+{{Key|O}}, then hit {{Key|Enter}} to accept the file name. Use {{Key|Ctrl}}+{{Key|X}} to exit {{Code|nano}}.
To save the changes, use {{Key|Ctrl}}+{{Key|O}}, then hit {{Key|Enter}} to accept the file name. Use {{Key|Ctrl}}+{{Key|X}} to exit {{C|nano}}.


Now every time you login, the GNU compiler and Open MPI compiled with the GNU compiler will be the default.
Now every time you login, the GNU compiler and Open MPI compiled with the GNU compiler will be the default.
[[Category:Tutorials]]

Latest revision as of 13:32, 8 February 2018

Important
This page is specific to the Flory cluster

In High-Performance computing (HPC) there are several compiler and message passing interface systems for users to choose. Selecting a combination of compiler and message passing interface systems for a particular computational job is an involved process that depends upon the intricacies of the calculation and the specifications of the HPC system. This initial setup tutorial will guide the user in setting up a new account to use the most common combination of compiler and message passing interface system: GNU Compiler Collection, and Open MPI.

Startup Scripts

When logging into the cluster, the bash shell starts, and a series of dot files are executed. To setup aliases, functions, or to run specific commands every time a new shell starts, the .bashrc dot file, located in the home directory ( ${HOME}/.bashrc, or, equivalently, ~/.bashrc ), is used.

The .bashrc File

This file will be used in subsequent sections of this article to setup an environment suitable for computational calculations on the cluster. To see the contents of your .bashrc file, use the following command:

user $cat ${HOME}/.bashrc

Your file should look similar to this:

FILE ${HOME}/.bashrc
# .bashrc

# Test for an interactive shell.  There is no need to set anything              
# past this point for scp and rcp, and it's important to refrain from           
# outputting anything in those cases.                                           
if [[ $- != *i* ]] ; then                                                       
        # Shell is non-interactive.  Be done now!                               
        return                                                                  
fi                                                                              

# Source global definitions
if [ -f /etc/bashrc ]; then
	. /etc/bashrc
fi

# User specific aliases and functions

If you'd like to make changes to your .bashrc file, use the nano editor:

user $nano -w ${HOME}/.bashrc

To save any change you make, use Ctrl+O, then hit Enter to accept the file name. Use Ctrl+X to exit nano.

Important
To avoid problems copying files to and from the cluster, make sure that the test for an interactive shell is included at the top of the ${HOME}/.bashrc file.


Set Defaults

Most of the scripts and programs are located in the /home/bgo/shared path and it's sub-directories. For the curious, to see what is currently in /home/bgo/shared/bin, use the ls command. You should see something like this:

user $ls /home/bgo/shared/bin
lmp-config  pw.x-config  qe-config  qpeek  selector-compiler  selector-mpi

The selector-compiler and selector-mpi scripts will be used for setting up a new account.

GNU Compiler Collection

To setup the GNU Compiler Collection for the currently running environment:

user $source /home/bgo/shared/bin/selector-compiler -g
Setup GNU compiler and libraries.

Open MPI

To setup Open MPI that has been compiled with the GNU compiler, along with the default version of all packages, for the currently running environment:

user $source /home/bgo/shared/bin/selector-mpi -i gnu-openmpi
Implementation: gnu-openmpi
Package: lammps/default (20161117)
Package: qe/default (5.0.2)

For details on selecting an MPI implementation, see Selector-mpi.

Note
The settings so far affect only the currently running shell and will be lost when logging back in at a later date. To make changes persistent across logins, see the next section.

Make Changes Persistent and Global

Before a job can be successfully submitted to the cluster it is required to have:

  • Persistent settings across logins
  • Same settings on all nodes of cluster

To make the compiler and message passing system settings persistent across logins and all nodes we need to edit the .bashrc file in the home directory. The directions below will be for using the nano editor, but you are free to use whatever text editor you are most comfortable with.

Use the following to start editing the file:

user $nano -w ${HOME}/.bashrc

If this is your first time editing the .bashrc file, it should look just like the one shown in the .bashrc section.

We need to add the two commands we typed earlier (here and here) to the bottom of this file. It should look like this:

FILE ${HOME}/.bashrc(after editing)
# .bashrc

# Test for an interactive shell.  There is no need to set anything              
# past this point for scp and rcp, and it's important to refrain from           
# outputting anything in those cases.                                           
if [[ $- != *i* ]] ; then                                                       
        # Shell is non-interactive.  Be done now!                               
        return                                                                  
fi                                                                              

# Source global definitions
if [ -f /etc/bashrc ]; then
	. /etc/bashrc
fi

# User specific aliases and functions
source /home/bgo/shared/bin/selector-compiler -g -q
source /home/bgo/shared/bin/selector-mpi -i gnu-openmpi -q

You'll notice that the two commands are slightly different than what we used before; they both have a -q added. The -q tells the commands to be 'quiet' and not show any output. You can omit the -q from each command if you'd like to be reminded of what compiler and message passing system is set during each login.

To save the changes, use Ctrl+O, then hit Enter to accept the file name. Use Ctrl+X to exit nano.

Now every time you login, the GNU compiler and Open MPI compiled with the GNU compiler will be the default.

Retrieved from "http://magnolia.usm.edu/index.php?title=Initial_Setup_(flory)&oldid=867"