Selector-mpi (flory): Difference between revisions

From HPC Wiki
Jump to:navigation Jump to:search
(Initial commit)
 
No edit summary
 
(16 intermediate revisions by the same user not shown)
Line 1: Line 1:
The script {{C|selector-mpi}}, located in the {{Path|{{HPC-Base-Dir|flory}}/bin}} directory, makes it easy to setup the environment for supported MPI implementations and packages.  
{{Important|This page is specific to the '''Flory''' cluster}}
The script {{C|selector-mpi}}, located in the {{Path|{{HPC-Base-Dir|flory}}/bin}} directory, makes it easy to setup the environment for supported Message Passing Interface (MPI) implementations and packages.  


= Command line parameters =
= Usage =


{{Template Parameter Table|1=
A typical command line would look like this:
{{Template Parameter|-i <implementation>|Set MPI implementation|required=no}}
 
{{Template Parameter|-o                 | Shortcut for '-i gnu-openmpi'|required=no}}
{{GenericCmd|source {{HPC-Base-Dir|flory}}/bin/selector-mpi --quiet -f -i gnu-openmpi lammps/20161117 qe/5.0.2}}
{{Template Parameter|-s                 | Shows available MPI implementations or package versions for <implementation>. To see package versions, -i or -o  must specified first, otherwise MPI implementations will be shown.|required=no}}
 
{{Template Parameter|-f                 | Full (almost) clean of path. Cleans out extraneous path entries that the typical user does not need (e.g., sbin directories).|required=no}}
Below is a table of all options available from the command line. The long form options are preceded with <code>--</code>, and short form a single <code>-</code>. For options with both a long and short form, either one may be used on the command line.
{{Template Parameter|-c                 | Cleans all paths by removing entries that this script adds, also does a '-f' for a full cleanup. NOTE: If this is specified, the script will clean the path and ignore all other options, except for verbose and quiet.|required=no}}
 
{{Template Parameter|-v                 | Be Verbose|required=no}}
= Command Line Options =
{{Template Parameter|-q                 | Be quiet.|required=no}}
{{Command Option Table|1=
{{Template Parameter|-h                 | Show the help text.|required=no}}
{{Command Option|long=implementation|short=i|type=string|Set MPI implementation}}
{{Template Parameter|package versions  | List of package versions to use. The default version will be used for any unspecified package. See examples below.|default=default|required=no}}
{{Command Option|                   short=o           |Shortcut for <code>-i gnu-openmpi</code>|required=no}}
{{Command Option|long=show          |short=s           |Shows available MPI implementations or package versions for implementation. To see package versions, <code>-i</code> or <code>-o</code> must specified first, otherwise MPI implementations will be shown.}}
{{Command Option|long=fullclean    |short=f           |Full (almost) clean of path. Cleans out extraneous path entries that the typical user does not need (e.g., sbin directories).}}
{{Command Option|long=clean        |short=c           |Cleans all paths by removing entries that this script adds. '''NOTE''': If this is specified, the script will clean the path and ignore all other options, except for verbose and quiet.}}
{{Command Option|long=verbose      |short=v           | Be verbose}}
{{Command Option|long=quiet        |short=q           | Be quiet.}}
{{Command Option|long=help          |short=h           | Show the help text.}}
}}
}}


A list of packages and their versions to use can be added to the end of the command line, the default version will be used for any unspecified package. In the example above, {{C|lammps/20161117}} and {{C|qe/5.0.2}} package/versions are selected. See examples below for details.


= Examples =
= Examples =


== Example 1 ==
== Show Available MPI Implementations ==
 
To get a list of available MPI implementations:
 
{{Cmd|source {{HPC-Base-Dir|flory}}/bin/selector-mpi -s|output=<pre>gnu-openmpi: GNU compiler and OpenMPI
icc-openmpi: Intel compiler and OpenMPI (Not currently available)</pre>}}
 
== Show Available Package Versions ==
 
To get a list of packages and their versions available for the ''gnu-openmpi'' MPI implementation:
 
{{Cmd|source {{HPC-Base-Dir|flory}}/bin/selector-mpi -i gnu-openmpi -s|output=<pre>Implementation: gnu-openmpi
lammps:
  lammps/20140201
* lammps/20161117
 
qe:
* qe/5.0.2
  qe/5.3.0
  qe/5.3.0-environ-0.2
  qe/6.0
 
 
*: Default version.</pre>}}
 
== Use Default Package Version ==
 
To use the default version of all packages available for the ''gnu-openmpi'' MPI implementation:
 
{{Cmd|source {{HPC-Base-Dir|flory}}/bin/selector-mpi -i gnu-openmpi|output=<pre>Implementation: gnu-openmpi
Package: lammps/default (20161117)
Package: qe/default (5.0.2)</pre>}}
 
== Use Specific Package Versions ==
 
To use LAMMPS version 20161117, Quantum ESPRESSO version 5.0.2, and the default version for all other available packages for the ''gnu-openmpi'' MPI implementation:
 
{{Cmd|source {{HPC-Base-Dir|flory}}/bin/selector-mpi -i gnu-openmpi lammps/20161117 qe/5.0.2|output=<pre>Implementation: gnu-openmpi
Package: lammps/20161117
Package: qe/5.0.2</pre>}}
 
=== See Details ===
 
To see in more detail what {{C|selector-mpi}} is doing, the {{C|verbose}} flag can be added to the command line. The previous command, with the {{C|verbose}} flag, gives:
 
{{Cmd|source {{HPC-Base-Dir|flory}}/bin/selector-mpi --verbose -i gnu-openmpi lammps/20161117 qe/5.0.2|output=<pre>Implementation: gnu-openmpi
Cleaning paths.
Removing previous implementation paths.
  Updated PATH
  Updated MANPATH
  Updated LD_LIBRARY_PATH
Updating environmental variables for base implementation.
        Adding to PATH: /home/bgo/shared/bin
Updating environmental variables for gnu-openmpi implementation.
        Adding to PATH: /home/bgo/shared/gnu-openmpi/bin
        Adding to LD_LIBRARY_PATH: /home/bgo/shared/gnu-openmpi/lib
        Adding to LD_LIBRARY_PATH: /home/bgo/shared/gnu-openmpi/usr/lib
        Adding to MANPATH: /home/bgo/shared/gnu-openmpi/share/man
Package: lammps/default (20161117)
        Adding to PATH: /home/bgo/shared/gnu-openmpi/versioned/lammps/default/bin
Package: qe/default (5.0.2)
        Adding to PATH: /home/bgo/shared/gnu-openmpi/versioned/qe/default/bin</pre>}}
 
== Automatically Select Packages on Login ==
 
To automatically select Open MPI compiled with the GNU compiler and the default version of all packages for every login, add the following line to the bottom of the {{C|$HOME/.bashrc}} file:
 
{{GenericCmd|source {{HPC-Base-Dir|flory}}/bin/selector-mpi -i gnu-openmpi -f -q}}
 
See [[Initial_Setup#Make_Changes_Persistent_and_Global|Make Changes Persistent and Global]] on the [[Initial Setup]] page for details.
 
{{Note|It is recommended to put a line similar to this into all queue submission scripts to be sure which MPI implementation and package version is used.}}
 
== Cleaning <var>PATH</var> ==
 
The [//www.wikipedia.org/wiki/PATH_(variable) <var>PATH</var>]  [//www.wikipedia.org/wiki/Environment_variable environmental variable], which defines a list of directories that is searched when looking for an executable program, can sometimes become cluttered. The <code>--clean</code> (or <code>-c</code>) command line option to {{C|selector-mpi}} is used to help clean up the path. For example, when the current path is:
 
{{Cmd|echo $PATH|output=<pre>/usr/lib64/qt-3.3/bin:/usr/NX/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/pssc/bin:/opt/openmpi/bin:/opt/torque/bin:/opt/torque/sbin:/opt/mpich2/bin</pre>}}
 
Running {{C|selector-mpi}} with the <code>--clean</code> and <code>--quiet</code> command line options gives a <var>PATH</var> of:
 
{{Cmd|source {{HPC-Base-Dir|flory}}/bin/selector-mpi --clean --quiet|echo $PATH|output=<pre>/usr/lib64/qt-3.3/bin:/usr/NX/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/pssc/bin:/opt/torque/bin:/opt/torque/sbin:</pre>}}
 
where {{Path|/opt/openmpi/bin}} and {{Path|/opt/mpich2/bin}} have been removed from <var>PATH</var>. Adding the <code>-f</code> command line option gives:
 
{{Cmd|source {{HPC-Base-Dir|flory}}/bin/selector-mpi --clean -f --quiet|echo $PATH|output=<pre>/usr/lib64/qt-3.3/bin:/usr/NX/bin:/usr/local/bin:/bin:/usr/bin:/opt/torque/bin</pre>}}
 
where {{Path|/usr/local/sbin}}, {{Path|/usr/sbin}},  {{Path|/sbin}}, and {{Path|/opt/pssc/bin}} have been removed.
 
=== Advanced ===
 
The user can specify additional paths to be cleaned by defining an environmental variable called <var>SELECTOR_USER_TO_CLEAR_PATH</var>. For example, to have the {{Path|/usr/lib64/qt-3.3/bin}} and {{Path|/usr/NX/bin}} paths removed from <var>PATH</var> which is currently
 
{{Cmd|echo $PATH|output=<pre>/usr/lib64/qt-3.3/bin:/usr/NX/bin:/usr/local/bin:/bin:/usr/bin:/opt/torque/bin</pre>}}
 
run the following commands:
 
{{Cmd|1=export SELECTOR_USER_TO_CLEAR_PATH=/usr/lib64/qt-3.3/bin:/usr/NX/bin|2=source {{HPC-Base-Dir|flory}}/bin/selector-mpi --clean -f --quiet|3=echo $PATH|output=<pre>/usr/local/bin:/bin:/usr/bin:/opt/torque/bin</pre>}}


== Example 2 ==
'''NOTE''': Care must be taken when setting the <var>SELECTOR_USER_TO_CLEAR_PATH</var> environmental variable because it is used to match the prefix of each path in <var>PATH</var>. For example, with <code>SELECTOR_USER_TO_CLEAR_PATH=/usr/local</code>, any path beginning with {{C|/usr/local}} will be removed from <var>PATH</var>, like {{Path|/usr/local/bin}} and {{Path|/usr/local/lib64}}.


== Example 3 ==
[[Category:Tutorials]]

Latest revision as of 13:41, 8 February 2018

Important
This page is specific to the Flory cluster

The script selector-mpi, located in the /home/bgo/shared/bin directory, makes it easy to setup the environment for supported Message Passing Interface (MPI) implementations and packages.

Usage

A typical command line would look like this:

source /home/bgo/shared/bin/selector-mpi --quiet -f -i gnu-openmpi lammps/20161117 qe/5.0.2

Below is a table of all options available from the command line. The long form options are preceded with --, and short form a single -. For options with both a long and short form, either one may be used on the command line.

Command Line Options

1Long form options are prefixed with --; 2Short form options are prefixed with -.
Long form1 Short form2 Option type Required? Description
implementation i string No Set MPI implementation
o No Shortcut for -i gnu-openmpi
show s No Shows available MPI implementations or package versions for implementation. To see package versions, -i or -o must specified first, otherwise MPI implementations will be shown.
fullclean f No Full (almost) clean of path. Cleans out extraneous path entries that the typical user does not need (e.g., sbin directories).
clean c No Cleans all paths by removing entries that this script adds. NOTE: If this is specified, the script will clean the path and ignore all other options, except for verbose and quiet.
verbose v No Be verbose
quiet q No Be quiet.
help h No Show the help text.

A list of packages and their versions to use can be added to the end of the command line, the default version will be used for any unspecified package. In the example above, lammps/20161117 and qe/5.0.2 package/versions are selected. See examples below for details.

Examples

Show Available MPI Implementations

To get a list of available MPI implementations:

user $source /home/bgo/shared/bin/selector-mpi -s
gnu-openmpi: GNU compiler and OpenMPI
icc-openmpi: Intel compiler and OpenMPI (Not currently available)

Show Available Package Versions

To get a list of packages and their versions available for the gnu-openmpi MPI implementation:

user $source /home/bgo/shared/bin/selector-mpi -i gnu-openmpi -s
Implementation: gnu-openmpi
lammps: 
   lammps/20140201
 * lammps/20161117

qe: 
 * qe/5.0.2
   qe/5.3.0
   qe/5.3.0-environ-0.2
   qe/6.0


*: Default version.

Use Default Package Version

To use the default version of all packages available for the gnu-openmpi MPI implementation:

user $source /home/bgo/shared/bin/selector-mpi -i gnu-openmpi
Implementation: gnu-openmpi
Package: lammps/default (20161117)
Package: qe/default (5.0.2)

Use Specific Package Versions

To use LAMMPS version 20161117, Quantum ESPRESSO version 5.0.2, and the default version for all other available packages for the gnu-openmpi MPI implementation:

user $source /home/bgo/shared/bin/selector-mpi -i gnu-openmpi lammps/20161117 qe/5.0.2
Implementation: gnu-openmpi
Package: lammps/20161117
Package: qe/5.0.2

See Details

To see in more detail what selector-mpi is doing, the verbose flag can be added to the command line. The previous command, with the verbose flag, gives:

user $source /home/bgo/shared/bin/selector-mpi --verbose -i gnu-openmpi lammps/20161117 qe/5.0.2
Implementation: gnu-openmpi
Cleaning paths.
Removing previous implementation paths.
  Updated PATH
  Updated MANPATH
  Updated LD_LIBRARY_PATH
Updating environmental variables for base implementation.
         Adding to PATH: /home/bgo/shared/bin
Updating environmental variables for gnu-openmpi implementation.
         Adding to PATH: /home/bgo/shared/gnu-openmpi/bin
         Adding to LD_LIBRARY_PATH: /home/bgo/shared/gnu-openmpi/lib
         Adding to LD_LIBRARY_PATH: /home/bgo/shared/gnu-openmpi/usr/lib
         Adding to MANPATH: /home/bgo/shared/gnu-openmpi/share/man
Package: lammps/default (20161117)
         Adding to PATH: /home/bgo/shared/gnu-openmpi/versioned/lammps/default/bin
Package: qe/default (5.0.2)
         Adding to PATH: /home/bgo/shared/gnu-openmpi/versioned/qe/default/bin

Automatically Select Packages on Login

To automatically select Open MPI compiled with the GNU compiler and the default version of all packages for every login, add the following line to the bottom of the $HOME/.bashrc file:

source /home/bgo/shared/bin/selector-mpi -i gnu-openmpi -f -q

See Make Changes Persistent and Global on the Initial Setup page for details.

Note
It is recommended to put a line similar to this into all queue submission scripts to be sure which MPI implementation and package version is used.

Cleaning PATH

The PATH environmental variable, which defines a list of directories that is searched when looking for an executable program, can sometimes become cluttered. The --clean (or -c) command line option to selector-mpi is used to help clean up the path. For example, when the current path is:

user $echo $PATH
/usr/lib64/qt-3.3/bin:/usr/NX/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/pssc/bin:/opt/openmpi/bin:/opt/torque/bin:/opt/torque/sbin:/opt/mpich2/bin

Running selector-mpi with the --clean and --quiet command line options gives a PATH of:

user $source /home/bgo/shared/bin/selector-mpi --clean --quiet
user $echo $PATH
/usr/lib64/qt-3.3/bin:/usr/NX/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/pssc/bin:/opt/torque/bin:/opt/torque/sbin:

where /opt/openmpi/bin and /opt/mpich2/bin have been removed from PATH. Adding the -f command line option gives:

user $source /home/bgo/shared/bin/selector-mpi --clean -f --quiet
user $echo $PATH
/usr/lib64/qt-3.3/bin:/usr/NX/bin:/usr/local/bin:/bin:/usr/bin:/opt/torque/bin

where /usr/local/sbin, /usr/sbin, /sbin, and /opt/pssc/bin have been removed.

Advanced

The user can specify additional paths to be cleaned by defining an environmental variable called SELECTOR_USER_TO_CLEAR_PATH. For example, to have the /usr/lib64/qt-3.3/bin and /usr/NX/bin paths removed from PATH which is currently

user $echo $PATH
/usr/lib64/qt-3.3/bin:/usr/NX/bin:/usr/local/bin:/bin:/usr/bin:/opt/torque/bin

run the following commands:

user $export SELECTOR_USER_TO_CLEAR_PATH=/usr/lib64/qt-3.3/bin:/usr/NX/bin
user $source /home/bgo/shared/bin/selector-mpi --clean -f --quiet
user $echo $PATH
/usr/local/bin:/bin:/usr/bin:/opt/torque/bin

NOTE: Care must be taken when setting the SELECTOR_USER_TO_CLEAR_PATH environmental variable because it is used to match the prefix of each path in PATH. For example, with SELECTOR_USER_TO_CLEAR_PATH=/usr/local, any path beginning with /usr/local will be removed from PATH, like /usr/local/bin and /usr/local/lib64.

Retrieved from "http://magnolia.usm.edu/index.php?title=Selector-mpi_(flory)&oldid=873"