Schrödinger Command Line Job Submissions: Difference between revisions

Revision as of 21:14, 17 June 2020

Schrödinger was originally created to run on a workstation, not in an HPC environment like Magnolia. This results in the loss of common job submission controls and more importantly the requirement that the GUI be running while the job is running on the cluster. In common HPC environments the GUI is completely independent of the running job meaning that the GUI can be closed while the job is running on the cluster. When the job completes, an email is automatically sent to the user who then starts the GUI and imports the results. This page explains how to run Schrodinger jobs in a manner more closely resembling the server/client environment of HPC systems.

Initial Setup

An initial setup, which consists of one-time settings that rarely need to be changed, is required.

Queue Settings

The workstation environment that Schrodinger was originally designed for removes the ability of HPC cluster users to control the job queuing system. To allow for users to use two common settings, email notifications and job runtime limits, we have modified the Schrodinger scripts. The modification allows for the use of a configuration file where email an runtime limits can be sent. Each user needs to have this configuration file in their home directory, the easiest way to do this is by copying a default configuration. The instructions in this section assume you are connected Magnolia (see the Connect to Magnolia page).

Copy a default configuration into your home directory on Magnolia using this command:

user $cp /modules/schrodinger/queue.local-example ${HOME}/.schrodinger/queue.local

To show the contents of the file, use:

user $cat ${HOME}/.schrodinger/queue.local

The default file should look something like:

FILE ${HOME}/.schrodinger/queue.local

# Configuration for per user job settings
# This file must be located in your $HOME/.schrodinger directory.
#
# To enable, use:
#
# cp /modules/schrodinger/queue.local-example ~/.schrodinger/queue.local
#
# SCHROD_SLURM_TIME
#    Max run time for jobs submitted to queue.
#
#    0-05:00:00 means 0 days and 5 hours.
#
#    See https://slurm.schedmd.com/sbatch.html#OPT_time
#    for time format.
#
# Errors like 'Unable to submit job for batch execution'
# may mean that the value entered below is invalid.
#
SCHROD_SLURM_TIME=0-05:00:00
#SCHROD_SLURM_EMAIL=me@usm.edu
#SCHROD_SLURM_EMAIL_TYPE=END

The queue.local file should be modified to have your email and maximum job runtime. To edit the file, use the nano editor:

user $nano -w ${HOME}/.bashrc

Use the cursor keys to move around the file and make changes. To save any changes you make, use Ctrl+O, then hit Enter to accept the file name. Use Ctrl+X to exit nano.

FILE ${HOME}/.schrodinger/queue.local(after editing)

# Configuration for per user job settings
# This file must be located in your $HOME/.schrodinger directory.
#
# To enable, use:
#
# cp /modules/schrodinger/queue.local-example ~/.schrodinger/queue.local
#
# SCHROD_SLURM_TIME
#    Max run time for jobs submitted to queue.
#
#    0-05:00:00 means 0 days and 5 hours.
#
#    See https://slurm.schedmd.com/sbatch.html#OPT_time
#    for time format.
#
# Errors like 'Unable to submit job for batch execution'
# may mean that the value entered below is invalid.
#
SCHROD_SLURM_TIME=2-01:10:11
SCHROD_SLURM_EMAIL=example@gmail.com
SCHROD_SLURM_EMAIL_TYPE=END

This sends an email when a job is complete to example@gmail.com, you should change this to where you'd like email sent. The maximum job run time is set to 2 days, 1 hour, 10 minutes, and 11 seconds.

Note
If # is the first character on a line, then the whole line is considered a comment and ignored. Therefore, remove any # for settings you wish to use. Notice that there is no # before the last three (3) lines of the example.

Job Settings

The following settings control show the files for jobs are saved. By default, the files will be saved in your home directory. To help organize the job files, it will be convenient to have jobs saved in the project directory of Schrodinger. These settings need to be set in the GUI. Launch Schrodinger and open the preferences window (Edit->Preferences) and change the Default directory for job startup and i/o to Project Directory:

This completes the initial setup, the following is required for each job you wish to submit.

Create Job for Command Line Submission

Setup a Molecular Dynamics jobs in the GUI as usual, but instead of directly running the job click the gear icon next to the Job Name to bring up the Molecular Dynamics - Job Settings dialog.

The job name field will be the name of a directory created, so fill this in with something easy to remember, and without spaces.

At this point, normally the 'Run' button is clicked and the job is sent through the GUI. However, since we want to submit our job through the command line, here we click the Write button and then close the Molecular Dynaimcs - Job Settings dialog window.

Now we are done with the GUI for now, so close all of the Schrodinger GUI windows

Submit Job via Command Line

Connect to Magnolia and go to the directory that was created in the Create Job for Command Line Submission section. If you followed the instructions in the job settings section the directory will be inside the project directory. For example, if the project is named TestProject and the job was named TestJob001 use:

user $cd ${HOME}/TestProject/TestJob01

Once there, check which files are present in that directory with the ls command. You should see something like:

user $ls -l

TestRun001.cfg
TestRun001.cms
TestRun001.msj
TestRun001.sh

Most of the files define your structure and molecular dynamics job type, but the file ending in .sh is an executable which will build the submission script and submit it to the cluster. Execute the script with

user $./TestRun001.sh

Note
The leading dot of the previous command is important.

The command will take a few moments to complete, then drop you back to a prompt where you can type. Your job should now be submitted.

Wait for job to Complete

Your job should be submitted to the cluster now and is either running or waiting to run. To check the status of your job, run a command like:

user $squeue -p gpu

This command shows all jobs on the GPU nodes. To see jobs only belonging to yourself, use a command like:

user $squeue -u user

where the text user needs to be replace with your Magnolia username.

If you completed the initial setup steps and configured the queue to send you an email when jobs complete, simply wait until you receive the email. Otherwise you'll have to use the two squeue commands above to check if you job has completed.

@@ Line 9: / Line 9: @@
 === Queue Settings ===
-The workstation environment that Schrodinger was originally designed for removes the ability of HPC cluster users to control the job queuing system. To allow for users to use two common settings, email notifications and job runtime limits, we have modified the Schrodinger scripts. The modification allows for the use of a configuration file where email an runtime limits can be sent. Each user needs to have this configuration file in their home directory, the easiest way to do this is by copying a default configuration. The instructions below assume you are connected to a Magnolia terminal (see the [[Connect_to_Magnolia#Connecting_.2F_Login|Connect to Magnolia]] page).
+The workstation environment that Schrodinger was originally designed for removes the ability of HPC cluster users to control the job queuing system. To allow for users to use two common settings, email notifications and job runtime limits, we have modified the Schrodinger scripts. The modification allows for the use of a configuration file where email an runtime limits can be sent. Each user needs to have this configuration file in their home directory, the easiest way to do this is by copying a default configuration. The instructions in this section assume you are connected Magnolia (see the [[Connect_to_Magnolia#Connecting_.2F_Login|Connect to Magnolia]] page).
 Copy a default configuration into your home directory on Magnolia using this command:
@@ Line 72: / Line 72: @@
 }}
-This sends email when a job is complete to `example@gmail.com`, you should change this to where you'd like email sent. The maximum job run time is set to 2 days, 1 hour, 10 minutes, and 11 seconds.
+This sends an email when a job is complete to ''example@gmail.com'', you should change this to where you'd like email sent. The maximum job run time is set to 2 days, 1 hour, 10 minutes, and 11 seconds.
 {{Note|If ''#'' is the first character on a line, then the whole line is considered a comment and ignored. Therefore, remove any ''#'' for settings you wish to use. Notice that there is no ''#'' before the last three (3) lines of the example.}}
 === Job Settings ===
-It will be convenient to have jobs saved in the project directory of Schrodinger. Open the preferences window (Edit->Preferences) and change the''Default directory for job startup and i/o'' to ''Project Directory'':
+The following settings control show the files for jobs are saved. By default, the files will be saved in your home directory. To help organize the job files, it will be convenient to have jobs saved in the project directory of Schrodinger. These settings need to be set in the GUI. Launch Schrodinger and open the preferences window (Edit->Preferences) and change the ''Default directory for job startup and i/o'' to ''Project Directory'':
 [[File:Schrodinger-Preferences-Directories.png]]
+This completes the initial setup, the following is required for each job you wish to submit.
 == Create Job for Command Line Submission ==
-Click the gear icon next to the ''Job Name'' to bring up the ''Molecular Dynamics - Job Settings'' dialog.
+Setup a Molecular Dynamics jobs in the GUI as usual, but instead of directly running the job click the gear icon next to the ''Job Name'' to bring up the ''Molecular Dynamics - Job Settings'' dialog.
 [[File:Schrodinger-JobSettings-Dialog.png]]
-The job name field will also be the name of a directory created, so fill this in with something easy to remember, and without spaces.
+The job name field will be the name of a directory created, so fill this in with something easy to remember, and without spaces.
 At this point, normally the 'Run' button is clicked and the job is sent through the GUI. However, since we want to submit our job through the command line, here we click the ''Write'' button and then close the ''Molecular Dynaimcs - Job Settings'' dialog window.
-Now we are done with the GUI, so close all of the Schrodinger GUI windows
+Now we are done with the GUI for now, so close all of the Schrodinger GUI windows
 == Submit Job via Command Line ==
-Connect to the command line of the Cluster CITE and go to the directory that was created in the `Create Job for Command Line Submission` step. For example
+[[Connect_to_Magnolia#Connecting_.2F_Login|Connect to Magnolia]] and go to the directory that was created in the [[#Create Job for Command Line Submission|Create Job for Command Line Submission]] section. If you followed the instructions in the [[#Job Settings|job settings]] section the directory will be inside the project directory. For example, if the project is named ''TestProject'' and the job was named ''TestJob001'' use:
 {{Cmd|cd ${HOME}/TestProject/TestJob01}}
-Once there check which files are present in that directory, you should see something like:
+Once there, check which files are present in that directory with the {{C|ls}} command. You should see something like:
 {{Cmd|ls -l|output=<pre>
 TestRun001.cfg
@@ Line 106: / Line 108: @@
 </pre>}}
-Most of the files define your structure and molecular dynamics job type, but the file ending in ''.sh'' is an executable job which will build the submission script and submit it to the cluster. Execute the script with
+Most of the files define your structure and molecular dynamics job type, but the file ending in ''.sh'' is an executable which will build the submission script and submit it to the cluster. Execute the script with
 {{Cmd|./TestRun001.sh}}
@@ Line 121: / Line 123: @@
 {{Cmd|squeue -u user}}
-where the text `user` needs to be replace with your Magnolia username.
+where the text ''user'' needs to be replace with your Magnolia username.
-If you completed the `Initial Setup` steps and configured the queue to send you an email when jobs complete, simply wait until you receive the email. Otherwise you'll have to use the two `squeue` commands above to check if you job has completed.
+If you completed the [[#Initial Setup|initial setup]] steps and configured the queue to send you an email when jobs complete, simply wait until you receive the email. Otherwise you'll have to use the two {{C|squeue}} commands above to check if you job has completed.
 == Load Completed Job into the GUI ==