Schrödinger Command Line Job Submissions

Schrödinger was originally created to run on a workstation, not in an HPC environment like Magnolia. This results in the loss of common job submission controls and more importantly the requirement that the GUI be running while the job is running on the cluster. In common HPC environments the GUI is completely independent of the running job meaning that the GUI can be closed while the job is running on the cluster. When the job completes, an email is automatically sent to the user who then starts the GUI and imports the results. This page explains how to run Schrodinger jobs in a manner more closely resembling the server/client environment of HPC systems.

Initial Setup

An initial setup, which consists of one-time settings that rarely need to be changed, is required.

Queue Settings

The workstation environment that Schrodinger was originally designed for removes the ability of HPC cluster users to control the job queuing system. To allow for users to use two common settings, email notifications and job runtime limits, we have modified the Schrodinger scripts. The modification allows for the use of a configuration file where email an runtime limits can be sent. Each user needs to have this configuration file in their home directory, the easiest way to do this is by copying a default configuration. The instructions below assume you are connected to a Magnolia terminal (see the Connect to Magnolia page).

Copy a default configuration into your home directory on Magnolia using this command:

user $cp /modules/schrodinger/queue.local-example ${HOME}/.schrodinger/queue.local

To show the contents of the file, use:

user $cat ${HOME}/.schrodinger/queue.local

The default file should look something like:

FILE ${HOME}/.schrodinger/queue.local

# Configuration for per user job settings
# This file must be located in your $HOME/.schrodinger directory.
#
# To enable, use:
#
# cp /modules/schrodinger/queue.local-example ~/.schrodinger/queue.local
#
# SCHROD_SLURM_TIME
#    Max run time for jobs submitted to queue.
#
#    0-05:00:00 means 0 days and 5 hours.
#
#    See https://slurm.schedmd.com/sbatch.html#OPT_time
#    for time format.
#
# Errors like 'Unable to submit job for batch execution'
# may mean that the value entered below is invalid.
#
SCHROD_SLURM_TIME=0-05:00:00
#SCHROD_SLURM_EMAIL=me@usm.edu
#SCHROD_SLURM_EMAIL_TYPE=END

The queue.local file should be modified to have your email and maximum job runtime. To edit the file, use the nano editor:

user $nano -w ${HOME}/.bashrc

Use the cursor keys to move around the file and make changes. To save any changes you make, use Ctrl+O, then hit Enter to accept the file name. Use Ctrl+X to exit nano.

FILE ${HOME}/.schrodinger/queue.local(after editing)

# Configuration for per user job settings
# This file must be located in your $HOME/.schrodinger directory.
#
# To enable, use:
#
# cp /modules/schrodinger/queue.local-example ~/.schrodinger/queue.local
#
# SCHROD_SLURM_TIME
#    Max run time for jobs submitted to queue.
#
#    0-05:00:00 means 0 days and 5 hours.
#
#    See https://slurm.schedmd.com/sbatch.html#OPT_time
#    for time format.
#
# Errors like 'Unable to submit job for batch execution'
# may mean that the value entered below is invalid.
#
SCHROD_SLURM_TIME=2-01:10:11
SCHROD_SLURM_EMAIL=example@gmail.com
SCHROD_SLURM_EMAIL_TYPE=END

This sends email when a job is complete to `example@gmail.com`, you should change this to where you'd like email sent. The maximum job run time is set to 2 days, 1 hour, 10 minutes, and 11 seconds.

Note
If # is the first character on a line, then the whole line is considered a comment and ignored. Therefore, remove any # for settings you wish to use. Notice that there is no # before the last three (3) lines of the example.

Job Settings

It will be convenient to have jobs saved in the project directory of Schrodinger. Open the preferences window (Edit->Preferences) and change theDefault directory for job startup and i/o to Project Directory:

Create Job for Command Line Submission

Click the gear icon next to the Job Name to bring up the Molecular Dynamics - Job Settings dialog.

The job name field will also be the name of a directory created, so fill this in with something easy to remember, and without spaces.

At this point, normally the 'Run' button is clicked and the job is sent through the GUI. However, since we want to submit our job through the command line, here we click the Write button and then close the Molecular Dynaimcs - Job Settings dialog window.

Now we are done with the GUI, so close all of the Schrodinger GUI windows

Submit Job via Command Line

Connect to the command line of the Cluster CITE and go to the directory that was created in the `Create Job for Command Line Submission` step. For example

user $cd ${HOME}/TestProject/TestJob01

Once there check which files are present in that directory, you should see something like:

user $ls -l

TestRun001.cfg
TestRun001.cms
TestRun001.msj
TestRun001.sh

Most of the files define your structure and molecular dynamics job type, but the file ending in .sh is an executable job which will build the submission script and submit it to the cluster. Execute the script with

user $./TestRun001.sh

Note
The leading dot of the previous command is important.

The command will take a few moments to complete, then drop you back to a prompt where you can type. Your job should now be submitted.

Wait for job to Complete

Your job should be submitted to the cluster now and is either running or waiting to run. To check the status of your job, run a command like:

user $squeue -p gpu

This command shows all jobs on the GPU nodes. To see jobs only belonging to yourself, use a command like:

user $squeue -u user

where the text `user` needs to be replace with your Magnolia username.

If you completed the `Initial Setup` steps and configured the queue to send you an email when jobs complete, simply wait until you receive the email. Otherwise you'll have to use the two `squeue` commands above to check if you job has completed.