Examples for submit_command

In this section we present several examples that you can use to set the submit_command parameter in your Execution Scenario. Notice that the Running module is agnostic to the execution environment, so you can create other submission commands for your specific system.

We recommend to use a bash script to encapsulate the submission command. This script will recieve the following parameters, and in this specific order:

  • cmd: Command that will be executed

  • name: Name of the submitted job

  • cpu_seconds: Job CPU limit in seconds

  • mem: Job Memory limit (in MB)

  • slots: Number of slots consumed by the job

  • output: Path where the job’s output will be stored

  • err: Path where the job’s error will be stored

  • Remaining args for cmd

To ease job management it is recommended to use a queue manager. In the following we present examples for Task Spooler (to run experiments in local) and for SGE (to run experiments in a cluster).

Task Spooler

Task Spooler (tsp) [LluisBiRossell21] is a simple unix batch system. This is an example script that can be passed to submit_command to run experiments in a local machine through tsp:

#!/bin/bash

cmd=$1
name=$2
slots=$5
output=$6
err=$7
shift 7

EXECUTING_COMMAND="${cmd} ${@} > ${output} 2> ${err}"
tsp -L $name -n -N ${slots} sh -c "$EXECUTING_COMMAND"

Warning

Task Spooler will not enforce time or memory limits, so it is required to set the runsolver parameter to True in the Scenario generation constructor.

SGE

Sun Grid Engine (SGE) [Microsystems01] is a batch-queuing system usually used in computer clusters. You can use the following script as submit_command to run experiments in a cluster:

#!/bin/bash

cmd=$1
name=$2
cpu_seconds=$3
mem=$4
slots=$5
output=$6
err=$7
shift 7

RQUEUE=<YOUR-QUEUE-NAME>

EXECUTING_COMMAND="${cmd} ${@}"
echo "${EXECUTING_COMMAND}"     \
    | qsub -cwd -V -q ${RQUEUE} \
    -pe smp ${slots}            \
    -l h_cpu=${cpu_seconds}     \
    -l h_vmem=${mem}M           \
    -N $name -o $output -e $err

Note

SGE automatically enforces the required time and memory limits, so the usage of runsolver in the Scenario generation constructor is not required.

Notice also that you must specify in the script the name of the queue in SGE where the jobs will be submitted.