QMUL Multicore Gridengine Configuration

Create parallel environment (PE) with dynamic cores PE configuration and attach this to the queue that will run the multicore jobs

For more info see https://twiki.cern.ch/twiki/bin/view/LCG/SGE

qconf -sp smp, (man sge_pe for more info)

pe_name            smp 
slots              2448    
user_lists         NONE
xuser_lists        NONE
start_proc_args    /bin/true
stop_proc_args     /bin/true
allocation_rule    $pe_slots
control_slaves     FALSE
job_is_first_task  TRUE
urgency_slots      min
accounting_summary FALSE

Reservations (draining nodes for multi core jobs) needs to be enabled. '-R y' flag enabling reservations is not added by 'sge_submit.sh' current workaround is running 'qalter -R y $list_of_mc_jobs' by a cron.

#!/bin/bash
# reserve slots for jobs requesting 8 cores
. /etc/profile.d/sge.sh
/usr/bin/qstat -u \* | grep " 8 " | awk '{ system("/usr/bin/qalter " $1 " -R y")}'
/usr/bin/qstat -u \* | grep " 2 " | awk '{ system("/usr/bin/qalter " $1 " -R y")}'
/usr/bin/qstat -u \* | grep " 4 " | awk '{ system("/usr/bin/qalter " $1 " -R y")}'

While you are at it why not enable mpi, you then also get the mpi nagios check for free. note you need to make sure that the PE SMP is listed first in order to make sure the multicore job is run only on one node.

https://wiki.egi.eu/wiki/MAN03_MPI-Start_Installation_and_Configuration