QMUL Multicore Gridengine Configuration
From GridPP Wiki
Revision as of 11:25, 11 February 2016 by Christopher Walker 5f07fc1200 (Talk | contribs)
Create parallel environment (PE) with dynamic cores PE configuration and attach this to the queue that will run the multicore jobs
For more info see https://twiki.cern.ch/twiki/bin/view/LCG/SGE
qconf -sp smp, (man sge_pe for more info)
pe_name smp slots 2448 user_lists NONE xuser_lists NONE start_proc_args /bin/true stop_proc_args /bin/true allocation_rule $pe_slots control_slaves FALSE job_is_first_task TRUE urgency_slots min accounting_summary FALSE
Reservations (draining nodes for multi core jobs) needs to be enabled. '-R y' flag enabling reservations is not added by 'sge_submit.sh' current workaround is running 'qalter -R y $list_of_mc_jobs' by a cron.
#!/bin/bash # reserve slots for jobs requesting 8 cores . /etc/profile.d/sge.sh /usr/bin/qstat -u \* | grep " 8 " | awk '{ system("/usr/bin/qalter " $1 " -R y")}' /usr/bin/qstat -u \* | grep " 2 " | awk '{ system("/usr/bin/qalter " $1 " -R y")}' /usr/bin/qstat -u \* | grep " 4 " | awk '{ system("/usr/bin/qalter " $1 " -R y")}'
While you are at it why not enable mpi, you then also get the mpi nagios check for free. note you need to make sure that the PE SMP is listed first in order to make sure the multicore job is run only on one node.
https://wiki.egi.eu/wiki/MAN03_MPI-Start_Installation_and_Configuration