Difference between revisions of "ARC HTCondor Accounting"

Revision as of 11:09, 5 June 2014

This page explains how APEL accounting is done at the RAL Tier-1 in order to ensure that CPU benchmarks are taken into account properly.

HTCondor configuration

The HTCondor configuration on the worker nodes includes the following, for example:

RalScaling = 2.14
STARTD_ATTRS = $(STARTD_ATTRS) RalScaling

Here RalScaling is a scaling factor used to normalize the CPU benchmarks. Quattor is configured to do this automatically.

The HTCondor configuration on the ARC CEs contains:

# CPU benchmark scaling factor
MachineRalScaling = "$$([ifThenElse(isUndefined(RalScaling), 1.00, RalScaling)])"
SUBMIT_EXPRS = $(SUBMIT_EXPRS) MachineRalScaling

This causes the attribute MachineRalScaling to be added to job ClassAds, and is available as MATCH_EXP_MachineRalScaling.

ARC CE configuration

We include the following line in the [grid-manager] section of /etc/arc.conf:

authplugin="FINISHING timeout=60,onfailure=pass,onsuccess=pass /usr/local/bin/scaling_factors_plugin.py %S %C %I"

where scaling_factors_plugin.py is available here: https://gist.github.com/alahiff/3a427620e73b42d9282d#file-scaling_factors_plugin-py. This scales the CPU and walltimes in the files from which the ARC CE extracts accounting information.

The following lines should be included in /etc/arc.conf in order to ensure that the ARC CE will send its accounting records to APEL:

jobreport="APEL:http://mq.cro-ngi.hr:6162"
jobreport_options="urbatch:1000,archiving:/var/run/arc/urs,topic:/queue/global.accounting.cpu.central,gocdb_name:RAL-LCG2,use_ssl:true,Network:PROD,benchmark_type:Si2k,benchmark_value:1006.00"
jobreport_credentials="/etc/grid-security/hostkey.pem /etc/grid-security/hostcert.pem /etc/grid-security/certificates"
jobreport_publisher="jura"

APEL

In order for an ARC CE to have permission to send accounting records to the production APEL broker, the CE must be registered in the GOC DB as a glite-APEL node type with the host DN specified.