Some simple test jobs

Here are some simple test jobs to help debug parts of the VO's use of the grid.

WMS and queue test

Command line version

This job submits this hello world job to all the queues in the UK via all the WMSs in the UK.

#!/bin/bash

VO=dteam
# remember to do:
#voms-proxy-init --voms $VO 

for wms in $(lcg-infosites --vo $VO wms | grep ac.uk); do
export GLITE_WMS_WMPROXY_ENDPOINT=$wms
    for ce in $(lcg-infosites --vo $VO ce | awk '/ac.uk/{print $6}'); do
        glite-wms-job-submit -a -o jobIDfile  -r $ce helloworld.jdl
    done
done 

# Then to check job status do:
#glite-wms-job-status -i jobIDfile

# THen get output with:
#glite-wms-job-output -i jobIDfile

submits the following hello world job:

#############Hello World#################
Executable = "/bin/echo";
Arguments = "Hello welcome to Dteam ";
StdOutput = "hello.out";
StdError = "hello.err";
OutputSandbox = {"hello.out","hello.err"};
##########################################

Ganga Version

# submit a hello world job to all WMS's and all CEs 

# Don't forget to specify your VO in [LCG][VirtualOrganisation]

# get wms and ce lists                                                                                                                                              
b = LCG()
wms_list = b.get_wms_list()
ce_list = b.get_ce_list()

# setup the job                                                                     
job = Job()

job.application.exe = "/bin/echo"
job.application.args = "Hello welcome to Dteam "
job.backend = LCG(requirements = LCGRequirements())

# loop over WMS list                         
for wms in wms_list:
    print wms
    for ce in ce_list:
        print ce
        j = job.copy()

        # only run on this wms and this ce         
        config['LCG']['GLITE_ALLOWED_WMS_LIST'] = [ wms ]
        j.backend.requirements.allowedCEs = ce
        try:
           j.submit()
        except:
           pass       

# view the output with              
#jobs(<id>).peek("stdout.gz", "emacs")  

For more info on using Ganga as an automatic blacklisting service, see Setting_up_an_Automatic_Blacklisting_Service

Proxy testing

Getting proxy renewal working seems to cause problems. Here are some scripts to use that will allow you to test that proxy renewal is working.

In essence, what you should do is deliberately create a short lived proxy (1 hour in the example here), then check that it is renewed.

This test job will run on a worker node for more than the normal proxy lifetime, and copy reports onto a grid storage element as it goes. Ideally we'll be enhancing it with a non-grid output method so that even jobs with expired proxies can get the word out. To use it you will need to do several things (these instructions are written with the gridpp VO in mind, some things will need to be changed for other VOs).

- Get your proxies ready:

 voms-proxy-init --voms gridpp --valid 1:0
 myproxy-init -d -n
 glite-wms-job-delegate-proxy  -d ewan

• Find a Logical File Catalogue

 export LFC_HOST=$(lcg-infosites --vo gridpp lfc) 

• and make yourslf a suitable directory in it

 lfc-mkdir /grid/gridpp/myproxytests-2011/ewan
 export LFC_HOME=/grid/gridpp/myproxytests-2011/ewan

• Update the entry in proxytest.sh to match that and to point to a suitable Storage Element

• Submit the job(s)

 glite-wms-job-submit -d ewan -o jobIDfile myproxytest.jdl
 glite-wms-job-status -i jobIDfile

• myproxytest.jdl

walker@heppc400:~/grid/myproxy-test-ewan/myproxytest-April2012$ cat  myproxytest.jdl 
# You might need to change the myproxy server (but 
# probably not):
MyProxyServer = "lcgrbp01.gridpp.rl.ac.uk";

# If you want to force a particular Computing Element, set it here:
# Requirements = other.GlueCEInfoHostName == "t2ce06.physics.ox.ac.uk";

# The following should be OK as is:
Executable = "proxytest.sh";
StdOutput = "env.out";
StdError = "env.err";
InputSandbox = {"proxytest.sh","burnP6"};
OutputSandbox = {"env.out","env.err"};

• proxytest.sh

#!/bin/bash
#
# proxytest.sh
#
# This should use lcg-infosites, but that seems not
# to work for the gridpp VO for now, but the actual
# LFC does.


#export LFC_HOST=lfc.gridpp.rl.ac.uk

export LFC_HOST=prod-lfc-shared-central.cern.ch
# Tweak the last component of this for other people, 
# change the whole thing for other VOs

export LFC_HOME=/grid/dteam/myproxytests-2011/cwalker3

# Pick an SE. Not using the sites' default close SE
# though we could really since everything's getting
# registered in the LFC anyway.

targetSE=se03.esc.qmul.ac.uk
#targetSE=t2se01.physics.ox.ac.uk

# Function to run a CPU burn process while we sleep 
# to avoid tripping inefficient jobs killers
busywait() {
        ./burnP6 &
        pid=$!
        sleep $1
        kill ${pid}
}


chmod u+x ./burnP6
myhost=$(hostname -f)
echo ${LFC_HOST}
echo ${LFC_HOME}

for ((h=0;h<=36;h++)); do
    for ((m=0;m<=50;m=m+10)); do
        date
        voms-proxy-info --all

        date >proxyinfo
        echo ${GRID_JOBID} >>proxyinfo
        voms-proxy-info --all >>proxyinfo
        lcg-cr -d ${targetSE} -D srmv2 -l myproxytest-${myhost}.${h}.${m} file://$(pwd)/proxyinfo
        busywait 600
    done
done

You should delete these files once you are finished.

Check if the proxy has been renewed on the WMS.

# Get info on the proxy of your current job
glite-wms-job-info --proxy <job Id>
</pro