Enabling the DNS VO style test.gridpp.ac.uk

From GridPP Wiki
Jump to: navigation, search

Objective:

To enable support of the test.gridpp.ac.uk vo hosted on the following VOMS in Manchester:

https://voms03.gpp.hep.man.ac.uk:8443/voms/test.gridpp.ac.uk/

on UI, CE, WN, DPM SE and RB and to test the job submission chain on the pre-production service.

Yaim version:

3.0.1-17 (but should work with version 3.0.1-* too)

Yaim configuration:

I would like to map users of this vo to the local accounts gridpp[[:digit:]]{3} with unix group gridpp. I would also like to bind test.gridpp.ac.uk to the gridpp and short queues. I cannot use the test.gridpp.ac.uk name as this is too long for torque.

I've setup my users.conf file as follows:

...
85001:gridpp001:8500:gridpp:test.gridpp.ac.uk::
85002:gridpp002:8500:gridpp:test.gridpp.ac.uk::
...

and I've got only one line (only one in group and no role but VO-Admin) in my groups.conf file

"/VO=test.gridpp.ac.uk/GROUP=/test.gridpp.ac.uk"::::

The relevant portion of the site-info.def file is:

VOS="alice atlas biomed calice cms dteam test.gridpp.ac.uk lhcb ops"
#
ALICE_GROUP_ENABLE="alice"
ATLAS_GROUP_ENABLE="atlas"
CMS_GROUP_ENABLE="cms"
DTEAM_GROUP_ENABLE="dteam"
GRIDPP_GROUP_ENABLE="test.gridpp.ac.uk"
LHCB_GROUP_ENABLE="lhcb"
OPS_GROUP_ENABLE="ops"
SHORT_GROUP_ENABLE="alice atlas cms dteam test.gridpp.ac.uk lhcb ops"
#
QUEUES="alice atlas biomed calice cms dteam gridpp lhcb ops short"

Note about potentially common pitfall: I originally set QUEUES="$VOS short"

In the directory of my site-info.def file, I created the vo.d directory and the test.gridpp.ac.uk file with the details of this VO:

$ ls site-info.def; cat vo.d/test.gridpp.ac.uk
site-info.def
SW_DIR=$VO_SW_DIR/test.gridpp.ac.uk
VO_DEFAULT_SE=$DPM_HOST
VO_STORAGE_DIR=$SE_ACCESSPOINT/test.gridpp.ac.uk
VO_VOMS_SERVERS="vomss://voms.gridpp.ac.uk:8443/voms/gridpp?/gridpp/"
VO_VOMSES="'test.gridpp.ac.uk voms03.gpp.hep.man.ac.uk 15015 
            /C=UK/O=eScience/OU=Manchester/L=HEP/CN=voms.gridpp.ac.uk/Email=hostmaster@hep.man.ac.uk test.gridpp.ac.uk'"

The yaim configuration proceeds as usual. I noticed that no entry was added to edg-mkgridmap.conf for the test.gridpp.ac.uk VO. In hindsight, this is correct as no LDAP compatibility mode is required for the new DNS style VOs. The environment is not set for that VO in lcgenv.sh and wrongly set in lcgenv.csh.

From the following yaim fragment:

if [ x`eval echo ${VO} | egrep "\.|-"` == 'x' ] ; then
#       Export only if VO name is not DNS style
   echo "export VO_${VO}_DEFAULT_SE=$default_se" >> ${LCG_ENV_LOC}/lcgenv.sh
fi

I'm under the impression that not setting the DEFAULT_SE and SW_DIR is the intended behaviour? I feel this is bug, but no reply yet on the PPS mailing list.

Testing:

VOMS proxy instantiation on the UI:

$ voms-proxy-init -voms test.gridpp.ac.uk
Your identity: /C=UK/O=eScience/OU=Birmingham/L=ParticlePhysics/CN=yves coppens
Enter GRID pass phrase:
Creating temporary proxy ....................................................... Done
Contacting  voms03.gpp.hep.man.ac.uk:15015 [/C=UK/O=eScience/OU=Manchester/L=HEP/CN=voms.gridpp.ac.uk/Email=hostmaster@hep.man.ac.uk]
"test.gridpp.ac.uk" Done
Creating proxy ...................................... Done
Your proxy is valid until Tue Jun  5 02:20:53 2007
$

Are my queues properly published?

$ edg-job-list-match -vo test.gridpp.ac.uk env.jdl
Selected Virtual Organisation name (from --vo option): test.gridpp.ac.uk
Connecting to host epbf007.ph.bham.ac.uk, port 7772
***************************************************************************
                         COMPUTING ELEMENT IDs LIST
 The following CE(s) matching your job requirements have been found:

                   *CEId*
 epbf005.ph.bham.ac.uk:2119/jobmanager-lcgpbs-gridpp
 epbf005.ph.bham.ac.uk:2119/jobmanager-lcgpbs-short
***************************************************************************
$

That's ok. So can I run a job and retrieve its output?

$ edg-job-submit -vo test.gridpp.ac.uk env.jdl
Selected Virtual Organisation name (from --vo option): test.gridpp.ac.uk
Connecting to host epbf007.ph.bham.ac.uk, port 7772
Logging to host epbf007.ph.bham.ac.uk, port 9002
*********************************************************************************************
                               JOB SUBMIT OUTCOME
 The job has been successfully submitted to the Network Server.
 Use edg-job-status command to check job current status. Your job identifier (edg_jobId) is:
 - https://epbf007.ph.bham.ac.uk:9000/ysapsRTYVc6Rf_rgX6bWOw
*********************************************************************************************
$
$ edg-job-status  https://epbf007.ph.bham.ac.uk:9000/ysapsRTYVc6Rf_rgX6bWOw
*************************************************************
BOOKKEEPING INFORMATION:
Status info for the Job : https://epbf007.ph.bham.ac.uk:9000/ysapsRTYVc6Rf_rgX6bWOw
Current Status:     Done (Success)
Exit code:          0
Status Reason:      Job terminated successfully
Destination:        epbf005.ph.bham.ac.uk:2119/jobmanager-lcgpbs-short
reached on:         Mon Jun  4 13:30:32 2007
*************************************************************
$ edg-job-get-output  https://epbf007.ph.bham.ac.uk:9000/ysapsRTYVc6Rf_rgX6bWOw
Retrieving files from host: epbf007.ph.bham.ac.uk ( for https://epbf007.ph.bham.ac.uk:9000/ysapsRTYVc6Rf_rgX6bWOw )
*********************************************************************************
                        JOB GET OUTPUT OUTCOME
Output sandbox files for the job:
- https://epbf007.ph.bham.ac.uk:9000/ysapsRTYVc6Rf_rgX6bWOw
have been successfully retrieved and stored in the directory:
/tmp/jobOutput/yrc_ysapsRTYVc6Rf_rgX6bWOw
*********************************************************************************
$

A simple DPM test:

$ dpns-mkdir /dpm/ph.bham.ac.uk/home/test.gridpp.ac.uk/atest
$ lcg-cp -v  file:/home/yrc/yrcTestFile gsiftp://epbf004.ph.bham.ac.uk//dpm/ph.bham.ac.uk/home/test.gridpp.ac.uk/atest/yrcTestFile
Source URL: file:/home/yrc/yrcTestFile
File size: 197
Source URL for copy: file:/home/yrc/yrcTestFile
Destination URL: gsiftp://epbf004.ph.bham.ac.uk//dpm/ph.bham.ac.uk/home/test.gridpp.ac.uk/atest/yrcTestFile
# streams: 1
# set timeout to  0 (seconds)
          197 bytes      0.08 KB/sec avg      0.08 KB/sec inst
Transfer took 3090 ms
$
$ 
$ srm-advisory-delete  srm://epbf004.ph.bham.ac.uk:8443/dpm/ph.bham.ac.uk/home/test.gridpp.ac.uk/atest/yrcTestFile
Storage Resource Manager (SRM) CP Client version 1.21
Copyright (c) 2002-2006 Fermi National Accelerator Laboratory
SRM Configuration:
       debug=true
<snip>
Mon Jun 04 14:38:51 BST 2007:  srm.advisoryDelete() returned
$
$ dpns-ls /dpm/ph.bham.ac.uk/home/test.gridpp.ac.uk/atest
$

So, all is well in these rudimentary tests!