Difference between revisions of "Quick Guide to Dirac"

From GridPP Wiki
Jump to: navigation, search
Line 11: Line 11:
 
* Please consider signing up to the gridpp dirac users mailing list: [https://mailman.ic.ac.uk/mailman/listinfo/gridpp-dirac-users Sign up !]
 
* Please consider signing up to the gridpp dirac users mailing list: [https://mailman.ic.ac.uk/mailman/listinfo/gridpp-dirac-users Sign up !]
 
* For operational issues please consider filing a [https://ggus.eu/ GGUS ticket]. If you set "Notify site" to UKI-LT2-IC-HEP, we'll see it quicker. If you don't know if your issue is operational, just use the mailing list :-)
 
* For operational issues please consider filing a [https://ggus.eu/ GGUS ticket]. If you set "Notify site" to UKI-LT2-IC-HEP, we'll see it quicker. If you don't know if your issue is operational, just use the mailing list :-)
−
* If something isn't working, it may help to include the output from your commands using the debug flag, which can be done with the <code>-dd</code> flag, e.g.
+
* If something isn't working, it may help to include the output from your commands using the debug flag, which can be done with the <code>-ddd</code> flag, e.g.
  
 
<pre>
 
<pre>
−
$ dirac-proxy-init -dd -g gridpp_user -M
+
$ dirac-proxy-init -ddd -g gridpp_user -M
 
</pre>
 
</pre>
  
Line 20: Line 20:
 
The Dirac server is installed at Imperial College. The Web interface is at:
 
The Dirac server is installed at Imperial College. The Web interface is at:
  
−
   https://dirac.grid.hep.ph.ic.ac.uk:8443
+
   https://dirac.gridpp.ac.uk
  
 
You need a certificate installed in your browser. <br>
 
You need a certificate installed in your browser. <br>
−
To check that you are correctly registered with this dirac instance, check the bottom righthand corner. If this shows you as "visitor" and you cannot change this, you are not registered. In this case, please contact Janusz.
+
To check that you are correctly registered with this dirac instance, check the bottom righthand corner. If this shows you as "visitor" and you cannot change this, you are not registered. In this case, please contact lcg-site-admin@imperial.ac.uk
  
 
= Dirac client installation =  
 
= Dirac client installation =  
 +
You need an SL6 machine, with your user cert and key located in the $HOME/.globus directory. You also need to be a member of a dirac supported VO. <br>
 +
You can try and run this python script (which indeed should move to a better location): [http://www.hep.ph.ic.ac.uk/~dbauer/dirac/ui/install_dirac_ui.py dirac ui install script]. <br>
 +
Or you can run the command sequence by hand: <br>
 
<pre>
 
<pre>
−
mkdir dirac
+
mkdir dirac_ui
−
cd dirac
+
cd dirac_ui
 
wget -np -O dirac-install http://lhcbproject.web.cern.ch/lhcbproject/dist/Dirac_project/dirac-install
 
wget -np -O dirac-install http://lhcbproject.web.cern.ch/lhcbproject/dist/Dirac_project/dirac-install
−
chmod +x dirac-install
+
dirac-install -r v6r13p9 -i 27 -g 2015-07-09
−
./dirac-install -V gridpp -r v6r11p24
+
 
source bashrc  
 
source bashrc  
−
</pre>
+
dirac-proxy-init -x (needs user cert password)
 +
dirac-configure -F -S GridPP -C dips://dirac01.grid.hep.ph.ic.ac.uk:9135/Configuration/Server -I
 +
dirac-proxy-init -g [your_vo_goes_here]_user' + ' -M' (e.g. dirac-proxy-init -g comet.j-parc.jp_user -M)
  
−
Note 1: The 'gridpp' in the command above does not refer to the gridpp VO - please use the command as is, independently of which VO you will use later on."
 
−
 
−
'''Note 2: The version (v6r11p24) should match the version given at the bottom of the [https://dirac00.grid.hep.ph.ic.ac.uk:8443/DIRAC/ GridPP DIRAC portal web pages]. CHECK THIS BEFORE CREATING A NEW DIRAC CLIENT.'''
 
−
 
−
= Configure the client =
 
−
 
−
Create a dumb, groupless proxy (mind -x). This option is to be used only when configuring the client.
 
−
 
−
You need your keys (userkey.pem and usercert.pem) in $HOME/.globus/.
 
−
You will also need the [http://en.wikipedia.org/wiki/Certificate_authority CA] certs. When you install the dirac UI, the script will automatically download a current copy of these certs for you and store them in dirac/etc/grid-security/certificates. This will be sufficient for a couple of weeks of  testing. If you are planning on using your dirac UI for longer, you need to update these certificates regularly. Most grid sites maintain a current copy of these files somewhere on their systems and you should link to those (try /etc/grid-security/certificates or at Imperial /vols/grid/certificates -- if in doubt ask your friendly sys admin :-) instead.
 
−
 
−
Make a proxy.
 
−
<pre>
 
−
$ dirac-proxy-init -x
 
−
 
−
Generating proxy...
 
−
Enter Certificate password:
 
−
Proxy generated:
 
−
subject      : /C=UK/O=eScience/OU=Imperial/L=Physics/CN=janusz martyniak/CN=proxy
 
−
issuer      : /C=UK/O=eScience/OU=Imperial/L=Physics/CN=janusz martyniak
 
−
identity    : /C=UK/O=eScience/OU=Imperial/L=Physics/CN=janusz martyniak
 
−
timeleft    : 23:59:59
 
−
path        : /tmp/x509up_u500
 
−
username    : unknown
 
−
properties  : 
 
 
</pre>
 
</pre>
  
−
 
+
If you see an error like below then you probably are not registered with the dirac instance.
−
Configure defaults. (Also use this command (-F == force update) when updating the UI).
+
−
 
+
−
<pre>
+
−
$ dirac-configure -F defaults-gridpp.cfg
+
−
</pre>
+
−
 
+
−
if you see an error like below then you probably are not registered with the dirac instance.
+
 
<pre>
 
<pre>
 
Could not sync dir Cannot get URL for Framework/BundleDelivery in setup MyDIRAC-Production: Option /DIRAC/Setups/MyDIRAC-Production/Framework is not defined
 
Could not sync dir Cannot get URL for Framework/BundleDelivery in setup MyDIRAC-Production: Option /DIRAC/Setups/MyDIRAC-Production/Framework is not defined
 
</pre>
 
</pre>
  
−
= Create a proxy for a community=  
+
= A note on CA certificates =
−
 
+
When you install the dirac UI, the script will automatically download a current copy of these certs for you and store them in dirac/etc/grid-security/certificates. This will be sufficient for a couple of weeks of testing. If you are planning on using your dirac UI for longer, you need to update these certificates regularly. Most grid sites maintain a current copy of these files somewhere on their systems and you should link to those (try /etc/grid-security/certificates or at Imperial /vols/grid/certificates -- if in doubt ask your friendly sys admin :-) instead.
−
To be able to submit a job, you need to be a member of a community (VO), here '''gridpp'''. If you are, you can create a proxy:
+
−
 
+
−
<pre>
+
−
$ dirac-proxy-init -g gridpp_user -M
+
−
Generating proxy...
+
−
Enter Certificate password:
+
−
Uploading proxy for gridpp_user...
+
−
 
+
−
Proxy generated:
+
−
subject      : /C=UK/O=eScience/OU=Imperial/L=Physics/CN=janusz martyniak/CN=proxy
+
−
issuer      : /C=UK/O=eScience/OU=Imperial/L=Physics/CN=janusz martyniak
+
−
identity    : /C=UK/O=eScience/OU=Imperial/L=Physics/CN=janusz martyniak
+
−
timeleft    : 23:59:58
+
−
DIRAC group  : gridpp_user
+
−
path        : /tmp/x509up_u500
+
−
username    : martynia
+
−
properties  : NormalUser
+
−
</pre>
+
  
−
You can easily verify what VOs you are a member of, by going to the Dirac website and clicking at the drop-down menu at the bottom right, next to your username@ text. If your username shows "Anonymous" it means
+
== Submitting a 'Hello World' job via dirac ==
−
that you are not registered as a Dirac user. Please contact janusz.martyniak at imperial.ac.uk. <br>
+
−
'''Other VOs: vo.londongrid.ac.uk: use londongrid_user, for t2k.org please use t2k_user, for pheno use pheno_user, for snoplus use snoplus_user, for cernatschool use cernatschool_user, comet: comet_user''' <br>
+
−
Note: We are aware that the VO name and the dirac group should match, but for the time being, please bear with us and use the dirac groups ......
+
  
 
= Create a JDL =
 
= Create a JDL =

Revision as of 14:16, 3 August 2015

Note: These instructions have been updated to use the 'new' dirac server at dirac.gridpp.ac.uk. If you are using the 'old' dirac server at dirac.grid.hep.ph.ic.ac.uk, please email Janusz with a cc to lcg-site-admin@imperial.ac.uk in case of problems.

Introduction

This document describes how to quickly set up a simple Dirac UI client, create a job script and then submit it to a Dirac server for execution. You need to have a valid grid certificate and be a member of a supported VO. You do not need root access to the machine you are installing the Dirac UI on.

If you want to check if the dirac server recognizes you, please go to the GridPP Dirac Server and check if your name appears in the bottom right corner with the correct VO associated with it. You can be a member of more than one VO.

Getting support

  • Please consider signing up to the gridpp dirac users mailing list: Sign up !
  • For operational issues please consider filing a GGUS ticket. If you set "Notify site" to UKI-LT2-IC-HEP, we'll see it quicker. If you don't know if your issue is operational, just use the mailing list :-)
  • If something isn't working, it may help to include the output from your commands using the debug flag, which can be done with the -ddd flag, e.g. 
$ dirac-proxy-init -ddd -g gridpp_user -M

Server URL

The Dirac server is installed at Imperial College. The Web interface is at: 

 https://dirac.gridpp.ac.uk

You need a certificate installed in your browser.
To check that you are correctly registered with this dirac instance, check the bottom righthand corner. If this shows you as "visitor" and you cannot change this, you are not registered. In this case, please contact lcg-site-admin@imperial.ac.uk

Dirac client installation

You need an SL6 machine, with your user cert and key located in the $HOME/.globus directory. You also need to be a member of a dirac supported VO.
You can try and run this python script (which indeed should move to a better location): dirac ui install script.
Or you can run the command sequence by hand: 

mkdir dirac_ui
cd dirac_ui
wget -np -O dirac-install http://lhcbproject.web.cern.ch/lhcbproject/dist/Dirac_project/dirac-install
dirac-install -r v6r13p9 -i 27 -g 2015-07-09
source bashrc 
dirac-proxy-init -x (needs user cert password)
dirac-configure -F -S GridPP -C dips://dirac01.grid.hep.ph.ic.ac.uk:9135/Configuration/Server -I
dirac-proxy-init -g [your_vo_goes_here]_user' + ' -M' (e.g. dirac-proxy-init -g comet.j-parc.jp_user -M)

If you see an error like below then you probably are not registered with the dirac instance. 

Could not sync dir Cannot get URL for Framework/BundleDelivery in setup MyDIRAC-Production: Option /DIRAC/Setups/MyDIRAC-Production/Framework is not defined

A note on CA certificates

When you install the dirac UI, the script will automatically download a current copy of these certs for you and store them in dirac/etc/grid-security/certificates. This will be sufficient for a couple of weeks of testing. If you are planning on using your dirac UI for longer, you need to update these certificates regularly. Most grid sites maintain a current copy of these files somewhere on their systems and you should link to those (try /etc/grid-security/certificates or at Imperial /vols/grid/certificates -- if in doubt ask your friendly sys admin :-) instead.

Submitting a 'Hello World' job via dirac

Create a JDL

Now we are ready to create a simple Dirac jdl. Put the lines below into a file Simple.jdl: 

[
JobName = "Simple_Job";
Executable = "/bin/ls";
Arguments = "-ltr";
StdOutput = "StdOut";
StdError = "StdErr";
OutputSandbox = {"StdOut","StdErr"};
]

Submit a job

$ dirac-wms-job-submit Simple.jdl 
JobID = 236

Check job status: 

$ dirac-wms-job-status 236

JobID=236 Status=Waiting; MinorStatus=Pilot Agent Submission; Site=ANY

and eventually: 

$ dirac-wms-job-status 236
JobID=236 Status=Done; MinorStatus=Execution Complete; Site=LCG.Glasgow.uk;

The jobs can also be monitored using the Web interface. Go to Jobs ->Job monitor You can also submit a job using the Web interface (Tools->Job Launchpad).


Back to GridPP Dirac overview page.

Retrieved from "https://www.gridpp.ac.uk/w/index.php?title=Quick_Guide_to_Dirac&oldid=9174"