Difference between revisions of "Quick Guide to Dirac"

From GridPP Wiki
Jump to: navigation, search
(Dirac client installation)
(Dirac client installation)
(27 intermediate revisions by 2 users not shown)
Line 4: Line 4:
 
This document describes how to quickly set up a simple Dirac UI client, create a job script and then submit it to a Dirac server for execution. '''You need to have a valid grid certificate and be a member of a [https://www.gridpp.ac.uk/wiki/VOs_supported_by_gridpp_dirac supported VO].''' You '''do not''' need root access to the machine you are installing the Dirac UI on.
 
This document describes how to quickly set up a simple Dirac UI client, create a job script and then submit it to a Dirac server for execution. '''You need to have a valid grid certificate and be a member of a [https://www.gridpp.ac.uk/wiki/VOs_supported_by_gridpp_dirac supported VO].''' You '''do not''' need root access to the machine you are installing the Dirac UI on.
  
''Note: new users may wish to start with the [https://www.gridpp.ac.uk/userguide/getting-on-the-grid/getting-on-the-grid.html GridPP UserGuide], which includes complete instructions for getting setup with GridPP DIRAC.''
+
''Note: If you are completely new to grid computing, please either talk to computing support in your experiment and/or your local grid admin first to make sure you have the basics covered. Any attempts at producing "one size fits all" documentation have failed, for good reason. If you have no idea who to contact, please email tb-support/at/jiscmail.ac.uk with as much detail as you are able to provide and we'll work it out from there.''
  
 
===Getting support===
 
===Getting support===
Line 22: Line 22:
 
= Dirac client installation =  
 
= Dirac client installation =  
 
'''16/11/2020: Major version upgrade. Please update your UI.''' <br>
 
'''16/11/2020: Major version upgrade. Please update your UI.''' <br>
You need an '''SL7''' (SL6 might still work, but we do not recommned it) machine, with your user cert and key located in the $HOME/.globus directory. You also need to be a member of a GridPP supported VO. <br>
+
If you do not want to install a client and have cvmfs available on your machine you can do:
 +
<pre>
 +
source /cvmfs/dirac.egi.eu/dirac/bashrc_gridpp
 +
dirac-proxy-init -g [your_vo_goes_here]_user -M -U # (e.g. dirac-proxy-init -g comet.j-parc.jp_user -M -U)
 +
</pre>
 +
and skip the rest of this paragraph.
 +
 
 +
You need an '''SL7''' (SL6 might still work, but we do not recommend it) machine, with your user cert and key located in the $HOME/.globus directory. You also need to be a member of a GridPP supported VO. <br>
 
Please check that your firewall is open on the appropriate ports: <br>
 
Please check that your firewall is open on the appropriate ports: <br>
 
nc -v dirac01.grid.hep.ph.ic.ac.uk 9135 <br>
 
nc -v dirac01.grid.hep.ph.ic.ac.uk 9135 <br>
Line 30: Line 37:
 
''Note: the installation works on SL7 only. If your cluster does not support SL7, you may wish to consider using a [https://www.gridpp.ac.uk/userguide/gridpp-cernvm/gridpp-cernvm.html GridPP CernVM].''
 
''Note: the installation works on SL7 only. If your cluster does not support SL7, you may wish to consider using a [https://www.gridpp.ac.uk/userguide/gridpp-cernvm/gridpp-cernvm.html GridPP CernVM].''
  
To install the UI, run this command sequence in a bash shell:
+
To install the UI, run this command sequence in a '''bash''' shell:
  
 
<pre>
 
<pre>
 
mkdir dirac_ui
 
mkdir dirac_ui
 
cd dirac_ui
 
cd dirac_ui
wget -np -O dirac-install https://raw.githubusercontent.com/DIRACGrid/DIRAC/integration/Core/scripts/dirac-install.py  
+
wget -np -O dirac-install https://raw.githubusercontent.com/DIRACGrid/management/master/dirac-install.py
 
# add --no-check-certificate to wget if needed
 
# add --no-check-certificate to wget if needed
 
chmod u+x dirac-install
 
chmod u+x dirac-install
./dirac-install -r v7r1p17
+
./dirac-install -r v7r2p28
. bashrc  
+
source bashrc  
dirac-proxy-init -x # (needs user cert password)
+
dirac-proxy-init -x -N # (needs user cert password; note that for higher versions of DIRAC you need to add the -N option to this command, in addition to -x)
 
dirac-configure -F -S GridPP -C dips://dirac01.grid.hep.ph.ic.ac.uk:9135/Configuration/Server -I
 
dirac-configure -F -S GridPP -C dips://dirac01.grid.hep.ph.ic.ac.uk:9135/Configuration/Server -I
dirac-proxy-init -g [your_vo_goes_here]_user -M # (e.g. dirac-proxy-init -g comet.j-parc.jp_user -M)
+
 
 +
Note: If you are still using  v7r1p45, please add:
 +
export X509_VOMS_DIR="$DIRAC/etc/grid-security/vomsdir"
 +
export X509_VOMSES="$DIRAC/etc/grid-security/vomses"
 +
at the end of your bashrc. This should be fixed in v7r2p28
 +
 
 +
dirac-proxy-init -g [your_vo_goes_here]_user -M -U # (e.g. dirac-proxy-init -g comet.j-parc.jp_user -M -U)
 
</pre>
 
</pre>
  
  
If you see an error like below then you probably are not registered with the dirac instance. Usually you are automatically registered, but there is a delay of several hours between registering with the VO and the dirac server picking this up. If after 24h you still get the same error message, please send an email to lcg-site-admin -at- imperial.ac.uk and we'll have a look.
+
If you see an error like below then you probably are not registered with the dirac instance. Usually you are automatically registered, but there is a delay of several hours between registering with the VO and the dirac server picking this up. If after 12h you still get the same error message, please send an email to lcg-site-admin -at- imperial.ac.uk and we'll have a look.
  
 
<pre>
 
<pre>
 
Could not sync dir Cannot get URL for Framework/BundleDelivery in setup MyDIRAC-Production: Option /DIRAC/Setups/MyDIRAC-Production/Framework is not defined
 
Could not sync dir Cannot get URL for Framework/BundleDelivery in setup MyDIRAC-Production: Option /DIRAC/Setups/MyDIRAC-Production/Framework is not defined
 +
</pre>
 +
If you have cvmfs available on your SL7 machine you can use the cvmfs ui instead:
 +
<pre>
 +
/cvmfs/dirac.egi.eu/dirac/bashrc_gridpp
 +
</pre>
 +
You should be able to run dirac-proxy-init -g [your_vo_goes_here]_user -M -U immediately after that.
 +
 +
'''New in v7r2: DIRAC UI is now available in python3'''
 +
<pre>
 +
mkdir dirac_py3
 +
cd dirac_py3
 +
curl -LO https://github.com/DIRACGrid/DIRACOS2/releases/latest/download/DIRACOS-Linux-$(uname -m).sh
 +
bash DIRACOS-Linux-$(uname -m).sh
 +
rm DIRACOS-Linux-$(uname -m).sh
 +
source diracos/diracosrc
 +
pip install DIRAC==7.2.28
 +
dirac-configure -F -S GridPP -C dips://dirac01.grid.hep.ph.ic.ac.uk:9135/Configuration/Server -I
 
</pre>
 
</pre>
  
Line 124: Line 154:
 
By default GridPP DIRAC will only run jobs at sites providing an EL7 (as in "Enterprise Linux 7", treat it as a synonym for SL7) based operating system.
 
By default GridPP DIRAC will only run jobs at sites providing an EL7 (as in "Enterprise Linux 7", treat it as a synonym for SL7) based operating system.
 
If you would like to submit a job that requires a different operating system (such as EL6)
 
If you would like to submit a job that requires a different operating system (such as EL6)
this can be set using the Platform option in the JDL:
+
this can be set using the Platform option in the JDL (note that by now (Jan 2021) there are basically no EL6 resources left:
  
 
<pre>Platform = "EL6";</pre>
 
<pre>Platform = "EL6";</pre>
Line 223: Line 253:
 
</pre>
 
</pre>
  
=Data Handling within a job=  
+
=Output data handling within a job=  
  
 
Please see the documentation here: [[DIRAC_Data_Handling_within_a_Job]]
 
Please see the documentation here: [[DIRAC_Data_Handling_within_a_Job]]
  
 
=DIRAC command reference=
 
=DIRAC command reference=
[http://dirac.readthedocs.io/en/rel-v6r22/UserGuide/CommandReference/index.html Docs for v6r22]
+
[https://dirac.readthedocs.io/en/latest/ DIRAC on ReadTheDocs]
  
 
<br />
 
<br />
 
<hr>
 
<hr>
 
Back to [https://www.gridpp.ac.uk/wiki/Dirac GridPP Dirac] overview page.
 
Back to [https://www.gridpp.ac.uk/wiki/Dirac GridPP Dirac] overview page.

Revision as of 15:23, 13 December 2021

These instruction are for the dirac server at https://dirac.gridpp.ac.uk/DIRAC.

Introduction

This document describes how to quickly set up a simple Dirac UI client, create a job script and then submit it to a Dirac server for execution. You need to have a valid grid certificate and be a member of a supported VO. You do not need root access to the machine you are installing the Dirac UI on.

Note: If you are completely new to grid computing, please either talk to computing support in your experiment and/or your local grid admin first to make sure you have the basics covered. Any attempts at producing "one size fits all" documentation have failed, for good reason. If you have no idea who to contact, please email tb-support/at/jiscmail.ac.uk with as much detail as you are able to provide and we'll work it out from there.

Getting support

  • Please consider signing up to the gridpp dirac users mailing list: Sign up ! (It's fairly low traffic.) If you are using a non-institutional email, please let us know, it's hard to seperate the spam from the real requests. Thank you.
  • If something isn't working, please try the mailing list first. It may help to include the output from your commands using the debug flag, which can be done with the -ddd flag, e.g.: dirac-proxy-init -ddd -g gridpp_user -M
  • For operational issues ("Dirac server has died") please consider filing a GGUS ticket. If you set "Notify site" to UKI-LT2-IC-HEP, we'll see it quicker. If you don't know if your issue is operational, just use the mailing list :-)
  • If your jobs don't run at a specific site that you think they should be running at (but run on other sites), please email lcg-site-admin at imperial.ac.uk. Please include your DN and VO. We will then check if it is a configuration issue on the dirac side and either fix it, or advise you whom to contact instead. Alternatively you can use GGUS for this issue. Please set "Notify site" to UKI-LT2-IC-HEP, we will reroute it to the appropriate site, if applicable.

Server URL

The GridPP DIRAC server is installed at Imperial College. The Web interface can be found at:

 https://dirac.gridpp.ac.uk

You need a certificate installed in your browser.
To check that you are correctly registered with this dirac instance and that you are a member of a dirac supported VO, visit [the server] and check the bottom righthand corner. If this shows you as "visitor" and you cannot change this, you are not registered. In this case, please contact lcg-site-admin@imperial.ac.uk

Dirac client installation

16/11/2020: Major version upgrade. Please update your UI.
If you do not want to install a client and have cvmfs available on your machine you can do:

source /cvmfs/dirac.egi.eu/dirac/bashrc_gridpp
dirac-proxy-init -g [your_vo_goes_here]_user -M -U # (e.g. dirac-proxy-init -g comet.j-parc.jp_user -M -U)

and skip the rest of this paragraph.

You need an SL7 (SL6 might still work, but we do not recommend it) machine, with your user cert and key located in the $HOME/.globus directory. You also need to be a member of a GridPP supported VO.
Please check that your firewall is open on the appropriate ports:
nc -v dirac01.grid.hep.ph.ic.ac.uk 9135
Connection to dirac01.grid.hep.ph.ic.ac.uk 9135 port [tcp/*] succeeded!


Note: the installation works on SL7 only. If your cluster does not support SL7, you may wish to consider using a GridPP CernVM.

To install the UI, run this command sequence in a bash shell:

mkdir dirac_ui
cd dirac_ui
wget -np -O dirac-install https://raw.githubusercontent.com/DIRACGrid/management/master/dirac-install.py
# add --no-check-certificate to wget if needed
chmod u+x dirac-install
./dirac-install -r v7r2p28
source  bashrc 
dirac-proxy-init -x -N # (needs user cert password; note that for higher versions of DIRAC you need to add the -N option to this command, in addition to -x)
dirac-configure -F -S GridPP -C dips://dirac01.grid.hep.ph.ic.ac.uk:9135/Configuration/Server -I

Note: If you are still using  v7r1p45, please add:
export X509_VOMS_DIR="$DIRAC/etc/grid-security/vomsdir"
export X509_VOMSES="$DIRAC/etc/grid-security/vomses"
at the end of your bashrc. This should be fixed in v7r2p28

dirac-proxy-init -g [your_vo_goes_here]_user -M -U # (e.g. dirac-proxy-init -g comet.j-parc.jp_user -M -U)


If you see an error like below then you probably are not registered with the dirac instance. Usually you are automatically registered, but there is a delay of several hours between registering with the VO and the dirac server picking this up. If after 12h you still get the same error message, please send an email to lcg-site-admin -at- imperial.ac.uk and we'll have a look.

Could not sync dir Cannot get URL for Framework/BundleDelivery in setup MyDIRAC-Production: Option /DIRAC/Setups/MyDIRAC-Production/Framework is not defined

If you have cvmfs available on your SL7 machine you can use the cvmfs ui instead:

/cvmfs/dirac.egi.eu/dirac/bashrc_gridpp

You should be able to run dirac-proxy-init -g [your_vo_goes_here]_user -M -U immediately after that.

New in v7r2: DIRAC UI is now available in python3

mkdir dirac_py3
cd dirac_py3
curl -LO https://github.com/DIRACGrid/DIRACOS2/releases/latest/download/DIRACOS-Linux-$(uname -m).sh
bash DIRACOS-Linux-$(uname -m).sh
rm DIRACOS-Linux-$(uname -m).sh
source diracos/diracosrc
pip install DIRAC==7.2.28
dirac-configure -F -S GridPP -C dips://dirac01.grid.hep.ph.ic.ac.uk:9135/Configuration/Server -I

Submitting a 'Hello World' job via dirac

Create a proxy

If you don't already have a valid proxy you should create one:

source  bashrc 
dirac-proxy-init -g [your_vo_goes_here]_user -M (e.g. dirac-proxy-init -g comet.j-parc.jp_user -M)

Create a JDL

Now we are ready to create a simple Dirac jdl. Put the lines below into a file Simple.jdl:

[
JobName = "Simple_Job";
Executable = "/bin/ls";
Arguments = "-ltr";
StdOutput = "StdOut";
StdError = "StdErr";
OutputSandbox = {"StdOut","StdErr"};
]

Submit a job

$ dirac-wms-job-submit -f logfile Simple.jdl 
JobID = 236

Check job status:

$ dirac-wms-job-status -f logfile

JobID=236 Status=Waiting; MinorStatus=Pilot Agent Submission; Site=ANY 

and eventually:

$ dirac-wms-job-status -f logfile 
JobID=236 Status=Done; MinorStatus=Execution Complete; Site=LCG.Glasgow.uk;

The jobs can also be monitored using the Web interface. Go to Jobs ->Job monitor You can also submit a job using the Web interface (Tools->Job Launchpad).

Once the job is done, retrieve the output:

$ dirac-wms-job-get-output -f logfile

A list of JDL parameters used by DIRAC can be found in the DIRAC documentation.

Advanced job management

For any real work, we recommend that you develop your code against the DIRAC python API. Please also read the "Data Handling Within a Job" section before you produce any kind of data to be stored on a storage element.

Using the DIRAC API to submit a Job

An example of a hello world job can be found here.
Another example with more options: Options!. You will also need this shell script which is used as an executable within the example.
The corresponding section in the DIRAC user guide can be found here and the Job API here.

Selecting a site platform

By default GridPP DIRAC will only run jobs at sites providing an EL7 (as in "Enterprise Linux 7", treat it as a synonym for SL7) based operating system. If you would like to submit a job that requires a different operating system (such as EL6) this can be set using the Platform option in the JDL (note that by now (Jan 2021) there are basically no EL6 resources left:

Platform = "EL6";

If is also possible to specify AnyPlatform as the platform name to disable this check:

Platform = "AnyPlatform";

Note that all parts of this JDL string are case sensitive (the word Platform and the platform string itself).

Users of the DIRAC API can use the job.setPlatform("EL6") function to configure this behaviour.

Ganga users can set extra options, including the platform, via the backend settings attribute:

j = Job(backend=Dirac())
j.backend.settings['Platform'] = "EL6"

Advanced Site/CE selection

If you want to run at a specific subset of available sites for your VO, you can specify a list of Sites using the usual Site clause with a list:

Site = {"Site1", "Site2", "Site3"};

It is also possible to select specific CEs (generally this is used for targeting special resources):

GridCE = {"ceprod05.grid.hep.ph.ic.ac.uk", "ceprod06.grid.hep.ph.ic.ac.uk"};

Note that selecting an invalid combination of Sites & CEs may result in your job staying in the waiting state forever.

Basic data management

(using the UI and proxy you just setup)
List files in the dirac file catalogue (please do not use the file catalogue for anything else):

$ dirac-dms-filecatalog-cli
Starting FileCatalog client
File Catalog Client $Revision: 1.17 $Date: 
FC:/> ls
(use 'exit' to, well, exit)

Add a file to an SE (here UKI-LT2-IC-HEP-disk). Please replace /gridpp with the name of the VO you are using, and firstname.lastname with your first and last name (or alias or superhero, it just has to be unique):

dirac-dms-add-file /gridpp/user/f/firstname.lastname/myfile.txt myfile.txt UKI-LT2-IC-HEP-disk

List all replicas:

dirac-dms-lfn-replicas /gridpp/user/f/firstname.lastname/myfile.txt

Copy file from SE to disk:

dirac-dms-get-file /gridpp/user/f/firstname.lastname/myfile.txt

Replicate file from on SE to another (file is currently at UKI-LT2-IC-HEP-disk):

dirac-dms-replicate-lfn /gridpp/user/f/firstname.lastname/myfile.txt UKI-LT2-QMUL2-disk

Ask for the access URL (you should only need this for very special cases):

dirac-dms-lfn-accessURL  /gridpp/user/f/firstname.lastname/myfile.txt UKI-LT2-IC-HEP-disk

Remove file from a specific SE:

dirac-dms-remove-replicas /gridpp/user/f/firstname.lastname/myfile.txt UKI-LT2-IC-HEP-disk

Remove all replicas (?):

dirac-dms-remove-files  /gridpp/user/f/firstname.lastname/myfile.txt

Using the asynchronous files transfer tools.
Replicate a file to a given SE:

dirac-dms-replicate-and-register-request [make up an id for this transfer] [LFN] [target SE]

e.g. replicate a file to the RHUL SE (files must exist and be registered in the file catalogue):

dirac-dms-replicate-and-register-request daniela1 /gridpp/user/d/daniela.bauer/small.text.file.txt UKI-LT2-RHUL-disk

Querying the transfers:

dirac-rms-request [transfer if from above, e.g. daniela1]
dirac-rms-request--Status='Done' --Since='2016-05-05'

Output data handling within a job

Please see the documentation here: DIRAC_Data_Handling_within_a_Job

DIRAC command reference

DIRAC on ReadTheDocs



Back to GridPP Dirac overview page.