Difference between revisions of "Imperial Dirac TransformationSystem"

From GridPP Wiki
Jump to: navigation, search
(GridPP DIRAC Transformation System)
 
 
(6 intermediate revisions by one user not shown)
Line 1: Line 1:
 +
== Documentation ==
 
[https://github.com/DIRACGrid/DIRAC/wiki/Transformation-System-Tutorial Tutorial] <br>
 
[https://github.com/DIRACGrid/DIRAC/wiki/Transformation-System-Tutorial Tutorial] <br>
 
[http://dirac.readthedocs.io/en/latest/AdministratorGuide/Systems/Transformation/index.html Install Guide]
 
[http://dirac.readthedocs.io/en/latest/AdministratorGuide/Systems/Transformation/index.html Install Guide]
 +
 +
== Install Log ==
 +
Instance Name is Devel <br>
 +
Setup Name is GridPP <br>
 +
 +
<pre>
 +
dirac-admin-sysadmin-cli --host dirac00.grid.hep.ph.ic.ac.uk
 +
Pinging dirac00.grid.hep.ph.ic.ac.uk...
 +
[dirac00.grid.hep.ph.ic.ac.uk]> add instance Transformation Devel
 +
Adding Transformation system as Devel self.instance for GridPP self.setup to dirac.cfg and CS
 +
Transformation system instance Devel added successfully
 +
 +
[dirac00.grid.hep.ph.ic.ac.uk]> install db TransformationDB
 +
Adding to CS Transformation/TransformationDB
 +
Database TransformationDB from DIRAC/TransformationSystem installed successfully
 +
</pre>
 +
It turns out the TransformationManagerHandler is only a TransformationManager.
 +
<pre>
 +
[dirac00.grid.hep.ph.ic.ac.uk]> install service Transformation TransformationManager
 +
Loading configuration template /home/hep/dbauer/check_test_dirac/2018_May_30_1452_skatelescope.eu_EL7/DIRAC/TransformationSystem/ConfigTemplate.cfg
 +
Adding to CS service Transformation/TransformationManager
 +
service Transformation_TransformationManager is installed, runit status: Run
 +
</pre>
 +
Now go and try to install all the agents: <br>
 +
<pre>
 +
[dirac00.grid.hep.ph.ic.ac.uk]> install agent Transformation TransformationAgent
 +
Loading configuration template /home/hep/dbauer/check_test_dirac/2018_May_30_1452_skatelescope.eu_EL7/DIRAC/TransformationSystem/ConfigTemplate.cfg
 +
Adding to CS agent Transformation/TransformationAgent
 +
agent Transformation_TransformationAgent is installed, runit status: Run
 +
 +
[dirac00.grid.hep.ph.ic.ac.uk]> install agent Transformation WorkflowTaskAgent
 +
Loading configuration template /home/hep/dbauer/check_test_dirac/2018_May_30_1452_skatelescope.eu_EL7/DIRAC/TransformationSystem/ConfigTemplate.cfg
 +
Adding to CS agent Transformation/WorkflowTaskAgent
 +
agent Transformation_WorkflowTaskAgent is installed, runit status: Run
 +
 +
[dirac00.grid.hep.ph.ic.ac.uk]> install agent Transformation RequestTaskAgent
 +
Loading configuration template /home/hep/dbauer/check_test_dirac/2018_May_30_1452_skatelescope.eu_EL7/DIRAC/TransformationSystem/ConfigTemplate.cfg
 +
Adding to CS agent Transformation/RequestTaskAgent
 +
agent Transformation_RequestTaskAgent is installed, runit status: Run
 +
 +
[dirac00.grid.hep.ph.ic.ac.uk]> install agent Transformation MCExtensionAgent
 +
Loading configuration template /home/hep/dbauer/check_test_dirac/2018_May_30_1452_skatelescope.eu_EL7/DIRAC/TransformationSystem/ConfigTemplate.cfg
 +
Adding to CS agent Transformation/MCExtensionAgent
 +
agent Transformation_MCExtensionAgent is installed, runit status: Run
 +
 +
[dirac00.grid.hep.ph.ic.ac.uk]> install agent Transformation TransformationCleaningAgent
 +
Loading configuration template /home/hep/dbauer/check_test_dirac/2018_May_30_1452_skatelescope.eu_EL7/DIRAC/TransformationSystem/ConfigTemplate.cfg
 +
Adding to CS agent Transformation/TransformationCleaningAgent
 +
agent Transformation_TransformationCleaningAgent is installed, runit status: Run
 +
 +
[dirac00.grid.hep.ph.ic.ac.uk]> install agent Transformation InputDataAgent
 +
Loading configuration template /home/hep/dbauer/check_test_dirac/2018_May_30_1452_skatelescope.eu_EL7/DIRAC/TransformationSystem/ConfigTemplate.cfg
 +
Adding to CS agent Transformation/InputDataAgent
 +
agent Transformation_InputDataAgent is installed, runit status: Run
 +
</pre>
 +
 +
In the config Systems/Transformation/Devel/Agents: All newly installed agents have default values, stick with them for now, apart from the
 +
RequestTaskAgent which acquires all the config options of the tutorial server:
 +
<pre>
 +
SubmitTasks = yes
 +
CheckReserved = yes
 +
MonitorTasks = yes
 +
MonitorFiles = yes
 +
TaskUpdateStatus = Submitted, Received, Waiting, Running, Matched, Completed, Failed
 +
TransType = Removal, Replication
 +
</pre>
 +
 +
In Transformation/Devel/Databases/TransformationDB change host to 127.0.0.1 (on dirac00).
 +
 +
 +
shifterProxy = ProductionManager
 +
 +
 +
Metadata: The fields to be used by the transformation have to be indexed in the FileCatalogDB.
 +
From a DIRAC UI as an admin do e.g. <br>
 +
<pre>
 +
FC:/> meta index -f owner string
 +
Added metadata field owner of type string
 +
</pre>
 +
(use 'help' to get list of all options for 'meta')

Latest revision as of 13:51, 20 June 2018

Documentation

Tutorial
Install Guide

Install Log

Instance Name is Devel
Setup Name is GridPP 

dirac-admin-sysadmin-cli --host dirac00.grid.hep.ph.ic.ac.uk
Pinging dirac00.grid.hep.ph.ic.ac.uk...
[dirac00.grid.hep.ph.ic.ac.uk]> add instance Transformation Devel
Adding Transformation system as Devel self.instance for GridPP self.setup to dirac.cfg and CS
Transformation system instance Devel added successfully

[dirac00.grid.hep.ph.ic.ac.uk]> install db TransformationDB
Adding to CS Transformation/TransformationDB
Database TransformationDB from DIRAC/TransformationSystem installed successfully

It turns out the TransformationManagerHandler is only a TransformationManager. 

[dirac00.grid.hep.ph.ic.ac.uk]> install service Transformation TransformationManager
Loading configuration template /home/hep/dbauer/check_test_dirac/2018_May_30_1452_skatelescope.eu_EL7/DIRAC/TransformationSystem/ConfigTemplate.cfg
Adding to CS service Transformation/TransformationManager
service Transformation_TransformationManager is installed, runit status: Run

Now go and try to install all the agents: 

[dirac00.grid.hep.ph.ic.ac.uk]> install agent Transformation TransformationAgent
Loading configuration template /home/hep/dbauer/check_test_dirac/2018_May_30_1452_skatelescope.eu_EL7/DIRAC/TransformationSystem/ConfigTemplate.cfg
Adding to CS agent Transformation/TransformationAgent
agent Transformation_TransformationAgent is installed, runit status: Run

[dirac00.grid.hep.ph.ic.ac.uk]> install agent Transformation WorkflowTaskAgent
Loading configuration template /home/hep/dbauer/check_test_dirac/2018_May_30_1452_skatelescope.eu_EL7/DIRAC/TransformationSystem/ConfigTemplate.cfg
Adding to CS agent Transformation/WorkflowTaskAgent
agent Transformation_WorkflowTaskAgent is installed, runit status: Run

[dirac00.grid.hep.ph.ic.ac.uk]> install agent Transformation RequestTaskAgent
Loading configuration template /home/hep/dbauer/check_test_dirac/2018_May_30_1452_skatelescope.eu_EL7/DIRAC/TransformationSystem/ConfigTemplate.cfg
Adding to CS agent Transformation/RequestTaskAgent
agent Transformation_RequestTaskAgent is installed, runit status: Run

[dirac00.grid.hep.ph.ic.ac.uk]> install agent Transformation MCExtensionAgent
Loading configuration template /home/hep/dbauer/check_test_dirac/2018_May_30_1452_skatelescope.eu_EL7/DIRAC/TransformationSystem/ConfigTemplate.cfg
Adding to CS agent Transformation/MCExtensionAgent
agent Transformation_MCExtensionAgent is installed, runit status: Run

[dirac00.grid.hep.ph.ic.ac.uk]> install agent Transformation TransformationCleaningAgent
Loading configuration template /home/hep/dbauer/check_test_dirac/2018_May_30_1452_skatelescope.eu_EL7/DIRAC/TransformationSystem/ConfigTemplate.cfg
Adding to CS agent Transformation/TransformationCleaningAgent
agent Transformation_TransformationCleaningAgent is installed, runit status: Run

[dirac00.grid.hep.ph.ic.ac.uk]> install agent Transformation InputDataAgent
Loading configuration template /home/hep/dbauer/check_test_dirac/2018_May_30_1452_skatelescope.eu_EL7/DIRAC/TransformationSystem/ConfigTemplate.cfg
Adding to CS agent Transformation/InputDataAgent
agent Transformation_InputDataAgent is installed, runit status: Run

In the config Systems/Transformation/Devel/Agents: All newly installed agents have default values, stick with them for now, apart from the RequestTaskAgent which acquires all the config options of the tutorial server: 

SubmitTasks = yes
CheckReserved = yes
MonitorTasks = yes
MonitorFiles = yes
TaskUpdateStatus = Submitted, Received, Waiting, Running, Matched, Completed, Failed
TransType = Removal, Replication

In Transformation/Devel/Databases/TransformationDB change host to 127.0.0.1 (on dirac00).


shifterProxy = ProductionManager


Metadata: The fields to be used by the transformation have to be indexed in the FileCatalogDB. From a DIRAC UI as an admin do e.g. 

FC:/> meta index -f owner string
Added metadata field owner of type string

(use 'help' to get list of all options for 'meta')

Retrieved from "https://www.gridpp.ac.uk/w/index.php?title=Imperial_Dirac_TransformationSystem&oldid=17896"