Difference between revisions of "Imperial Dirac TransformationSystem"
From GridPP Wiki
| (5 intermediate revisions by one user not shown) | |||
| Line 13: | Line 13: | ||
Adding Transformation system as Devel self.instance for GridPP self.setup to dirac.cfg and CS | Adding Transformation system as Devel self.instance for GridPP self.setup to dirac.cfg and CS | ||
Transformation system instance Devel added successfully | Transformation system instance Devel added successfully | ||
| + | |||
[dirac00.grid.hep.ph.ic.ac.uk]> install db TransformationDB | [dirac00.grid.hep.ph.ic.ac.uk]> install db TransformationDB | ||
Adding to CS Transformation/TransformationDB | Adding to CS Transformation/TransformationDB | ||
| Line 30: | Line 31: | ||
Adding to CS agent Transformation/TransformationAgent | Adding to CS agent Transformation/TransformationAgent | ||
agent Transformation_TransformationAgent is installed, runit status: Run | agent Transformation_TransformationAgent is installed, runit status: Run | ||
| + | |||
[dirac00.grid.hep.ph.ic.ac.uk]> install agent Transformation WorkflowTaskAgent | [dirac00.grid.hep.ph.ic.ac.uk]> install agent Transformation WorkflowTaskAgent | ||
Loading configuration template /home/hep/dbauer/check_test_dirac/2018_May_30_1452_skatelescope.eu_EL7/DIRAC/TransformationSystem/ConfigTemplate.cfg | Loading configuration template /home/hep/dbauer/check_test_dirac/2018_May_30_1452_skatelescope.eu_EL7/DIRAC/TransformationSystem/ConfigTemplate.cfg | ||
Adding to CS agent Transformation/WorkflowTaskAgent | Adding to CS agent Transformation/WorkflowTaskAgent | ||
agent Transformation_WorkflowTaskAgent is installed, runit status: Run | agent Transformation_WorkflowTaskAgent is installed, runit status: Run | ||
| + | |||
[dirac00.grid.hep.ph.ic.ac.uk]> install agent Transformation RequestTaskAgent | [dirac00.grid.hep.ph.ic.ac.uk]> install agent Transformation RequestTaskAgent | ||
Loading configuration template /home/hep/dbauer/check_test_dirac/2018_May_30_1452_skatelescope.eu_EL7/DIRAC/TransformationSystem/ConfigTemplate.cfg | Loading configuration template /home/hep/dbauer/check_test_dirac/2018_May_30_1452_skatelescope.eu_EL7/DIRAC/TransformationSystem/ConfigTemplate.cfg | ||
Adding to CS agent Transformation/RequestTaskAgent | Adding to CS agent Transformation/RequestTaskAgent | ||
agent Transformation_RequestTaskAgent is installed, runit status: Run | agent Transformation_RequestTaskAgent is installed, runit status: Run | ||
| + | |||
[dirac00.grid.hep.ph.ic.ac.uk]> install agent Transformation MCExtensionAgent | [dirac00.grid.hep.ph.ic.ac.uk]> install agent Transformation MCExtensionAgent | ||
Loading configuration template /home/hep/dbauer/check_test_dirac/2018_May_30_1452_skatelescope.eu_EL7/DIRAC/TransformationSystem/ConfigTemplate.cfg | Loading configuration template /home/hep/dbauer/check_test_dirac/2018_May_30_1452_skatelescope.eu_EL7/DIRAC/TransformationSystem/ConfigTemplate.cfg | ||
Adding to CS agent Transformation/MCExtensionAgent | Adding to CS agent Transformation/MCExtensionAgent | ||
agent Transformation_MCExtensionAgent is installed, runit status: Run | agent Transformation_MCExtensionAgent is installed, runit status: Run | ||
| + | |||
[dirac00.grid.hep.ph.ic.ac.uk]> install agent Transformation TransformationCleaningAgent | [dirac00.grid.hep.ph.ic.ac.uk]> install agent Transformation TransformationCleaningAgent | ||
Loading configuration template /home/hep/dbauer/check_test_dirac/2018_May_30_1452_skatelescope.eu_EL7/DIRAC/TransformationSystem/ConfigTemplate.cfg | Loading configuration template /home/hep/dbauer/check_test_dirac/2018_May_30_1452_skatelescope.eu_EL7/DIRAC/TransformationSystem/ConfigTemplate.cfg | ||
Adding to CS agent Transformation/TransformationCleaningAgent | Adding to CS agent Transformation/TransformationCleaningAgent | ||
agent Transformation_TransformationCleaningAgent is installed, runit status: Run | agent Transformation_TransformationCleaningAgent is installed, runit status: Run | ||
| + | |||
[dirac00.grid.hep.ph.ic.ac.uk]> install agent Transformation InputDataAgent | [dirac00.grid.hep.ph.ic.ac.uk]> install agent Transformation InputDataAgent | ||
Loading configuration template /home/hep/dbauer/check_test_dirac/2018_May_30_1452_skatelescope.eu_EL7/DIRAC/TransformationSystem/ConfigTemplate.cfg | Loading configuration template /home/hep/dbauer/check_test_dirac/2018_May_30_1452_skatelescope.eu_EL7/DIRAC/TransformationSystem/ConfigTemplate.cfg | ||
| Line 51: | Line 57: | ||
agent Transformation_InputDataAgent is installed, runit status: Run | agent Transformation_InputDataAgent is installed, runit status: Run | ||
</pre> | </pre> | ||
| + | |||
| + | In the config Systems/Transformation/Devel/Agents: All newly installed agents have default values, stick with them for now, apart from the | ||
| + | RequestTaskAgent which acquires all the config options of the tutorial server: | ||
| + | <pre> | ||
| + | SubmitTasks = yes | ||
| + | CheckReserved = yes | ||
| + | MonitorTasks = yes | ||
| + | MonitorFiles = yes | ||
| + | TaskUpdateStatus = Submitted, Received, Waiting, Running, Matched, Completed, Failed | ||
| + | TransType = Removal, Replication | ||
| + | </pre> | ||
| + | |||
| + | In Transformation/Devel/Databases/TransformationDB change host to 127.0.0.1 (on dirac00). | ||
| + | |||
| + | |||
| + | shifterProxy = ProductionManager | ||
| + | |||
| + | |||
| + | Metadata: The fields to be used by the transformation have to be indexed in the FileCatalogDB. | ||
| + | From a DIRAC UI as an admin do e.g. <br> | ||
| + | <pre> | ||
| + | FC:/> meta index -f owner string | ||
| + | Added metadata field owner of type string | ||
| + | </pre> | ||
| + | (use 'help' to get list of all options for 'meta') | ||
Latest revision as of 13:51, 20 June 2018
Documentation
Install Log
Instance Name is Devel
Setup Name is GridPP
dirac-admin-sysadmin-cli --host dirac00.grid.hep.ph.ic.ac.uk Pinging dirac00.grid.hep.ph.ic.ac.uk... [dirac00.grid.hep.ph.ic.ac.uk]> add instance Transformation Devel Adding Transformation system as Devel self.instance for GridPP self.setup to dirac.cfg and CS Transformation system instance Devel added successfully [dirac00.grid.hep.ph.ic.ac.uk]> install db TransformationDB Adding to CS Transformation/TransformationDB Database TransformationDB from DIRAC/TransformationSystem installed successfully
It turns out the TransformationManagerHandler is only a TransformationManager.
[dirac00.grid.hep.ph.ic.ac.uk]> install service Transformation TransformationManager Loading configuration template /home/hep/dbauer/check_test_dirac/2018_May_30_1452_skatelescope.eu_EL7/DIRAC/TransformationSystem/ConfigTemplate.cfg Adding to CS service Transformation/TransformationManager service Transformation_TransformationManager is installed, runit status: Run
Now go and try to install all the agents:
[dirac00.grid.hep.ph.ic.ac.uk]> install agent Transformation TransformationAgent Loading configuration template /home/hep/dbauer/check_test_dirac/2018_May_30_1452_skatelescope.eu_EL7/DIRAC/TransformationSystem/ConfigTemplate.cfg Adding to CS agent Transformation/TransformationAgent agent Transformation_TransformationAgent is installed, runit status: Run [dirac00.grid.hep.ph.ic.ac.uk]> install agent Transformation WorkflowTaskAgent Loading configuration template /home/hep/dbauer/check_test_dirac/2018_May_30_1452_skatelescope.eu_EL7/DIRAC/TransformationSystem/ConfigTemplate.cfg Adding to CS agent Transformation/WorkflowTaskAgent agent Transformation_WorkflowTaskAgent is installed, runit status: Run [dirac00.grid.hep.ph.ic.ac.uk]> install agent Transformation RequestTaskAgent Loading configuration template /home/hep/dbauer/check_test_dirac/2018_May_30_1452_skatelescope.eu_EL7/DIRAC/TransformationSystem/ConfigTemplate.cfg Adding to CS agent Transformation/RequestTaskAgent agent Transformation_RequestTaskAgent is installed, runit status: Run [dirac00.grid.hep.ph.ic.ac.uk]> install agent Transformation MCExtensionAgent Loading configuration template /home/hep/dbauer/check_test_dirac/2018_May_30_1452_skatelescope.eu_EL7/DIRAC/TransformationSystem/ConfigTemplate.cfg Adding to CS agent Transformation/MCExtensionAgent agent Transformation_MCExtensionAgent is installed, runit status: Run [dirac00.grid.hep.ph.ic.ac.uk]> install agent Transformation TransformationCleaningAgent Loading configuration template /home/hep/dbauer/check_test_dirac/2018_May_30_1452_skatelescope.eu_EL7/DIRAC/TransformationSystem/ConfigTemplate.cfg Adding to CS agent Transformation/TransformationCleaningAgent agent Transformation_TransformationCleaningAgent is installed, runit status: Run [dirac00.grid.hep.ph.ic.ac.uk]> install agent Transformation InputDataAgent Loading configuration template /home/hep/dbauer/check_test_dirac/2018_May_30_1452_skatelescope.eu_EL7/DIRAC/TransformationSystem/ConfigTemplate.cfg Adding to CS agent Transformation/InputDataAgent agent Transformation_InputDataAgent is installed, runit status: Run
In the config Systems/Transformation/Devel/Agents: All newly installed agents have default values, stick with them for now, apart from the RequestTaskAgent which acquires all the config options of the tutorial server:
SubmitTasks = yes CheckReserved = yes MonitorTasks = yes MonitorFiles = yes TaskUpdateStatus = Submitted, Received, Waiting, Running, Matched, Completed, Failed TransType = Removal, Replication
In Transformation/Devel/Databases/TransformationDB change host to 127.0.0.1 (on dirac00).
shifterProxy = ProductionManager
Metadata: The fields to be used by the transformation have to be indexed in the FileCatalogDB.
From a DIRAC UI as an admin do e.g.
FC:/> meta index -f owner string Added metadata field owner of type string
(use 'help' to get list of all options for 'meta')
