Difference between revisions of "RucioDocker"
From GridPP Wiki
(Created page with " = Testing RUCIO deployment using Docker = == Resources == This is a collection of resources used as reference in throwing up a RUCIO service using containers: [https://gi...") |
(No difference)
|
Revision as of 13:59, 8 June 2021
Contents
Testing RUCIO deployment using Docker
Resources
This is a collection of resources used as reference in throwing up a RUCIO service using containers:
[1] SKA RUCIO prototype project
[2] RUCIO ReadTheDocs
[3] Talk on RUCIO in containers
Running basic commands in test RUCIO
Checkout RUCIO
cd $WORKDIR git clone https://github.com/rucio/rucio.git cd rucio/etc/docker/dev
Launch containers needed for test
# Update the docker containers on your system docker-compose --file docker-compose-storage-alldb.yml pull # Launch the containers docker-compose --file docker-compose-storage-alldb.yml up -d
Run dev tests
# Launch interactive shell on RUCIO main image (Akin to bastion server) docker exec -it dev_rucio_1 /bin/bash # Setup simple configs, setup databases and add 3 RSEs which speak XRootD [root@bb88b6f88320 rucio]# tools/run_tests_docker.sh -ir # To run the full test suite if you've already initialised the DB [root@bb88b6f88320 rucio]# tools/run_tests_docker.sh -a # This will go away and run the full suite of dev tests for RUCIO and takes some time but can be informative
Remove containers after finished
# Destroy the containers and network docker-compose --file docker-compose-storage-alldb.yml down
Manually play with placing files and such in RSEs
docker exec -ti dev_rucio_1 bash # Setup simple configs, setup databases and add 3 RSEs which speak XRootD [root@bb88b6f88320 rucio]# tools/run_tests_docker.sh -ir # NB it's only really possible to run this script once depite what the docs suggest otherwise it will put at least some components into a funny state # If you want to start again it's best to remove the container instances and start again [root@bb88b6f88320 rucio]# rucio add-dataset test:mynewdataset Added test:mynewdataset [root@bb88b6f88320 rucio]# rucio attach test:mynewdataset test:file1 test:file2 test:file3 test:file4 DIDs successfully attached to test:mynewdataset # Shamelessly borrowed from https://rucio.readthedocs.io/en/latest/demo_environment.html#start-the-daemons # You now have 4 files in your dataset. E.g: [root@bb88b6f88320 rucio]# rucio list-file-replicas test:mynewdataset +---------+--------+------------+-----------+------------------------------------------------+ | SCOPE | NAME | FILESIZE | ADLER32 | RSE: REPLICA | |---------+--------+------------+-----------+------------------------------------------------| | test | file1 | 10.486 MB | d4ed8299 | XRD1: root://xrd1:1094//rucio/test/80/25/file1 | | test | file2 | 10.486 MB | 7822b5da | XRD1: root://xrd1:1094//rucio/test/f3/14/file2 | | test | file3 | 10.486 MB | 2bac1f0e | XRD2: root://xrd2:1095//rucio/test/a9/23/file3 | | test | file4 | 10.486 MB | f6af16a9 | XRD2: root://xrd2:1095//rucio/test/2b/c2/file4 | +---------+--------+------------+-----------+------------------------------------------------+ # To check this file exists, exit the current container and go into a RSE container with a file: [root@gridpp01 dev]# docker exec -ti dev_xrd1_1 bash [root@bb88b6f88320 /]# ls /rucio/test/80/25/file1 /rucio/test/80/25/file1 # NB these files will be lost when the container is destroyed by construction as of writing based on the container configs