ClassicSE To DPM Pool Node
If your site already has a DPM install or you have a new machine on which you would like to run the core DPM services, then it is still possible to migrate the current Classic SE entries into the DPM and then convert the Classic SE into a DPM pool node. This is what has been done at Edinburgh and what follows below is a detailed description of the procedure, as outlined in the LCG twiki here.
Contents
Migration Script
Download the script from here. Unpack the source and build it using:
# make all
There is a README file.
Problem with script
The initial release of the migration script caused problems when migrating Classic SE entries to an existing DPM server that already contained entries. This manifested itself by the DPNS transfers starting, but then immediately stopping before any file entries were migrated. This problem does not appear to affect Classic SE to DPM migrations on a fresh/empty install of DPM. If you are using the initial release of the script, then the problem can be resolved by removing the options -D_SIMULATION_ in the Makefile before making the script (this option prevents all the dpns_* functions being called inside the program). After that, the operation of the script is as described in the README file and the remainder of this page.
A more robust version of the migration script is currently being worked on (Oct 2005).
Install and configure DPM software
- To convert the Classic SE into a DPM pool node, you must download and install the following rpms (make sure you get the latest versions) on the Classic SE host:
# wget http://grid-deployment.web.cern.ch/grid-deployment/RpmDir_i386-sl3/external/DPM-client-1.3.8-1sec_sl3.i386.rpm # wget http://grid-deployment.web.cern.ch/grid-deployment/RpmDir_i386-sl3/external/DPM-gridftp-server-1.3.8-2sec_sl3.i386.rpm # wget http://grid-deployment.web.cern.ch/grid-deployment/RpmDir_i386-sl3/external/DPM-rfio-server-1.3.8-2sec_sl3.i386.rpm # wget http://grid-deployment.web.cern.ch/grid-deployment/RpmDir_i386-sl3/external/lcg-dm-common-1.3.8-1_sl3.i386.rpm
# rpm -Uvh lcg-dm-common-1.3.8-1_sl3.i386.rpm DPM-client-1.3.8-1sec_sl3.i386.rpm DPM-rfio-server-1.3.8-2sec_sl3.i386.rpm DPM-gridftp-server-1.3.8-2sec_sl3.i386.rpm
lcg-dm-common is needed to satisfy dependencies. The DPM-rfio-server conflicts with CASTOR-client-1.7.1.5-1.longname (from the lcg-SE_classic metapackage) and this must be removed before the install can proceed.
- Stop the Classic SE globus-gridftp service and prevent it starting at boot time:
# /sbin/service globus-gridftp stop # /sbin/chkconfig globus-gridftp off
You can start the relevant DPM services:
# /sbin/service rfiod restart # /sbin/service dpm-gsiftp restart /etc/init.d/dpm-gsiftp: line 19: /etc/sysconfig/dpm-gsiftp: No such file or directory
I was not sure what the significance of this error message is.
- Set up the DPNS server to trust the new pool node. In /etc/shift.conf add:
DPNS TRUST glenmorangie glenmorangie.epcc.ed.ac.uk DPM TRUST glenmorangie glenmorangie.epcc.ed.ac.uk
The DPM TRUST line was not mentioned in the LCG instructions, but should exist according to the DPM admin guide. In /etc/shift.conf on the new pool node add:
RFIOD TRUST dpm dpm.epcc.ed.ac.uk RFIOD RTRUST dpm dpm.epcc.ed.ac.uk RFIOD WTRUST dpm dpm.epcc.ed.ac.uk RFIOD XTRUST dpm dpm.epcc.ed.ac.uk RFIOD FTRUST dpm dpm.epcc.ed.ac.uk
Note, that the LCG instructions do not mention adding the first line (RFIOD TRUST). It is also necessary to open up the RFIO port (5001) on the new pool node if not already open.
Setup DPM
Make sure that the uids/gids are the same on the Classic SE and DPNS host. Create the dpmmgr user.
# /usr/sbin/groupadd -g GID dpmmgr # /usr/sbin/useradd -c "DPM manager" -g dpmmgr -u UID dpmmgr
# chown root:dpmmgr /etc/grid-security/gridmapdir # mkdir /etc/grid-security/dpmmgr # chown dpmmgr:dpmmgr /etc/grid-security/dpmmgr # cp -p /etc/grid-security/hostcert.pem /etc/grid-security/dpmmgr/dpmcert.pem # cp -p /etc/grid-security/hostkey.pem /etc/grid-security/dpmmgr/dpmkey.pem
I had to create the biomed VO on my DPM so that the Classic SE files can be owned by the correct users. Use dpns-mkdir to make the correct VO directories (i.e. /dpm/epcc.ed.ac.uk/home/biomed/)
I hacked together a script from the YAIM directory that took a file containing the uids/gids for the biomed VO from the Classic SE.
- Create a pool that corresponds to the Classic SE:
# dpm-addpool --poolname classicSE --def_filesize 200M
- Add in the Classic SE filesystem:
# dpm-addfs --poolname classicSE --server glenmorangie --fs /storage
# dpm-qryconf POOL RAID22 DEFSIZE 200.00M GC_START_THRESH 0 GC_STOP_THRESH 0 DEFPINTIME 600 PUT_RETENP 86400 FSS_POLICY maxfreespace GC_POLICY lru RS_POLICY fifo GID 0 S_TYPE - CAPACITY 938.66G FREE 888.31G ( 94.6%) dpm /storage2/dpmmgr CAPACITY 938.66G FREE 888.31G ( 94.6%) POOL SANSTOR4 DEFSIZE 904.00M GC_START_THRESH 0 GC_STOP_THRESH 0 DEFPINTIME 600 PUT_RETENP 86400 FSS_POLICY maxfreespace GC_POLICY lru RS_POLICY fifo GID 0 S_TYPE - CAPACITY 806.72G FREE 796.02G ( 98.7%) dpm /sanstor/stor4/dpmmgr/ CAPACITY 806.72G FREE 796.02G ( 98.7%) POOL dpmPart DEFSIZE 200.00M GC_START_THRESH 0 GC_STOP_THRESH 0 DEFPINTIME 600 PUT_RETENP 86400 FSS_POLICY maxfreespace GC_POLICY lru RS_POLICY fifo GID 0 S_TYPE - CAPACITY 9.86G FREE 7.67G ( 77.8%) dpm /dpmStorage0/dpmmgr CAPACITY 9.86G FREE 7.67G ( 77.8%) POOL classicSE DEFSIZE 200.00M GC_START_THRESH 0 GC_STOP_THRESH 0 DEFPINTIME 0 PUT_RETENP 86400 FSS_POLICY maxfreespace GC_POLICY lru RS_POLICY fifo GID 0 S_TYPE - CAPACITY 36.09G FREE 20.36G ( 56.4%) glenmorangie /storage CAPACITY 36.09G FREE 20.36G ( 56.4%)
You can now see the extra capacity due to the new disk pool (the old Classic SE).
- To prevent any data from being written into the classicSE filessystem, set to readonly:
$ dpm-modifyfs --server glenmorangie --fs /storage --st RDONLY
Migration
Use the script to migrate the files, using the following syntax:
# ./migration classic_SE_HOST /path/to/classicSE/data/directory DPM_HOST /dpm/DOMAIN_NAME/home/data/directory poolname
As an example: re-running dteam migration.
# ./migration glenmorangie /storage/dteam dpm /dpm/epcc.ed.ac.uk/home/dteam classicSE Classic SE->DPM migration classicSE hostname = glenmorangie classicSE directory = /storage/dteam dpm hostname = dpm dpm directory = /dpm/epcc.ed.ac.uk/home/dteam dpm poolname = classicSE Beginning of migration: Fri Oct 21 13:43:41 2005 /storage/dteam/generated/ ==> /dpm/epcc.ed.ac.uk/home/dteam/generated /storage/dteam/generated/2004-09-21/ ==> /dpm/epcc.ed.ac.uk/home/dteam/generated/2004-09-21 /storage/dteam/generated/2004-09-22/ ==> /dpm/epcc.ed.ac.uk/home/dteam/generated/2004-09-22 /storage/dteam/generated/2004-09-23/ ==> /dpm/epcc.ed.ac.uk/home/dteam/generated/2004-09-23 /storage/dteam/generated/2004-09-24/ ==> /dpm/epcc.ed.ac.uk/home/dteam/generated/2004-09-24 /storage/dteam/generated/2004-09-25/ ==> /dpm/epcc.ed.ac.uk/home/dteam/generated/2004-09-25 /storage/dteam/generated/2004-09-26/ ==> /dpm/epcc.ed.ac.uk/home/dteam/generated/2004-09-26 /storage/dteam/generated/2004-09-27/ ==> /dpm/epcc.ed.ac.uk/home/dteam/generated/2004-09-27
...
output very verbose.
...
End of migration: Fri Oct 21 14:04:20 2005 Number of files migrated to dpm: 1171 Duration: 0 days 0 hours 20 minutes 2439 seconds Number of files migrated per second: 0.95
- Once the migration has taken place, you must change ownership of the Classic SE files. This is very important as you will not be able to use globus-url-copy or srmcp the files out of the DPM without it:
# chown -R dpmmgr:dpmmgr /storage # chmod -R 660 /storage # ls -ld /storage/dteam/ drw-rw---- 353 dpmmgr dpmmgr 8192 Oct 16 16:45 generated
- Now migrate the other VOs.
DPNS file permissions
As yet, I haven't changed the DPNS permissions.
$ dpns-ls -l /dpm/epcc.ed.ac.uk/home/lhcb/generated /dpm/epcc.ed.ac.uk/home/lhcb/generated: Permission denied
You will be able to view the contents of the directories as root. If necessary, you can use dpns-chmod to set the permissions in DPNS (dpns-chmod does not support recursive mode).
File transfer tests
Classic SE method
You should still be able to access the Classic SE files via the traditional commands.
$ lcg-cp --vo dteam sfn://glenmorangie.epcc.ed.ac.uk/storage/dteam/generated/2005-10-14/file42f3e584-42e8-43cf-9bd1-6c0eac723d06 file:/tmp/greig_test7.txt $ cat /tmp/greig_test7.txt heplnx49.pp.rl.ac.uk 10/14:12:38
Globus-url-copy
$ globus-url-copy gsiftp://dpm.epcc.ed.ac.uk/dpm/epcc.ed.ac.uk/home/dteam/generated/2005-10-14/file42f3e584-42e8-43cf-9bd1-6c0eac723d06 file:/tmp/greig_test6.txt $ cat /tmp/greig_test6.txt heplnx49.pp.rl.ac.uk 10/14:12:38
SRM copy
$ srmcp srm://dpm.epcc.ed.ac.uk:8443/dpm/epcc.ed.ac.uk/home/dteam/generated/2005-10-14/file42f3e584-42e8-43cf-9bd1-6c0eac723d06 file:////tmp/greig_test8.txt $ cat /tmp/greig_test8.txt heplnx49.pp.rl.ac.uk 10/14:12:38