ClassicSE To DPM Pool Node

If your site already has a DPM install or you have a new machine on which you would like to run the core DPM services, then it is still possible to migrate the current Classic SE entries into the DPM and then convert the Classic SE into a DPM pool node. This is what has been done at Edinburgh and what follows below is a detailed description of the procedure, as outlined in the LCG twiki here.

Migration Script

Download the script from here. Unpack the source and build it using:

# make all

There is a README file.

Problem with script

The initial release of the migration script caused problems when migrating Classic SE entries to an existing DPM server that already contained entries. This manifested itself by the DPNS transfers starting, but then immediately stopping before any file entries were migrated. This problem does not appear to affect Classic SE to DPM migrations on a fresh/empty install of DPM. If you are using the initial release of the script, then the problem can be resolved by removing the options -D_SIMULATION_ in the Makefile before making the script (this option prevents all the dpns_* functions being called inside the program). After that, the operation of the script is as described in the README file and the remainder of this page.

A more robust version of the migration script is currently being worked on (Oct 2005).

Install and configure DPM software

• To convert the Classic SE into a DPM pool node, you must download and install the following rpms (make sure you get the latest versions) on the Classic SE host:

# wget http://grid-deployment.web.cern.ch/grid-deployment/RpmDir_i386-sl3/external/DPM-client-1.3.8-1sec_sl3.i386.rpm
# wget http://grid-deployment.web.cern.ch/grid-deployment/RpmDir_i386-sl3/external/DPM-gridftp-server-1.3.8-2sec_sl3.i386.rpm
# wget http://grid-deployment.web.cern.ch/grid-deployment/RpmDir_i386-sl3/external/DPM-rfio-server-1.3.8-2sec_sl3.i386.rpm
# wget http://grid-deployment.web.cern.ch/grid-deployment/RpmDir_i386-sl3/external/lcg-dm-common-1.3.8-1_sl3.i386.rpm

# rpm -Uvh lcg-dm-common-1.3.8-1_sl3.i386.rpm 
DPM-client-1.3.8-1sec_sl3.i386.rpm 
DPM-rfio-server-1.3.8-2sec_sl3.i386.rpm
DPM-gridftp-server-1.3.8-2sec_sl3.i386.rpm

lcg-dm-common is needed to satisfy dependencies. The DPM-rfio-server conflicts with CASTOR-client-1.7.1.5-1.longname (from the lcg-SE_classic metapackage) and this must be removed before the install can proceed.

• Stop the Classic SE globus-gridftp service and prevent it starting at boot time:

# /sbin/service globus-gridftp stop
# /sbin/chkconfig globus-gridftp off

You can start the relevant DPM services:

# /sbin/service rfiod restart
# /sbin/service dpm-gsiftp restart
/etc/init.d/dpm-gsiftp: line 19: /etc/sysconfig/dpm-gsiftp: No such file or directory

I was not sure what the significance of this error message is.

• Set up the DPNS server to trust the new pool node. In /etc/shift.conf add:

DPNS TRUST glenmorangie glenmorangie.epcc.ed.ac.uk
DPM TRUST glenmorangie glenmorangie.epcc.ed.ac.uk

The DPM TRUST line was not mentioned in the LCG instructions, but should exist according to the DPM admin guide. In /etc/shift.conf on the new pool node add:

RFIOD TRUST  dpm dpm.epcc.ed.ac.uk
RFIOD RTRUST  dpm dpm.epcc.ed.ac.uk 
RFIOD WTRUST  dpm dpm.epcc.ed.ac.uk
RFIOD XTRUST  dpm dpm.epcc.ed.ac.uk
RFIOD FTRUST  dpm dpm.epcc.ed.ac.uk

Note, that the LCG instructions do not mention adding the first line (RFIOD TRUST). It is also necessary to open up the RFIO port (5001) on the new pool node if not already open.

Setup DPM

Make sure that the uids/gids are the same on the Classic SE and DPNS host. Create the dpmmgr user.

# /usr/sbin/groupadd -g GID dpmmgr
# /usr/sbin/useradd -c "DPM manager" -g dpmmgr -u UID dpmmgr

# chown root:dpmmgr /etc/grid-security/gridmapdir
# mkdir /etc/grid-security/dpmmgr
# chown dpmmgr:dpmmgr /etc/grid-security/dpmmgr
# cp -p /etc/grid-security/hostcert.pem /etc/grid-security/dpmmgr/dpmcert.pem
# cp -p /etc/grid-security/hostkey.pem /etc/grid-security/dpmmgr/dpmkey.pem

I had to create the biomed VO on my DPM so that the Classic SE files can be owned by the correct users. Use dpns-mkdir to make the correct VO directories (i.e. /dpm/epcc.ed.ac.uk/home/biomed/)

I hacked together a script from the YAIM directory that took a file containing the uids/gids for the biomed VO from the Classic SE.

• Create a pool that corresponds to the Classic SE:

# dpm-addpool --poolname classicSE --def_filesize 200M

• Add in the Classic SE filesystem:

# dpm-addfs --poolname classicSE --server glenmorangie --fs /storage

# dpm-qryconf 
POOL RAID22 DEFSIZE 200.00M GC_START_THRESH 0 GC_STOP_THRESH 0 DEFPINTIME 600 PUT_RETENP 86400 FSS_POLICY maxfreespace GC_POLICY lru RS_POLICY fifo GID 0 S_TYPE -
                              CAPACITY 938.66G FREE 888.31G ( 94.6%)
 dpm /storage2/dpmmgr CAPACITY 938.66G FREE 888.31G ( 94.6%)
POOL SANSTOR4 DEFSIZE 904.00M GC_START_THRESH 0 GC_STOP_THRESH 0 DEFPINTIME 600 PUT_RETENP 86400 FSS_POLICY maxfreespace GC_POLICY lru RS_POLICY fifo GID 0 S_TYPE -
                              CAPACITY 806.72G FREE 796.02G ( 98.7%)
 dpm /sanstor/stor4/dpmmgr/ CAPACITY 806.72G FREE 796.02G ( 98.7%)
POOL dpmPart DEFSIZE 200.00M GC_START_THRESH 0 GC_STOP_THRESH 0 DEFPINTIME 600 PUT_RETENP 86400 FSS_POLICY maxfreespace GC_POLICY lru RS_POLICY fifo GID 0 S_TYPE -
                              CAPACITY 9.86G FREE 7.67G ( 77.8%)
 dpm /dpmStorage0/dpmmgr CAPACITY 9.86G FREE 7.67G ( 77.8%)
POOL classicSE DEFSIZE 200.00M GC_START_THRESH 0 GC_STOP_THRESH 0 DEFPINTIME 0 PUT_RETENP 86400 FSS_POLICY maxfreespace GC_POLICY lru RS_POLICY fifo GID 0 S_TYPE -
                              CAPACITY 36.09G FREE 20.36G ( 56.4%)
 glenmorangie /storage CAPACITY 36.09G FREE 20.36G ( 56.4%)

You can now see the extra capacity due to the new disk pool (the old Classic SE).

• To prevent any data from being written into the classicSE filessystem, set to readonly:

$ dpm-modifyfs --server glenmorangie --fs /storage --st RDONLY

Migration

Use the script to migrate the files, using the following syntax:

# ./migration classic_SE_HOST /path/to/classicSE/data/directory DPM_HOST /dpm/DOMAIN_NAME/home/data/directory poolname

As an example: re-running dteam migration.

# ./migration glenmorangie /storage/dteam dpm /dpm/epcc.ed.ac.uk/home/dteam classicSE
Classic SE->DPM migration
classicSE hostname  = glenmorangie
classicSE directory = /storage/dteam
dpm hostname        = dpm
dpm directory       = /dpm/epcc.ed.ac.uk/home/dteam
dpm poolname        = classicSE

Beginning of migration: Fri Oct 21 13:43:41 2005
/storage/dteam/generated/ ==> /dpm/epcc.ed.ac.uk/home/dteam/generated
/storage/dteam/generated/2004-09-21/ ==> /dpm/epcc.ed.ac.uk/home/dteam/generated/2004-09-21
/storage/dteam/generated/2004-09-22/ ==> /dpm/epcc.ed.ac.uk/home/dteam/generated/2004-09-22
/storage/dteam/generated/2004-09-23/ ==> /dpm/epcc.ed.ac.uk/home/dteam/generated/2004-09-23
/storage/dteam/generated/2004-09-24/ ==> /dpm/epcc.ed.ac.uk/home/dteam/generated/2004-09-24
/storage/dteam/generated/2004-09-25/ ==> /dpm/epcc.ed.ac.uk/home/dteam/generated/2004-09-25
/storage/dteam/generated/2004-09-26/ ==> /dpm/epcc.ed.ac.uk/home/dteam/generated/2004-09-26
/storage/dteam/generated/2004-09-27/ ==> /dpm/epcc.ed.ac.uk/home/dteam/generated/2004-09-27

...

output very verbose.

...

End of migration: Fri Oct 21 14:04:20 2005

Number of files migrated to dpm: 1171
Duration: 0 days 0 hours 20 minutes 2439 seconds
Number of files migrated per second: 0.95

• Once the migration has taken place, you must change ownership of the Classic SE files. This is very important as you will not be able to use globus-url-copy or srmcp the files out of the DPM without it:

 # chown -R dpmmgr:dpmmgr /storage
 # chmod -R 660 /storage
 # ls -ld /storage/dteam/
 drw-rw----  353 dpmmgr   dpmmgr       8192 Oct 16 16:45 generated

• Now migrate the other VOs.

DPNS file permissions

As yet, I haven't changed the DPNS permissions.

$ dpns-ls -l /dpm/epcc.ed.ac.uk/home/lhcb/generated
/dpm/epcc.ed.ac.uk/home/lhcb/generated: Permission denied

You will be able to view the contents of the directories as root. If necessary, you can use dpns-chmod to set the permissions in DPNS (dpns-chmod does not support recursive mode).

File transfer tests

Classic SE method

You should still be able to access the Classic SE files via the traditional commands.

$ lcg-cp --vo dteam 
sfn://glenmorangie.epcc.ed.ac.uk/storage/dteam/generated/2005-10-14/file42f3e584-42e8-43cf-9bd1-6c0eac723d06 file:/tmp/greig_test7.txt
$ cat /tmp/greig_test7.txt 
heplnx49.pp.rl.ac.uk 
10/14:12:38

Globus-url-copy

$ globus-url-copy 
gsiftp://dpm.epcc.ed.ac.uk/dpm/epcc.ed.ac.uk/home/dteam/generated/2005-10-14/file42f3e584-42e8-43cf-9bd1-6c0eac723d06 file:/tmp/greig_test6.txt
$ cat /tmp/greig_test6.txt
heplnx49.pp.rl.ac.uk
10/14:12:38

SRM copy

$ srmcp srm://dpm.epcc.ed.ac.uk:8443/dpm/epcc.ed.ac.uk/home/dteam/generated/2005-10-14/file42f3e584-42e8-43cf-9bd1-6c0eac723d06     
file:////tmp/greig_test8.txt
$ cat /tmp/greig_test8.txt
heplnx49.pp.rl.ac.uk
10/14:12:38