Difference between revisions of "XRootD"

From GridPP Wiki
Jump to: navigation, search
(Instruction to setup a disk server for xrootd)
Line 28: Line 28:
 
** cp /etc/grid-security/hostcert.pem /etc/grid-security/xrd/xrdcert.pem
 
** cp /etc/grid-security/hostcert.pem /etc/grid-security/xrd/xrdcert.pem
 
** chown -R xrootd:xrootd /etc/grid-security/xrd
 
** chown -R xrootd:xrootd /etc/grid-security/xrd
* create a proper home directory for user xrootd
+
* create a proper home directory for user xrootd: /home/xrootd
 
** change ownership of it to xrootd:xrootd
 
** change ownership of it to xrootd:xrootd
 
** enable it as xrootd's homedirectory in /etc/passwd
 
** enable it as xrootd's homedirectory in /etc/passwd
Line 37: Line 37:
 
** use /etc/shadow to disable logins for xrootd (preferred and probably default)
 
** use /etc/shadow to disable logins for xrootd (preferred and probably default)
 
* create cronjob for root to execute edg-mkgridmap
 
* create cronjob for root to execute edg-mkgridmap
**  10 */6 * * *  /usr/sbin/edg-mkgridmap --conf=~xrootd/config/edg-mkgridmap.conf --output=/etc/grid-security/grid-mapfile --safe
+
**  10 */6 * * *  /usr/sbin/edg-mkgridmap --conf=/home/xrootd/config/edg-mkgridmap.conf --output=/etc/grid-security/grid-mapfile --safe
 
* create as user xrootd needed directories
 
* create as user xrootd needed directories
 
** $HOME/config
 
** $HOME/config
Line 53: Line 53:
 
** xrdcp -f xroot://gridpp09.ecdf.ed.ac.uk//etc/check-running.sh $HOME/bin/check-running.sh
 
** xrdcp -f xroot://gridpp09.ecdf.ed.ac.uk//etc/check-running.sh $HOME/bin/check-running.sh
 
* make all the downloaded scripts executable
 
* make all the downloaded scripts executable
** chmod +x ~xrootd/bin/*
+
** chmod +x /home/xrootd/bin/*
 
* as user xrootd create a $HOME/local.cfg with the content: export localstorage=/PATH/To/Your/Grid/Storage/Space
 
* as user xrootd create a $HOME/local.cfg with the content: export localstorage=/PATH/To/Your/Grid/Storage/Space
 
** it should point to the directory where you have mounted the space that xrootd can use
 
** it should point to the directory where you have mounted the space that xrootd can use
 
** e.g.: export localstorage=/gridstorage  
 
** e.g.: export localstorage=/gridstorage  
 
* make sure your grid storage space is owned and writeable by user&group xrootd
 
* make sure your grid storage space is owned and writeable by user&group xrootd
** as root: source ~xrootd/local.cfg && chown -R xrootd:xrootd $localstorage
+
** as root: source /home/xrootd/local.cfg && chown -R xrootd:xrootd $localstorage
 
* as root execute edg-mkgridmap once
 
* as root execute edg-mkgridmap once
**  edg-mkgridmap --conf=~xrootd/config/edg-mkgridmap.conf --output=/etc/grid-security/grid-mapfile --safe
+
**  edg-mkgridmap --conf=/home/xrootd/config/edg-mkgridmap.conf --output=/etc/grid-security/grid-mapfile --safe
 
* as root disable xrootd as a system service
 
* as root disable xrootd as a system service
 
**  chkconfig xrootd off
 
**  chkconfig xrootd off

Revision as of 16:13, 29 March 2017

This instructions are an example of how to setup a multi-VO xrootd server that participates in a GridPP-wide storage network.
Right now (03/2017) it is only between servers withing ScotGrid and in testing mode.
A presentation about different setup possibilities can be found ---> here

to get the most recent install instructions while in development: xrdcp xroot://gridpp09.ecdf.ed.ac.uk//etc/xrootd-install.txt  ./

general information about xrootd/cmsd are available ---> here

Instruction to setup a disk server for xrootd

  • do a basic/minimal OS install (SL6 tested, CentOS7 used in Glasgow too)
  • open ports 1094 and 3121
  • install general GridPP infrastructure
    • hostkey/cert in /etc/grid-security with correct permissions
    • install edg-mkgridmap
    • install certificates: ca_*
    • install the gridpp-voms package (all)
  • create unix groups for all VOs
    • list of all VOs is given by: ls -1 /etc/grid-security/vomsdir
    • One of the main principles of this storage is that all sites support all VOs for storage, there is no more coupling between what is supported for computing on a site and the local storage. However, quotas can be used at the file system level (e.g. through ZFS) to restrict usage for specific VOs.
  • install packages needed for xrootd to work
    • xrootd
    • vomsxrd
    • xrootd-client
    • this list will be updated once http and gridftp transfers through xrootd are supported (and needed)
  • copy hostkey/cert where xrootd expect it
    • mkdir /etc/grid-security/xrd
    • cp /etc/grid-security/hostkey.pem /etc/grid-security/xrd/xrdkey.pem
    • cp /etc/grid-security/hostcert.pem /etc/grid-security/xrd/xrdcert.pem
    • chown -R xrootd:xrootd /etc/grid-security/xrd
  • create a proper home directory for user xrootd: /home/xrootd
    • change ownership of it to xrootd:xrootd
    • enable it as xrootd's homedirectory in /etc/passwd
  • change shell for xrootd in /etc/passwd to /bin/bash
  • make sure user xrootd can't login in remotely through ssh
    • create a strong password for user xrootd, OR
    • disable password based ssh logins, OR
    • use /etc/shadow to disable logins for xrootd (preferred and probably default)
  • create cronjob for root to execute edg-mkgridmap
    • 10 */6 * * * /usr/sbin/edg-mkgridmap --conf=/home/xrootd/config/edg-mkgridmap.conf --output=/etc/grid-security/grid-mapfile --safe
  • create as user xrootd needed directories
    • $HOME/config
    • $HOME/bin
    • $HOME/log
    • $HOME/spool
    • $HOME/run
  • as user xrootd copy needed config and administrating files to your server
    • xrdcp -f xroot://gridpp09.ecdf.ed.ac.uk//etc/xrootd-gridpp.cfg $HOME/config/xrootd-gridpp.cfg
    • xrdcp -f xroot://gridpp09.ecdf.ed.ac.uk//etc/auth_file $HOME/config/auth_file
    • xrdcp -f xroot://gridpp09.ecdf.ed.ac.uk//etc/edg-mkgridmap.conf $HOME/config/edg-mkgridmap.conf
    • xrdcp -f xroot://gridpp09.ecdf.ed.ac.uk//etc/check-config-gridpp.sh $HOME/bin/check-config-gridpp.sh
    • xrdcp -f xroot://gridpp09.ecdf.ed.ac.uk//etc/start-xrootd-server.sh $HOME/bin/start-xrootd-server.sh
    • xrdcp -f xroot://gridpp09.ecdf.ed.ac.uk//etc/end-xrootd-server.sh $HOME/bin/end-xrootd-server.sh
    • xrdcp -f xroot://gridpp09.ecdf.ed.ac.uk//etc/check-running.sh $HOME/bin/check-running.sh
  • make all the downloaded scripts executable
    • chmod +x /home/xrootd/bin/*
  • as user xrootd create a $HOME/local.cfg with the content: export localstorage=/PATH/To/Your/Grid/Storage/Space
    • it should point to the directory where you have mounted the space that xrootd can use
    • e.g.: export localstorage=/gridstorage
  • make sure your grid storage space is owned and writeable by user&group xrootd
    • as root: source /home/xrootd/local.cfg && chown -R xrootd:xrootd $localstorage
  • as root execute edg-mkgridmap once
    • edg-mkgridmap --conf=/home/xrootd/config/edg-mkgridmap.conf --output=/etc/grid-security/grid-mapfile --safe
  • as root disable xrootd as a system service
    • chkconfig xrootd off
    • different for CentOS7
  • as user xrootd create a cronjob
    • */30 * * * * $HOME/bin/check-running.sh
    • it checks (every 30min here) if all services are running and restarts everything if needed
    • also makes sure after a power outage or reboot the service comes up by its own after a while
  • as user xrootd start the service
    • su - xrootd
    • $HOME/bin/start-xrootd-server.sh
    • exit

To check if your server is registered with the redirector: xrdfs xroot://dev2.gridpp.ecdf.ed.ac.uk locate -m \*
To (re)start the service, do as user xrootd: $HOME/bin/start-xrootd-server
To end the service, do as user xrootd: $HOME/bin/end-xrootd-server