Revision as of 14:59, 2 November 2015

Note: The title is misleading - due to security concerns glexec can't be truely relocatable, but it can be built to use a different binary and config path to the defaults, allowing the exporting and use of glexec in a tarball environment.

Building GLEXEC to suit your site's tarball needs

(with reference to EMITarball)

Work in Progress

Please note that we are unable to support glexec directly within the tarball, for many reasons. Listed below is a possible method (still being tested) for a site to build their own relocatable glexec. A group of sites using the same convention for tarball mount points could share the same glexec build to lower the total workload.

We welcome all feedback on the tickets listed below, or to the tarball support e-mail ( tarball-grid-support atSPAMNOT cern.ch ).

Acknowledgements and Further Reading

Please refer to the glexec web pages for more information:
https://wiki.nikhef.nl/grid/GLExec

with particular thanks to the writers of:
https://wiki.nikhef.nl/grid/Building_gLExec_from_src_rpm

(the script I use is an updated version of the example given).

and: https://wiki.nikhef.nl/grid/GLExec_Argus_Quick_Installation_Guide

Requirements

• A clean SL6 system, similar to the nodes that you will run on. It will need network connectivity.
• gcc and rpm-build packages installed, as well as the glexec user that you will use on your cluster.
• The script below, or one like it:

#!/bin/sh

# SET CUSTOM BUILD ARGUMENTS HERE


# EMI and EPEL directories
glexec_pfx=/opt/gridapps/glexec
glexec_etc=/opt/gridapps/glexec/etc
glexec_doc=/opt/gridapps/glexec/doc

# END OF BUILD ARGUMENTS

# Setup build infrastructure
export TOPDIR=`pwd`
mkdir -p $TOPDIR/{SRPMS,SOURCES,SPECS,BUILD,RPMS/x86_64,RPMS/i386}

# Download and install lcmaps-interface and glexec src
rpm2cpio http://software.nikhef.nl/dist/mwsec/rpm/epel6/x86_64/lcmaps-basic-interface-1.6.1-1.el6.noarch.rpm | cpio -id
rpm --define "_topdir $TOPDIR" -i http://software.nikhef.nl/dist/mwsec/rpm/epel6/SRPMS/glexec-0.9.11-1.el6.src.rpm

# Patch spec file to match module directories for LCAS and LCMAPS
sed -i "s+^\(%configure\).*+\1 --with-lcmaps-moduledir-sfx=$lcmaps_moddir_sfx --with-lcas-moduledir-sfx=$lcas_moddir_sfx+" $TOPDIR/SPECS/glexec.spec

# Build the RPM
CFLAGS=-I$TOPDIR/usr/include rpmbuild \
    --nodeps \
    -ba --define "_topdir $TOPDIR" \
    --define "_prefix $glexec_pfx" \
    --define "_sysconfdir $glexec_etc" \
    --define "_defaultdocdir $glexec_doc" \
    $TOPDIR/SPECS/glexec.sp

The important site variables are glexec prefix, which should be your tarball mount point (the glexec binary will be in $prefix/sbin). The glexec_etc variable should point to where the glexec.conf file will be kept. The two rpm urls should be checked before building to make sure they are current and point to the latest and greatest release.

Once run this will give you an rpm to unpack in RPMS. You can do this with an:

rpm2cpio RPMS/x86_64/$glexec_rpm | cpio -dim

You will then probably need to do some directory pruning before you have something you can load into your shared area. The glexec.conf file will need to have its ownership and permissions changed, probably to glexec.glexec, 0400. The glexec/sbin directory will likely need to be put into your $PATH environment variable.

If planning on using the (recommended) setuid mode you will need to export and mount your tarballs so that glexec's suid properties aren't squashed. To this end it is recommended that you consider exporting glexec in parallel to instead of on the same mount as the "regular" tarball.

Other Dependencies

Currently this isn't sufficient to get glexec working - one needs to have additional lcas dependencies "installed" for glexec to work. The list is currently thought to be:

lcmaps
lcmaps-plugins-basic
lcmaps-plugins-c-pep
lcmaps-plugins-tracking-groupid
lcmaps-plugins-verify-proxy
lcmaps-plugins-voms

Our suggested place to install these is within the glexec path, and point glexec at them by editing the "lcas_libdir" and "lcmaps_libdir" variables, as well as possibly the "lcas_moduledir_sfx" and "lcas_moduledir_sfx" settings in the glexec.conf.

Update 15 October 2015:

The list of needed libraries is expanding, requiring globus gsi libraries on top of lcas/lcmaps. We are attempting to keep the number of packages needed down.

Library Path Problems

Update 2nd Nov 2015: For obvious reasons glexec does not respect the LD_LIBRARY_PATH environment variable. This leads to errors in execution when using glexec outside of the normal paths (as libraries fail to dynamically link).

An easy fix to this is add into /etc/ld.so.conf.d/ a file called glexec.conf that contains the full path to the usr/lib64 directory. This is however not a very "tarball-y" or relocatable solution. With a lot of help from the glexec developers (who we are indebted to) we are working on a method to modify the necessary library's RPATHs.

Notes on glexec.conf settings

The lcas and lcmap _lidir variables are very particular, needing to be of the form of an absolute directory, i.e.:

lcas_libdir                         = /opt/gridapps/glexec/usr/lib64
lcas_moduledir_sfx                  = /lcas/
lcmaps_libdir                       = /opt/gridapps/glexec/usr/lib64
lcmaps_moduledir_sfx                = /lcmaps/

glexec in cvmfs

With reference to the ticket 116154 we are investigating making glexec available through cvmfs - although it is early days yet, and we cannot at this juncture recommend sites mounting cvmfs with suid enabled.

ggus ticket

Please also see the original glexec tarball ticket 95832 (submitted by the tarball devs to themselves).