From GridPP Wiki
Revision as of 15:02, 26 February 2016 by Matthew Doidge 09da329419 (Talk | contribs)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Note: The title is misleading - due to security concerns glexec cannot be built to be truely relocatable, but it can be built to use a different binary and config path to the defaults, which may be useful for a site trying to support a tarball environment.

Please note that a true "tarball glexec" isn't any longer being developed by the UI/WN tarball support unit. This page simply documents are attempts at building one - as it's typed up notes it's a little rambling, my apologies.


We'll start with the conclusion - glexec is built too securely to be made to work in a traditional tarball environment. This along with other factors has made the tarball support group decide to "stop trying", any offering would be either not technically-viable or not security-sound.

Our recommendation to tarball sites who want to use glexec is to build glexec with a binary and config path suited to your site's needs, as detailed below or (better) on the Nikhef site. Then either install all the dependencies on the node from the EPEL and the normal SL repositories OR edit the to point to the libraries within your non-default locations.

Our apologies that we couldn't offer a proper solution.

Building GLEXEC to suit your site's path needs

(with reference to EMITarball)

Please note that we are unable to support glexec directly within the tarball, for many reasons. Documented below was the attempt to find a method for a site to build their own "relocatable glexec". However the task ended unsatisfactorily - a solution for a full relocatable version couldn't be found. However hopefully documenting our failure will help someone who needs some form of glexec solution for their tarball site.

A group of sites using the same convention for paths could share the same glexec build to lower the total workload.

Acknowledgements and Further Reading

Please refer to the glexec web pages for more information:

with particular thanks to the writers of:

(the script I used is an updated version of the example given).



  • A clean SL6 system, similar to the nodes that you will run on. It will need network connectivity.
  • gcc and rpm-build packages installed, as well as the glexec user that you will use on your cluster.
  • The script below, or one like it:


# EMI and EPEL directories


# Setup build infrastructure
export TOPDIR=`pwd`

# Download and install lcmaps-interface and glexec src
rpm2cpio | cpio -id
rpm --define "_topdir $TOPDIR" -i

# Patch spec file to match module directories for LCAS and LCMAPS
sed -i "s+^\(%configure\).*+\1 --with-lcmaps-moduledir-sfx=$lcmaps_moddir_sfx --with-lcas-moduledir-sfx=$lcas_moddir_sfx+" $TOPDIR/SPECS/glexec.spec

# Build the RPM
CFLAGS=-I$TOPDIR/usr/include rpmbuild \
    --nodeps \
    -ba --define "_topdir $TOPDIR" \
    --define "_prefix $glexec_pfx" \
    --define "_sysconfdir $glexec_etc" \
    --define "_defaultdocdir $glexec_doc" \

The important site variables are glexec prefix, which should be your tarball mount point (the glexec binary will be in $prefix/sbin). The glexec_etc variable should point to where the glexec.conf file will be kept. The two rpm urls should be checked before building to make sure they are current and point to the latest and greatest release.

Once run this will give you an rpm to unpack in RPMS. You can do this with an:

rpm2cpio RPMS/x86_64/$glexec_rpm | cpio -dim

You will then probably need to do some directory pruning before you have something you can load into your shared area. The glexec.conf file will need to have its ownership and permissions changed, probably to glexec.glexec, 0400. The glexec/sbin directory will likely need to be put into your $PATH environment variable.

If planning on using the (recommended) setuid mode you will need to export and mount your tarballs so that glexec's suid properties aren't squashed. To this end it is recommended that you consider exporting glexec in parallel to instead of on the same mount as the "regular" tarball.

Other Dependencies

The list of dependencies required to get glexec to work is (possibly not complete):


Originally our suggested place to install these is within the glexec path, and point glexec at them by editing the "lcas_libdir" and "lcmaps_libdir" variables, as well as possibly the "lcas_moduledir_sfx" and "lcas_moduledir_sfx" settings in the glexec.conf. However this didn't work- see below.

Library Path Problems

For obvious reasons glexec does not respect the LD_LIBRARY_PATH environment variable. This leads to errors in execution when using glexec outside of the normal paths (as libraries fail to dynamically link).

An possible fix to this is add into /etc/ a file called glexec.conf that contains the full path to the usr/lib64 directory. This is however not a very "tarball-y" or relocatable solution, and would not be of use to most tarball sites.

Second option would be to install all the OS/EPEL dependencies onto the machine in the normal place, having glexec built to work from the mounted tarball path. Again this is not a universal solution.

Notes on glexec.conf settings

The lcas and lcmap _lidir variables are very particular, needing to be of the form of an absolute directory, i.e.:

lcas_libdir                         = /opt/gridapps/glexec/usr/lib64
lcas_moduledir_sfx                  = /lcas/
lcmaps_libdir                       = /opt/gridapps/glexec/usr/lib64
lcmaps_moduledir_sfx                = /lcmaps/

glexec in cvmfs

With reference to this unsolved ticket 116154 - at this point we cannot get glexec to work within cvmfs, and have been also advised that this is not a good idea, in part as we cannot recommend sites mounting cvmfs with suid enabled.

ggus ticket

Please also see the original glexec tarball ticket 95832 (submitted by the tarball devs to themselves), set unsolved February 2016.