Difference between revisions of "HTCondor Jobs In Containers"

From GridPP Wiki
Jump to: navigation, search
(Docker)
(Docker)
Line 2: Line 2:
  
 
== Docker ==
 
== Docker ==
The SL7 or Centos7 worker nodes should have CVMFS, HTCondor and Docker engine installed. Usually autofs is used with CVMFS but this will not work with Docker - you will likely get errors of the form:
+
The SL7 or Centos7 worker nodes should have:CVMFS, HTCondor, Docker engine, CA certs and fetch-crl installed. Usually autofs is used with CVMFS but this will not work with Docker - you will likely get errors of the form:
 
  ls: cannot open directory /cvmfs/cms.cern.ch: Too many levels of symbolic links
 
  ls: cannot open directory /cvmfs/cms.cern.ch: Too many levels of symbolic links
 
Each CVMFS repository must be mounted manually, e.g.
 
Each CVMFS repository must be mounted manually, e.g.

Revision as of 21:35, 16 January 2017

This page explains two methods of running LHC jobs in SL6 containers using HTCondor with SL7 worker nodes.

Docker

The SL7 or Centos7 worker nodes should have:CVMFS, HTCondor, Docker engine, CA certs and fetch-crl installed. Usually autofs is used with CVMFS but this will not work with Docker - you will likely get errors of the form:

ls: cannot open directory /cvmfs/cms.cern.ch: Too many levels of symbolic links

Each CVMFS repository must be mounted manually, e.g.

mount -t cvmfs grid.cern.ch /cvmfs/grid.cern.ch
mount -t cvmfs grid.cern.ch /cvmfs/cms.cern.ch

Add the condor user to the Docker group:

usermod -G docker condor

Some additional HTCondor configuration required in order to automatically bind mount CVMFS and /etc/grid-security into all Docker containers run by HTCondor:

DOCKER_MOUNT_VOLUMES=CVMFS, GRID_SECURITY, PASSWD, GROUP
DOCKER_VOLUMES=CVMFS, GRID_SECURITY
DOCKER_VOLUME_DIR_CVMFS=/cvmfs:/cvmfs:ro
DOCKER_VOLUME_DIR_GRID_SECURITY=/etc/grid-security:/etc/grid-security:ro
DOCKER_VOLUME_DIR_PASSWD=/etc/passwd:/etc/passwd:ro
DOCKER_VOLUME_DIR_GROUP=/etc/group:/etc/group:ro

Here we also bind mount /etc/passwd and /etc/group into the containers so that pool accounts are available. The pool accounts must be configured on the host.

Singularity