Difference between revisions of "HTCondor Jobs In Containers"
From GridPP Wiki
(Created page with "This page explains two methods of running LHC jobs in SL6 containers using HTCondor with SL7 worker nodes. == Docker == == Singularity ==") |
(→Docker) |
||
| Line 2: | Line 2: | ||
== Docker == | == Docker == | ||
| + | The SL7 or Centos7 worker nodes should have CVMFS, HTCondor and Docker engine installed. Usually autofs is used with CVMFS but this will not work with Docker - you will likely get errors of the form: | ||
| + | ls: cannot open directory /cvmfs/cms.cern.ch: Too many levels of symbolic links | ||
| + | Each CVMFS repository must be mounted manually, e.g. | ||
| + | mount -t cvmfs grid.cern.ch /cvmfs/grid.cern.ch | ||
| + | mount -t cvmfs grid.cern.ch /cvmfs/cms.cern.ch | ||
| + | Add the condor user to the Docker group: | ||
| + | usermod -G docker condor | ||
| + | Some additional HTCondor configuration required in order to automatically bind mount CVMFS and /etc/grid-security into all Docker containers run by HTCondor: | ||
| + | DOCKER_MOUNT_VOLUMES=CVMFS, GRID_SECURITY, PASSWD, GROUP | ||
| + | DOCKER_VOLUMES=CVMFS, GRID_SECURITY | ||
| + | DOCKER_VOLUME_DIR_CVMFS=/cvmfs:/cvmfs:ro | ||
| + | DOCKER_VOLUME_DIR_GRID_SECURITY=/etc/grid-security:/etc/grid-security:ro | ||
| + | DOCKER_VOLUME_DIR_PASSWD=/etc/passwd:/etc/passwd:ro | ||
| + | DOCKER_VOLUME_DIR_GROUP=/etc/group:/etc/group:ro | ||
| + | Here we also bind mount /etc/passwd and /etc/group into the containers so that pool accounts are available. The pool accounts must be configured on the host. | ||
== Singularity == | == Singularity == | ||
Revision as of 21:34, 16 January 2017
This page explains two methods of running LHC jobs in SL6 containers using HTCondor with SL7 worker nodes.
Docker
The SL7 or Centos7 worker nodes should have CVMFS, HTCondor and Docker engine installed. Usually autofs is used with CVMFS but this will not work with Docker - you will likely get errors of the form:
ls: cannot open directory /cvmfs/cms.cern.ch: Too many levels of symbolic links
Each CVMFS repository must be mounted manually, e.g.
mount -t cvmfs grid.cern.ch /cvmfs/grid.cern.ch mount -t cvmfs grid.cern.ch /cvmfs/cms.cern.ch
Add the condor user to the Docker group:
usermod -G docker condor
Some additional HTCondor configuration required in order to automatically bind mount CVMFS and /etc/grid-security into all Docker containers run by HTCondor:
DOCKER_MOUNT_VOLUMES=CVMFS, GRID_SECURITY, PASSWD, GROUP DOCKER_VOLUMES=CVMFS, GRID_SECURITY DOCKER_VOLUME_DIR_CVMFS=/cvmfs:/cvmfs:ro DOCKER_VOLUME_DIR_GRID_SECURITY=/etc/grid-security:/etc/grid-security:ro DOCKER_VOLUME_DIR_PASSWD=/etc/passwd:/etc/passwd:ro DOCKER_VOLUME_DIR_GROUP=/etc/group:/etc/group:ro
Here we also bind mount /etc/passwd and /etc/group into the containers so that pool accounts are available. The pool accounts must be configured on the host.
