GridPP and the CERN VM service

From GridPP Wiki
Jump to: navigation, search

Introduction

This is a (work-in-progress) guide to setting up a CERN VM (Virtual Machine) using the CERN VM service and the GridPP CERN VM context. Following this guide, you should be able to quickly(!) setup a User Interface (UI) for running grid jobs via gLite or DIRAC.

Requirements

Overview of the process

  • Download the CERN VM image from the CernVM downloads page;
  • Create a new guest VM with this image;
  • Enable shared folders and add a shared folder;
  • Start the VM;
  • Pair the VM with the GridPP context;
  • Log in to the VM (username: gridpp password: gridpp);
  • Add the gridpp user to the vboxsf security group (for Shared Folder access);
  • Log out and log in again;
  • Put your grid certificate files in ~/.globus;
  • Get the GridPP setup files from GitHub;
  • Run the grid UI initialisation script;
  • Generate a proxy to test that it's all working!

Your Grid UI VM

Rather than try to configure your own system - i.e. your laptop, university cluster, etc. - to use the grid, people have gradually converged on the rather neat idea of creating a guest Virtual Machine (VM) that's pretty much ready to go once booted up and configured. The CERN VM team have done something even smarter - rather than having to install an experiment's software onto a bunch of guest VMs, a VM can be linked to the CERN VM File System (CVMFS) which has everything the experiment needs to run. These next steps will show you how to create your own CERN VM on your own system ready for use with GridPP resources.

  • Download the CERN VM SL6 image: visit this page and select your image. We will use VirtualBox (which you will need to install) iso from here.
  • Create a guest VM with this image: choose a suitable name, amount of system memory, virtual hard disk size (30GB?). Ideally choose '64 bit Red Hat Enterprise Linux' as the OS type; if you only have options to create 32 bit VMs, you may need to enable hardware virtualization support in your system's BIOS.
  • Enable bidirectional clipboard sharing and Shared Folders: you can do this in "Settings" -> "Advanced". Choose a folder on your host system (e.g. /Desktop/mycernvm/" to share with the Virtual Machine. This will be useful for transferring files between your host and your guest (CERN) VM.
  • Start the VM: with VirtualBox, this is done by pressing the large "Start" button.

You will be presented with the CERN VM login screen for the guest (CERN) VM.

Pairing with the GridPP VM context

The CERN VM service offers the ability to contextualise a VM with pre-defined settings, environment variables, CVMFS repositories, etc. that are put in place when the VM is first booted. This is known as pairing the VM. You can create your own contextuallisations, but it is also possible for individuals to create public contexts (e.g. for experiments, open data initiatives, etc.) that anyone can use. We have created such a context for GridPP - it's a work in progress, but you can use it to get going with the grid. Importantly, you do not need a CERN account to do this, so it is possible for anyone with a grid certificate to use it!

  • Visit the CERN VM context Market Place: this can be found here.
  • Pair with the GridPP context: select "Experimental" from the panel on the right, and then choose the gridpp_cernvm context. Then click on the "Pair" button from the panel on the right. This will generate a six-figure pin number.
  • Apply the context to your VM: enter the pin number (preceeded by a hash, as instructed in the login). The CERN VM webpage will now update indicating the VM has been successfully paired. Your VM will then be contextualised.

Preparing your VM for grid usage

NOTE: WORK IN PROGRESS! Ideally this would all be configured by the CERN VM context or provided via a custom CVMFS repository. For now, we'll use the following steps to get a grid UI up and running.

  • Log in the to the VM: after re-booting, you should be presented with the Desktop log-in screen. The default user account is gridpp with password gridpp. Log on using these details.
  • Start a terminal session: by pressing the terminal button in the panel at the bottom of the guest CERN VM window. (Note how there's also ROOT there! They couldn't resist!)
  • Get access to the Shared Folders: by default you don't have permission to see these. Enter the following command and then log out and log in again to apply the changes:
$ sudo usermod -a -G vboxsf gridpp
  • Put your grid certificate files in ~/.globus: either via the Shared Folder or scp, or whatever works for you. You'll need usercert.pem and userkey.pem files (as usual).
  • Retrieve the GridPP configuration files from GitHub: for now, we'll use the script and configuration files in the GridPP GitHub repository:
$ git clone https://github.com/gridpp/gridpp-setup.git
  • Run the setup script provided therein: there is a setup.sh script that will source the CVMFS grid UI setup script and set a few custom environment variables too:
$ cd gridpp-setup
$ source setup.sh
$ cd ../
  • Generate a proxy: you should now be able to generate a grid proxy with your certificate, e.g. if you are a member of the cernastchool.org:
$ voms-proxy-init --voms cernatschool.org -dont-verify-ac

NOTE: if you are a CMS, ATLAS, etc. user you may need to remove the last few lines of setup.sh to keep the default VOMS configuration - these will work for you!

And that's it! Your CERN VM is now grid-ready. Note that the DIRAC installation described here will work out of the box without the VOMS configuration/GitHub stuff.

Gotchas

  • Uploading to CVMFS as described here will also work from this VM, as the gsi* tools are provided via the grid CVMFS repository. However, gsiscp doesn't work straight away as it looks for gsissh in /usr/bin/. You can fix (hack) this by creating a symlink as follows:
$ sudo ln -s /cvmfs/grid.cern.ch/emi-ui-2.10.4-1_sl6v1/usr/bin/gsissh /usr/bin/gsissh

An overview

A (draft) schematic of how the CERN VM fits into the DIRAC and CVMFS framework.

Twhyntie DRN000024-v1-0 DIRAC-CVMFS-CERNVM mk01.jpg

Useful links