BaBar Skimming Creating a New Tarball

This ios an outline of the procedure used to create a new tarball for installation on the grid from a basic Release + Tags.

Set up your Base Release

Set up the new release and tags as normal

newrel -t 24.2.1d Test-24.2.1d
cd Test-24.2.1d
srtpath
gmake installdirs
addpkg BbkJobWrappers cajb20080225a
addpkg BbkTaskManager tja080225a
addpkg BbkTMGridTools cajb20080225a
addpkg BbkSqlAbstractor tja071123a
addpkg BbkTools tja071123a
addpkg workdir
workdir V00-04-21
gmake workdir.setup
gmake binscripts

Next create a test task in the normal way and make sure you can run skims locally.

Create and run an strace Task

In BbkTaskManager/site/RAL there is a task template file RAL-strace-Task-Template. You need to create a task based on that template + whatever is needed to get your release to run over it's validation dataset. You need to keep

SkimShellTemplateFile = runTemplate-strace

and make sure that the SkimMaxEvents is set high enough that only one skim event per input collection is created, runTemplate-strace will mangle the tcl file to only run on the first ten events of the input file.

Now create and submit the skim jobs for the strace task, the should be submitted to the express queue and run pretty quickly creating a .strace file in each skim run directory.

Create the file list and the Grid Release

Create a file with the list of .strace files, something like:

find /stage/xrootd-data6/bbrprod/skimming/tasks/RAL-strace-24.2.1d-1.2/28/ -name '*.strace' > trace-files.lis

Then use the mk-lists.sh script to process the strace output and produce a list of files:

./mk-lists.sh trace-files.lis

Next edit the mk-tarball.sh script to change the output directory and the versions of BbkTaskmanager, BbkJobWrappers, BbkTools and BbkSqlAbstractor it will add to the grid release, and run the script on the output of the mk-lists.sh script:

./mk-tarball.pl files.lis

Testing the Grid Release

Create a new Test Task that would run on the grid at the RAL using the RAL xrootd for data input and output.

The create a test run with:

BbkSubmitSkims --test -n 1 <TaskName>

Create a work and SW_DIR directory under /pool/bbrprod

in SW_DIR run:
<pre>
ln -s /stage/sl3-lcg-exp/babarsgm/bin bin
ln -s /stage/sl3-lcg-exp/babarsgm/kanga kanga

Copy the grid release directory to /pool/bbrprod/SW_DIR

In the grid release directory run

ln -s ../kanga kanga

Create a run directory under work and copy all the files from the run directory of the test run into it.

Edit run_<run_id>.sh to remove everything after the line:

echo copy the skimming output to the SE

and edit the tcl file to make it only run on the first 10 events.

Run the test job by running from a new login shell (i.e. one where you haven't run srtpath):

./run_<run_id>.sh

Fix and problems (preferably by adding whatever is needed into either the mk-lists.sh or mk-tarball.pl).

And rerun the job after cleaning up with:

rm -rf <run_id>/ skim<run_id>-01.tar

Once the job runs cleanly and saves it's output to xrootd you're ready to tar up the grid release directory, save it to the grid then install it at remote sites.

Installing the Tarball on a Grid Site

First tar up the release directory: </pre> tar -czf skim-tarball-24.2.1d.tar.gz skim-tarball-24.2.1d </pre>

Then copy that file to a grid storage element:

lcg-cr -v --vo babar -d heplnx204.pp.rl.ac.uk \
   -P SPGrid/bbrgrid-soft/babar-skim-tarball-24.2.1d.tar.gz \
   -l lfn:/grid/babar/SPGrid/bbrgrid-soft/babar-skim-tarball-24.2.1d.tar.gz \
   file:/pool/bbrprod/skim-tarball-24.2.1d.tar.gz

Then in /stage/xrootd-data6/work/babarmc/babarsgm/SkimMiniApp/<site> where <site> is the grid site you want to install on edit the JDL file for the compute element you wish to use to change the version and lfn of the tarball:

cat skim-tarball-install-man-ce02.jdl
Executable = "skim-tarball-install.sh";
VirtualOrganisation = "babar";
StdOutput = "install.out";
StdError = "install.out";
Environment = {
                "VER=skim-tarball-24.2.1f",
                "LFN=lfn:/grid/babar/SPGrid/bbrgrid-soft/babar-skim-tarball-24.2.1f.tar.gz",
                "REINSTALL=YES",
                "MKKANACCESS=YES"
              };
InputSandbox = {"../skim-tarball-install.sh","./KanAccess.cfg","./local-grid-setup","./local-grid-wrapup"};
OutputSandbox = {"install.out"};
Requirements = ( other.GlueCEUniqueID == "ce02.tier2.hep.manchester.ac.uk:2119/jobmanager-lcgpbs-babar" )

and submit the JDL file with a Software Group Manager proxy.

---

Chris brew 10:10, 14 Mar 2008 (GMT)