BaBar Skimming Commands and Notes
Contents
The Skimming Cycle
One Skim Super job corresponds to one input dataset and produces X streams (depending on the SkimCycle). One Merge Super job merges the inputs on 1 to n Skim Super Jobs and produces X output files (One for each stream).
After running BbkCreateTask to set up the initial tasks the basic skim and merge cycles are:
BbkCreateSkims BbkSubmitSkims BbkCheckSkims
BbkCreateMerges BbkSubmitMerges BbkCheckMerges BbkTMLoad BbkTMExport
These run independently apart from the fact that the Merge cycle obviously takes the output of the skim cycle as it's input.
for running on the Grid two new commands BbkTMGridJobStatus and BbkRetrieveGridJob are inserted between the BbkSubmit[Skim|Merge] and BbkCheck[Skim|Merge] commands to check the status of the Skim or Merge job on the grid and to recover the output to the local setup.
BaBar Grid Skimming Commands
BbkTaskManager Commands
BbkCreateTask
Takes the text tasak description file and creates a new task, loading the task settings into the database.
BbkEditTask
Updates and already created task.
BbkCreateSkims and BbkCreateMerges
Create the Skim and Merge SuperJobs respectively, the -n option on BbkCreateSkims creates that many skim super jobs not individual jobs. It appears to be non functional on the BbkCreateMerges command
BbkSubmitSkims and BbkSubmitMerges
Submits the individual skim or merge jobs to the batch system of choice.
BbkSubmitSkims -n 50 MAN-Task01-funny04-R24a1 BbkSubmitMerges -n 10 MAN-Task01-funny04-R24a1
BbkCheckSkims and BbkCheckMerges
Checks the output of the individual jobs once the jobs has finised either marking it as successful or failed. Also used for reseting failed jobs for resubmission. BbkCheckMerges will also go back and reset the component skims of a merge job if the job failed because of missing input files
BbkCheckMerges MAN-Task01-funny04-R24a1 BbkCheckSkims MAN-Task01-funny04-R24a1
BbkTMLoad
Loads the infomation for the successfully completed merge jobs into the main bookkeepping database at SLAC
BbkTMExport
Exports the successful merge collections to SLAC.
BbkTMSprite
Automation tool, will create, submit and check skims then create, submit, check, load and export skims
BbkTMUser
Used to query the Task database, good for finding out detailed infomation and debugging
BbkTMGridTools Commands
BbkTMGridJobStatus
Used to check the status of skim or merge jobs submitted to the grid.
BbkTMGridJobStatus --check --print MAN-Task01-funny04-R24a1
BbkTMGridJobsStatusDaemon
Daemon that sits in the background updateing the status of skim and merge jobs submitted to the grid
BbkRetrieveGridJob
Once a skim job has runn on the grid the output files (needed to check the status) are stored on a remote server. BbkRetrieveGridJob copies the files back and puts them in the job directory so the Check commands can check the job. Can be run interactively against one or all completed jobs or as a daemon to retrieve jobs as the finish.
BbkRetrieveGridJob --temp /var/tmp <--background> MAN-Task01-funny04-R24a1 <JobID>
