BaBar Local Bookkeeping

From GridPP Wiki
Revision as of 13:07, 23 June 2006 by Tim adye (Talk | contribs)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search


Recreating Lost Symlinks

Cautious method

(bbkr18 --file_status=3 file -q; bbkr14 --file_status=3 file -q) |\
   sort -u | find-imported-files > imported-pfn.lis

Check files are all OK, then

lnlist imported-pfn.lis -r '/stage/bdata-data*/import' -n ~/dummylinks

Cavalier method

... or all at once, just for bbkr18...

bbkr18 --file_status=3 file -q | find-imported-files | lnlist - -r '/stage/bdata-data*/import' -n ~/dummylinks
bbkr18 --file_status=3 file -q | sed 's=^=/home/csf/bbdatsrv/dummylinks=' | fexists !

If that's all OK

shimp2 --file_status=3 --set file_status=1A --dbuser=bfactory
/home/csf/bbdatsrv/bin/import-r18.cron.sh b 1A
job -M /home/csf/bbdatsrv/bin/sweep-cm2.cron.sh

Listing Untagged Bad Files

comm -23 \
 <(BbkUser --dbname=bbkr18 --dse_status=2 --file_status=0 file -q |\
   sort -u) \
 <(BbkUser --dbname=bbkr18 --dse_status=2 --file_status=0 file -q \
      --ds_is_local=1 --tag=v02,v03,v04 --ds_name='^prelim-*,^test-*' |\
   sort -u) \
 > badstat-untagged.lis

Only works with BbkTools tja060320a or later, but that's now the default in 18.6.4b. The second BbkUser command took more than 2 hours until we added indices to the MySQL tables; thereafter it only takes a few seconds.

Listing Queued Imports

% BbkUser --dbname=bbkr18 --file_status='1*' \
          file_status dse_run_cycle dse_type components file_id gbytes --summary \
          --combine 'components:full=0+*A*,pointer=*'
FILE_STATUS DSE_RUN_CYCLE DSE_TYPE COMPONENTS #FILE_ID +GBYTES
=========== ============= ======== ========== ======== =======
1J          run3          PRskims  full             38    11.1
1K          run3          PR       full           1113   488.3
1L          run3          PRskims  pointer         128   120.2
1N          run5          PR       full           2092  1369.0
1P          run2          PRskims  full           1353  1149.6
1Q          run2          PR       full           1960  1045.0
1R          run2          PRskims  pointer         733   689.7
=========== ============= ======== ========== ======== =======
Totals                                            7417  4873.1

Marking Datasets Local

Datasets should be marked local (ds_is_local=1) if they will be of any use to users - even if they are not all (yet) available at RAL.

R18

This is now done automatically in ~bbdatsrv/bin/mark-r18.cron.sh

BbkUser --site=ral --dbuser=bfactory --dbname=bbkr18 \
        --set ds_is_local=1 --ds_is_local=0 \
        --ds_type=PR,SP,Bkg --ds_skim=AllEvents,SP,BkgTriggers

BbkUser --site=ral --dbuser=bfactory --dbname=bbkr18 \
        --set ds_is_local=1 --ds_is_local=0 \
        --ds_type=PRskims,SPskims -i ds_skim=/home/csf/bbdatsrv/ral-skims.lis \
        --ds_skim_cycle=^R18a

R14/R16

BbkUser --site=ral --dbuser=bfactory --dbname=bbkr14 \
        --set ds_is_local=1 --ds_is_local=0 \
        --ds_type=PR,SP,Bkg --ds_skim=AllEvents,SP,BkgTriggers

BbkUser --site=ral --dbuser=bfactory --dbname=bbkr14 \
        --set ds_is_local=1 --ds_is_local=0 \
        --ds_type=PRskims,SPskims -i ds_skim=/home/csf/bbdatsrv/ral-skims.lis \
        --ds_name '^testSkim*'

We also have the BchToD0KstarAll data skims, so I marked them local too.

BbkUser --site=ral --dbuser=bfactory --dbname=bbkr14 \
        --set ds_is_local=1 --ds_is_local=0 \
        --ds_type=PRskims --ds_skim=BchToD0KstarAll --ds_skim_cycle=R14


Listing just skimmed data

In this example, we list run5 SP files that have been skimmed (and are hence needed by the pointer skims).

BbkUser --dbname=bbkr18 \
        --dse_status=1 --dse_run_cycle=run5 \
        --dse_type=SPskims --skim=AllEvents input_dse -q |\
sort -nu |\
BbkUser --dbname=bbkr18 \
        -i dse_id=-  --file_suffix='^.??E' \
        file_status file_id gbytes --summary