BaBar Bookkeeping Changelog
Contents
24/05/07
Marked micro+mini files for 35 PR/SP collections with file_status='0K' to indicate that they should not be removed from disk as they are used for skimming tests.
FILE_STATUS DSE_TYPE DSE_MODENUM COMPONENTS #FILE_ID +GBYTES =========== ======== =========== ========== ======== ======= 0K PR full 12 4.2 0K PR mini 12 7.6 0K SP generic full 5 4.3 0K SP generic mini 5 5.5 0K SP signal full 18 1.1 0K SP signal mini 18 1.4 =========== ======== =========== ========== ======== ======= Totals 70 24.1
20/03/07
Marked a few more release>=19.0.0 file_status=0S (were imported yesterday).
RELEASE FILE_STATUS DSE_TYPE DSE_MODENUM COMPONENTS #FILE_ID +GBYTES ======= =========== ======== =========== ========== ======== ======= 22.0.4 0S SP signal full 37 0.6 22.0.4a 0S SP signal full 37 16.8 ======= =========== ======== =========== ========== ======== ======= Totals 74 17.4
19/03/07
Changed mark-r18.cron.sh to exclude R22a/b skimming and mini2mini datasets, include all Nonevent datasets as they appear, mark Bkg and Nonevent files for import with status 1C, include other files only if they have release <19.0.0, and exclude files only in skimming and mini2mini datasets datasets (any release).
Marked everything already imported with release>=19.0.0 as file_status=0S.
RELEASE FILE_STATUS DSE_TYPE DSE_MODENUM COMPONENTS #FILE_ID +GBYTES ======= =========== ======== =========== ========== ======== ======= 22.0.3 0S PR full 2 0.9 19.4.0 0S SP generic full 89 25.9 19.4.1b 0S SP generic full 91 26.4 22.0.3 0S SP generic full 1763 1469.8 22.0.4 0S SP generic full 153 44.1 22.0.4 0S SP signal full 7285 1156.1 22.0.4a 0S SP signal full 583 276.0 ======= =========== ======== =========== ========== ======== ======= Totals 9966 2999.1
06/02/07
Changed mark-r18.cron.sh to exclude Giuliano's R18dTestUK datasets.
02/02/07
Changed mark-r18.cron.sh to exclude R20 and R22 PR, SP, and skims. Also don't import files just because they are in the SP-AllOldSignalSkim dataset (like the test and prelim datasets).
Mark everything that was awaiting import in these categories back to file_status '2'.
Mark everything that was already imported in these categories file_status '0S' (to remove from disk later):-
SKIM_CYCLE FILE_STATUS DSE_TYPE DSE_MODENUM COMPONENTS #FILE_ID +GBYTES ========== =========== ======== =========== ========== ======== ======= R20a 0S PRskims full 177 236.6 R20a 0S PRskims generic full 13 7.9 R20a 0S PRskims signal full 2 0.0 R20a 0S SPskims generic full 245 319.7 R20a 0S SPskims signal full 5 0.2 ========== =========== ======== =========== ========== ======== ======= Totals 442 564.4
30/01/07
Changed mark-r18.cron.sh to select files with --components=^E instead of --components='^*E*'. This gives us files that have mini included with the micro. We don't have any skims in this category, but maybe we get them in future. We still exclude mini-only files.
13/06/06
Changed all run2 AllEvents PR collections to have is_local=0
BbkUser --dbname=bbkr18 --dbuser=bfactory \
--dse_type=PR --dse_run_cycle=run2 \
--is_local=1 --set is_local=0
except for the few that have the mini
BbkUser --dbname=bbkr18 --dbuser=bfactory \
--dse_type=PR --dse_run_cycle=run2 \
--file_status='0*' --components=E --dse_status=1 \
--is_local=0 --set is_local=1
This is to allow for skimming tests, since now AllEvents-Run2-R18 includes just collections we can skim.
BbkDatasetTcl AllEvents-Run2-R18 BbkDatasetTcl: wrote AllEvents-Run2-R18.tcl Selected 27 collections, 5972410/13964715 events, ~406.6/pb, from bbkr18 at ral ****************** WARNING ****************** 5714 collections (of 5741) are not available in bbkr18 at ral. tcl file only lists local collections. Specify BbkDatasetTcl --nolocal to include all collections. ****************** WARNING ******************
The whole dataset is on tape anyway (file_status=0T), so it doesn't really matter that the rest of it isn't really accessible with BbkDatasetTcl.
30/01/06
Files that turned up in the import areas that were marked with status=0, but weren't on disk or tape.
% BbkUser --dbname=bbkr14 -i file=tosweep-noxrd0.lis \
file_status file_id gbytes --summary --set file_status=2
FILE_STATUS #FILE_ID +GBYTES
=========== ======== =======
0 8 2.1
=========== ======== =======
Totals 8 2.1
% BbkUser --dbname=bbkr18 -i file=tosweep-noxrd0.lis \
file_status file_id gbytes --summary --set file_status=2
FILE_STATUS #FILE_ID +GBYTES
=========== ======== =======
0 253 103.1
=========== ======== =======
Totals 253 103.1
15/11/05
Mark next batch of R14/R16 skims on tape:-
BbkUser --dse_type=PRskims,SPskims --file_status=0 \
--where "(skim IN ('BCCEta03a3body','BFourHHHE','BFourHHHH','BFourHHHK','BFourHHHP','BFourHHKE','BFourHHKK','BFourHHPE','BFourHHPK','BFourHHPP','BToA1Kst','ExclEtaP','TwoPhotonPentaquark','TwoPhotonPi0Pi0','TwoPhotonTwoTrack') OR (skim='BFourBody' AND dse_type='SPskims') OR (skim='AllEvents' and skim_cycle<>'R16c'))" \
--set file_status=0T --dbuser=bfactory
Check bbkr18 database:-
% BbkModifyDseFlags --dbname=bbkr18 --checkislocal --all --maxquery 20000 2005/11/15 20:45:01-is_local should be 1 for the following 8 collections...
Mark most R18 PR and SP mini file_status=-1 so collections become local:-
% BbkUser --dbname=bbkr18 --components=E --file_status=2 \
--set file_status=-1 --dbuser=bfactory
2005/11/15 20:55:19-Report-set file_status=-1, dfile_modified=20051115205517 for 43598 rows
% BbkUser --dbname=bbkr18 --components=0 --file_suffix=.??E --file_status=2 \
--set file_status=-1 --dbuser=bfactory
2005/11/15 20:56:51-Report-set file_status=-1, dfile_modified=20051115205651 for 661 rows
% BbkModifyDseFlags --dbname=bbkr18 --dbuser=bfactory --autosetislocal --all --maxquery 20000
2005/11/15 21:31:51-Report-9023 collections had the wrong is_local
Mark some R14 SP mini file_status=-1 so collections become local:-
% BbkUser --dbname=bbkr14 --components=E --file_status=2 --set file_status=-1 --dbuser=bfactory 2005/11/15 21:15:23-Report-set file_status=-1, dfile_modified=20051115211523 for 1714 rows % BbkModifyDseFlags --dbname=bbkr14 --dbuser=bfactory --autosetislocal --all --maxquery 20000 2005/11/15 21:25:55-Report-1372 collections had the wrong is_local (0->1)
Some bad R18 files deleted from disk
% BbkUser --dbname=bbkr18 --dbuser=bfactory --file_suffix='.??E' \
--components=0 --file_status=0 --set file_status=0T
2005/11/15 21:56:34-Report-set file_status='0T', dfile_modified=20051115215634 for 122 rows
% BbkUser --dbname=bbkr18 --dbuser=bfactory --dse_status=2 \
--file_suffix '^.??E' --file_status=0 --set file_status=0T
2005/11/15 21:59:13-Report-set file_status='0T', dfile_modified=20051115215913 for 2248 rows
R14 mini files deleted from disk
BbkUser --dbname=bbkr14 --dbuser=bfactory --file_status=0 --components=E --set file_status=0T 2005/11/15 22:23:35-Report-set file_status='0T', dfile_modified=20051115222335 for 507 rows
R14 dse_status=2 files deleted from disk
% cd ~adye/bbk
% BbkUser --file_status=0 --dse_status=2 file -q | sort -u > 051115-r14-badstat.lis
% BbkUser --file_status=0 --dse_status=2 --noprune --link_status=1 \
--ds_is_local=1 --ds_name '^prelim-*,^SP-SignalModesForSkims,^testSP,^SLACSkimR16a' \
--dtd_modified '>=2004/09/30' file -q | sort -u > 051115-r14-badstat-recent.lis
% comm -23 051115-r14-badstat.lis 051115-r14-badstat-recent.lis > 051115-r14-badstat-del.lis
% BbkUser --dbname=bbkr14 -b file=051115-r14-badstat-del.lis file_id -q | \
BbkUser --dbname=bbkr14 --dbuser=bfactory -i file_id=- --file_status=0 \
--set file_status=0T
2005/11/15 22:57:47-Report-set file_status='0T', dfile_modified=20051115225747 for 1416 rows
(Checked for collections that were tagged with GreenCircle, BlueSquarePrime, BlackDiamond, Total, or HEAD (but not BlueSquare) as follows
% ( for t in GreenCircle BlueSquarePrime BlackDiamond Total; do \
BbkUser --dse_status=2 --tag $t dse_id -q; \
done; \
BbkUser --ds_name '^prelim-*,^test*,^SlacSkim*,^FixSkim-Run4-R14,^GridKaSkim,^SLACSkimR16a,^SP-SignalModesForSkims' \
--dse_status=2 dse_id -q \
) | sort -nu > taggedbad3-allbutBS.lis
% BbkUser -b dse_id=taggedbad3-allbutBS.lis --file_status=0 \
dse_type skim dse_id file_id gbytes --summary
where the first command was done at SLAC (for speed) and the second at RAL. However the 20 files that resulted were all in 051115-r14-badstat-recent.lis, so that wasn't needed.)
Don't import bad data either. All the run5 PRskims to date have been marked bad (dse_status=2).
% BbkUser --dbname=bbkr18 --dbuser=bfactory --dse_status=2 --file_status='1*' \
--set file_status=-1
2005/11/16 00:27:05-Report-set file_status=-1, dfile_modified=20051116002705 for 1142 rows
Change run5 PR currently queued for import, from priority E to N (same as run3 SP, which doesn't yet exist). This is because there are currently no run5 PRskims, so we should prioritrise the run3.
% BbkUser --dbname=bbkr18 --dbuser=bfactory --file_status=1E \
--set file_status=1N \
file_status dse_run_cycle dse_type components file_id gbytes --summary
2005/11/16 00:38:51-Report-set file_status='1N', dfile_modified=20051116003850 for 2092 rows
FILE_STATUS DSE_RUN_CYCLE DSE_TYPE COMPONENTS #FILE_ID +GBYTES
=========== ============= ======== ========== ======== =======
1E run5 PR HBCA 2092 1369.0
Here's what is now queued to import...
% BbkUser --dbname=bbkr18 --file_status='1*' \
file_status dse_run_cycle dse_type components file_id gbytes --summary \
--combine 'components:full=0+*A*,pointer=*'
FILE_STATUS DSE_RUN_CYCLE DSE_TYPE COMPONENTS #FILE_ID +GBYTES
=========== ============= ======== ========== ======== =======
1J run3 PRskims full 38 11.1
1K run3 PR full 1113 488.3
1L run3 PRskims pointer 128 120.2
1N run5 PR full 2092 1369.0
1P run2 PRskims full 1353 1149.6
1Q run2 PR full 1960 1045.0
1R run2 PRskims pointer 733 689.7
=========== ============= ======== ========== ======== =======
Totals 7417 4873.1
