Difference between revisions of "BaBar Data Management"
(No difference)
|
Latest revision as of 11:39, 28 March 2007
BaBar Data Management
[move03] /home/csf/bbdatsrv > shimp FILE_STATUS DSE_TYPE DSE_MODENUM COMPONENTS #FILE_ID +GBYTES =========== ======== =========== ========== ======== ======= 1P SP signal full 8 5.9 =========== ======== =========== ========== ======== ======= Totals 8 5.9 8 rows returned from bbkr18 at ral
or add dfile_modified for mark date.
ral-disk-status ral-import-status # when total goes to 0TB, import stops
shimp2 --file_status=3 # shows files to sweep
shimp == shimp2 --file_status='1*'
shjobs # lists processes run via "job" command for current user shjobs -l # just list logfile names
Thus, to see which jobs are stuck
cd ~bbdatsrv/logs ls -lt `shjobs -l`
If any logfiles haven't been updated for some time, they will show at the bottom of the list with an old time (less than an hour or two should be OK).
To start/restart import jobs, use
restart-r18
(with -h shows possibilities), but restart-r18 can only stop jobs that are still running normally and notice the signal (~/BbkImport.stop.$$ file).
Freeing disk space
shimp2 --file_status=0S # data to delete shimp2 --file_status=0 --dse_type=PR,SP --set file_status=0S --dbuser=bfactory # change status to 0S
cd ~bbdatsrv/lists ads-files -z | sort -k2,2 > ads.lis
BbkUser --dbname=bbkr18 --file_status=0S file bytes file_status -q \ | sort -k1,1 \ | join -11 -22 - ads.lis \ | ads-size-check \ > 070125a-del.lis
files-in-catalog < 070125a-del.lis | sort -u > 070125a-del.pfn
sed 's=^.*/store/=/store/=' 070125a-del.pfn \ | sort | diff - 070125a-del.lis | wc # to check that just 1 copy and all there shimp2 -i file=070125a-del.lis --set file_status=0T --dbuser=bfactory # mark as being on tape only
cd ~bbdatsrv job -NQ cat lists/070125a-del.pfn \| xargs -ti /home/csf/bbdatsrv/mps/mps_Xeq rm '{}'
This sends a message when the job is complete. Check logfile for errors.
Allocating datastore tapes for user archiving
Users can archive their data to the datastore, using tape names with a 3-character prefix, allocated, on request, by us. To allocate a prefix for a user, chose a prefix like "UX1", where "U" is used for all user tapes, "X" is the first character of their username, and "1" is chosen for a unique prefix for each user who asks.
Use the command
datastore rule query babar
to list all of our existing Datastore access settings. The prefixes should be listed in entries like
Address Username Group Name Mode %ATLAS MOHANTY * UM1* RWCPAL
Pick a new prefix and add permissions like this
datastore rule add babar %ATLAS TLATHAM + UT1+ RWCPAL
("+" can be used in place of "*", but isn't expanded by the shell). Check it is in the list with a new
lcgui0360> datastore rule query babar Rule BABAR owned by BABAR with components:- Address Username Group Name Mode %ATLAS MOHANTY * UM1* RWCPAL %ATLAS TLATHAM * UT1* RWCPAL ...
This allows user tlatham to create and write to tapes named UT1001 etc. Other users have read access.