RAL Tier1 weekly operations castor 07/06/2019
1. Achievements this week
2. Problems encountered this week
3. What are we planning to do next week?
4. Long-term project updates (if not already covered)
5. Special topics
7. Review Fabric tasks
9. Availability for next week
Achievements this week
- LHCb decommissioning ongoing.
- ATLAS are periodically submitting SAM tests that impact availability and cause pointless callouts - Currently with TA
- Last Thursday, a change to resolv.conf necessary to test the new Facilities CASTOR headnodes caused the production tape servers to go down.
- This was an accidental change that wasn't supposed to go into production.
- Proposal: Move VCert into the Facilities domain so we have a facilities test instance.
- Question over the quality of the monitoring - the CANBEMIGR test caught the problem, but didn't make a ticket. Also drives were UNKNOWN rather than DOWN so the drive test didn't catch them.
Plans for next few weeks
- Examine further standardisation of CASTOR pool settings.
- CASTOR team to generate a list of nonstandard settings and consider whether they are justified.
- Castor tape testing to continue after the production tape robot networking is installed
- Decommission lhcbDst then move them to wlcgTape
- Agreed with Raja to keep the data in lhcbDst pool till the 7th of June (2 months after the last storage element was copied to Echo)
- Agreed with Darren to actually do this on Monday.
- Kevin has done some storageD functional tests with the new tape robot
- He has a plan for CEDA to do this testing.
- New CASTOR WLCGTape instance.
- Migration of name server to VMs on 2.1.17-xx is waiting until aliceDisk is decommissioned.
- CASTOR disk server migration to Aquilon.
- Need to work with Fabric to get a stress test (see above)
- The task of moving gridmap-file functionality away from castor-functional-test1 is complete
- RA to confirm with JA that the Echo Gateways have also been migrated.
- SL7 VM headnodes need changes to their personalities for the facilities.
- SL7 headnodes are being tested by GP
- Implementing DUNE on Spectralogic robot is paused.
- Migrate VCert to VMWare.
- AD wants us to make sure that experiments cannot write to that part of namespace that was used for d1t0 data: namespace cleanup/deletion of empty dirs.
- Some discussion about what exactly is required and how this can be actually implemented.
- CASTOR team proposal is either:
- to switch all of these directories to a fileclass with a requirement for a tape copy but no migration route; this will cause an error whenever any writes are attempted.
- to run a recursive nschmod on all the unneeded directories to make them read only.
- CASTOR team split over the correct approach.
- Problem with functional test node using a personal proxy which runs out some time in July.
- RA met with JJ, requested an appropriate certificate.
- Follow up with JJ or ST next week
- RA and DM to sit down to sort out storage metric question
- Plan to create new metrics for GridPP6.
- RA in Lyon Monday evening to Wednesday evening.
- GP is back on the 24th.
RA on call