RAL Tier1 weekly operations castor 21/06/2019

Standing agenda

1. Achievements this week

2. Problems encountered this week

3. What are we planning to do next week?

4. Long-term project updates (if not already covered)

5. Special topics

6. Actions

7. Review Fabric tasks

  1.   Link

8. AoTechnicalB

9. Availability for next week

10. On-Call

11. AoOtherB

LHCb decommissioning ongoing.
- Wrinkle over files which are on both d1t0 and tape.
Odd behaviour of xrdadler32 for files that have become corrupt on disk explained.
- If there's a checksum in the extended attributes, it will return that instead of actually checksumming the file.
- Agreed to stop the on-disk checksumming.

ATLAS are periodically submitting SAM tests that impact availability and cause pointless callouts - Currently with TA
New Facilities headnodes called out. They are not supported 24/7, and weren't in production, so this was doubly wrong!
- The reasons were twofold:
  - CASTOR team asked for the machines to have the same Nagios config as Tier 1 headnodes, but neglected to mention that these machines should not call out.
  - The person who created the config forgot to put them in downtime.

Proposal: Move VCert into the Facilities domain so we have a facilities test instance.
Decommissioned lhcbDst; hardware awaiting retirement.
Kevin has done some storageD functional tests with the new tape robot
- Castor tape testing has started again for CEDA

New CASTOR WLCGTape instance.
- Migration of name server to VMs on 2.1.17-xx is waiting until aliceDisk is decommissioned.
CASTOR disk server migration to Aquilon.
- Need to work with Fabric to get a stress test (see above)
SL7 VM headnodes need changes to their personalities for the facilities.
- SL7 headnodes are being tested by GP
Implementing DUNE on Spectralogic robot is paused.
Migrate VCert to VMWare.

AD wants us to make sure that experiments cannot write to that part of namespace that was used for d1t0 data: namespace cleanup/deletion of empty dirs.
- Some discussion about what exactly is required and how this can be actually implemented.
- CASTOR team proposal is either:
  - to switch all of these directories to a fileclass with a requirement for a tape copy but no migration route; this will cause an error whenever any writes are attempted.
  - to run a recursive nschmod on all the unneeded directories to make them read only.
  - CASTOR team split over the correct approach.
Problem with functional test node using a personal proxy which runs out some time in July.
- RA met with JJ, requested an appropriate certificate.
- Follow up with JJ or ST next week