RAL Tier1 weekly operations castor 24/11/2017

From GridPP Wiki
Jump to: navigation, search

Draft agenda

1. Problems encountered this week

2. Upgrades/improvements made this week

3. What are we planning to do next week?

  1. Facilities Headnode Replacement

4. Long-term project updates (if not already covered)

  1. SL7 upgrade on tape servers
  2. SL5 elimination from CASTOR functional test boxes and tape verification server
  3. CASTOR stress test improvement
  4. Generic CASTOR headnode setup

5. Special topics

6. Actions

7. Anything for CASTOR-Fabric?

8. AoTechnicalB

9. Availability for next week

10. On-Call

11. AoOtherB

Operation problems

Increased migration trafic for CMS

gdss896 out of prod

gdss771 out prod

Corrupted/incompletely written files due to 20/11 power outage

Continuing problems with SOLID writes to CASTOR

Operation news

The old atlasScratchDisk disk servers were finally decommissioned

Patched Facilities disk servers to latest available errata/kernel.

Continue rolling SL7 upgrade on Facilities tape servers

Plans for next week

Complete SL7 upgrade of the Facilities tape servers

Long-term projects

CASTOR stress test improvement - Script writing, awaiting testing and validation

Tape server migration to Aquilon - Still need to do Facilities - GP to co-ordinate with GTF

Headnode migration to Aquilon - Templates for the stager written, but not compiled or tested. Focus on features but not personalities

Target: Combined headnodes running on SL7/Aquilon - implement CERN-style 'Macro' headnodes.

Draining of remainder of 12 generation HW - waiting on CMS migration to Echo. No more draining ongoing.

Special topics

Patching - consider turning repeatedly re-used patching plan into standard procedure.

Actions

RA to check if the disk server setting, changed to bring disk servers back more quickly after a CASTOR shutdown, is still in place

RA/GP: Run GFAL unit tests against CASTOR. Get them here: https://gitlab.cern.ch/dmc/gfal2/tree/develop/test/

GP: Turn CASTOR SRMs & other Aquilonised nodes from clients to servers

GP to chase up Fabric team about getting action on RT197296(use fdscspr05 as the preprod stager - replacement for ccse08)

Staffing

GP on call

RA out until 6th December