Difference between revisions of "RAL Tier1 weekly operations castor 01/12/2017"

From GridPP Wiki
Jump to: navigation, search
(Long-term projects)
 
(5 intermediate revisions by one user not shown)
Line 32: Line 32:
 
gdss746 went out of production
 
gdss746 went out of production
  
lhbDst is almost full (%2 free space is left)
+
lcgsrm11 (gen) failed and had to be re-installed [https://helpdesk.gridpp.rl.ac.uk/Ticket/Display.html?id=198211 RT198211]
 +
 
 +
lhbDst is almost full (%2 free space is left) which caused problems to LHCb [https://ggus.eu/index.php?mode=ticket_info&ticket_id=132094 GGUS132094]
  
 
Corrupted/incompletely written files due to 20/11 power outage
 
Corrupted/incompletely written files due to 20/11 power outage
 +
 +
SOLID access/ceritificate issues [https://ggus.eu/index.php?mode=ticket_info&ticket_id=131840 GGUS131840]
  
 
== Operation news ==
 
== Operation news ==
  
Completed SL7 upgrade of the Facilities tape servers
+
Completed SL7 upgrade of the Facilities tape servers. Memory issues due to wrong configuration on Aquilon resolved
  
 
== Plans for next week ==
 
== Plans for next week ==
  
Upgrade of non-LHCb SRMs to 2.1.16-18 on Wed 06/12
+
Upgrade of the non-LHCb SRMs to 2.1.16-18 on Wed 06/12
  
 
== Long-term projects ==
 
== Long-term projects ==
Line 53: Line 57:
  
 
Draining of remainder of 12 generation HW - waiting on CMS migration to Echo. No more draining ongoing.
 
Draining of remainder of 12 generation HW - waiting on CMS migration to Echo. No more draining ongoing.
 
== Special topics ==
 
 
Patching - consider turning repeatedly re-used patching plan into standard procedure.
 
  
 
== Actions ==  
 
== Actions ==  

Latest revision as of 14:22, 1 December 2017

Draft agenda

1. Problems encountered this week

2. Upgrades/improvements made this week

3. What are we planning to do next week?

  1. Facilities Headnode Replacement

4. Long-term project updates (if not already covered)

  1. SL7 upgrade on tape servers
  2. SL5 elimination from CASTOR functional test boxes and tape verification server
  3. CASTOR stress test improvement
  4. Generic CASTOR headnode setup

5. Special topics

6. Actions

7. Anything for CASTOR-Fabric?

8. AoTechnicalB

9. Availability for next week

10. On-Call

11. AoOtherB

Operation problems

gdss746 went out of production

lcgsrm11 (gen) failed and had to be re-installed RT198211

lhbDst is almost full (%2 free space is left) which caused problems to LHCb GGUS132094

Corrupted/incompletely written files due to 20/11 power outage

SOLID access/ceritificate issues GGUS131840

Operation news

Completed SL7 upgrade of the Facilities tape servers. Memory issues due to wrong configuration on Aquilon resolved

Plans for next week

Upgrade of the non-LHCb SRMs to 2.1.16-18 on Wed 06/12

Long-term projects

CASTOR stress test improvement - Script writing, awaiting testing and validation

Headnode migration to Aquilon - Stager configuration almost complete; functional tests pass. Focus on features but not personalities

Target: Combined headnodes running on SL7/Aquilon - implement CERN-style 'Macro' headnodes.

Draining of remainder of 12 generation HW - waiting on CMS migration to Echo. No more draining ongoing.

Actions

RA to check if the disk server setting, changed to bring disk servers back more quickly after a CASTOR shutdown, is still in place

RA/GP: Run GFAL unit tests against CASTOR. Get them here: https://gitlab.cern.ch/dmc/gfal2/tree/develop/test/

GP: Turn CASTOR SRMs & other Aquilonised nodes from clients to servers

GP to chase up Fabric team about getting action on RT197296(use fdscspr05 as the preprod stager - replacement for ccse08)

Staffing

GP on call

RA out until 6th December