Difference between revisions of "RAL Tier1 weekly operations castor 24/08/2018"

From GridPP Wiki
Jump to: navigation, search
(Operation problems)
(Operation problems)
 
(3 intermediate revisions by one user not shown)
Line 37: Line 37:
 
   * gdss747 is irretrievably lost - work in progress to clean up stager db
 
   * gdss747 is irretrievably lost - work in progress to clean up stager db
  
   * Unusual high failure rate in LHCb transfers from RAL WN to CASTOR ggus ticket
+
   * Unusual high failure rate in LHCb transfers from RAL WN to CASTOR [https://ggus.eu/index.php?mode=ticket_info&ticket_id=136701 GGUS ticket]
  
 
== Operation news ==
 
== Operation news ==
  
   * CIP is *actually* decommissioned
+
   * The endpoint for the new CASTOR instance is published on GOCDB as monitored
 
+
  * Draining of 2014 ds from cmsDisk and atlasStripInput is complete and all of them are deployed in the gridTape pool of the new CASTOR instance
+
 
+
  *  The final prpoduction configuration of the tape-backed pool of the new CASTOR instance has been finalised and it will include all the OCF-2014 and the Dell 2015 ds 
+
  
 
== Plans for next few weeks ==
 
== Plans for next few weeks ==
 
  * T
 
 
    
 
    
 
   * Push gsi authentication for xrootd in production
 
   * Push gsi authentication for xrootd in production
Line 60: Line 54:
 
3) Perform a stress test
 
3) Perform a stress test
  
Things need doing: log rotation, DB link to pluto set up and tested.
+
Things need doing: log rotation, xrootd for ALICE
  
 
== Actions ==  
 
== Actions ==  

Latest revision as of 09:51, 24 August 2018

Draft agenda

1. Problems encountered this week

2. Upgrades/improvements made this week

3. What are we planning to do next week?

4. Long-term project updates (if not already covered)

  1. SL5 elimination from CIP and tape verification server
  2. CASTOR stress test improvement
  3. Generic CASTOR headnode setup
  4. Aquilonised headnodes

5. Special topics

6. Actions

7. Review Fabric tasks

  1.   Link

8. AoTechnicalB

9. Availability for next week

10. On-Call

11. AoOtherB

Operation problems

 * gdss771 (lhcbDst) failed and were removed from prod
 * gdss757 (cmsDisk) was removed from prod as a precaution
 * gdss747 is irretrievably lost - work in progress to clean up stager db
 * Unusual high failure rate in LHCb transfers from RAL WN to CASTOR GGUS ticket

Operation news

 * The endpoint for the new CASTOR instance is published on GOCDB as monitored

Plans for next few weeks

  * Push gsi authentication for xrootd in production
  * Decommission gdss747

Long-term projects

gsi authentication for xrootd: 1) Return preprod to a working state 2) Ask Raja and Tim to test the gsi config on preprod 3) Perform a stress test

Things need doing: log rotation, xrootd for ALICE

Actions

JK to investigate why ralreplicas fails and submit a "bug" report

RA/BD: Run GFAL unit tests against CASTOR. Get them here: https://gitlab.cern.ch/dmc/gfal2/tree/develop/test/

GP/RA to write a Nagios test to check for large number of requests on the NewRequests DB table that remain for a long time - the check will run every xxx min (to be aggreed with Production) and issue an alarm if the number exceeds 1,000 for an extended period of time (15 min)

Staffing

GP on call