Difference between revisions of "RAL Tier1 weekly operations castor 28/09/2018"

From GridPP Wiki
Jump to: navigation, search
 
Line 36: Line 36:
  
 
A list of irregular files on fdsdss35 [[https://helpdesk.gridpp.rl.ac.uk/Ticket/Display.html?id=213302][RT213302]]
 
A list of irregular files on fdsdss35 [[https://helpdesk.gridpp.rl.ac.uk/Ticket/Display.html?id=213302][RT213302]]
 +
confirm that tape copies are good
  
 
== Operation news ==
 
== Operation news ==
  
 
   * ATLAS tape test completed successfully on the new CASTOR instance  
 
   * ATLAS tape test completed successfully on the new CASTOR instance  
 
  * 8 tape servers are allocated to the new CASTOR instance and upgraded to 2.1.17.35. They are ready
 
    to be used for the ATLAS tape test [https://elog.gridpp.rl.ac.uk/Tier1/6840 elog]
 
  
 
== Plans for next few weeks ==
 
== Plans for next few weeks ==
Line 48: Line 46:
 
   * Push gsi authentication for xrootd in production
 
   * Push gsi authentication for xrootd in production
  
   * Decommission gdss747
+
   * Decommission gdss747 after Rob finishes with cleaning it up
 +
 +
  * Proceed with the atlasStripinput decommissioning
  
 
== Long-term projects ==
 
== Long-term projects ==
Line 62: Line 62:
  
 
== Actions ==  
 
== Actions ==  
 
GP/RA to draft an email to Giuseppe about setting up xrootd for ALICE on the new CASTOR instance
 
  
 
GP to add ldif entry for the new srm-wlcgtape endpoint
 
GP to add ldif entry for the new srm-wlcgtape endpoint
  
JK to investigate why ralreplicas fails and submit a "bug" report
+
JK to investigate why ralreplicas fails and submit a "bug" report link
  
 
RA/BD:  Run GFAL unit tests against CASTOR. Get them here: https://gitlab.cern.ch/dmc/gfal2/tree/develop/test/
 
RA/BD:  Run GFAL unit tests against CASTOR. Get them here: https://gitlab.cern.ch/dmc/gfal2/tree/develop/test/
Line 75: Line 73:
 
== Staffing ==
 
== Staffing ==
  
GP on call
+
RA on call

Latest revision as of 10:10, 28 September 2018

Standing agenda

1. Problems encountered this week

2. Upgrades/improvements made this week

3. What are we planning to do next week?

4. Long-term project updates (if not already covered)

  1. SL5 elimination from CIP and tape verification server
  2. CASTOR stress test improvement
  3. Generic CASTOR headnode setup
  4. Aquilonised headnodes

5. Special topics

6. Actions

7. Review Fabric tasks

  1.   Link

8. AoTechnicalB

9. Availability for next week

10. On-Call

11. AoOtherB

Operation problems

Lost na62 and snoplus files that never migrated to tape

OPS tests for WLCGTape are failing and a change in static ldif file needs to be made

A list of irregular files on fdsdss35 [[1][RT213302]] confirm that tape copies are good

Operation news

 * ATLAS tape test completed successfully on the new CASTOR instance 

Plans for next few weeks

  * Push gsi authentication for xrootd in production
  * Decommission gdss747 after Rob finishes with cleaning it up

  * Proceed with the atlasStripinput decommissioning

Long-term projects

gsi authentication for xrootd: 1) Return preprod to a working state (done) 2) Ask Raja and Tim to run a functional test on preprod to make sure their prod users work 3) Perform a stress test

New CASTOR WLCGTape instance: Upgraded stagers and ns nodes to 2.1.17-35. Need to upgrade tape servers to make migrations/recalls to work. Because of CASTOR version depedencies, tape servers cannot all be upgraded and allocated to the new instance at the same time. Rob suggested we do 5 tape servers for every VO migration to the new CASTOR instance. Also need to negotiate with TimA how many tape drives ATLAS would be happy to use for the tape "carouselle" test.

Things need doing: log rotation(RT212634), xrootd for ALICE

Actions

GP to add ldif entry for the new srm-wlcgtape endpoint

JK to investigate why ralreplicas fails and submit a "bug" report link

RA/BD: Run GFAL unit tests against CASTOR. Get them here: https://gitlab.cern.ch/dmc/gfal2/tree/develop/test/

GP/RA to write a Nagios test to check for large number of requests on the NewRequests DB table that remain for a long time - the check will run every xxx min (to be aggreed with Production) and issue an alarm if the number exceeds 1,000 for an extended period of time (15 min)

Staffing

RA on call