Difference between revisions of "RAL Tier1 weekly operations castor 12/10/2018"

From GridPP Wiki
Jump to: navigation, search
(Operation problems)
Line 42: Line 42:
 
== Operation news ==
 
== Operation news ==
  
  * ATLAS tape test completed successfully on the new CASTOR instance
+
Two drives were added to fdscts01 and fdscts04
  
 
== Plans for next few weeks ==
 
== Plans for next few weeks ==
Line 51: Line 51:
 
   
 
   
 
   * Proceed with the atlasStripInput decommissioning
 
   * Proceed with the atlasStripInput decommissioning
 +
 +
  * Proceed with the cmsDisk decommissioning
  
 
== Long-term projects ==
 
== Long-term projects ==
Line 56: Line 58:
 
gsi authentication for xrootd: 1) Ask Raja and Tim to run a functional test on preprod to make sure their prod users work 2) Perform a stress test
 
gsi authentication for xrootd: 1) Ask Raja and Tim to run a functional test on preprod to make sure their prod users work 2) Perform a stress test
  
New CASTOR WLCGTape instance: Upgraded stagers and ns nodes to 2.1.17-35. Need to upgrade tape servers to make migrations/recalls
+
New CASTOR WLCGTape instance. Things need doing: 1) Update of the CASTOR ldif file and 2) Xrootd functional test on castor-fuctional-test1 3) Fix a misconfiguration on the Eris disk array (cannot bring WLCGTape into prod before this is done)
to work. Because of CASTOR version depedencies, tape servers cannot all be upgraded and allocated to the new instance at the same
+
4) xrootd for ALICE
time. Rob suggested we do 5 tape servers for every VO migration to the new CASTOR instance. Also need to negotiate with TimA how
+
many tape drives ATLAS would be happy to use for the tape "carouselle" test. 
+
 
+
Things need doing: log rotation([https://helpdesk.gridpp.rl.ac.uk/Ticket/Display.html?id=212634 RT212634]), xrootd for ALICE
+
  
 
== Actions ==  
 
== Actions ==  
  
 
GP to add ldif entry for the new srm-wlcgtape endpoint
 
GP to add ldif entry for the new srm-wlcgtape endpoint
 
JK to investigate why ralreplicas fails and submit a "bug" report link
 
 
RA/BD:  Run GFAL unit tests against CASTOR. Get them here: https://gitlab.cern.ch/dmc/gfal2/tree/develop/test/
 
 
GP/RA to write a Nagios test to check for large number of requests on the NewRequests DB table that remain for a long time - the check will run every xxx min (to be aggreed with Production) and issue an alarm if the number exceeds 1,000 for an extended period of time (15 min)
 
  
 
== Staffing ==
 
== Staffing ==
  
 
GP on call
 
GP on call

Revision as of 10:04, 12 October 2018

Standing agenda

1. Problems encountered this week

2. Upgrades/improvements made this week

3. What are we planning to do next week?

4. Long-term project updates (if not already covered)

  1. SL5 elimination from CIP and tape verification server
  2. CASTOR stress test improvement
  3. Generic CASTOR headnode setup
  4. Aquilonised headnodes

5. Special topics

6. Actions

7. Review Fabric tasks

  1.   Link

8. AoTechnicalB

9. Availability for next week

10. On-Call

11. AoOtherB

Operation problems

OPS tests for WLCGTape are failing and a change in static ldif file needs to be made

AAA issues with CMS GGUS137650

gdss734 and gdss738 failed and temporarily removed from prod; they are back now

tnsnames.ora issue on vCert and WLCGTape (the file was updated by a cron job script provided by oracle-instantclient-tnsnames.ora-1-3.el7.cern.noarch). Now fixed

Operation news

Two drives were added to fdscts01 and fdscts04

Plans for next few weeks

  * Push gsi authentication for xrootd in production
  * Decommission gdss747 & gdss753

  * Proceed with the atlasStripInput decommissioning
  * Proceed with the cmsDisk decommissioning

Long-term projects

gsi authentication for xrootd: 1) Ask Raja and Tim to run a functional test on preprod to make sure their prod users work 2) Perform a stress test

New CASTOR WLCGTape instance. Things need doing: 1) Update of the CASTOR ldif file and 2) Xrootd functional test on castor-fuctional-test1 3) Fix a misconfiguration on the Eris disk array (cannot bring WLCGTape into prod before this is done) 4) xrootd for ALICE

Actions

GP to add ldif entry for the new srm-wlcgtape endpoint

Staffing

GP on call