Difference between revisions of "RAL Tier1 weekly operations castor 21/09/2018"
(→Staffing) |
|||
| (12 intermediate revisions by one user not shown) | |||
| Line 31: | Line 31: | ||
== Operation problems == | == Operation problems == | ||
| − | + | Lost snoplus files that failed to migrate to tape [[https://ggus.eu/index.php?mode=ticket_info&ticket_id=137294][GGUS137294]]. Disk server decommissioning procedure has been modified to avoid this happening in the future. | |
| − | + | ||
| − | + | gdss747 called out; needs to be decommissioned | |
| − | + | Argo tests for atlasStripInput are failing since the pool was set to read only; a change in GOCDB needs to be made | |
| − | + | ||
== Operation news == | == Operation news == | ||
| − | * | + | * 8 tape servers are allocated to the new CASTOR instance and upgraded to 2.1.17.35. They are ready |
| − | + | to be used for the ATLAS tape test [https://elog.gridpp.rl.ac.uk/Tier1/6840 elog] | |
| − | + | ||
| − | + | ||
| − | + | ||
== Plans for next few weeks == | == Plans for next few weeks == | ||
| Line 55: | Line 50: | ||
== Long-term projects == | == Long-term projects == | ||
| − | gsi authentication for xrootd: 1) Return preprod to a working state (done) 2) Ask Raja and Tim to test | + | gsi authentication for xrootd: 1) Return preprod to a working state (done) 2) Ask Raja and Tim to run a functional test on preprod to make sure their prod users work 3) Perform a stress test |
| − | 3) Perform a stress test | + | |
New CASTOR WLCGTape instance: Upgraded stagers and ns nodes to 2.1.17-35. Need to upgrade tape servers to make migrations/recalls | New CASTOR WLCGTape instance: Upgraded stagers and ns nodes to 2.1.17-35. Need to upgrade tape servers to make migrations/recalls | ||
| Line 66: | Line 60: | ||
== Actions == | == Actions == | ||
| + | |||
| + | GP/RA to draft an email to Giuseppe about setting up xrootd for ALICE on the new CASTOR instance | ||
GP to add ldif entry for the new srm-wlcgtape endpoint | GP to add ldif entry for the new srm-wlcgtape endpoint | ||
| Line 77: | Line 73: | ||
== Staffing == | == Staffing == | ||
| − | + | GP on call | |
Latest revision as of 10:18, 21 September 2018
Contents
Standing agenda
1. Problems encountered this week
2. Upgrades/improvements made this week
3. What are we planning to do next week?
4. Long-term project updates (if not already covered)
1. SL5 elimination from CIP and tape verification server 2. CASTOR stress test improvement 3. Generic CASTOR headnode setup 4. Aquilonised headnodes
5. Special topics
6. Actions
7. Review Fabric tasks
1. Link
8. AoTechnicalB
9. Availability for next week
10. On-Call
11. AoOtherB
Operation problems
Lost snoplus files that failed to migrate to tape [[1][GGUS137294]]. Disk server decommissioning procedure has been modified to avoid this happening in the future.
gdss747 called out; needs to be decommissioned
Argo tests for atlasStripInput are failing since the pool was set to read only; a change in GOCDB needs to be made
Operation news
* 8 tape servers are allocated to the new CASTOR instance and upgraded to 2.1.17.35. They are ready to be used for the ATLAS tape test elog
Plans for next few weeks
* Push gsi authentication for xrootd in production
* Decommission gdss747
Long-term projects
gsi authentication for xrootd: 1) Return preprod to a working state (done) 2) Ask Raja and Tim to run a functional test on preprod to make sure their prod users work 3) Perform a stress test
New CASTOR WLCGTape instance: Upgraded stagers and ns nodes to 2.1.17-35. Need to upgrade tape servers to make migrations/recalls to work. Because of CASTOR version depedencies, tape servers cannot all be upgraded and allocated to the new instance at the same time. Rob suggested we do 5 tape servers for every VO migration to the new CASTOR instance. Also need to negotiate with TimA how many tape drives ATLAS would be happy to use for the tape "carouselle" test.
Things need doing: log rotation(RT212634), xrootd for ALICE
Actions
GP/RA to draft an email to Giuseppe about setting up xrootd for ALICE on the new CASTOR instance
GP to add ldif entry for the new srm-wlcgtape endpoint
JK to investigate why ralreplicas fails and submit a "bug" report
RA/BD: Run GFAL unit tests against CASTOR. Get them here: https://gitlab.cern.ch/dmc/gfal2/tree/develop/test/
GP/RA to write a Nagios test to check for large number of requests on the NewRequests DB table that remain for a long time - the check will run every xxx min (to be aggreed with Production) and issue an alarm if the number exceeds 1,000 for an extended period of time (15 min)
Staffing
GP on call
