Difference between revisions of "Tier1 Operations Report 2019-09-04"
From GridPP Wiki
(→) |
(→) |
||
(7 intermediate revisions by one user not shown) | |||
Line 8: | Line 8: | ||
{| width="100%" cellspacing="0" cellpadding="0" style="background-color: #ffffff; border: 1px solid silver; border-collapse: collapse; width: 100%; margin: 0 0 1em 0;" | {| width="100%" cellspacing="0" cellpadding="0" style="background-color: #ffffff; border: 1px solid silver; border-collapse: collapse; width: 100%; margin: 0 0 1em 0;" | ||
|- | |- | ||
− | | style="background-color: #b7f1ce; border-bottom: 1px solid silver; text-align: center; font-size: 1em; font-weight: bold; margin-top: 0; margin-bottom: 0; padding-top: 0.1em; padding-bottom: 0.1em;" | Review of Issues during the week | + | | style="background-color: #b7f1ce; border-bottom: 1px solid silver; text-align: center; font-size: 1em; font-weight: bold; margin-top: 0; margin-bottom: 0; padding-top: 0.1em; padding-bottom: 0.1em;" | Review of Issues during the week 29th August2019 to the 3rd September 2019. |
|} | |} | ||
− | * | + | * gridmapfile update issue led to new job starts to fail |
− | + | * blip in echo activity for 15 minutes due to internal authentication issue. | |
− | * | + | |
<!-- ***********End Review of Issues during last week*********** -----> | <!-- ***********End Review of Issues during last week*********** -----> | ||
Line 225: | Line 224: | ||
| style="background-color: #b7f1ce; border-bottom: 1px solid silver; text-align: center; font-size: 1em; font-weight: bold; margin-top: 0; margin-bottom: 0; padding-top: 0.1em; padding-bottom: 0.1em;" | GGUS Tickets Closed Last week | | style="background-color: #b7f1ce; border-bottom: 1px solid silver; text-align: center; font-size: 1em; font-weight: bold; margin-top: 0; margin-bottom: 0; padding-top: 0.1em; padding-bottom: 0.1em;" | GGUS Tickets Closed Last week | ||
|} | |} | ||
+ | |||
+ | |||
{| border=1 align=center | {| border=1 align=center | ||
|- bgcolor="#7c8aaf" | |- bgcolor="#7c8aaf" | ||
Line 238: | Line 239: | ||
! Scope | ! Scope | ||
|- | |- | ||
− | | | + | | 142815 |
| USER | | USER | ||
− | | | + | | cms |
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
| RAL-LCG2 | | RAL-LCG2 | ||
| urgent | | urgent | ||
| NGI_UK | | NGI_UK | ||
| solved | | solved | ||
− | | 2019-08-14 | + | | 2019-08-29 14:28:00 |
− | | | + | | PhEDEx deletions pending since 10+ days at T1_UK_RAL_Disk |
| WLCG | | WLCG | ||
|- | |- | ||
− | | | + | | 142782 |
− | | | + | | TEAM |
− | | | + | | lhcb |
| RAL-LCG2 | | RAL-LCG2 | ||
− | | urgent | + | | very urgent |
| NGI_UK | | NGI_UK | ||
| solved | | solved | ||
− | | 2019-08- | + | | 2019-08-30 09:34:00 |
− | | | + | | FTS3 transfers Failed to RAL-RDST at RAL-LCG2 |
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
| WLCG | | WLCG | ||
|- | |- | ||
− | | | + | | 142710 |
| TEAM | | TEAM | ||
| lhcb | | lhcb | ||
Line 289: | Line 268: | ||
| NGI_UK | | NGI_UK | ||
| verified | | verified | ||
− | | 2019-08- | + | | 2019-08-30 09:51:00 |
− | | | + | | Staging problems |
| WLCG | | WLCG | ||
|- | |- | ||
− | | | + | | 142694 |
| TEAM | | TEAM | ||
| atlas | | atlas | ||
Line 300: | Line 279: | ||
| NGI_UK | | NGI_UK | ||
| closed | | closed | ||
− | | 2019-08- | + | | 2019-08-28 23:59:00 |
− | | RAL- | + | | RAL-LCG2 transfer errors at source |
+ | | WLCG | ||
+ | |- | ||
+ | | 142665 | ||
+ | | USER | ||
+ | | cms | ||
+ | | RAL-LCG2 | ||
+ | | urgent | ||
+ | | NGI_UK | ||
+ | | closed | ||
+ | | 2019-08-28 23:59:00 | ||
+ | | Failing to transfer few files to RAL_Disk from CERN | ||
| WLCG | | WLCG | ||
|- | |- | ||
Line 310: | Line 300: | ||
| less urgent | | less urgent | ||
| NGI_UK | | NGI_UK | ||
− | | | + | | closed |
− | | 2019-08- | + | | 2019-08-28 23:59:00 |
| mice LFC to DFC transition | | mice LFC to DFC transition | ||
| EGI | | EGI | ||
Line 329: | Line 319: | ||
Availability Report | Availability Report | ||
|} | |} | ||
− | |||
{| border=1 align=center | {| border=1 align=center | ||
|- bgcolor="#7c8aaf" | |- bgcolor="#7c8aaf" | ||
Line 339: | Line 328: | ||
! Comments | ! Comments | ||
|- | |- | ||
− | | 2019-08- | + | | 2019-08-28 |
| 100 | | 100 | ||
| 99 | | 99 | ||
Line 346: | Line 335: | ||
| | | | ||
|- | |- | ||
− | | 2019-08- | + | | 2019-08-29 |
| 100 | | 100 | ||
| 100 | | 100 | ||
− | |||
− | |||
− | |||
− | |||
− | |||
| 100 | | 100 | ||
| 100 | | 100 | ||
− | |||
− | |||
| | | | ||
|- | |- | ||
− | | 2019-08- | + | | 2019-08-30 |
| 100 | | 100 | ||
| 100 | | 100 | ||
Line 367: | Line 349: | ||
| | | | ||
|- | |- | ||
− | | 2019-08- | + | | 2019-08-31 |
| 100 | | 100 | ||
| 100 | | 100 | ||
Line 374: | Line 356: | ||
| | | | ||
|- | |- | ||
− | | 2019- | + | | 2019-09-01 |
| 100 | | 100 | ||
| 100 | | 100 | ||
Line 381: | Line 363: | ||
| | | | ||
|- | |- | ||
− | | 2019- | + | | 2019-09-02 |
| 100 | | 100 | ||
+ | | 97 | ||
| 100 | | 100 | ||
+ | | 96 | ||
+ | | | ||
+ | |- | ||
+ | | 2019-09-03 | ||
+ | | 100 | ||
+ | | 99 | ||
| 100 | | 100 | ||
| 100 | | 100 | ||
Line 405: | Line 394: | ||
! Day !! Atlas HC !! CMS HC !! Comment | ! Day !! Atlas HC !! CMS HC !! Comment | ||
|- | |- | ||
− | | 2019-08- | + | | 2019-08-29 || 100 || 98 || |
|- | |- | ||
− | | 2019-08- | + | | 2019-08-29 || 100 || 96 || |
|- | |- | ||
− | | 2019-08- | + | | 2019-08-30 || 100 || 99 || |
|- | |- | ||
− | | 2019-08- | + | | 2019-08-31 || 100 || 100 || |
|- | |- | ||
− | | 2019- | + | | 2019-09-01 || 100 || 98|| |
|- | |- | ||
− | | 2019- | + | | 2019-09-02|| 96 || 99|| |
|- | |- | ||
− | | 2019- | + | | 2019-09-03 || 96 || 99 || |
|- | |- | ||
|} | |} |
Latest revision as of 13:12, 4 September 2019
RAL Tier1 Operations Report for 04th September 2019
Review of Issues during the week 29th August2019 to the 3rd September 2019. |
- gridmapfile update issue led to new job starts to fail
- blip in echo activity for 15 minutes due to internal authentication issue.
Current operational status and issues |
Notable Changes made since the last meeting. |
- NTR
Entries in GOC DB starting since the last report. |
Service | ID | Scheduled? | Outage/At Risk | Start | End | Duration | Reason |
---|
Declared in the GOC DB |
Service | ID | Scheduled? | Outage/At Risk | Start | End | Duration | Reason |
---|---|---|---|---|---|---|---|
- | - | - | - | - | - | - | - |
- No ongoing downtime
Advanced warning for other interventions |
The following items are being discussed and are still to be formally scheduled and announced. |
Listing by category:
- DNS servers will be rolled out within the Tier1 network.
Open GGUS Tickets |
Ticket-ID | Type | VO | Site | Priority | Responsible Unit | Status | Last Update | Subject | Scope |
---|---|---|---|---|---|---|---|---|---|
142981 | USER | mice | RAL-LCG2 | less urgent | NGI_UK | in progress | 2019-09-03 13:00:00 | mice; LFC to DFC transition | EGI |
142955 | USER | ops | RAL-LCG2 | less urgent | NGI_UK | in progress | 2019-09-02 10:26:00 | [Rod Dashboard] Issues detected at RAL-LCG2 | EGI |
142835 | USER | snoplus.snolab.ca | RAL-LCG2 | less urgent | NGI_UK | waiting for reply | 2019-08-30 09:25:00 | Connection Issues | EGI |
142689 | USER | cms | RAL-LCG2 | very urgent | NGI_UK | in progress | 2019-09-02 17:22:00 | Transfer failing to RAL_Disk | WLCG |
142350 | TEAM | lhcb | RAL-LCG2 | top priority | NGI_UK | in progress | 2019-09-03 12:41:00 | Proble accessing some LHCb files at RAL | WLCG |
140447 | USER | dteam | RAL-LCG2 | less urgent | NGI_UK | on hold | 2019-08-22 10:04:00 | packet loss outbound from RAL-LCG2 over IPv6 | EGI |
GGUS Tickets Closed Last week |
Ticket-ID | Type | VO | Site | Priority | Responsible Unit | Status | Last Update | Subject | Scope |
---|---|---|---|---|---|---|---|---|---|
142815 | USER | cms | RAL-LCG2 | urgent | NGI_UK | solved | 2019-08-29 14:28:00 | PhEDEx deletions pending since 10+ days at T1_UK_RAL_Disk | WLCG |
142782 | TEAM | lhcb | RAL-LCG2 | very urgent | NGI_UK | solved | 2019-08-30 09:34:00 | FTS3 transfers Failed to RAL-RDST at RAL-LCG2 | WLCG |
142710 | TEAM | lhcb | RAL-LCG2 | very urgent | NGI_UK | verified | 2019-08-30 09:51:00 | Staging problems | WLCG |
142694 | TEAM | atlas | RAL-LCG2 | urgent | NGI_UK | closed | 2019-08-28 23:59:00 | RAL-LCG2 transfer errors at source | WLCG |
142665 | USER | cms | RAL-LCG2 | urgent | NGI_UK | closed | 2019-08-28 23:59:00 | Failing to transfer few files to RAL_Disk from CERN | WLCG |
140220 | USER | mice | RAL-LCG2 | less urgent | NGI_UK | closed | 2019-08-28 23:59:00 | mice LFC to DFC transition | EGI |
Availability Report |
Day | Atlas | CMS | LHCB | Alice | Comments |
---|---|---|---|---|---|
2019-08-28 | 100 | 99 | 100 | 100 | |
2019-08-29 | 100 | 100 | 100 | 100 | |
2019-08-30 | 100 | 100 | 100 | 100 | |
2019-08-31 | 100 | 100 | 100 | 100 | |
2019-09-01 | 100 | 100 | 100 | 100 | |
2019-09-02 | 100 | 97 | 100 | 96 | |
2019-09-03 | 100 | 99 | 100 | 100 |
Hammercloud Test Report |
Target Availability for each site is 97.0% |
Day | Atlas HC | CMS HC | Comment |
---|---|---|---|
2019-08-29 | 100 | 98 | |
2019-08-29 | 100 | 96 | |
2019-08-30 | 100 | 99 | |
2019-08-31 | 100 | 100 | |
2019-09-01 | 100 | 98 | |
2019-09-02 | 96 | 99 | |
2019-09-03 | 96 | 99 |
Key: Atlas HC = Atlas HammerCloud (Queue RAL-LCG2_UCORE, Template 841); CMS HC = CMS HammerCloud
Notes from Meeting. |