Difference between revisions of "Tier1 Operations Report 2019-10-02"

From GridPP Wiki
Jump to: navigation, search
()
()
 
(3 intermediate revisions by one user not shown)
Line 125: Line 125:
 
| style="background-color: #b7f1ce; border-bottom: 1px solid silver; text-align: center; font-size: 1em; font-weight: bold; margin-top: 0; margin-bottom: 0; padding-top: 0.1em; padding-bottom: 0.1em;" | Open GGUS Tickets  
 
| style="background-color: #b7f1ce; border-bottom: 1px solid silver; text-align: center; font-size: 1em; font-weight: bold; margin-top: 0; margin-bottom: 0; padding-top: 0.1em; padding-bottom: 0.1em;" | Open GGUS Tickets  
 
|}
 
|}
 
  
 
{| border=1 align=center
 
{| border=1 align=center
Line 139: Line 138:
 
! Subject
 
! Subject
 
! Scope
 
! Scope
 +
|-
 +
| 143402
 +
| USER
 +
| none
 +
| RAL-LCG2
 +
| urgent
 +
| NGI_UK
 +
| in progress
 +
| 2019-09-30 12:23:00
 +
| CVMFS IPv6 connection issues at RAL
 +
| EGI
 +
|-
 +
| 143387
 +
| USER
 +
| snoplus.snolab.ca
 +
| RAL-LCG2
 +
| less urgent
 +
| NGI_UK
 +
| in progress
 +
| 2019-10-01 10:29:00
 +
| Transfer issues to RAL
 +
| EGI
 
|-
 
|-
 
| 143323
 
| 143323
Line 146: Line 167:
 
| top priority
 
| top priority
 
| NGI_UK
 
| NGI_UK
| waiting for reply
+
| in progress
| 2019-09-25 08:07:00
+
| 2019-09-27 15:20:00
 
| File deletion at RAL ECHO
 
| File deletion at RAL ECHO
| WLCG
 
|-
 
| 142689
 
| USER
 
| cms
 
| RAL-LCG2
 
| very urgent
 
| NGI_UK
 
| waiting for reply
 
| 2019-09-25 11:21:00
 
| Transfer failing to RAL_Disk
 
 
| WLCG
 
| WLCG
 
|-
 
|-
Line 172: Line 182:
 
| Proble accessing some LHCb files at RAL
 
| Proble accessing some LHCb files at RAL
 
| WLCG
 
| WLCG
|-
 
| 140447
 
| USER
 
| dteam
 
| RAL-LCG2
 
| less urgent
 
| NGI_UK
 
| in progress
 
| 2019-09-11 10:37:00
 
| packet loss outbound from RAL-LCG2 over IPv6
 
| EGI
 
 
|}
 
|}
  
Line 215: Line 214:
 
! Scope
 
! Scope
 
|-
 
|-
| 143324
+
| 143406
| TEAM
+
| USER
| lhcb
+
| cms
 
| RAL-LCG2
 
| RAL-LCG2
| very urgent
+
| urgent
 
| NGI_UK
 
| NGI_UK
 
| solved
 
| solved
| 2019-09-20 14:31:00
+
| 2019-10-01 06:27:00
| File recreation canceled since the file cannot be routed to tape
+
| transfers failing to T1_UK_RAL_Disk
 
| WLCG
 
| WLCG
 
|-
 
|-
| 143269
+
| 143384
| ALARM
+
| TEAM
| none
+
| atlas
 
| RAL-LCG2
 
| RAL-LCG2
| top priority
+
| very urgent
 
| NGI_UK
 
| NGI_UK
| verified
+
| solved
| 2019-09-18 10:57:00
+
| 2019-09-25 22:08:00
| This TEST ALARM has been raised for testing GGUS alarm work flow after a new GGUS release.
+
| Low efficiency of Atlas transfers to sites in UK cloud
 
| WLCG
 
| WLCG
 
|-
 
|-
| 143231
+
| 143379
 
| USER
 
| USER
| other
+
| cms
 
| RAL-LCG2
 
| RAL-LCG2
 
| urgent
 
| urgent
| EGI CVMFS Service
+
| NGI_UK
 
| solved
 
| solved
| 2019-09-20 07:53:00
+
| 2019-09-26 06:40:00
| CVMFS repo dirac.egi.eu updates are not propagated
+
| issues with RAL FTS?
| EGI
+
| WLCG
 
|-
 
|-
 
| 143225
 
| 143225
Line 259: Line 258:
 
| WLCG
 
| WLCG
 
|-
 
|-
| 143218
+
| 143198
| TEAM
+
| USER
| lhcb
+
| cms
 
| RAL-LCG2
 
| RAL-LCG2
 
| urgent
 
| urgent
 
| NGI_UK
 
| NGI_UK
| solved
+
| closed
| 2019-09-24 13:45:00
+
| 2019-09-27 23:59:00
| FTS3 transfers problem to GRIDKA for transfers executing at RAL FTS3 server
+
| issues with RAL FTS?
 
| WLCG
 
| WLCG
 
|-
 
|-
| 142981
+
| 142689
 
| USER
 
| USER
| mice
+
| cms
 
| RAL-LCG2
 
| RAL-LCG2
| less urgent
+
| very urgent
 
| NGI_UK
 
| NGI_UK
| closed
+
| solved
| 2019-09-18 23:59:00
+
| 2019-10-01 15:33:00
| mice; LFC to DFC transition
+
| Transfer failing to RAL_Disk
| EGI
+
| WLCG
 
|-
 
|-
| 142835
+
| 140447
 
| USER
 
| USER
| snoplus.snolab.ca
+
| dteam
 
| RAL-LCG2
 
| RAL-LCG2
 
| less urgent
 
| less urgent
 
| NGI_UK
 
| NGI_UK
 
| solved
 
| solved
| 2019-09-18 12:52:00
+
| 2019-09-27 08:34:00
| Connection Issues
+
| packet loss outbound from RAL-LCG2 over IPv6
 
| EGI
 
| EGI
 
|}
 
|}
Line 315: Line 314:
 
! Comments
 
! Comments
 
|-
 
|-
| 2019-09-18
+
| 2019-09-25
 
| 100
 
| 100
| 100
+
| 87
| 100
+
| 92
| 43
+
| 81
 
|  
 
|  
 
|-
 
|-
| 2019-09-19
+
| 2019-09-26
 +
| 100
 
| 100
 
| 100
 
| 100
 
| 100
 
| 100
 
| 100
| 44
 
 
|  
 
|  
 
|-
 
|-
| 2019-09-20
+
| 2019-09-27
 +
| 100
 
| 100
 
| 100
 
| 100
 
| 100
 
| 100
 
| 100
| 96
 
 
|  
 
|  
 
|-
 
|-
| 2019-09-21
+
| 2019-09-28
 
| 100
 
| 100
 
| 100
 
| 100
Line 343: Line 342:
 
|  
 
|  
 
|-
 
|-
| 2019-09-22
+
| 2019-09-29
 
| 100
 
| 100
 
| 100
 
| 100
Line 350: Line 349:
 
|  
 
|  
 
|-
 
|-
| 2019-09-23
+
| 2019-09-30
 
| 100
 
| 100
 
| 100
 
| 100
Line 357: Line 356:
 
|  
 
|  
 
|-
 
|-
| 2019-09-24
+
| 2019-10-01
 +
| 100
 
| 100
 
| 100
| 99
 
 
| 100
 
| 100
 
| 100
 
| 100
Line 381: Line 380:
 
! Day !! Atlas HC !! CMS HC !! Comment
 
! Day !! Atlas HC !! CMS HC !! Comment
 
|-
 
|-
| 2019-09-25 || 88 || 100 ||  
+
| 2019-09-25 || 89 || 100 ||  
 
|-
 
|-
| 2019-09-26 || 86 || 100 ||  
+
| 2019-09-26 || 100 || 100 ||  
 
|-
 
|-
| 2019-09-27 || 100 || 100 ||  
+
| 2019-09-27 || 89 || 100 ||  
 
|-
 
|-
 
| 2019-09-28 || 100 || 100 ||  
 
| 2019-09-28 || 100 || 100 ||  
Line 391: Line 390:
 
| 2019-09-29 || 100 || 100||  
 
| 2019-09-29 || 100 || 100||  
 
|-
 
|-
| 2019-09-30|| 100|| 100||  
+
| 2019-09-30|| 92|| 100||  
 
|-
 
|-
| 2019-10-01 || 86 || 97 ||  
+
| 2019-10-01 || 100 || 100 ||  
 
|-
 
|-
 
|}  
 
|}  

Latest revision as of 09:53, 2 October 2019

RAL Tier1 Operations Report for 2nd October 2019

Review of Issues during the week 25th September 2019 to the 1st October 2019.
  • IPv6 packet loss on SuperJanet solved with network intervention.
Current operational status and issues
Notable Changes made since the last meeting.
  • NTR
Entries in GOC DB starting since the last report.
Service ID Scheduled? Outage/At Risk Start End Duration Reason
Declared in the GOC DB
Service ID Scheduled? Outage/At Risk Start End Duration Reason
- - - - - - - -
  • No ongoing downtime
Advanced warning for other interventions
The following items are being discussed and are still to be formally scheduled and announced.


Listing by category:

  • DNS servers will be rolled out within the Tier1 network.
Open GGUS Tickets
Ticket-ID Type VO Site Priority Responsible Unit Status Last Update Subject Scope
143402 USER none RAL-LCG2 urgent NGI_UK in progress 2019-09-30 12:23:00 CVMFS IPv6 connection issues at RAL EGI
143387 USER snoplus.snolab.ca RAL-LCG2 less urgent NGI_UK in progress 2019-10-01 10:29:00 Transfer issues to RAL EGI
143323 TEAM lhcb RAL-LCG2 top priority NGI_UK in progress 2019-09-27 15:20:00 File deletion at RAL ECHO WLCG
142350 TEAM lhcb RAL-LCG2 top priority NGI_UK in progress 2019-09-18 14:09:00 Proble accessing some LHCb files at RAL WLCG



GGUS Tickets Closed Last week


Ticket-ID Type VO Site Priority Responsible Unit Status Last Update Subject Scope
143406 USER cms RAL-LCG2 urgent NGI_UK solved 2019-10-01 06:27:00 transfers failing to T1_UK_RAL_Disk WLCG
143384 TEAM atlas RAL-LCG2 very urgent NGI_UK solved 2019-09-25 22:08:00 Low efficiency of Atlas transfers to sites in UK cloud WLCG
143379 USER cms RAL-LCG2 urgent NGI_UK solved 2019-09-26 06:40:00 issues with RAL FTS? WLCG
143225 USER cms RAL-LCG2 very urgent NGI_UK verified 2019-09-25 06:04:00 some of RAL FTS servers are not running? WLCG
143198 USER cms RAL-LCG2 urgent NGI_UK closed 2019-09-27 23:59:00 issues with RAL FTS? WLCG
142689 USER cms RAL-LCG2 very urgent NGI_UK solved 2019-10-01 15:33:00 Transfer failing to RAL_Disk WLCG
140447 USER dteam RAL-LCG2 less urgent NGI_UK solved 2019-09-27 08:34:00 packet loss outbound from RAL-LCG2 over IPv6 EGI


Availability Report

Day Atlas CMS LHCB Alice Comments
2019-09-25 100 87 92 81
2019-09-26 100 100 100 100
2019-09-27 100 100 100 100
2019-09-28 100 100 100 100
2019-09-29 100 100 100 100
2019-09-30 100 100 100 100
2019-10-01 100 100 100 100
Hammercloud Test Report
Target Availability for each site is 97.0%
Day Atlas HC CMS HC Comment
2019-09-25 89 100
2019-09-26 100 100
2019-09-27 89 100
2019-09-28 100 100
2019-09-29 100 100
2019-09-30 92 100
2019-10-01 100 100

Key: Atlas HC = Atlas HammerCloud (Queue RAL-LCG2_UCORE, Template 841); CMS HC = CMS HammerCloud

Notes from Meeting.