12.09.22
Present: David Britton (Chair), Tony Cass, Peter Clarke, David Colling, Davide Costanzo , Alastair Dewhurst, Tony Doyle, Katy Ellis, Alessandra Forti , Peter Gronbech, Roger Jones, David Kelsey, Steve Lloyd, Andrew McNab, Andrew Sansum, Sam Skipsey (Minutes), Jill Sambrook (Minutes) ,Vivien Hamilton
Apologies: Jonathan Hays
Items:
1) Any comments/feedback on GridPP48 Ambleside
The Collaboration meeting in Ambleside was given positive feedback by attendees and it was noted the livestream also went smoothly (previously had some technical difficulties in this venue). No talk of moving this elsewhere for future meetings.
DB noted the agenda was quite full and perhaps more discussion could be arranged for future meetings, so this is something to keep in mind for GridPP49 and GridPP50 next year.
Thanks to Sam and Jill (and Roger) for arranging the agenda and event content and with the organisation throughout.
Brief discussions around GridPP49. Pitlochry (Atholl Palace hotel) could be one option, but seems to have increased in price and may now be out of reach. Further investigation required.
AD has been in touch with Cosner’s House to ask for availability for some dates end of March into April. He is hoping to get a list of potential options for hosting the event there. It was noted the group would need to consider teaching end dates and other conferences before finalising date and venue.
Also, option of co-location should be considered.
SS, DB and VH will plan to meet and discuss organisation and plans for GridPP49.
2) Pledge sign-off
DB circulated a document to the group prior to the meeting regarding pledge adjustments for 2023. This was a continuation of discussions held at GridPP48. This document is due to be submitted this week (A couple of weeks earlier than normal).
The group discussed various points in the document and it was agreed DB would tidy/correct any numbers and figures and send to AD to check tier 1 pledges by Wednesday 14th. Tier 2 pledges would also be written up and would tie into discussion held in Ambleside.
DB will circulate the document to AD and the group and wait for any comments by Wednesday. This will then be submitted and DB will close this item.
Standing items:
SI-0 Tier-1 Manager’s weekly report & Technical Meetings [AD]
Alastair sent round an email with the Tier-1 Managers Report.
- Technical Meetings
I Will be restarting these. We have a perfSonar meeting on the 23rd September (https://indico.cern.ch/event/1190370/). We also need to arrange one on Rocky 8 (now that the Tier-1 has migrated), Rucio / DIRAC integration and tokens. - Tier-1 Operations
In the last couple of weeks starting just before GridPP48 there had been a few separate incidents of Echo gateways having problems (hardware) and causing transfers and test failures that affected most VOs. These appear to have been resolved and several tickets will be closed. We will be keeping a close eye on things in case any of the problems had other causes.
On Thursday we upgraded one of our production Echo gateways to XRootD 5.5.0. It has been running over the weekend without any issues and so the upgrade will start to be rolled out more widely. Once this is done, we will enable the configuration change that stops the deletes being serialised.
Tier-1 Batch farm has now completely migrated to HTCondor 9 and Rocky 8.
Katy is planning on running a mini data challenge against Antares in the next few weeks.
There is a long running ticket with LHCb on Webdav transfers (https://ggus.eu/index.php?mode=ticket_info&ticket_id=156492). Historically, every time LHCb have switched to Webdav they have run into operational problems. We have had a fairly extensive period of stable running with Webdav (and when there have been problems, we have identified clear causes). There is also significantly more hardware available than ~6 months ago. We are therefore planning on reaching out to Chris Haen shortly to arrange a time for a (hopefully) successful transition to Webdav. - Procurement
Procurement update circulated to the group including delivery times and costs.
PC asked for an update on LHCONE and IPv6 routing for the LHCONE meeting in 5/6 weeks.
AD agreed to ask James A for input for this meeting.
SI-1 ATLAS Weekly Review and Plans [DCos]
DCos had to depart meeting a little early.
RJ gave a brief update.
Some Rucio host certificates expired over the weekend. This caused a few issues, but was well recovered in the end.
There were also some RAL failures earlier in the week. Once this was restarted it was fairly stable.
It was noted that the IGTF certificate bundle / ARGUS “commas in DNs” issue had hit some site availability until updates could be reverted [and currently some sites not updating ARGUS to avoid hitting this issue]
SI-2 CMS Weekly Review and Plans [DC/KE]
-KE mentioned the Webdav issue at RAL Tuesday and Wednesday (previously discussed by AD). KE overrode that status and everything looked much better from Thursday
-KE Starting Tape test (read only test) Thursday/Friday this week to check improvement over previous CMS challenge, and will update over this next week.
SI-3 LHCb Weekly Review and Plans [AM]
AM asked for clarity around the WebDav issue. Chris Haen LHCb had raised a ticket. He noted that while everything looked a little better, there were still some issues.
DB asked if this had now fully been resolved and it was unclear whether there might still be a few underlying issues – communication on the ticket could be better.
It was suggested the new Liaison, scheduled to start on the 26th of September, could try and look into this issue further and ensure it is now all sorted.
SI-4 Operations Meeting Report [SS,PG,PC] –
It was noted that sites were encouraged to support the EUCLID VO [now that their work is moving from the GridPP VO which is supported everywhere], but the aforementioned IGTF certs / ARGUS issue meant that some sites were waiting for ARGUS fix to be available and tested before making config changes necessary.
SI-5 LCG Management Board Report of Issues [DB]
NTR
SI-6 External Contexts (eg NGI/EGI) [PC/JH]
PC – EGR meeting today.
REVIEW OF ACTIONS
800.5 AD – Arrange in person DIRAC/Rucio meeting at IC (Jan22). [on-going] – Plans to re-establish virtual meetings and then set up an in person meeting. AD to ask Tim Noble if he can arrange this.
818.5 – DB/SS Group to formalise VOs to be added to the approved list. Ongoing
DB and SS have begun to look into this. 3 in particular were flagged as perhaps needing to be removed.
MaGrid (Morocco institute)
ComplexSystems
Raised with TB-Support for site feedback on who supports these and SS will wait until he has some feedback about use.
Discussion also around Microboone.
Close to closing this off.
SS has had a couple of talks with Gerard. We still need to decide if we are going to approve MaGrid and SS needs to find out if ComplexSystems needs to be removed.
Dave Colling to “investigate Complex System VO use at IC – adding as a new action.
823.1 – AD to Invite Rob to the Wednesday Liaison meeting for a few weeks to discuss LHCb tickets and leave as a standing item at the start of the agenda for a few weeks.
824.1 – SS will take lead on OsC action:
Collaboration to provide brief, narrative information on data recovery processes in the report for the next OsC meeting.
Ongoing
824.2 – DK and AD to draft a short narrative addressing OsC action:
Collaboration to provide narrative in the OsC report on the process that is used to determine the networking costs for GridPP6 for the next OsC meeting
DB asked for there to be a brief statement for GridPP6 with more details and plan for GridPP7.
DK and AD can send draft to DB.
Ongoing
826.2 – SS to put together a few slides on technical situation around Token Transition. These could then be used at either Ambleside or a specific technical meeting about the subject.
SS to also discuss with AD possibility of technical meeting on this subject.
Ongoing.
827.1 – DC to investigate Complex System VO use at IC
828.1 – RJ to ask the CB for interest in hosting GridPP49 [interest by end of Sept] so we can plan this in good time if RAL has competition for hosting.
828.1 – PC + AM to bring up the issue of Vladmir’s cert being the one that LHCb pilots use.
828.3 – Expt Reps (DCos, DC, AM) to report back on discussions with Experiment PIs on policy for use of Russian resources/availability of resources to Russia.
828.4 – All: email DB with suggestions for International Reviewers to suggest. (GridPP7 proposal)
828.5 – DB to add Security to Strategic Objectives in GridPP7 proposal.
828.6 – PC to consider WP4 content and how to refactor for GridPP7 proposal.