| General updates
|
Monday 20th April
Tuesday 14th April
- There was a WLCG workshop over the weekend in Okinawa: agenda.
- CHEP has started. Many interesting talks. Timetable/Talks. Will review at GridPP34.
- GridPP34 agenda is developing.
- GOCDB has a number of Service Types that appear to be unused. We need to confirm for the following list: OpsTool; dg.TargetSystemFactory; dg.ARC-CE; eu.egi.cloud.broker.proprietary.slipstream; globus-RLS; RGMA-IC; eu.egi.egran; SRM.online; egi.VODashboard; CUSTOM.egi.HTTPserver; egi.NetworkPortal; ch.cern.cvmfs.stratum.1; CUSTOM.pl.plgrid.Bazaar.
- User Community DIRAC meeting that will take place in May. Survey in progress for all communities who use DIRAC. Let Jeremy know if you need the URL.
- A potentially useful LHCb monitoring page.
- Reminder of Tom's VO cleanup campaign.
- Quarterly reports for Q115 now due.
- March 2015 WLCG A/R results circulated:
- ALICE. No problems.
- ATLAS. Brunel: 85%:96%; QMUL: 62%:62%; UCL: 27%:27%; Sussex: 39%:39.
- CMS Brunel: 86%:96%
- LHCb. All good.
Tuesday 7th April
- Proton beams are back in the LHC
- DIRAC (framework) users survey circulated.
- ATLAS: Reprocessing of full 2012 dataset - resources should have been busy over the Easter weekend.
- Training Course, run by the authors, on ISO 16363 for Auditors and Managers of Digital Repositories - CERN 15th-19th June.
Tuesday 31st March
- Winnie has an issue in that 'SL5 (works) vs SL6 (not) emi-3 UI'.
- There is a DPM Collaboration Board this Tuesday.
- The GridPP34 agenda is now being created; are there specific topics anyone wishes to put forward?
- NGI-FR report Argus cannot run with the last version of the Java JDK which requires argus-pap-1.6.2-1.el6.noarch.rpm only available in EMI not UMB repos. (See tickets 111505 and 112606).
- A look through the HEPiX talks given last week is recommended.
- There was an EGI OMB last Thursday 26th March: agenda.
- Operations
- Request to register iRODS endpoints in GOCDB.
- Advisory groups being created for ops and accounting portals.
- Some EGI/NGI ARGUS servers are 'exposed'. Please check. Also NGI Argus service type should be "emi.argus" not "ngi.argus".
- Plans to use ARGO for monitoring of central series
- Updated technology provider UA (please review)
- Reminder of EGI conference 18th-22nd May.
- Security: Some FedCloud security packaged defined; OpenSSL Debian patch impacted VOMS API
- Some RODS are re-enabling suspended sites without fixing them! Process change in order.
- Action: Comment on change to make it mandatory for sites to download and run Pakiti on highlighted WNs. (see slides)
- EGI-Engage update: started in March. New platforms and business models being explored. See slides for details.
- Developing the infrastructure: includes plans for HPC provisioning, data preservation and a storage federation pilot. Testbed to explore areas such as persistent IDs.
- EGI Commons: Talk explained the new architecture and how core services will evolve. EGI Engage WP3 will meet in Amsterdam 14th-15th Arpil.
- Survey of core activities: In progress till 24th April.
|
| WLCG Operations Coordination - Agendas
|
Tuesday 14th April
Monday 6th April
- There was a WLCG ops coordination meeting on Thursday 2nd April: Agenda | Minutes.
- News: Ops survey responses reviewed. Results overview available.
- Data Preservation Training: Need more WLCG sites involved to justify central funding. See slides.
- Baselines: missing ARGUS PAP 1.6.2 in UMD.
- MW Issues: NTR
- T0 & T1 services: CERN CASTOR upgrades 6th and 7th April. T1s various SE upgrades.
- T0 news/T1/T2 feedback: NTR
- ALICE: High activity. Most VO Boxes now use RFC proxies. Various CERN CASTOR instabilities. CRL URL being firewalled for IPv6.
- ATLAS: Cosmics export smooth. Grid should be full. Triumf-RAL link problems 50kB/s. WhiteHat challenge comments.
- CMS: Cosmic run finished. T1s and T2s busy. Problems with frontier/squid infrastructure. Scheduling (conflict) of SAM tests. T1 tape stage tests done.
- LHCb: NTR
- gLExec: 61/94 panda site coverage. Pilot to be enhanced - RAL/RALPP results to be investigated.
- SHA-2: Old VOMS servers to be retired (kept to forward user requests). Exceptions (till 5th May) to stop lcg-voms.cern.ch queries hanging. Propose to remove aliases on 27th April.
- Machine/Job features: NTR
- M/W readiness WG: Setup working well - and finding issues. Actions list updated. Next meeting 6th May. Need to clarify ECDF DPM contribution.
- Multicore: NTR.
- IPv6: FTS3 testbed operational (KIT & IC). 8 sites now activated IPv6. OSG testing glideinWMS. There is an IPv6 dualstack mesh.
- Squid/HTTP proxy: NTR
- Network & transfers WG: CVE released for cassandra. PS 3.4.2 released - anyone at 3.4.1 encouraged to check disk space and enable auto-update. Full mesh trace paths now at 80%. Network performance follow-up proposal
- New mailing list and GGUS SU will be established to follow up (wlcg-network-throughput)
- Experiments can report to the GGUS SU/mailing list potential network performance incidents/degradations, WLCG perfSONAR support unit will investigate and confirm if this is a network related issue.
- Affected sites will be contacted. Sites observing a network performance problem should follow their standard procedure, i.e. report to their network team and if necessary escalate to their network provider while informing the wlcg-network-throughput mailing list.
- Issues will be tracked in the twiki.
- Next WG meeting 8th April.
- HTTP deployment: First meeting will be on 29th April.
- Next meeting 23rd April.
|
| Tier-1 - Status Page
|
Tuesday 21st April
- Castor update to 2.1.14-15 successfully took place on 15th April.
- There will be an interruption to the LFC on Tuesday next week (28th April) for an update to Oracle on the back end database.
- We will turn off our remainining CREAM CEs on 5th May.
- The problems with our primary network router are still being followed up.
|
| Storage & Data Management - Agendas/Minutes
|
Wednesday 15 April
- Backing up data from DiRAC to GridPP (tape)
- More case studies on supporting non-LHC VOs on GridPP: we have a lot of great stuff that can do great stuff - non-LHC VOs tend to have less regimented data models so maybe we need more case studies.
Wednesday 8 April
Tuesday 7th April
- There was a DPM collaboration meeting last week. (Blog update)
Tuesday 31st March
Tuesday 17th March
- An annual DPM collaboration board will take place in coming weeks. Are there issues that sites want raised (in relation to things like roadmap, concerns about DPM, requests etc.)?
|
| Accounting - UK Grid Metrics HEPSPEC06 Atlas Dashboard HS06
|
Tuesday 14th April
- APEL delays for UCL; Sheffield; RALPP and Bristol
Tuesday 10th March
- Possible delay at Birmingham. Worth checking.
Tuesday 24th February
- UKI-SOUTHGRID-SUSX now highlighted as not publishing for 3 months.
|
| Documentation - KeyDocs
|
See the worst KeyDocs list for documents needing review now and the names of the responsible people.
Tuesday 17th March
- Documents reviewed at core-ops meeting last week. Agreed to remove 4 documents and add 2 (tarball status).
Tuesday 3rd March
- Proposals for about 4 keydocs to be removed/downgraded. For discussion at next core-ops or focus meeting.
Tuesday 24th February
- New section in Wiki called "Project Management Pages".
The idea is to cluster all Self-Edited Site Tracking Tables
in here. Sites should keep entries in Current Activities
up to date. Once a Self-Edited Site Tracking Tables has
served its purpose, PM to move it to Historical Archive
or otherwise dispose of the table.
|
| Interoperation - EGI ops agendas
|
Monday 9th March
- The agenda for February's EGI ops meeting is here. Minutes are here
- APEL 1.4.0
- Added Month and Year columns to primary key of CloudSummaries table in cloud schema.
- DPM-Xrootd 3.5.2 is in EPEL stable - this is the first version of the component compatible with xrootd4
- gLExec-wn - v. 1.2.3: lcmaps-plugins-c-pep 1.3.0-1 & mkgltempdir 0.0.5-1
- "The lcmaps-plugins-c-pep-1.3.0-1 preferably needs the argus-pep-api-c-2.3.0. This version will be released into EMI & UMD repositories in a near future."
- UMD 3.11.0 released on 16.02.2014, UMD 3.11.1 released on 4.03.2014
- lcg-CA 1.62 noted with an intention to broadcast these as they occur as opposed to monthly.
- EGI looking at the decommissioning of SL5, possibly by end of 2015, as a byproduct of adding CentOS 7 to UMD. NGIs to make a note if extended SL5 support is required.
- Vincenzo Spinoso has joined EGI Ops team from NGI_IT. Vincenzo will chair EGI Ops.
- Next meeting is April 20th.
|
| Monitoring - Links MyWLCG
|
Tuesday 31st March
Monday 7th December
|
| On-duty - Dashboard ROD rota
|
Monday 31st March
- Dashboard has been problematic - following recent UI changes. Alarms not appearing.
- ROD rota to be updated this week.
Monday 16th March
- Low availability alarm for UCL, but no alarm for the actual underlying cause (disk full on DPM).
- Ticket now issued.
- Kashif -> Daniela
|
| Rollout Status WLCG Baseline
|
Tuesday 17th March
- Daniela has updated the [ https://www.gridpp.ac.uk/wiki/Staged_rollout_emi3 EMI-3 testing table]. Please check it is correct for your site. We want a clear view of where we are contributing.
- There is a middleware readiness meeting this Wednesday. Would be good if a few site representatives joined.
- Machine job features solution testing. Fed back that we will only commence tests if more documentation made available. This stops the HTC solution until after CHEP. Is there interest in testing other batch systems? Raul mentioned SLURM. There is also SGE and Torque.
Tuesday 17th February
- A message from Maria Dimou: A request to you and the sites you collaborate with, to please install the MW Package Reporter.
The current version, documented in the wiki , it
is very easy to install and addresses all site and security requirements.
- Quick check on: 1. ARC CE readiness testing. 2. Machine job features testing.
References
|
| Security - Incident Procedure Policies Rota
|
Tuesday 7th April
- EGI Trust Anchor release 1.63-1 (see announcement 30th March).
Tuesday 31st March
Tuesday 17th March
- There will be a security team meeting this week. Are there any site issues to be raised?
- Note a new OpenSSL release is coming.
Monday 9th March
- Trust anchor release 1.62 (missing announcement!).
- From DG: "1.62 is indeed out and was released by the IGTF on Feb 23rd. The EGI release by an unfortunate coincidence got linked to a VOMS update for UMD (v3.3.3), so was released only today."
|
|
| Services - PerfSonar dashboard | GridPP VOMS
|
- This includes notifying of (inter)national services that will have an outage in the coming weeks or will be impacted by work elsewhere. (Cross-check the Tier-1 update).
Tuesday 31st March
Tuesday 10th March
- From the recent WLCG meeting, two slides (1 & 2) give the direction of the network monitoring and metrics progress: integration of perfSONAR event types into experiment monitoring and an architecture for data to get from RSV probes to client. Components described on slide 3.
- The next LHCOPN and LHCONE joint meeting will take place on Monday 1st and Tuesday 2nd of June 2015 in Berkeley (US) (hosted by LBL and ESnet).
|
| Tickets
|
Monday 20th April 2015, 14.30 BST
24 Open UK tickets this week, only a light review.
NGI/TIER 1
113150 (20/4)
Fresh in - the NGI has been ticketed to change the regional VO from emi.argus to ngi.argus in the gocdb. Seems a bit pedantic, but hey! I assigned it to the NGI ops, and notified RAL as keepers of the regional argus. Assigned (20/4)
TIER 1
113035 (14/4)
Just for people's interest, the ticket tracking the decommissioning of the last of the RAL CREAM CEs. In progress (14/4)
112819 (2/4)
A SNO+ ticket I must of somehow missed last week, concerning SNO+'s manual renewing of proxies on ARC machines. Matt M has noticed that ArcSync occasionally hangs rather then timeouts smoothly (although he later notes that he doesn't see the initial problems working from a different network). I'm thinking that this should be redirected at the arc devs, but I don't think they have a GGUS support group (I could be wrong, I'm well behind on the ARC curve). In progress (7/4)
EDINBURGH
113110 (17/4)
Looks like this atlas low transfer efficiency ticket can be closed. Waiting for reply (20/4)
GLASGOW
113095 (17/4)
ROD ticket for some BDII misreporting at Glasgow. The botheration seems to be ephemeral in nature, the blunders passing with the abating of their batch system's burden. This ticket can probably be solved. In progress (17/4)
|
| Tools - MyEGI Nagios
|
Tuesday 17th February
- Another period where message brokers were temporarily unavailable seen yesterday. Any news on the last follow-up?
Tuesday 27th January
- Unscheduled outage of the EGI message broker (GRNET) caused a short-lived disruption to GridPP site monitoring (jobs failed) last Thursday 22nd January. Suspect BDII caching meant no immediate failover to stomp://mq.cro-ngi.hr:6163/ from stomp://mq.afroditi.hellasgrid.gr:6163/
|
| VOs - GridPP VOMS VO IDs Approved VO table
|
Tuesday 31st March
- LIGO are in need of additional support for debugging some tests.
- LSST now enabled on 3 sites. No 'own' CVMFS yet.
Monday 9th March
- SIXT VOMS parameters have changed.
Tuesday 17th February
Two changes to approved VOs (https://www.gridpp.ac.uk/wiki/GridPP_approved_VOs)
- LSST uses port 15003 (had been 15002, clashing with dzero)
- t2k has included a note that it';s software is now distrinuted via CVMFS.
|
| Site Updates
|
Tuesday 24th February
- Next review of status today.
Tuesday 27th January
- Squids not in GOCDB for: UCL; ECDF; Birmingham; Durham; RHUL; IC; Sussex; Lancaster
- Squids in GOCDB for: EFDA-JET; Manchester; Liverpool; Cambridge; Sheffield; Bristol; Brunel; QMUL; T1; Oxford; Glasgow; RALPPD.
Tuesday 2nd December
- Multicore status. Queues available (63%)
- YES: RAL T1; Brunel; Imperial; QMUL; Lancaster; Liverpool; Manchester; Glasgow; Cambridge; Oxford; RALPP; Sussex (12)
- NO: RHUL (testing); UCL; Sheffield (testing); Durham; ECDF (testing); Birmingham; Bristol (7)
- According to our table for cloud/VMs (26%)
- YES: RAL T1; Brunel; Imperial; Manchester; Oxford (5)
- NO: QMUL; RHUL; UCL; Lancaster; Liverpool; Sheffield; Durham; ECDF; Glasgow; Birmingham; Bristol; Cambridge; RALPP; Sussex (14)
- GridPP DIRAC jobs successful (58%)
- YES: Bristol; Glasgow; Lancaster; Liverpool; Manchester; Oxford; Sheffield; Brunel; IC; QMUL; RHUL (11)
- NO: Cambridge; Durham; RALPP; RAL T1 (4) + ECDF; Sussex; UCL; Birmingham (4)
- IPv6 status
- Allocation - 42%
- YES: RAL T1; Brunel; IC; QMUL; Manchester; Sheffield; Cambridge; Oxford (8)
- NO: RHUL; UCL; Lancaster; Liverpool; Durham; ECDF; Glasgow; Birmingham; Bristol; RALPP; Sussex
- Dual stack nodes - 21%
- YES: Brunel; IC; QMUL; Oxford (4)
- NO: RHUL; UCL; Lancaster; Glasgow; Liverpool; Manchester; Sheffield; Durham; ECDF; Birmingham; Bristol; Cambridge; RALPP; Sussex, RAL T1 (15)
Tuesday 21st October
- High loads seen in xroot by several sites: Liverpool and RALT1... and also Bristol (see Luke's TB-S email on 16/10 for questions about changes to help).
Tuesday 9th September
- Intel announced the new generation of Xeon based on Haswell.
|
|
