Difference between revisions of "Tier1 Operations Report 2012-08-29"

From GridPP Wiki
Jump to: navigation, search
 
(No difference)

Latest revision as of 09:34, 29 August 2012

RAL Tier1 Operations Report for 29th August 2012

Review of Issues during the week 23rd to 29th August 2012
  • On Thursday 22nd August there was a break in the RAL-CERN backup link at Frankfurt. It was not service affecting. There is a temporary fix in place and a permanent fix is planned.
Resolved Disk Server Issues
  • None
Current operational status and issues
  • On 12th/13th June the first stage of switching ready for the work on the main site power supply took place. The work on the two transformers is expected to take until 18th December and involves powering off one half of the resilient supply for 3 months while being overhauled, then repeat with the other half. The work is running to schedule. In particular one half of the new switchboard has been refurbished and is on track to be brought into service by 17 September. Once this is operational then RAL will be switched over to using it and will no longer be dependent on the old switchgear.
Ongoing Disk Server Issues
  • None
Notable Changes made this last week
  • CIP upgraded to 2.2.12-1 on Tuesday 28th.
  • Castor Repack upgraded to 2.1.12-10 on Thursday 23rd and Friday 24th.
  • Continuing test of hyperthreading, one batch of worker nodes (the Dell 2011 batch) has number of jobs increased further (this week from 22 to 24).
  • As stated before: CVMFS available for testing by non-LHC VOs (including "stratum 0" facilities).
  • A test queue ("gridTest") is available with (currently) four worker nodes running EMI2/SL5.
Declared in the GOC DB
  • None
Advanced warning for other interventions
The following items are being discussed and are still to be formally scheduled and announced.

Listing by category:

  • Databases:
    • Switch LFC/FTS/3D to new Database Infrastructure.
  • Castor:
    • Migration of LHCb data from "A" to "C" tape media.
    • Upgrade to version 2.1.12.
  • Networking:
    • Install new Routing layer for Tier1 and update the way the Tier1 connects to the RAL network. (Plan to co-locate with replacement of UKlight network).
    • Update Spine layer for Tier1 network.
    • Replacement of UKLight Router.
    • Addition of caching DNSs into the Tier1 network.
  • Grid Services:
    • Updates of Grid Services as appropriate. (Services now on EMI/UMD versions unless there is a specific reason not.)
  • Infrastructure:
    • Intervention required on the "Essential Power Board". (Following further checks this should not require a power outage in UPS room, but should be an "At Risk").
    • Remedial work on three (out of four) transformers. Will require two "At Risk" periods.
    • Remedial work on the BMS (Building Management System) due to one its three modules being faulty. Will require a further “At Risk”.


Entries in GOC DB starting between 15th and 22nd August 2012

There were no Scheduled or Unscheduled entries in the GOC DB for this period.


Open GGUS Tickets
GGUS ID Level Urgency State Creation Last Update VO Subject
85438 yellow Less Urgent In progress 2012-08-28 2012-08-28 Atlas FTS errors from SRM srm-atlas.gridpp.rl.ac.uk
85077 yellow Less Urgent In progress 2012-08-13 2012-08-24 biomed CE lcgce05.gridpp.rl.ac.uk job cannot register file on SE srm-biomed.gridpp.rl.ac.uk
85023 Red Less Urgent Waiting Reply 2012-08-09 2012-08-10 SNO+ WMS
84492 Red Urgent In Progress 2012-07-24 2012-08-17 SNO+ Job time/memory requirements not provided
84408 Red Very Urgent In Progress 2012-07-20 2012-08-20 neurogrid Enable neurogrid.incf.org on WMS and LFC
68853 Red Less Urgent On hold 2011-03-22 2012-07-30 N/A Retirenment of SL4 and 32bit DPM Head nodes and Servers