Production Team Report 2011-02-07

From GridPP Wiki
Jump to: navigation, search

RAL Tier1 Production Team Report for 24th January 2011.

AoD This Week

Mon: John Tue: Gareth Wed - Thu: John Fri: Gareth

Last week

  • Gareth: AoD (2 days); Continuing Post Mortem work.
  • John: A/L (2 days); Looking at black hole WN detection;
  • Tiju: AoD (3 days); Monitoring updates (Puppetmaster02); NetFlow investigation

Changes to Operating procedures

  • None

Declared Outages in GOC DB

  • Monday 7th Feb: Castor GridFTP upgrade to fix checksums
  • Tuesday 8th Feb: WAN tuning - CMS WAN pools.
  • Wednesday 9th Feb: Oracle 10.2.0.5 upgrade for LFC, FTS & 3D.

Advanced Warning - dates provisional

  • Tuesday 15th Feb. Upgrade GEN Disk Servers to 64-bit.
  • Tuesday 15th Feb. WAN Tuning - rest of CMS.
  • Wednesday 16th Feb: FTS agent Quattorisation (part 1)
  • Tuesday 1st March: WAN Tuning - everything (except CMS which is done)
  • Tuesday 1st March: Castor Nameserver upgrade
  • Monday 28th March: Castor Atlas Upgrade (2.1.9-10 OR 2.1.10-0)
  • Tuesday 29th March: Castor CMS Upgrade (2.1.9-10 OR 2.1.10-0)
  • Wednesday 30th March: Castor LHCb & GEN Upgrade (2.1.9-10 OR 2.1.10-0)
  • Thursday 31st: - Switch castor to new database in R26

Other Changes

  • Fabric:
    • Swap out the older of the pair of SAN switches in the Tier1 Oracle databases for its new replacement. (Requires FTS, LFC, 3D stop).
  • Database:
    • Re-visit non-Castor database multipathing
    • Increase shared memory for LUGH & SOMNUS.
  • Grid Services:
    • Changes to increase resilience of the BDII service
  • Castor:
    • Change ATLAS castor permissions to prevent users deleting data
  • Networks:
    • Firmware updates for central networking components (likely to have some short network breaks - maybe in March)