RAL Tier1 weekly operations castor 29/09/2014
From GridPP Wiki
Contents
Operations News
- Plan to ensure PreProd represents production in terms of hardware generation are underway
- Disk server redeployments continue (i.e. D1T0 reused in D0T1 etc) ... 5 servers in LHCb left
- SL6 Headnode work progressing well - hoping for rollout in Nov
- xrootd security advisory with FAX component within xrootd
- useful breakout sessions at Castor face to face - deadlock analysis & bugs confirmed, discussions to simplify headnode configurations and remove a single point of failure
Operations Problems
- Atlas load, high wait I/O on several servers – partly understood and working towards some config changes
- Gdss720 crashed - currently draining to take out of production
- shellshock – latest errata installed on preprod to stress test
Blocking Issues
- grid ftp bug in SL6 - stops any globus copy if a client is using a particular library. This is a show stopper for SL6 on disk server.
Planned, Scheduled and Cancelled Interventions
- Juan to patch castor dbs starting next week (PSU patches) – standard change
- 2.1.14-14 testing in preprod
- A Tier 1 Database cleanup is planned so as to eliminate a number of excess tables and other entities left over from previous CASTOR versions. This will be change-controlled in the near future.
Advanced Planning
Tasks
- Possible future upgrade to CASTOR 2.1.14-15.
- Switch from admin machines: lcgccvm02 to lcgcadm05
- New VM configured to run against the standby CASTOR database will be created as a front-end for dark data etc queries.
- Replace DLF with Elastic Search
- Correct partitioning alignment issue (3rd CASTOR partition) on new castor disk servers
Interventions
- None
Staffing
- Castor on Call person
- Chris
- Staff absence/out of the office:
- Brian – Out all week
- Bruno - Out all week