Sections

About GridPP

Documents

Activities

Management

Wiki

Help Pages

Data Management Activity in GridPP Phase 1

Information on current data management work can be found here.

Spitfire

As part of the European Data Grid (EDG) project, a technology demonstrator service called Spitfire was developed to meet the challenge for managing metadata. There are a variety of papers and presentations discussing the progression of Spitfire and its web based technologies. The code and documentation for the Spitfire service can be obtained from the Spitfire Webpage. The Spitfire product has been a good testing ground for the technologies that we subsequently developed with the EDG Data Management group for the other replica management services.

We have also been involved in the Data Access and Integration Services (DAIS) working group of the Global Grid Forum to ensure that the Particle Physics use-cases are being adequately dealt with in this forum, such that we are in a position to adopt these standards when they become fully available.

Grid Query Optimisation

PPE Analysis jobs running in a grid environment will access a large number of data files and the majority of these files may not be local to the system on which the job is running. Therefore there is considerable scope for optimising the way in which the Grid will access these files for the end user. The purpose of this optimisation is two-fold:
  • to speed up the individual analysis job for the end user, so that they have to wait for a minimum of time for the results to be returned;
  • to increase the global throughput of the Grid, ensuring that all available hardware and grid resources are being utilised at maximum efficiency.
One method by which an optimum condition can be reached is by selectively replicating the data files around the grid so that they are available from more than location. Since there is only a finite amount of disk space and network capacity available, choosing exactly which files to replicate to achieve optimum usage can be very difficult. There are experimental optimisation algorithms being developed, and, in order to test and evaluate these, a data-centric grid simulation program (OptorSim) has been developed at Glasgow. The simulation code and results are available on the OptorSim web page. There are many papers, presentations and posters describing the OptorSim work available from the ScotGrid website.

We have applied some of the more basic optimisation work in the ROS (Replica Optimisation Service) service of the EDG's replica management system which implements the functionality of getBestFile, allowing the Grid to decide which replica to fetch for you based upon live network and storage monitoring statistics.

Useful Links

Last modified Wed 20 July 2005 . View page history
Switch to HTTPS . Website Help . Print View . Built with GridSite 1.4.3
For more about GridPP please contact Neasan O'Neill