Difference between revisions of "New Information System"
(→The JSON Standard) |
|||
(3 intermediate revisions by one user not shown) | |||
Line 5: | Line 5: | ||
== What's wrong with the current information system == | == What's wrong with the current information system == | ||
− | The current BDII system in use all over the grid has some problems. The schema is bloated. The information content is huge | + | The current BDII system in use all over the grid has some problems. The schema is bloated. The information content is huge. The software client implementation is complicated and weirdly written. There is ambiguity in the field semantics. For most users it is hard to query by hand. It is also hard for site admins to configure it in the first place and to keep it current on an on-going basis. The data content, when checked, is often unreliable. BDII clients regularly lock up or fall over. It is hard to verify. It is the source of many errors. It is difficult to extend for new requirements. In short, the BDII has had its day and it’s time to try something else. |
== Principles of operation == | == Principles of operation == | ||
Line 21: | Line 21: | ||
|} | |} | ||
− | A list of all sites’ | + | A list of all sites’ jsons can be obtained by querying the GOCDB in (say) a browser. |
https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&extensions=(InformationSystem=) | https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&extensions=(InformationSystem=) | ||
− | A standard exists for the JSON document that each site must create and deploy on some web-server. At present, the initiative covers site CEs, but it will later be | + | A standard exists for the JSON document that each site must create and deploy on some web-server. At present, the initiative covers site CEs, but it will later be extended to cover storage as well. |
− | For example, at Liverpool, we have installed a web-server on hepgrid4.ph.liv.ac.uk (which is also our BDII server, as it happens). Information consumers who wish to know about the Liverpool site will access the GOCDB, obtain the InformationSystem URL and access the information within the JSON. | + | For example, at Liverpool, we have installed a web-server on hepgrid4.ph.liv.ac.uk (which is also our BDII server, as it happens). Information consumers who wish to know about the Liverpool site will access the GOCDB, obtain the InformationSystem URL and access the information within the JSON. |
== The JSON Standard == | == The JSON Standard == | ||
Line 34: | Line 34: | ||
https://docs.google.com/document/d/1pg_5Kibc_-Z4JF4_HJyW5xL6GVYKwXxOU7DXf2QP9Ag/edit | https://docs.google.com/document/d/1pg_5Kibc_-Z4JF4_HJyW5xL6GVYKwXxOU7DXf2QP9Ag/edit | ||
+ | (Note that everything in the document after "Question:" is for historical reasons only.) | ||
== Various other deployment options == | == Various other deployment options == | ||
Line 39: | Line 40: | ||
TDB | TDB | ||
− | Deadlines/timelines for implementation | + | == Deadlines/timelines for implementation == |
TBD | TBD |
Latest revision as of 16:23, 21 February 2019
Contents
Introduction
This document describes a JSON static information system, that serves to replace parts of the BDII. See WLCG Information System Evolution
What's wrong with the current information system
The current BDII system in use all over the grid has some problems. The schema is bloated. The information content is huge. The software client implementation is complicated and weirdly written. There is ambiguity in the field semantics. For most users it is hard to query by hand. It is also hard for site admins to configure it in the first place and to keep it current on an on-going basis. The data content, when checked, is often unreliable. BDII clients regularly lock up or fall over. It is hard to verify. It is the source of many errors. It is difficult to extend for new requirements. In short, the BDII has had its day and it’s time to try something else.
Principles of operation
Site admins will be familiar with GOCDB, since each site must already have an entry describing it. A site entry in GOCDB has the idea of Extension Properties; an extension property is some name/value pair that can be used for arbitrary purposes. In this scheme, a site would enter an extension property named "InformationSystem". This extension property would have a value comprising a URL that links to a JSON document containing a description of the site in question. For example, the Liverpool site (UKI-NORTHGRID-LIV-HEP) has this entry:
Extension Property Name | Value |
---|---|
InformationSystem | http://hepgrid4.ph.liv.ac.uk/liv.json |
A list of all sites’ jsons can be obtained by querying the GOCDB in (say) a browser.
https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&extensions=(InformationSystem=)
A standard exists for the JSON document that each site must create and deploy on some web-server. At present, the initiative covers site CEs, but it will later be extended to cover storage as well.
For example, at Liverpool, we have installed a web-server on hepgrid4.ph.liv.ac.uk (which is also our BDII server, as it happens). Information consumers who wish to know about the Liverpool site will access the GOCDB, obtain the InformationSystem URL and access the information within the JSON.
The JSON Standard
For the time being, the standard for the JSON file (or files, if more than one is used) is available here.
https://docs.google.com/document/d/1pg_5Kibc_-Z4JF4_HJyW5xL6GVYKwXxOU7DXf2QP9Ag/edit
(Note that everything in the document after "Question:" is for historical reasons only.)
Various other deployment options
TDB
Deadlines/timelines for implementation
TBD
Further reading
GOCDB documentation can be found here:
https://wiki.egi.eu/wiki/GOCDB/PI/Technical_Documentation
The JSON schema was partially based on this proposal, which is now superceded:
https://twiki.cern.ch/twiki/pub/EGEE/WLCGISEvolution/CE-json-proposal.txt
There is a US standard here:
https://github.com/opensciencegrid/topology/blob/master/topology/Brookhaven%20National%20Laboratory/Brookhaven%20ATLAS%20Tier1/BNL-ATLAS.yaml