RAL Tier1 CASTOR Accounting

From GridPP Wiki
Jump to: navigation, search

Accounting for CASTOR

RAL runs RAL Tier1 CASTOR SRM CASTOR as (one of several) interfaces to its storage robots, and CASTOR is in turn interfaced to the Grid via, obviously, a Storage Element (SE).

SEs publish information in the Glue SE Schema -- and this page requires a little familiarity with this schema. They publish which Storage Areas (SAs) are available, and how much space is available in each SA, how much is used, etc.

The information is used for

  • resource discovery and selection,
  • accounting,
  • monitoring.

Generally, a new service class is published as a storage area soon after it is configured - it needs to filter its way through a sequence of tests and be redeployed via quattor.

Update for CIP 2.X

CIP 2 provides nearline (tape) accounting; the implementation is easier to configure and maintain for the administrator; and it is close to and converging towards InstalledCapacity.

Most recent release can be found here: [1]

Published Information

Values being published are usually "UNDEFINED" if the CIP is not sure what to publish, or it is sure that nothing should be published (or "/UNDEFINEDPATH" for paths, as per GLUE 2.0)

Short description of the objects being published:


  • Architecture : tape (even if an SE had only Disk1Tape0, CASTOR is a tapestore)
  • SizeTotal : Sum of all SA.TotalOnlineSize and SA.TotalNearlineSize
  • SizeFree : Sum of all SA.FreeOnlineSize and SA.FreeNearlineSize
  • InformationServiceURL : empty ("UNDEFINED")
  • Port : 8443 (not used)
  • StateOfCurrentIOLoad : 0 (Not used)
  • ImplementationName : CASTOR
  • ImplementationVersion : Returns version of STAGER and NAMESERVER
  • Status : Production : Currently hardwired, since nothing relies on this variable. Future releases may treat this in a cleverer way.
  • TotalOnlineSize : Sum of SA.TotalOnlineSize for all SAs (in GB)
  • TotalNearlineSize : Sum of SA.TotalNearlineSize for all SAs (in GB)
  • UsedOnlineSize : Sum of SA.UsedOnlineSize for all SAs (in GB)
  • UsedNearlineSize : Sum of SA.UsedNearlineSize for all SAs (in GB)


Each SA is a service class. As of CIP 2.0, service classes can share disk pools properly, and tape pools, too. As regards the accounting information, the CIP will divide the dynamic data evenly between the service classes - this is a simple heuristic which will add up to the right numbers.

The SA.Path is set if and only if all VOs using the SA have the same path.

Loosely speaking, online accounting is calculated as follows:

  • Free and Total are queried from the stager database's view of the disk pools;
  • Used is set to the difference between Free and Total: this is not necessarily the correct volume of used space but it satisfies the InstalledCapacity requirement that Free plus Used be equal to Total;
  • Reserved is summed over all disk servers, including those that may be disabled, as per InstalledCapacity, but only if it is associated with a service class.

Nearline' accounting, as of versions 2.2.+, is now very different. It takes into account the actual filesize of the files on the tapes. It then calculates the compression ratio for each tape, and assumes that the free space capacity on the tape is compressed in the same ratio. The compression ratio for a tape is calculated as "sum of actual sizes of files written to tape" divided by "sum of sizes of files as stored on the tape itself" - for each tape. The data is then summed for each tape pool.

The "size" of a tape can then vary according to how much data is written on it. What happens when the file is deleted? It is (check this) removed from both the numerator and denominator in the compression ratio calculation but also removed from the "free" space (probably).

Free size is added only for tapes which are not full, disabled, or read-only (or similar).

Since service classes may share tape pools (and disk pools), and conversely, a tape pool may be accessed by more than one service class, tape (and disk) pools are treated internally as shares (cf GLUE2) and divided equally amongst their "parents" (ie service classes, or equivalently SAs), the divided value then being added to the SA's accounting data. Future extensions may allow non-equal split. We also divide the "reserved" space, despite the whole lot being arguably available for the VO - all the values are divided by the number of parents.

Also the Capability is published.

A future extension may allow non-divided data to be published in one of the slots. Accounting data is published in GlueStateUsedSpace and GlueStateAvailableSpace, in GlueSACapability, and in GlueSA*NearlineSize and GlueSA*OnlineSize. Not all of these are used for summing up; it may make sense to publish sensible data in one of them.


VOInfo objects are dynamically generated in CIP 2, so have dynamically generated names which may differ from one pass to another - but are of course fully consistent within each pass.

A VOInfo object is used to publish space token descriptions. When a space token is published, we also publish a VOInfo object without space tokens, to enable VOs to write into the SA without the space token. This feature can be disabled but only for everyone, not by individual VO (as of 2.1.0).