RAL Tier1 CASTOR Accounting

From GridPPwiki

Table of contents

1 Accounting for CASTOR 2 Update for CIP 2.X 2.1 Published Information 2.1.1 SE 2.1.2 SA 2.1.3 VOInfo

Accounting for CASTOR

Last updated 25 Jan 2010 (update in progress).

RAL runs RAL Tier1 CASTOR SRM CASTOR as (one of several) interfaces to its storage robots, and CASTOR is in turn interfaced to the Grid via, obviously, a Storage Element (SE).

SEs publish information in the Glue SE Schema -- and this page requires a little familiarity with this schema. They publish which Storage Areas (SAs) are available, and how much space is available in each SA, how much is used, etc.

The information is used for resource discovery and selection, accounting, and monitoring.

Update for CIP 2.X

CIP 2 provides nearline (tape) accounting; the implementation is easier to configure and maintain for the administrator; and it is close to and converging towards InstalledCapacity (https://twiki.cern.ch/twiki/bin/view/LCG/WLCGCommonComputingReadinessChallenges).

Most recent release can be found here: [1] (http://storage.esc.rl.ac.uk/grid-deployment/cip/)

Published Information

Values being published are usually "UNDEFINED" if the CIP is not sure what to publish, or it is sure that nothing should be published (or "/UNDEFINEDPATH" for paths, as per GLUE 2.0)

Short description of the objects being published:

SE

• Architecture : tape (even if an SE had only Disk1Tape0, CASTOR is a tapestore)
• SizeTotal : Sum of all SA.TotalOnlineSize and SA.TotalNearlineSize
• SizeFree : Sum of all SA.FreeOnlineSize and SA.FreeNearlineSize
• InformationServiceURL : empty
• Port : empty
• StateOfCurrentLoad : empty
• ImplementationName : CASTOR
• ImplementationVersion : This string is actually hardwired for now (and therefore likely to be wrong), because it is not terribly important.
• Status : Production : Also currently hardwired, since nothing relies on this variable. Future releases may treat this in a cleverer way.
• TotalOnlineSize : Sum of SA.TotalOnlineSize for all SAs
• TotalNearlineSize : Sum of SA.TotalNearlineSize for all SAs
• UsedOnlineSize : Sum of SA.UsedOnlineSize for all SAs
• UsedNearlineSize : Sum of SA.UsedNearlineSize for all SAs

SA

Each SA is a service class. As of CIP 2.0, service classes can share disk pools properly, and tape pools, too. As regards the accounting information, the CIP will divide the dynamic data evenly between the service classes - this is a simple heuristic which will add up to the right numbers.

The SA.Path is set if and only if all VOs using the SA have the same path.

Loosely speaking, online accounting is calculated as follows:

• Free and Total are queried from the stager database's view of the disk pools;
• Used is set to the difference between Free and Total: this is not necessarily the correct volume of used space but it satisfies the InstalledCapacity requirement that Free plus Used be equal to Total;
• Reserved is summed over all disk servers, including those that may be disabled, as per InstalledCapacity, but only if it is associated with a service class.

Similarly, nearline accounting is handled as follows:

• Free is handled separately by summing the "estimated free space" over the relevant tape pools;
• Used and Total are handled by summing over tape capacities for the relevant tapes;
• Reserved is handled like Used and Total and also counting tapes which may be disabled.

These numbers are not necessarily "right":

• When a file is written to tape it may be compressed, so the used space is less than the filesize;
• When a file is deleted, the space on tape is not freed: the tape has to be repacked to free the space;
• The free space on a tape is not necessarily usable, since CASTOR doesn't split files across tapes, so in principle a file may not fit even if the free space adds up to more than the filesize;
• Etc...

VOInfo

VOInfo objects are dynamically generated in CIP 2, so have dynamically generated names which may differ from one pass to another - but are of course fully consistent within each pass.

A VOInfo object is used to publish space token descriptions. When a space token is published, we also publish a VOInfo object without space tokens, to enable VOs to write into the SA without the space token. This feature can be disabled but only for everyone, not by individual VO (as of 2.1.0).