Difference between revisions of "DIRAC FileCatalog MetaData"

Revision as of 15:41, 21 April 2022

The DIRAC FileCatalog has two types of Metadata: Metadata for files and for directories. Metadata should always be indexed. Unfortunately DIRAC currently allows you to create unindexed metadata. To avoid this, the correct procedure to create metadata is:

1. Through the CLI

For a file:

dirac-dms-filecatalog-cli

create index:
FC:/gridpp/user/d/daniela.bauer>meta index -f testfiles int
Added metadata field testfiles of type int

show will show you all the tags available for your VO (here:gridpp) 
FC:/gridpp/user/d/daniela.bauer>meta show
      FileMetaFields : {'testfiles': 'INT', 'experiment': 'VARCHAR(128)', 'JMMetaInt3': 'INT', 'JMMetaInt': 'INT'}
 DirectoryMetaFields : {'JMMetaInt2': 'INT'}

attach metadata to files:
FC:/gridpp/user/d/daniela.bauer>meta set test-man testfiles 1
/gridpp/user/d/daniela.bauer/test-man {'testfiles': '1'}
FC:/gridpp/user/d/daniela.bauer>meta set test-qmul testfiles 1
/gridpp/user/d/daniela.bauer/test-qmul {'testfiles': '1'}

find all files that are associated with a certain metadata tag:
FC:/gridpp/user/d/daniela.bauer>find /gridpp testfiles=1
Query: {'testfiles': 1}
/gridpp/user/d/daniela.bauer/test-man
/gridpp/user/d/daniela.bauer/test-qmul

For a directory:

dirac-dms-filecatalog-cli

FC:/gridpp/user/d/daniela.bauer>meta index -d testdir int
Added metadata field testdir of type int
FC:/gridpp/user/d/daniela.bauer>meta show
      FileMetaFields : {'testfiles': 'INT', 'experiment': 'VARCHAR(128)', 'JMMetaInt3': 'INT', 'JMMetaInt': 'INT'}
 DirectoryMetaFields : {'JMMetaInt2': 'INT', 'testdir': 'INT'}

FC:/gridpp/user/d/daniela.bauer>meta set /gridpp/user/d/daniela.bauer testdir 1
/gridpp/user/d/daniela.bauer {'testdir': '1'}

1. Through the API:

The official DIRAC documentation on the topic can be found here.