Access NFS Mounted dCache

The space from a dCache installation can be NFS mounted on to standard batch workers and accessed as a file system. For instance the batch workers at RAL automount the /pnfs/gridpp.rl.ac.uk area as a conventional NFS mount from the dCache head node.

dcache.conf

PNFS must be made aware of the locations of the hosts that run the dcap doors so that it can direct clients to the appropriate host when they try to access a file in the namespace. The dcache.conf file lists the hostnames:ports. Since this file is int PNFS, you have to redirect echo to add lines to it.

$ cat /pnfs/fs/admin/etc/config/dCache/dcache.conf 
pool1.epcc.ed.ac.uk:22125
pool2.epcc.ed.ac.uk:22125

Access with (gsi)dcap preload library

There is a (gsi)dcap preload library which allows for the OS to override functions provided by the standard C library, like open(). This means that you can continue to use standard commands like:

$ ls -l /pnfs/gridpp.rl.ac.uk/data/dteam 
$ cp /etc/group /pnfs/gridpp.rl.ac.uk/data/dteam/junk

in your application scripts/code, but the dcap lib will be used rather than the standard C lib. This is useful since dcap can take advantage of the advanced features of the dCache system and can handle things like staging in files from a tape backend. To use the preload library you must set the environment variable:

 export LD_PRELOAD=libpdcap.so
 export DCACHE_IO_TUNNEL=libgsiTunnel.so                <------------- required for gsidcap access

this assumes that /opt/d-cache/dcap/lib is in your LD_LIBRARY_PATH. If the site uses gsidcap then DCACHE_IO_TUNNEL must be set. If onlf dcap is used then DCACHE_IO_TUNNEL should not be set. The dcap API is documented here.

Creating directories

Using the preload library it is not possible to dynamically create new directories.

$ cp  /etc/group /pnfs/epcc.ed.ac.uk/data/dteam/dcap/testdirectory/test-wn1-cp-1
cp: cannot create regular file `/pnfs/epcc.ed.ac.uk/data/dteam/dcap/testdirectory/test-wn1-cp-1': No such file or directory
$ mkdir /pnfs/epcc.ed.ac.uk/data/dteam/dcap/testdirectory/
$ cp  /etc/group /pnfs/epcc.ed.ac.uk/data/dteam/dcap/testdirectory/test-wn1-cp-1
$ ls -l /pnfs/epcc.ed.ac.uk/data/dteam/dcap/testdirectory/
total 1
-rw-r--r--    1 dteam001 dteam         873 Jan 30 17:26 test-wn1-cp-1

Permissions

If the pnfs permissions are set correctly, then it should not be possible for a user to copy files into an area owned by another user/group.

$ cp  /etc/group /pnfs/epcc.ed.ac.uk/data/ops/test-wn1-cp-1
Failed create entry in pNFS.
cp: cannot create regular file `/pnfs/epcc.ed.ac.uk/data/ops/test-wn1-cp-1': Permission denied

Of course the files can still be world readable:

$ cp /pnfs/epcc.ed.ac.uk/data/ops/generated/2006-11-08/file00fddbb3-585b-4b72-bef7-fe57e9c3748c ./ops-1
$ ls -l ops-1
-rw-r--r--    1 dteam001 dteam       41472 Jan 30 17:29 ops-1

Access without the (gsi)dcap preload library

Some users may experience problems with the preload library so they can either use the dcap API directly in their applications or use the dCache.org provided client, dccp. With /pnfs mounted locally this works as follows:

$ dccp -d 63  /etc/group /pnfs/epcc.ed.ac.uk/data/dteam/dcap/test-1
$ dccp /pnfs/epcc.ed.ac.uk/data/dteam/dcap/test-1 ./test-dcap-12

If you want to use gsi security then you need this set:

$ export DCACHE_IO_TUNNEL=libgsiTunnel.so                <------------- required for gsidcap access

Creating directories

If you attempt to use dccp to write a file into a non-existent pnfs directory then an error will be returned:

$ dccp -d 63  /etc/group /pnfs/epcc.ed.ac.uk/data/dteam/dcap/testdir/test-1
Dcap Version version-1-2-39 Mar  9 2006 08:05:16
extra option: -alloc-size=873
Real file name: /etc/group.
Using system native open for /etc/group.
Real file name: /pnfs/epcc.ed.ac.uk/data/dteam/dcap/testdir/test-1.
Using system native open for /pnfs/epcc.ed.ac.uk/data/dteam/dcap/testdir/test-1.Can't open destination file : Not Pnfs file system
System error: No such file or directory

The directory must first be created and then the file copied in:

$ mkdir /pnfs/epcc.ed.ac.uk/data/dteam/dcap/testdir 
$ ls -l /pnfs/epcc.ed.ac.uk/data/dteam/dcap/
total 1
-rw-r--r--    1 dteam001 dteam         873 Jan 30 17:00 test-1
drwxr-xr-x    1 dteam001 dteam         512 Jan 30 17:14 testdir
$ dccp /etc/group /pnfs/epcc.ed.ac.uk/data/dteam/dcap/testdir/test-1
$ ls -l /pnfs/epcc.ed.ac.uk/data/dteam/dcap/testdir/
total 1
-rw-r--r--    1 dteam001 dteam         873 Jan 30 17:15 test-1

Permissions

If you try and write to a pnfs directory that is owned by another VO then dccp will fail:

[dteam001@wn1 dteam001]$ dccp -d 63  /etc/group /pnfs/epcc.ed.ac.uk/data/ops/test-ops-1
Dcap Version version-1-2-39 Mar  9 2006 08:05:16
extra option: -alloc-size=873
Real file name: /etc/group.
Using system native open for /etc/group.
Real file name: /pnfs/epcc.ed.ac.uk/data/ops/test-ops-1.
Using dCache open for /pnfs/epcc.ed.ac.uk/data/ops/test-ops-1.
Failed create entry in pNFS.
Can't open destination file : Can not create entry in pNfs
System error: Permission denied

since the permissions on the directory are:

$ ls -ld /pnfs/epcc.ed.ac.uk/data/ops/
drwxrwxr-x    1 ops001   ops           512 Sep 29 13:32 /pnfs/epcc.ed.ac.uk/data/ops/

However, these permissions can be changed by the dCache admin to suit particular needs. With the above permissions it is possible to read different user group files:

$ dccp /pnfs/epcc.ed.ac.uk/data/ops/generated/2006-11-08/file00fddbb3-585b-4b72-bef7-fe57e9c3748c ./ops-1