Transferring Files

This page describes how to transfer files into, out of, or across the GridPP infrastructure. Several tools are available; some experiments use their own tools (not documented here) usually layered on top of the lower level tools.

File Transfer Service

The file transfer service is based on the EMI FTS3. CERN runs an FTS3 service; the [RAL Tier 1] runs an FTS service as well albeit without a web front end. These instructions are for the use of the RAL FTS.

Proxies

• FTS will do a delegation based on the local proxy the user holds (grid or voms proxy) when:
  • the user runs a delegating command such as fts-delegation-init or fts-transfer-submit
  • AND the local proxy has a lifetime more than one hour
  • AND the server does not already hold any proxy for the user (DN) with a lifetime of more than four hours

• If the delegation is attempted and the local proxy lifetime is less than one hour then the delegation will fail.

• When a job is submitted, the transfers will use the proxy available at the time (delegated with the submission or the one on the server if validity is >4hrs) - if this proxy expires, every transfer will fail. There is no renewal of proxies (Ã la MyProxy).

When using plain old grid proxies, the best approach is thus to create a proxy with as long a lifetime as possible, delegate it to the server and then forget about it.

Now the thick plottens. Suppose we want to use a VOMS proxy. VOMS attributes (embedded in the voms proxy) have a maximal lifetime (by configuration/policy) of 24 hours, so the complete proxy can have a lifetime of at most 24 hours.

It follows that one should:

1. . Get as long a proxy as possible (e.g. 24 hrs) and delegate it to FTS
2. . Submit as many transfers as can comfortably fit into that lifetime minus four hours (so 20 hrs) - however, transfers can and do continue for the full lifetime of the proxy
3. . Keep an eye on the progress of the transfers (to submit more/fewer or even cancel as needed)
4. . When there is less than four hours left on the server's proxy, and just prior to a new job is being submitted, create a new proxy.