RFIO Performance Testing

From GridPP Wiki
Jump to: navigation, search


To test performance of LAN access to your DPM using the rfio protocol.


The basic idea is to seed an rfio client application onto N nodes of your cluster and get them to simultaneously begin reading N distinct files from the DPM. How this is done in practice depends somewhat on how your cluster and DPM are configured. If you have multiple disk servers, then you should ensure that the files being read are spread evenly among the disk servers. This should happen automatically when transferring data in due to the round-robin approach that DPM takes when assigning disk servers for writing.

To make the testing process as simple as possible, it is good if the cluster has certain infrastructure components already in place. For example:

  • a central admin node from which commands can be executed on all nodes using applications like dsh or pdsh. This is where you will launch the test on all nodes.
  • a central storage area that is visible on all nodes of the cluster. This is the location where the rfio binary app will be placed for easy access on the WNs.

An example client application and Makefile are included below. Also included are some wrapper scripts which you will likely require. Since RFIO uses GSI authentication, all client nodes will require a valid grid proxy and need to be told where this is using the X509_USER_PROXY environment variable.


Some ROOT macros and scripts are provided for analysing the results of your tests, producing some plots of quantities such as average open time per client, total read rate, etc.


Obviously all protocols will be bottlenecked according to the network topology, and here the most important thing for most sites will be to ensure that the worker nodes are connecting directly to the DPM disk servers, and not going through, e.g., a NAT server. There's some advice about how to do this in DPM Dual Homing.

RFIO client application

Takes an rfio file and attempts to read it, dumping the data. It will read the whole file (default) with a given block size, or will read only parts of the file. The following configuration options are available.

* -d,v: Debug/verbose mode
* -b X: Set block size for read to X (in unsigned ints!)
* -s X: Skip size - this number *blocks* (a la -b) skipped after each read
* -r X: Target read rate (MiB/s)
* -c  : Check mode: check reads against expectation.
* -r X: Do X random reads (can't be specified with skip)
* -n X: Read X times at each read point.
* -m X: Set the rfio read mode to X (READBUF, READAHEAD, STREAM)
* -z X: File size (used to determine random seek point)