DPM Enabling Gridview

GridView is a transfer monitor website which uses gridftp logs files, published into RGMA, in order to display the data transfers between sites on the grid.

In order for this to work sites have to enable a small RGMA python client to parse the gridftp logs files and publish them into RGMA.

To do this for Disk Pool Manager, sites have to:

1. Ensure that the lcg-mon-gridftp RPM is installed (it should be on all DPM nodes).
2. Make a simple change to /opt/lcg/etc/lcg-mon-gridftp.conf:
   1. Change the LOG_FILE entry to /var/log/dpm-gsiftp/dpm-gsiftp.log.
   2. Optionaly (but I found it helpful) to change log_level to DEBUG.
3. Restart the lcg-mon-gridftp service.

Then the /opt/lcg/etc/lcg-mon-gridftp.conf file should look like:

 RGMA_HOME = /opt/glite
 LOG_FILE = /var/log/dpm-gsiftp.log
 PARSER = gridftp
 TABLE = GridftpMonitor
 # Possible values: CRITICAL,ERROR,WARNING,INFO,DEBUG,NOTSET
 log_level = DEBUG
 # Possible values: 0, 1 
 debug = 0

As transfers take place the python script will check the /var/log/dpm-gsiftp.log file every minute and publish tuples for each completed transfer into RGMA. In the logfile /opt/lcg/var/log/lcg-mon-gridftp.log you should see entries like this:

 2006-03-21 12:50:49,489: [DEBUG] Next Entry
 2006-03-21 12:50:49,497: [DEBUG] while, insert
 2006-03-21 12:50:49,498: [DEBUG] INSERT INTO  GridftpMonitor (host,user_name,src
 ,dest,block_size,nstreams,buffer_size,nbytes,nstripes,transfer_rate,volume,file_
 name,code,start_time,start_time_micros,end_time,end_time_micros,MeasurementDate,
 MeasurementTime) VALUES ('pool1-gla.scotgrid.ac.uk','dteam001','pool1-gla.scotgr
 id.ac.uk','gftp0446.gridpp.rl.ac.uk',65536,2,16384,1000000000,1,10.2246,'?(rfio-
 file)','pool1-gla.scotgrid.ac.uk:/gridstorage/f1/dteam/2006-01-17/canned1G.19055
 .0',226,1142945326,425408,1142945419,698057,'2006-03-21','12:50:19');

N.B. GridView gets two records for file transfers - source and destination. At the moment it ignores destination records and only produces graphs based on source records. So you will see data flowing out of your site, but not flowing in (and in particular RAL, as a dCache site, don't publish their source records yet - dCache log files are in a different format).

LCG 2.7.0 and earlier

The location of the DPM gsiftp log file in LCG 2.7.0 and earlier was /var/log/dpm-gsiftp.log.