Difference between revisions of "DPM Enabling Gridview"
Latest revision as of 09:20, 14 June 2006
GridView is a transfer monitor website which uses gridftp logs files, published into RGMA, in order to display the data transfers between sites on the grid.
In order for this to work sites have to enable a small RGMA python client to parse the gridftp logs files and publish them into RGMA.
To do this for Disk Pool Manager, sites have to:
- Ensure that the lcg-mon-gridftp RPM is installed (it should be on all DPM nodes).
- Make a simple change to /opt/lcg/etc/lcg-mon-gridftp.conf:
- Change the LOG_FILE entry to /var/log/dpm-gsiftp/dpm-gsiftp.log.
- Optionaly (but I found it helpful) to change log_level to DEBUG.
- Restart the lcg-mon-gridftp service.
Then the /opt/lcg/etc/lcg-mon-gridftp.conf file should look like:
RGMA_HOME = /opt/glite LOG_FILE = /var/log/dpm-gsiftp.log PARSER = gridftp TABLE = GridftpMonitor # Possible values: CRITICAL,ERROR,WARNING,INFO,DEBUG,NOTSET log_level = DEBUG # Possible values: 0, 1 debug = 0
As transfers take place the python script will check the /var/log/dpm-gsiftp.log file every minute and publish tuples for each completed transfer into RGMA. In the logfile /opt/lcg/var/log/lcg-mon-gridftp.log you should see entries like this:
2006-03-21 12:50:49,489: [DEBUG] Next Entry 2006-03-21 12:50:49,497: [DEBUG] while, insert 2006-03-21 12:50:49,498: [DEBUG] INSERT INTO GridftpMonitor (host,user_name,src ,dest,block_size,nstreams,buffer_size,nbytes,nstripes,transfer_rate,volume,file_ name,code,start_time,start_time_micros,end_time,end_time_micros,MeasurementDate, MeasurementTime) VALUES ('pool1-gla.scotgrid.ac.uk','dteam001','pool1-gla.scotgr id.ac.uk','gftp0446.gridpp.rl.ac.uk',65536,2,16384,1000000000,1,10.2246,'?(rfio- file)','pool1-gla.scotgrid.ac.uk:/gridstorage/f1/dteam/2006-01-17/canned1G.19055 .0',226,1142945326,425408,1142945419,698057,'2006-03-21','12:50:19');
N.B. GridView gets two records for file transfers - source and destination. At the moment it ignores destination records and only produces graphs based on source records. So you will see data flowing out of your site, but not flowing in (and in particular RAL, as a dCache site, don't publish their source records yet - dCache log files are in a different format).
LCG 2.7.0 and earlier
The location of the DPM gsiftp log file in LCG 2.7.0 and earlier was /var/log/dpm-gsiftp.log.