RAL Tier1 Network Measurements
For the RAL Tier1 it is some timea nescesary to perform one of network measurments for particular links. These are allways recorded here so that there is a record for subsequent reference.
6th July 2006
Some network problems were reported so some random iperfs have been done. These were all gridmon.dl.ac.uk but subsequently it was understood that this is not a typical host to use since traffic to dl.ac.uk from rl.ac.uk is encrypted by the routers.
T1 gridftp server gridmon.dl.ac.uk
[root@gftp0440 root]# iperf -i 5 -c gridmon.dl.ac.uk ------------------------------------------------------------ Client connecting to gridmon.dl.ac.uk, TCP port 5001 TCP window size: 4.00 MByte (default) ------------------------------------------------------------ [ 3] local 130.246.179.1 port 37367 connected with 193.62.119.20 port 5001 [ 3] 0.0- 5.0 sec 12.0 MBytes 20.1 Mbits/sec [ 3] 5.0-10.0 sec 10.0 MBytes 16.8 Mbits/sec [ 3] 0.0-10.2 sec 22.0 MBytes 18.1 Mbits/sec
RAL PP Box to gridmon.ac.uk
$ ./iperf -i 5 -c gridmon.dl.ac.uk ------------------------------------------------------------ Client connecting to gridmon.dl.ac.uk, TCP port 5001 TCP window size: 16.0 KByte (default) ------------------------------------------------------------ [ 3] local 130.246.44.102 port 33513 connected with 193.62.119.20 port 5001 [ 3] 0.0- 5.0 sec 7.69 MBytes 12.9 Mbits/sec [ 3] 5.0-10.0 sec 7.58 MBytes 12.7 Mbits/sec [ 3] 0.0-10.0 sec 15.3 MBytes 12.8 Mbits/sec
Random Box in Oxford University Physics Department to gridmon.dl.ac.uk
./iperf -i 5 -c gridmon.dl.ac.uk ------------------------------------------------------------ Client connecting to gridmon.dl.ac.uk, TCP port 5001 TCP window size: 16.0 KByte (default) ------------------------------------------------------------ [ 3] local 163.1.243.202 port 45444 connected with 193.62.119.20 port 5001 [ 3] 0.0- 5.0 sec 43.9 MBytes 73.6 Mbits/sec [ 3] 5.0-10.0 sec 43.8 MBytes 73.4 Mbits/sec [ 3] 0.0-10.0 sec 87.7 MBytes 73.5 Mbits/sec
7th July 2006
Following the comments that dl.ac.uk is a non-typical destination to use the 6th July tests were repeated with a different destination. The summary is that from T1 to lancastor the rate is 17 Mbits/s but from Oxford to Lancastor the rate is 65 Mbit/s. The latter being a 100 Mbit/s connected box is reasnable where as the T1 figure is not.
From RAL T1 UI to Iperf Server in Lancastor
./iperf -i 5 -c fal-pygrid-29.lancs.ac.uk ------------------------------------------------------------ Client connecting to fal-pygrid-29.lancs.ac.uk, TCP port 5001 TCP window size: 16.0 KByte (default) ------------------------------------------------------------ [ 3] local 130.246.183.136 port 57580 connected with 148.88.137.206 port 5001 [ 3] 0.0- 5.0 sec 10.3 MBytes 17.3 Mbits/sec [ 3] 5.0-10.0 sec 10.4 MBytes 17.4 Mbits/sec [ 3] 0.0-10.0 sec 20.7 MBytes 17.3 Mbits/sec
From RAL GridFTP to Iperf Server at Lancastor over UK Light Link
./iperf -i 5 -c fal-pygrid-29.lancs.ac.uk [ 3] local 130.246.179.5 port 38636 connected with 148.88.137.206 port 5001 [ 3] 0.0- 5.0 sec 74.2 MBytes 124 Mbits/sec [ 3] 5.0-10.0 sec 73.5 MBytes 123 Mbits/sec [ 3] 0.0-10.1 sec 148 MBytes 123 Mbits/sec
Random Box in Oxford University Physics Department to Lancastor
./iperf -i 5 -c fal-pygrid-29.lancs.ac.uk ------------------------------------------------------------ Client connecting to fal-pygrid-29.lancs.ac.uk, TCP port 5001 TCP window size: 16.0 KByte (default) ------------------------------------------------------------ [ 3] local 163.1.5.238 port 49272 connected with 148.88.137.206 port 5001 [ 3] 0.0- 5.0 sec 38.0 MBytes 63.8 Mbits/sec [ 3] 5.0-10.0 sec 38.5 MBytes 64.7 Mbits/sec [ 3] 0.0-10.0 sec 76.6 MBytes 64.2 Mbits/sec
24th July 2006
We have had reports that things have improved so rerunning the tests. It looks like the link on the production (100Mbits) is now 63 Mbits to Lancastor so that is fine.
RAL T1 UI to Lancastor over the production network (100Mb/sec)
[traylens@lcgui0360:~]$ ./iperf -i 5 -c fal-pygrid-29.lancs.ac.uk ------------------------------------------------------------ Client connecting to fal-pygrid-29.lancs.ac.uk, TCP port 5001 TCP window size: 16.0 KByte (default) ------------------------------------------------------------ [ 3] local 130.246.183.187 port 41260 connected with 148.88.137.206 port 5001 [ 3] 0.0- 5.0 sec 36.9 MBytes 61.9 Mbits/sec [ 3] 5.0-10.0 sec 38.2 MBytes 64.1 Mbits/sec [ 3] 0.0-10.0 sec 75.1 MBytes 63.0 Mbits/sec
RAL T1 GridFTP to Lancastor over the light path network (1Gb/sec)
[root@gftp0440 root]# iperf -i 5 -c fal-pygrid-29.lancs.ac.uk ------------------------------------------------------------ Client connecting to fal-pygrid-29.lancs.ac.uk, TCP port 5001 TCP window size: 4.00 MByte (default) ------------------------------------------------------------ [ 3] local 130.246.179.1 port 43281 connected with 148.88.137.206 port 5001 [ 3] 0.0- 5.0 sec 107 MBytes 180 Mbits/sec [ 3] 5.0-10.0 sec 106 MBytes 178 Mbits/sec [ 3] 0.0-10.0 sec 213 MBytes 178 Mbits/sec
17 August 2006
RAS doing some tests as GRIDMON showing strange step funcyion effects between Tier-1 and Tier-2s:
Results from GRIDMON
Gridmon service can be found at: Gridmon Service
Gridmon tests are run using (for example):
iperf -c epnetmon.ph.bham.ac.uk -p 5001 -P 1 -w 1048576 -t 5 -fb
-c = run iperf as a client -p = remote server port to connect to (5001) -P = number of parallel iperf threads to run (= number of TCP streams) -w = TCP window size (socket buffer size), 1MB in this case -t = time to transmit for (length of test), 5 secs -fb = format to return data, in bytes in this case
Rate to Glasgow shows a step function on Monday:
Rate to Birmingham shows a similar step function:
Rate to RAL PPD does not show the problem:
UDP rates are not affected by this problem:
Tests from RAL UI to Glasgow iperf server
lcgui0357] /home/csf/ras/iperf-2.0.2 > ./src/iperf -c mon3-gla.scotgrid.ac.uk -p 27890 ------------------------------------------------------------ Client connecting to mon3-gla.scotgrid.ac.uk, TCP port 27890 TCP window size: 16.0 KByte (default) ------------------------------------------------------------ [ 3] local 130.246.183.136 port 43108 connected with 194.36.1.118 port 27890 [ 3] 0.0-10.0 sec 23.6 MBytes 19.8 Mbits/sec
Also tried 40 stream test and got rates of about 9 Mb/s per stream
Note that rates are probably limited by the maximum tcp window size. However using larger windows only gives a little gain:
[lcgui0357] /home/csf/ras/iperf-2.0.2 > ./src/iperf -c mon3-gla.scotgrid.ac.uk -p 27890 -w 1M ------------------------------------------------------------ Client connecting to mon3-gla.scotgrid.ac.uk, TCP port 27890 TCP window size: 256 KByte (WARNING: requested 1.00 MByte) ------------------------------------------------------------ [ 3] local 130.246.183.136 port 43123 connected with 194.36.1.118 port 27890 [ 3] 0.0-10.1 sec 26.8 MBytes 22.3 Mbits/sec
tests by Yves from Oxford to Glasgow
[root@t2se01 root]# date; iperf -c mon3-gla.scotgrid.ac.uk -p 27890 Thu Aug 17 09:59:13 BST 2006 ------------------------------------------------------------ Client connecting to mon3-gla.scotgrid.ac.uk, TCP port 27890 TCP window size: 16.0 KByte (default) ------------------------------------------------------------ [ 3] local 163.1.5.242 port 41585 connected with 194.36.1.118 port 27890 [ 3] 0.0-10.0 sec 32.0 MBytes 26.8 Mbits/sec