Difference between revisions of "RAL Tier1 Network Measurements"

From GridPP Wiki
Jump to: navigation, search
 
(No difference)

Latest revision as of 13:35, 17 August 2006

For the RAL Tier1 it is some timea nescesary to perform one of network measurments for particular links. These are allways recorded here so that there is a record for subsequent reference.


6th July 2006

Some network problems were reported so some random iperfs have been done. These were all gridmon.dl.ac.uk but subsequently it was understood that this is not a typical host to use since traffic to dl.ac.uk from rl.ac.uk is encrypted by the routers.

T1 gridftp server gridmon.dl.ac.uk

  [root@gftp0440 root]# iperf -i 5  -c gridmon.dl.ac.uk
  ------------------------------------------------------------
  Client connecting to gridmon.dl.ac.uk, TCP port 5001
  TCP window size: 4.00 MByte (default)
  ------------------------------------------------------------
  [  3] local 130.246.179.1 port 37367 connected with 193.62.119.20 port 5001
  [  3]  0.0- 5.0 sec  12.0 MBytes  20.1 Mbits/sec
  [  3]  5.0-10.0 sec  10.0 MBytes  16.8 Mbits/sec
  [  3]  0.0-10.2 sec  22.0 MBytes  18.1 Mbits/sec

RAL PP Box to gridmon.ac.uk

 $ ./iperf -i 5  -c gridmon.dl.ac.uk
 ------------------------------------------------------------
 Client connecting to gridmon.dl.ac.uk, TCP port 5001
 TCP window size: 16.0 KByte (default)
 ------------------------------------------------------------
 [  3] local 130.246.44.102 port 33513 connected with 193.62.119.20 port 5001
 [  3]  0.0- 5.0 sec  7.69 MBytes  12.9 Mbits/sec
 [  3]  5.0-10.0 sec  7.58 MBytes  12.7 Mbits/sec
 [  3]  0.0-10.0 sec  15.3 MBytes  12.8 Mbits/sec

Random Box in Oxford University Physics Department to gridmon.dl.ac.uk

./iperf -i 5  -c gridmon.dl.ac.uk
 ------------------------------------------------------------
 Client connecting to gridmon.dl.ac.uk, TCP port 5001
 TCP window size: 16.0 KByte (default)
 ------------------------------------------------------------
 [  3] local 163.1.243.202 port 45444 connected with 193.62.119.20 port 5001
 [  3]  0.0- 5.0 sec  43.9 MBytes  73.6 Mbits/sec
 [  3]  5.0-10.0 sec  43.8 MBytes  73.4 Mbits/sec
 [  3]  0.0-10.0 sec  87.7 MBytes  73.5 Mbits/sec


7th July 2006

Following the comments that dl.ac.uk is a non-typical destination to use the 6th July tests were repeated with a different destination. The summary is that from T1 to lancastor the rate is 17 Mbits/s but from Oxford to Lancastor the rate is 65 Mbit/s. The latter being a 100 Mbit/s connected box is reasnable where as the T1 figure is not.

From RAL T1 UI to Iperf Server in Lancastor

 ./iperf -i 5  -c fal-pygrid-29.lancs.ac.uk
 ------------------------------------------------------------
 Client connecting to fal-pygrid-29.lancs.ac.uk, TCP port 5001
 TCP window size: 16.0 KByte (default)
 ------------------------------------------------------------
 [  3] local 130.246.183.136 port 57580 connected with 148.88.137.206 port 5001
 [  3]  0.0- 5.0 sec  10.3 MBytes  17.3 Mbits/sec
 [  3]  5.0-10.0 sec  10.4 MBytes  17.4 Mbits/sec
 [  3]  0.0-10.0 sec  20.7 MBytes  17.3 Mbits/sec

From RAL GridFTP to Iperf Server at Lancastor over UK Light Link

   ./iperf -i 5  -c fal-pygrid-29.lancs.ac.uk
 [  3] local 130.246.179.5 port 38636 connected with 148.88.137.206 port 5001
 [  3]  0.0- 5.0 sec  74.2 MBytes    124 Mbits/sec
 [  3]  5.0-10.0 sec  73.5 MBytes    123 Mbits/sec
 [  3]  0.0-10.1 sec    148 MBytes    123 Mbits/sec

Random Box in Oxford University Physics Department to Lancastor

./iperf -i 5  -c fal-pygrid-29.lancs.ac.uk
 ------------------------------------------------------------
 Client connecting to fal-pygrid-29.lancs.ac.uk, TCP port 5001
 TCP window size: 16.0 KByte (default)
 ------------------------------------------------------------
 [  3] local 163.1.5.238 port 49272 connected with 148.88.137.206 port 5001
 [  3]  0.0- 5.0 sec  38.0 MBytes  63.8 Mbits/sec
 [  3]  5.0-10.0 sec  38.5 MBytes  64.7 Mbits/sec  
 [  3]  0.0-10.0 sec  76.6 MBytes  64.2 Mbits/sec

24th July 2006

We have had reports that things have improved so rerunning the tests. It looks like the link on the production (100Mbits) is now 63 Mbits to Lancastor so that is fine.

RAL T1 UI to Lancastor over the production network (100Mb/sec)

  [traylens@lcgui0360:~]$ ./iperf -i 5 -c fal-pygrid-29.lancs.ac.uk
  ------------------------------------------------------------
  Client connecting to fal-pygrid-29.lancs.ac.uk, TCP port 5001 
  TCP window size: 16.0 KByte (default)
  ------------------------------------------------------------
  [  3] local 130.246.183.187 port 41260 connected with 148.88.137.206 port 5001
  [  3]  0.0- 5.0 sec  36.9 MBytes  61.9 Mbits/sec
  [  3]  5.0-10.0 sec  38.2 MBytes  64.1 Mbits/sec
  [  3]  0.0-10.0 sec  75.1 MBytes  63.0 Mbits/sec

RAL T1 GridFTP to Lancastor over the light path network (1Gb/sec)

  [root@gftp0440 root]# iperf -i 5 -c fal-pygrid-29.lancs.ac.uk
  ------------------------------------------------------------
  Client connecting to fal-pygrid-29.lancs.ac.uk, TCP port 5001
  TCP window size: 4.00 MByte (default)
  ------------------------------------------------------------
  [  3] local 130.246.179.1 port 43281 connected with 148.88.137.206 port 5001
  [  3]  0.0- 5.0 sec    107 MBytes    180 Mbits/sec
  [  3]  5.0-10.0 sec    106 MBytes    178 Mbits/sec
  [  3]  0.0-10.0 sec    213 MBytes    178 Mbits/sec

17 August 2006

RAS doing some tests as GRIDMON showing strange step funcyion effects between Tier-1 and Tier-2s:

Results from GRIDMON

Gridmon service can be found at: Gridmon Service

Gridmon tests are run using (for example):

  iperf -c epnetmon.ph.bham.ac.uk -p 5001 -P 1 -w 1048576 -t 5 -fb
  -c  = run iperf as a client
  -p  = remote server port to connect to (5001)
  -P  = number of parallel iperf threads to run (= number of TCP streams)
  -w  = TCP window size (socket buffer size), 1MB in this case
  -t  = time to transmit for (length of test), 5 secs
  -fb = format to return data, in bytes in this case


Rate to Glasgow shows a step function on Monday:

File:160806T1-GLA.png

Rate to Birmingham shows a similar step function:

File:160806T1-BHAM.png

Rate to RAL PPD does not show the problem:

File:170806T1-RALPP.png


UDP rates are not affected by this problem:

File:170806RALGLAudp.png

Tests from RAL UI to Glasgow iperf server

  lcgui0357] /home/csf/ras/iperf-2.0.2 > ./src/iperf -c mon3-gla.scotgrid.ac.uk -p 27890
  ------------------------------------------------------------
  Client connecting to mon3-gla.scotgrid.ac.uk, TCP port 27890
  TCP window size: 16.0 KByte (default)
  ------------------------------------------------------------
  [  3] local 130.246.183.136 port 43108 connected with 194.36.1.118 port 27890
  [  3]  0.0-10.0 sec  23.6 MBytes  19.8 Mbits/sec


Also tried 40 stream test and got rates of about 9 Mb/s per stream

Note that rates are probably limited by the maximum tcp window size. However using larger windows only gives a little gain:

  [lcgui0357] /home/csf/ras/iperf-2.0.2 > ./src/iperf -c mon3-gla.scotgrid.ac.uk -p 27890 -w 1M
  ------------------------------------------------------------
  Client connecting to mon3-gla.scotgrid.ac.uk, TCP port 27890
  TCP window size:   256 KByte (WARNING: requested 1.00 MByte)
  ------------------------------------------------------------
  [  3] local 130.246.183.136 port 43123 connected with 194.36.1.118 port 27890
  [  3]  0.0-10.1 sec  26.8 MBytes  22.3 Mbits/sec


tests by Yves from Oxford to Glasgow

  [root@t2se01 root]# date; iperf -c mon3-gla.scotgrid.ac.uk -p 27890
  Thu Aug 17 09:59:13 BST 2006
  ------------------------------------------------------------
  Client connecting to mon3-gla.scotgrid.ac.uk, TCP port 27890
  TCP window size: 16.0 KByte (default)
  ------------------------------------------------------------
  [  3] local 163.1.5.242 port 41585 connected with 194.36.1.118 port 27890
  [  3]  0.0-10.0 sec  32.0 MBytes  26.8 Mbits/sec