DCache Billing Information

From GridPP Wiki
Jump to: navigation, search

The raw information about all dCache activities can be found in /opt/d-cache/billing/<YYYY>/<MM>/billing-<YYYY.MM.DD>. A typical output line is: 

 05.31 22:35:16 [pool:<pool-name>:transfer] [000100000000000000001320,24675] \
 myStore:STRING@osm 24675 474 true {GFtp-1.0 <client-host-fqn> 37592} {0:""}

The first bracket contains the pool name, the second the PNFS ID and the size of the file which is transferred. Then follows the storage class, the actual amount of bytes transferred, and the number of milliseconds the transfer took. The next entry is true if the transfer was a wrote data to the pool. The first braces contain the protocol, client FQN, and the client host data transfer listen port. The final bracket contains the return status and a possible error message.


Billing logs for dCache version 1.6.6

Slight changes have been made to the billing information that is recorded by dCache in its 'out-of-the-box' configuration. The DN of the user who initiates any transfer is now recorded, as is the hostname of the other machine. The filenames are not recorded, but the pnfs IDs are.

srmPut

 11.29 16:25:16 [pool:wn4_1@wn4Domain:transfer] [000400000000000000001698,17371] \
 dteam:dteam@osm 17371 289 true {GFtp-1.0 wn4.epcc.ed.ac.uk 46881} {0:""}
 11.29 16:25:16 [door:GFTP-wn4-Unknown-110@gridftp-wn4Domain:request] \
 ["/C=UK/O=eScience/OU=Edinburgh/L=NeSC/CN=greig cowan":18118:2688:ui.epcc.ed.ac.uk] \
 [000400000000000000001698,0] <unknown> 1133281516065 0 {0:""}

srmGet

 11.29 16:25:37 [pool:wn4_1:transfer] [000400000000000000001698,0] <unknown> true {null} {0:""}
 11.29 16:25:37 [pool:wn4_1@wn4Domain:transfer] [000400000000000000001698,17371] \
 dteam:dteam@osm 17371 1 false {GFtp-1.0 wn4.epcc.ed.ac.uk 46883} {0:""}
 11.29 16:25:37 [door:GFTP-wn4-Unknown-111@gridftp-wn4Domain:request] \
 ["/C=UK/O=eScience/OU=Edinburgh/L=NeSC/CN=greig cowan":18118:2688:ui.epcc.ed.ac.uk] \
 [000400000000000000001698,0] <unknown> 1133281537570 0 {0:""}

srmCopy out of the dCache

 11.29 16:27:06 [pool:wn4_1:transfer] [000400000000000000001698,0] <unknown> true {null} {0:""}
 11.29 16:27:06 [pool:wn4_1@wn4Domain:transfer] [000400000000000000001698,17371] \
 dteam:dteam@osm 17371 81 false {GFtp-1.0 148.88.137.201 20000} {0:""}
 11.29 16:27:05 [door:GFTP-wn4-Unknown-112@gridftp-wn4Domain:request] \
 ["/C=UK/O=eScience/OU=Edinburgh/L=NeSC/CN=greig cowan":18118:2688:fal-pygrid-24.lancs.ac.uk] \
 [000400000000000000001698,0] <unknown> 1133281625913 0 {0:""}

srmCopy into the dCache

Notice that the DN of the user is not logged here. 

 11.29 16:27:13 [pool:wn4_1@wn4Domain:transfer] [0004000000000000000016B8,17371] \
 dteam:dteam@osm 0 3420 true {RemoteGsiftpTransfer-1.1,fal-pygrid-25.lancs.ac.uk:0} {0:""}

srm-advisory-delete

Entry now made in the log file 

 11.29 16:28:25 Pool=broadcast;RemoveFiles=,000400000000000000001698


globus-url-copy (local -> dCache)

 11.29 16:32:02 [pool:wn4_1@wn4Domain:transfer] [0004000000000000000016E0,17371] \
 dteam:dteam@osm 17371 454 true {GFtp-1.0 wn4.epcc.ed.ac.uk 46905} {0:""}
 11.29 16:32:02 [door:GFTP-wn4-Unknown-113@gridftp-wn4Domain:request] \
 ["/C=UK/O=eScience/OU=Edinburgh/L=NeSC/CN=greig cowan":18118:2688:ui.epcc.ed.ac.uk] \
 [0004000000000000000016E0,0] <unknown> 1133281922143 0 {0:""}

globus-url-copy (dCache -> local)

 11.29 16:33:00 [pool:wn4_1:transfer] [0004000000000000000016E0,0] <unknown> true {null} {0:""}
 11.29 16:33:01 [pool:wn4_1@wn4Domain:transfer] [0004000000000000000016E0,17371] \
 dteam:dteam@osm 17371 1 false {GFtp-1.0 wn4.epcc.ed.ac.uk 46908} {0:""}
 11.29 16:33:00 [door:GFTP-wn4-Unknown-114@gridftp-wn4Domain:request] \
 ["/C=UK/O=eScience/OU=Edinburgh/L=NeSC/CN=greig cowan":18118:2688:ui.epcc.ed.ac.uk] \
 [0004000000000000000016E0,0] <unknown> 1133281980792 0 {0:""}

3rd party globus-url-copy from dCache

 11.29 16:33:48 [pool:wn4_1:transfer] [0004000000000000000016E0,0] <unknown> true {null} {0:""}
 11.29 16:33:48 [pool:wn4_1@wn4Domain:transfer] [0004000000000000000016E0,17371] \
 dteam:dteam@osm 17371 31 false {GFtp-1.0 130.246.47.204 50000} {0:""}
 11.29 16:33:47 [door:GFTP-wn4-Unknown-115@gridftp-wn4Domain:request] \
 ["/C=UK/O=eScience/OU=Edinburgh/L=NeSC/CN=greig cowan":18118:2688:ui.epcc.ed.ac.uk] \
 [0004000000000000000016E0,0] <unknown> 1133282027803 0 {0:""}

3rd party globus-url-copy into dCache

 11.29 16:35:02 [pool:wn4_2@wn4Domain:transfer] [000400000000000000001700,17371] \
 dteam:dteam@osm 17371 1114 true {GFtp-1.0 wn4.epcc.ed.ac.uk 46913} {0:""}
 11.29 16:35:02 [door:GFTP-wn4-Unknown-117@gridftp-wn4Domain:request] \
 ["/C=UK/O=eScience/OU=Edinburgh/L=NeSC/CN=greig cowan":18118:2688:ui.epcc.ed.ac.uk] \
 [000400000000000000001700,0] <unknown> 1133282102304 0 {0:""}


Billing logs for dCache version 1.5.{2,3}

srmPut

 11.11 12:09:54 [pool:wn4_2@wn4Domain:transfer] [0004000000000000000012D0,17371] \
 dteam:dteam@osm 17371 231 true {GFtp-1.0 wn4.epcc.ed.ac.uk 36857} {0:""}

srmGet

 11.11 12:10:50 [pool:wn4_2:transfer] [0004000000000000000012D0,0] <unknown> true {null} {0:""}
 11.11 12:10:55 [pool:wn4_2:transfer] [0004000000000000000012D0,0] <unknown> true {null} {0:""}
 11.11 12:10:55 [pool:wn4_2@wn4Domain:transfer] [0004000000000000000012D0,17371] \
 dteam:dteam@osm 17371 4 false {GFtp-1.0 wn4.epcc.ed.ac.uk 36859} {0:""}

srmCopy out of the dCache

 11.11 12:11:29 [pool:wn4_2:transfer] [0004000000000000000012D0,0] <unknown> true {null} {0:""}
 11.11 12:11:35 [pool:wn4_2:transfer] [0004000000000000000012D0,0] <unknown> true {null} {0:""}
 11.11 12:11:35 [pool:wn4_2@wn4Domain:transfer] [0004000000000000000012D0,17371] \
 dteam:dteam@osm 17371 6 false {GFtp-1.0 129.215.175.24 50000} {0:""}

srmCopy into the dCache

 11.11 12:17:23 [pool:wn4_1@wn4Domain:transfer] [000400000000000000001308,17371] \
 dteam:dteam@osm 0 2949 true {RemoteGsiftpTransfer-1.1,dcache.epcc.ed.ac.uk:0} {0:""}

No output is seen for srm-advisory-delete.

globus-url-copy (local -> dCache)

 11.11 12:34:45 [pool:wn4_1@wn4Domain:transfer] [000400000000000000001368,17371] \
 dteam:dteam@osm 17371 231 true {GFtp-1.0 wn4.epcc.ed.ac.uk 36894} {0:""}

globus-url-copy (dCache -> local)

 11.11 12:36:24 [pool:wn4_1:transfer] [000400000000000000001368,0] <unknown> true {null} {0:""}
 11.11 12:36:24 [pool:wn4_1@wn4Domain:transfer] [000400000000000000001368,17371] \
 dteam:dteam@osm 17371 4 false {GFtp-1.0 wn4.epcc.ed.ac.uk 36897} {0:""}

3rd party globus-url-copy from dCache

 11.11 12:38:48 [pool:wn4_1:transfer] [000400000000000000001368,0] <unknown> true {null} {0:""}
 11.11 12:38:48 [pool:wn4_1@wn4Domain:transfer] [000400000000000000001368,17371] \
 dteam:dteam@osm 17371 26 false {GFtp-1.0 148.88.137.199 20000} {0:""}

3rd party globus-url-copy into dCache

 11.11 12:41:10 [pool:wn4_2@wn4Domain:transfer] [000400000000000000001388,17371] \
 dteam:dteam@osm 17371 1971 true {GFtp-1.0 wn4.epcc.ed.ac.uk 36902} {0:""}


If an error occurs during a transfer, then corresponding /opt/d-cache/billing/<YYYY>/<MM>/billing-error-<YYYY.MM.DD> files are created.

Retrieved from "https://www.gridpp.ac.uk/w/index.php?title=DCache_Billing_Information&oldid=1132"