 |
 |
BTW recovery does not exhonerate xroots or the whole TLS stack. We still need the TLS tracing enabled and what the log says when a failure occurs. On Fri, 10 Mar 2023, jthiltges wrote: > It indeed did recover. So this issue is hopefully transparent to clients, outside of the delay. > > Here's a sample server log for when the issue appeared with `xrdcp roots://...`. It seems to match up with the initial ticket comment when I'd been testing with xrdfs: > > ``` > 230310 16:31:05.189706 31158 Xrd_Sched: running main accept inq=0 > 230310 16:31:05.195279 31158 Xrd_Inet: Accepted connection on port 1094 from ***@***.*** > 230310 16:31:05.203361 31158 Xrd_ProtLoad: matched port 1094 protocol xroot > 230310 16:31:05.203416 31158 ***@***.*** Xrd_Poll: FD 59 attached to poller 0; num=134 > 230310 16:31:05.203432 31158 ***@***.*** Xrootd_Protocol: 0000 req=protocol dlen=0 > 230310 16:31:05.203440 31158 ***@***.*** Xrootd_Response: 0000 sending 8 data bytes; status=0 > 230310 16:31:05.203491 31158 ***@***.*** TLS_Accept: Accepting a TLS connection... > 230310 16:31:05.228423 31158 XrdLinkXeq: ***@***.*** connection upgraded to TLSv1.2 > 230310 16:31:05.229509 31158 XrootdXeq: ***@***.*** disc 0:00:00 > ``` > > This was with my attempt at setting the server-side logging to maximum. Please let me know if further changes would help: > ``` > xrootd.trace all > xrd.trace all > > xrd.tls /etc/grid-security/xrd/xrdcert.pem /etc/grid-security/xrd/xrdkey.pem > xrd.tlsca noverify > xrootd.tls capable all -data > ``` > > Client command: > > ``` > export XRD_NETWORKSTACK=IPv4 > export XRD_LOGLEVEL=Dump > xrdcp roots://xrootd-local.unl.edu//store/hello_world.txt - > ``` > > <details> > <summary>Client output for the server logs above</summary> > > ``` > [2023-03-10 16:31:05.174083 -0600][Debug ][Utility ] Initializing xrootd client version: v5.5.3 > [2023-03-10 16:31:05.174666 -0600][Debug ][Utility ] Unable to process user config file: [ERROR] OS Error: no such file or directory > [2023-03-10 16:31:05.174826 -0600][Info ][Utility ] Env: Importing from shell XRD_NETWORKSTACK=IPv4 as networkstack > [2023-03-10 16:31:05.174877 -0600][Debug ][PlugInMgr ] Initializing plug-in manager... > [2023-03-10 16:31:05.174895 -0600][Debug ][PlugInMgr ] No default plug-in, loading plug-in configs... > [2023-03-10 16:31:05.174906 -0600][Debug ][PlugInMgr ] Processing plug-in definitions in /etc/xrootd/client.plugins.d... > [2023-03-10 16:31:05.174971 -0600][Dump ][PlugInMgr ] Processing: /etc/xrootd/client.plugins.d/recorder.conf > [2023-03-10 16:31:05.175033 -0600][Dump ][PlugInMgr ] Settings from '/etc/xrootd/client.plugins.d/recorder.conf': url='*', lib='/usr/lib64/libXrdClRecorder-5.so', enable='false' > [2023-03-10 16:31:05.175048 -0600][Debug ][PlugInMgr ] Trying to disable plug-in for '*' > [2023-03-10 16:31:05.175133 -0600][Debug ][PlugInMgr ] Processing plug-in definitions in /home/swanson/jthiltge/.xrootd/client.plugins.d... > [2023-03-10 16:31:05.175161 -0600][Debug ][PlugInMgr ] Unable to process directory /home/swanson/jthiltge/.xrootd/client.plugins.d: [ERROR] OS Error: no such file or directory > [2023-03-10 16:31:05.176939 -0600][Dump ][App ] Chunk size: 8388608, parallel chunks 4, streams: 1 > [2023-03-10 16:31:05.176962 -0600][Dump ][App ] Processing source entry: roots://xrootd-local.unl.edu//store/hello_world.txt, type other, target file: - > [2023-03-10 16:31:05.177134 -0600][Dump ][Utility ] Adding job with properties: 'checkSumMode' = 'none', 'checkSumPreset' = '', 'checkSumType' = '', 'chunkSize' = '8388608', 'coerce' = '0', 'continue' = '0', 'cpTimeout' = '0', 'delegate' = '0', 'doServer' = '0', 'dynamicSource' = '0', 'force' = '0', 'initTimeout' = '600', 'makeDir' = '0', 'parallelChunks' = '4', 'posc' = '0', 'preserveXAttr' = '0', 'rmOnBadCksum' = '0', 'source' = 'roots://xrootd-local.unl.edu//store/hello_world.txt', 'target' = '-', 'targetIsDir' = '0', 'thirdParty' = 'none', 'tpcTimeout' = '1800', 'xcp' = '0', 'xcpBlockSize' = '134217728', 'xrate' = '0', 'xrateThreshold' = '0', 'zipAppend' = '0', 'zipArchive' = '0' > [2023-03-10 16:31:05.177172 -0600][Debug ][Utility ] CopyProcess: 2 jobs to prepare > [2023-03-10 16:31:05.177211 -0600][Dump ][Utility ] URL: roots://xrootd-local.unl.edu//store/hello_world.txt > [2023-03-10 16:31:05.177211 -0600][Dump ][Utility ] Protocol: roots > [2023-03-10 16:31:05.177211 -0600][Dump ][Utility ] User Name: > [2023-03-10 16:31:05.177211 -0600][Dump ][Utility ] Password: > [2023-03-10 16:31:05.177211 -0600][Dump ][Utility ] Host Name: xrootd-local.unl.edu > [2023-03-10 16:31:05.177211 -0600][Dump ][Utility ] Port: 1094 > [2023-03-10 16:31:05.177211 -0600][Dump ][Utility ] Path: /store/hello_world.txt > [2023-03-10 16:31:05.177244 -0600][Dump ][Utility ] URL: - > [2023-03-10 16:31:05.177244 -0600][Dump ][Utility ] Protocol: stdio > [2023-03-10 16:31:05.177244 -0600][Dump ][Utility ] User Name: > [2023-03-10 16:31:05.177244 -0600][Dump ][Utility ] Password: > [2023-03-10 16:31:05.177244 -0600][Dump ][Utility ] Host Name: > [2023-03-10 16:31:05.177244 -0600][Dump ][Utility ] Port: 0 > [2023-03-10 16:31:05.177244 -0600][Dump ][Utility ] Path: - > [2023-03-10 16:31:05.177288 -0600][Dump ][Utility ] URL: roots://xrootd-local.unl.edu//store/hello_world.txt > [2023-03-10 16:31:05.177288 -0600][Dump ][Utility ] Protocol: roots > [2023-03-10 16:31:05.177288 -0600][Dump ][Utility ] User Name: > [2023-03-10 16:31:05.177288 -0600][Dump ][Utility ] Password: > [2023-03-10 16:31:05.177288 -0600][Dump ][Utility ] Host Name: xrootd-local.unl.edu > [2023-03-10 16:31:05.177288 -0600][Dump ][Utility ] Port: 1094 > [2023-03-10 16:31:05.177288 -0600][Dump ][Utility ] Path: /store/hello_world.txt > [2023-03-10 16:31:05.177326 -0600][Dump ][Utility ] URL: - > [2023-03-10 16:31:05.177326 -0600][Dump ][Utility ] Protocol: stdio > [2023-03-10 16:31:05.177326 -0600][Dump ][Utility ] User Name: > [2023-03-10 16:31:05.177326 -0600][Dump ][Utility ] Password: > [2023-03-10 16:31:05.177326 -0600][Dump ][Utility ] Host Name: > [2023-03-10 16:31:05.177326 -0600][Dump ][Utility ] Port: 0 > [2023-03-10 16:31:05.177326 -0600][Dump ][Utility ] Path: - > [2023-03-10 16:31:05.177344 -0600][Debug ][Utility ] Creating a classic copy job, from roots://xrootd-local.unl.edu:1094//store/hello_world.txt to stdio://- > [2023-03-10 16:31:05.177375 -0600][Debug ][Utility ] Monitor library name not set. No monitoring > [2023-03-10 16:31:05.177459 -0600][Debug ][Utility ] Opening roots://xrootd-local.unl.edu:1094//store/hello_world.txt for reading > [2023-03-10 16:31:05.177496 -0600][Dump ][Utility ] URL: roots://xrootd-local.unl.edu:1094//store/hello_world.txt > [2023-03-10 16:31:05.177496 -0600][Dump ][Utility ] Protocol: roots > [2023-03-10 16:31:05.177496 -0600][Dump ][Utility ] User Name: > [2023-03-10 16:31:05.177496 -0600][Dump ][Utility ] Password: > [2023-03-10 16:31:05.177496 -0600][Dump ][Utility ] Host Name: xrootd-local.unl.edu > [2023-03-10 16:31:05.177496 -0600][Dump ][Utility ] Port: 1094 > [2023-03-10 16:31:05.177496 -0600][Dump ][Utility ] Path: /store/hello_world.txt > [2023-03-10 16:31:05.177531 -0600][Dump ][Utility ] URL: roots://xrootd-local.unl.edu:1094//store/hello_world.txt > [2023-03-10 16:31:05.177531 -0600][Dump ][Utility ] Protocol: roots > [2023-03-10 16:31:05.177531 -0600][Dump ][Utility ] User Name: > [2023-03-10 16:31:05.177531 -0600][Dump ][Utility ] Password: > [2023-03-10 16:31:05.177531 -0600][Dump ][Utility ] Host Name: xrootd-local.unl.edu > [2023-03-10 16:31:05.177531 -0600][Dump ][Utility ] Port: 1094 > [2023-03-10 16:31:05.177531 -0600][Dump ][Utility ] Path: /store/hello_world.txt > [2023-03-10 16:31:05.177563 -0600][Dump ][Utility ] URL: roots://xrootd-local.unl.edu:1094//store/hello_world.txt > [2023-03-10 16:31:05.177563 -0600][Dump ][Utility ] Protocol: roots > [2023-03-10 16:31:05.177563 -0600][Dump ][Utility ] User Name: > [2023-03-10 16:31:05.177563 -0600][Dump ][Utility ] Password: > [2023-03-10 16:31:05.177563 -0600][Dump ][Utility ] Host Name: xrootd-local.unl.edu > [2023-03-10 16:31:05.177563 -0600][Dump ][Utility ] Port: 1094 > [2023-03-10 16:31:05.177563 -0600][Dump ][Utility ] Path: /store/hello_world.txt > [2023-03-10 16:31:05.177648 -0600][Debug ][File ] ***@***.***://xrootd-local.unl.edu:1094//store/hello_world.txt?xrdcl.requuid=49bd656c-2498-4298-b14d-b3036dc47ad9] Sending an open command > [2023-03-10 16:31:05.177695 -0600][Debug ][Utility ] Env: trying to get a non-existent string entry: pollerpreference > [2023-03-10 16:31:05.177713 -0600][Debug ][Poller ] Available pollers: built-in > [2023-03-10 16:31:05.177723 -0600][Debug ][Poller ] Attempting to create a poller according to preference: built-in > [2023-03-10 16:31:05.177733 -0600][Debug ][Poller ] Creating poller: built-in > [2023-03-10 16:31:05.177748 -0600][Debug ][Poller ] Creating and starting the built-in poller... > [2023-03-10 16:31:05.190940 -0600][Debug ][Poller ] Using 1 poller threads > [2023-03-10 16:31:05.190963 -0600][Debug ][TaskMgr ] Starting the task manager... > [2023-03-10 16:31:05.191032 -0600][Debug ][TaskMgr ] Task manager started > [2023-03-10 16:31:05.191058 -0600][Debug ][JobMgr ] Starting the job manager... > [2023-03-10 16:31:05.191210 -0600][Debug ][JobMgr ] Job manager started, 3 workers > [2023-03-10 16:31:05.191233 -0600][Debug ][TaskMgr ] Registering task: "FileTimer task" to be run at: [2023-03-10 16:31:05 -0600] > [2023-03-10 16:31:05.191246 -0600][Dump ][XRootD ] [xrootd-local.unl.edu:1094] Sending message kXR_open (file: /store/hello_world.txt, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) > [2023-03-10 16:31:05.191282 -0600][Debug ][ExDbgMsg ] [xrootd-local.unl.edu:1094] MsgHandler created: 0x262c680 (message: kXR_open (file: /store/hello_world.txt, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) ). > [2023-03-10 16:31:05.191338 -0600][Dump ][Utility ] URL: xrootd-local.unl.edu:1094 > [2023-03-10 16:31:05.191338 -0600][Dump ][Utility ] Protocol: root > [2023-03-10 16:31:05.191338 -0600][Dump ][Utility ] User Name: > [2023-03-10 16:31:05.191338 -0600][Dump ][Utility ] Password: > [2023-03-10 16:31:05.191338 -0600][Dump ][Utility ] Host Name: xrootd-local.unl.edu > [2023-03-10 16:31:05.191338 -0600][Dump ][Utility ] Port: 1094 > [2023-03-10 16:31:05.191338 -0600][Dump ][Utility ] Path: > [2023-03-10 16:31:05.191375 -0600][Dump ][Utility ] URL: roots://xrootd-local.unl.edu:1094/ > [2023-03-10 16:31:05.191375 -0600][Dump ][Utility ] Protocol: roots > [2023-03-10 16:31:05.191375 -0600][Dump ][Utility ] User Name: > [2023-03-10 16:31:05.191375 -0600][Dump ][Utility ] Password: > [2023-03-10 16:31:05.191375 -0600][Dump ][Utility ] Host Name: xrootd-local.unl.edu > [2023-03-10 16:31:05.191375 -0600][Dump ][Utility ] Port: 1094 > [2023-03-10 16:31:05.191375 -0600][Dump ][Utility ] Path: > [2023-03-10 16:31:05.191409 -0600][Debug ][PostMaster ] Creating new channel to: roots://xrootd-local.unl.edu:1094/ > [2023-03-10 16:31:05.191447 -0600][Debug ][PostMaster ] [xrootd-local.unl.edu:1094] Stream parameters: Network Stack: IPv4, Connection Window: 120, ConnectionRetry: 5, Stream Error Window: 1800 > [2023-03-10 16:31:05.191474 -0600][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: roots://xrootd-local.unl.edu:1094/" to be run at: [2023-03-10 16:31:20 -0600] > [2023-03-10 16:31:05.191495 -0600][Dump ][PostMaster ] [xrootd-local.unl.edu:1094] Sending message kXR_open (file: /store/hello_world.txt, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) (0x2626bb0) through substream 0 expecting answer at 0 > [2023-03-10 16:31:05.191784 -0600][Debug ][PostMaster ] [xrootd-local.unl.edu:1094] Found 1 address(es): [::ffff:129.93.239.164]:1094 > [2023-03-10 16:31:05.191829 -0600][Debug ][AsyncSock ] [xrootd-local.unl.edu:1094.0] Attempting connection to [::ffff:129.93.239.164]:1094 > [2023-03-10 16:31:05.191882 -0600][Debug ][Poller ] Adding socket 0x262cbb0 to the poller > [2023-03-10 16:31:05.199991 -0600][Debug ][AsyncSock ] [xrootd-local.unl.edu:1094.0] Async connection call returned > [2023-03-10 16:31:05.200049 -0600][Debug ][XRootDTransport ] [xrootd-local.unl.edu:1094.0] Sending out the initial hand shake + kXR_protocol > [2023-03-10 16:31:05.200102 -0600][Dump ][AsyncSock ] [xrootd-local.unl.edu:1094.0] Wrote a message: (0xf4000950), 44 bytes > [2023-03-10 16:31:05.204472 -0600][Dump ][XRootDTransport ] [msg: 0xf4000ab0] Expecting 8 bytes of message body > [2023-03-10 16:31:05.204490 -0600][Dump ][AsyncSock ] [xrootd-local.unl.edu:1094.0] Received message header, size: 8 > [2023-03-10 16:31:05.204505 -0600][Dump ][AsyncSock ] [xrootd-local.unl.edu:1094.0] Received a message of 16 bytes > [2023-03-10 16:31:05.204521 -0600][Debug ][XRootDTransport ] [xrootd-local.unl.edu:1094.0] Got the server hand shake response (type: manager [], protocol version 511) > [2023-03-10 16:31:05.204542 -0600][Dump ][XRootDTransport ] [msg: 0xf4000ab0] Expecting 8 bytes of message body > [2023-03-10 16:31:05.204568 -0600][Dump ][AsyncSock ] [xrootd-local.unl.edu:1094.0] Received message header, size: 8 > [2023-03-10 16:31:05.204583 -0600][Dump ][AsyncSock ] [xrootd-local.unl.edu:1094.0] Received a message of 16 bytes > [2023-03-10 16:31:05.204600 -0600][Debug ][XRootDTransport ] [xrootd-local.unl.edu:1094.0] kXR_protocol successful (type: manager [], protocol version 511) > [2023-03-10 16:31:05.205173 -0600][Debug ][XRootDTransport ] [xrootd-local.unl.edu:1094.0] Sending out kXR_login request, username: jthiltge, cgi: xrd.cc=us&xrd.tz=-6&xrd.appname=xrdcp&xrd.info=&xrd.hostname=t3.unl.edu&xrd.rn=v5.5.3, dual-stack: true, private IPv4: false, private IPv6: false > [2023-03-10 16:31:05.205206 -0600][Debug ][AsyncSock ] [xrootd-local.unl.edu:1094.0] TLS hand-shake exchange. > [2023-03-10 16:31:05.205968 -0600][Dump ][TaskMgr ] Running task: "FileTimer task" > [2023-03-10 16:31:05.206001 -0600][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2023-03-10 16:31:20 -0600] > [2023-03-10 16:31:05.221931 -0600][Debug ][AsyncSock ] [xrootd-local.unl.edu:1094.0] TLS hand-shake exchange. > [2023-03-10 16:31:05.226144 -0600][Debug ][AsyncSock ] [xrootd-local.unl.edu:1094.0] TLS hand-shake exchange. > [2023-03-10 16:31:05.226286 -0600][Info ][AsyncSock ] [xrootd-local.unl.edu:1094.0] TLS hand-shake done. > [2023-03-10 16:31:05.226354 -0600][Dump ][AsyncSock ] [xrootd-local.unl.edu:1094.0] Wrote a message: (0xf4000be0), 109 bytes > [2023-03-10 16:31:05.226639 -0600][Error ][TlsMsg ] [] TLS error rc=-1 ec=5 (error_syscall) errno=104. > [2023-03-10 16:31:05.226665 -0600][Error ][AsyncSock ] [xrootd-local.unl.edu:1094.0] Socket error while handshaking: [ERROR] TLS error > [2023-03-10 16:31:05.226679 -0600][Debug ][AsyncSock ] [xrootd-local.unl.edu:1094.0] Closing the socket > [2023-03-10 16:31:05.226694 -0600][Debug ][Poller ] <[::ffff:129.93.239.166]:64262><--><[::ffff:129.93.239.164]:1094> Removing socket from the poller > [2023-03-10 16:31:05.226784 -0600][Error ][PostMaster ] [xrootd-local.unl.edu:1094] elapsed = 0, pConnectionWindow = 120 seconds. > [2023-03-10 16:31:05.226800 -0600][Info ][PostMaster ] [xrootd-local.unl.edu:1094] Attempting reconnection in 120 seconds. > [2023-03-10 16:31:05.226813 -0600][Debug ][TaskMgr ] Registering task: "StreamConnectorTask for xrootd-local.unl.edu:1094" to be run at: [2023-03-10 16:33:05 -0600] > [2023-03-10 16:31:20.207211 -0600][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: roots://xrootd-local.unl.edu:1094/" > [2023-03-10 16:31:20.207296 -0600][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: roots://xrootd-local.unl.edu:1094/" at [2023-03-10 16:31:35 -0600] > [2023-03-10 16:31:20.207315 -0600][Dump ][TaskMgr ] Running task: "FileTimer task" > [2023-03-10 16:31:20.207338 -0600][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2023-03-10 16:31:35 -0600] > [2023-03-10 16:31:35.209873 -0600][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: roots://xrootd-local.unl.edu:1094/" > [2023-03-10 16:31:35.209953 -0600][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: roots://xrootd-local.unl.edu:1094/" at [2023-03-10 16:31:50 -0600] > [2023-03-10 16:31:35.209973 -0600][Dump ][TaskMgr ] Running task: "FileTimer task" > [2023-03-10 16:31:35.209993 -0600][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2023-03-10 16:31:50 -0600] > [2023-03-10 16:31:50.212835 -0600][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: roots://xrootd-local.unl.edu:1094/" > [2023-03-10 16:31:50.212900 -0600][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: roots://xrootd-local.unl.edu:1094/" at [2023-03-10 16:32:05 -0600] > [2023-03-10 16:31:50.212936 -0600][Dump ][TaskMgr ] Running task: "FileTimer task" > [2023-03-10 16:31:50.212958 -0600][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2023-03-10 16:32:05 -0600] > [2023-03-10 16:32:05.215786 -0600][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: roots://xrootd-local.unl.edu:1094/" > [2023-03-10 16:32:05.215846 -0600][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: roots://xrootd-local.unl.edu:1094/" at [2023-03-10 16:32:20 -0600] > [2023-03-10 16:32:05.215857 -0600][Dump ][TaskMgr ] Running task: "FileTimer task" > [2023-03-10 16:32:05.215870 -0600][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2023-03-10 16:32:20 -0600] > [2023-03-10 16:32:20.218888 -0600][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: roots://xrootd-local.unl.edu:1094/" > [2023-03-10 16:32:20.218963 -0600][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: roots://xrootd-local.unl.edu:1094/" at [2023-03-10 16:32:35 -0600] > [2023-03-10 16:32:20.218982 -0600][Dump ][TaskMgr ] Running task: "FileTimer task" > [2023-03-10 16:32:20.219001 -0600][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2023-03-10 16:32:35 -0600] > [2023-03-10 16:32:35.221853 -0600][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: roots://xrootd-local.unl.edu:1094/" > [2023-03-10 16:32:35.221925 -0600][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: roots://xrootd-local.unl.edu:1094/" at [2023-03-10 16:32:50 -0600] > [2023-03-10 16:32:35.221948 -0600][Dump ][TaskMgr ] Running task: "FileTimer task" > [2023-03-10 16:32:35.221967 -0600][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2023-03-10 16:32:50 -0600] > [2023-03-10 16:32:50.224723 -0600][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: roots://xrootd-local.unl.edu:1094/" > [2023-03-10 16:32:50.224790 -0600][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: roots://xrootd-local.unl.edu:1094/" at [2023-03-10 16:33:05 -0600] > [2023-03-10 16:32:50.224809 -0600][Dump ][TaskMgr ] Running task: "FileTimer task" > [2023-03-10 16:32:50.224828 -0600][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2023-03-10 16:33:05 -0600] > [2023-03-10 16:33:05.227458 -0600][Dump ][TaskMgr ] Running task: "StreamConnectorTask for xrootd-local.unl.edu:1094" > [2023-03-10 16:33:05.227909 -0600][Debug ][PostMaster ] [xrootd-local.unl.edu:1094] Found 1 address(es): [::ffff:129.93.239.164]:1094 > [2023-03-10 16:33:05.227982 -0600][Debug ][AsyncSock ] [xrootd-local.unl.edu:1094.0] Attempting connection to [::ffff:129.93.239.164]:1094 > [2023-03-10 16:33:05.228045 -0600][Debug ][Poller ] Adding socket 0x262cbb0 to the poller > [2023-03-10 16:33:05.228082 -0600][Debug ][TaskMgr ] Done with task: "StreamConnectorTask for xrootd-local.unl.edu:1094" > [2023-03-10 16:33:05.228093 -0600][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: roots://xrootd-local.unl.edu:1094/" > [2023-03-10 16:33:05.228106 -0600][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: roots://xrootd-local.unl.edu:1094/" at [2023-03-10 16:33:20 -0600] > [2023-03-10 16:33:05.228114 -0600][Dump ][TaskMgr ] Running task: "FileTimer task" > [2023-03-10 16:33:05.228125 -0600][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2023-03-10 16:33:20 -0600] > [2023-03-10 16:33:05.228361 -0600][Debug ][AsyncSock ] [xrootd-local.unl.edu:1094.0] Async connection call returned > [2023-03-10 16:33:05.228398 -0600][Debug ][XRootDTransport ] [xrootd-local.unl.edu:1094.0] Sending out the initial hand shake + kXR_protocol > [2023-03-10 16:33:05.228448 -0600][Dump ][AsyncSock ] [xrootd-local.unl.edu:1094.0] Wrote a message: (0xf40430f0), 44 bytes > [2023-03-10 16:33:05.228704 -0600][Dump ][XRootDTransport ] [msg: 0xf4020400] Expecting 8 bytes of message body > [2023-03-10 16:33:05.228726 -0600][Dump ][AsyncSock ] [xrootd-local.unl.edu:1094.0] Received message header, size: 8 > [2023-03-10 16:33:05.228744 -0600][Dump ][AsyncSock ] [xrootd-local.unl.edu:1094.0] Received a message of 16 bytes > [2023-03-10 16:33:05.228768 -0600][Debug ][XRootDTransport ] [xrootd-local.unl.edu:1094.0] Got the server hand shake response (type: manager [], protocol version 511) > [2023-03-10 16:33:05.228790 -0600][Dump ][XRootDTransport ] [msg: 0xf4020400] Expecting 8 bytes of message body > [2023-03-10 16:33:05.228799 -0600][Dump ][AsyncSock ] [xrootd-local.unl.edu:1094.0] Received message header, size: 8 > [2023-03-10 16:33:05.228809 -0600][Dump ][AsyncSock ] [xrootd-local.unl.edu:1094.0] Received a message of 16 bytes > [2023-03-10 16:33:05.228821 -0600][Debug ][XRootDTransport ] [xrootd-local.unl.edu:1094.0] kXR_protocol successful (type: manager [], protocol version 511) > [2023-03-10 16:33:05.229414 -0600][Debug ][XRootDTransport ] [xrootd-local.unl.edu:1094.0] Sending out kXR_login request, username: jthiltge, cgi: xrd.cc=us&xrd.tz=-6&xrd.appname=xrdcp&xrd.info=&xrd.hostname=t3.unl.edu&xrd.rn=v5.5.3, dual-stack: true, private IPv4: false, private IPv6: false > [2023-03-10 16:33:05.229434 -0600][Debug ][AsyncSock ] [xrootd-local.unl.edu:1094.0] TLS hand-shake exchange. > [2023-03-10 16:33:05.231258 -0600][Debug ][AsyncSock ] [xrootd-local.unl.edu:1094.0] TLS hand-shake exchange. > [2023-03-10 16:33:05.232929 -0600][Debug ][AsyncSock ] [xrootd-local.unl.edu:1094.0] TLS hand-shake exchange. > [2023-03-10 16:33:05.233011 -0600][Info ][AsyncSock ] [xrootd-local.unl.edu:1094.0] TLS hand-shake done. > [2023-03-10 16:33:05.233056 -0600][Dump ][AsyncSock ] [xrootd-local.unl.edu:1094.0] Wrote a message: (0xf40401d0), 109 bytes > [2023-03-10 16:33:05.233451 -0600][Dump ][XRootDTransport ] [msg: 0xf40430f0] Expecting 75 bytes of message body > [2023-03-10 16:33:05.233472 -0600][Dump ][AsyncSock ] [xrootd-local.unl.edu:1094.0] Received message header, size: 8 > [2023-03-10 16:33:05.233503 -0600][Dump ][AsyncSock ] [xrootd-local.unl.edu:1094.0] Received a message of 83 bytes > [2023-03-10 16:33:05.233521 -0600][Debug ][XRootDTransport ] [xrootd-local.unl.edu:1094.0] Logged in, session: 732400007f7900007302000088290000 > [2023-03-10 16:33:05.233530 -0600][Debug ][XRootDTransport ] [xrootd-local.unl.edu:1094.0] Authentication is required: &P=gsi,v:10600,c:ssl,ca:ba240aa8.0|f5f0dfc2.0&P=ztn,0:4096: > [2023-03-10 16:33:05.233540 -0600][Debug ][XRootDTransport ] [xrootd-local.unl.edu:1094.0] Sending authentication data > [2023-03-10 16:33:05.234929 -0600][Debug ][XRootDTransport ] [xrootd-local.unl.edu:1094.0] Trying to authenticate using gsi > [2023-03-10 16:33:07.099272 -0600][Dump ][AsyncSock ] [xrootd-local.unl.edu:1094.0] Wrote a message: (0xf40584f0), 136 bytes > [2023-03-10 16:33:07.106989 -0600][Dump ][XRootDTransport ] [msg: 0xf40439d0] Expecting 3252 bytes of message body > [2023-03-10 16:33:07.107017 -0600][Dump ][AsyncSock ] [xrootd-local.unl.edu:1094.0] Received message header, size: 8 > [2023-03-10 16:33:07.107081 -0600][Dump ][AsyncSock ] [xrootd-local.unl.edu:1094.0] Received a message of 3260 bytes > [2023-03-10 16:33:07.107096 -0600][Debug ][XRootDTransport ] [xrootd-local.unl.edu:1094.0] Sending more authentication data for gsi > [2023-03-10 16:33:07.117248 -0600][Dump ][AsyncSock ] [xrootd-local.unl.edu:1094.0] Wrote a message: (0xf4057df0), 9932 bytes > [2023-03-10 16:33:07.132028 -0600][Dump ][XRootDTransport ] [msg: 0xf40401d0] Expecting 0 bytes of message body > [2023-03-10 16:33:07.132059 -0600][Dump ][AsyncSock ] [xrootd-local.unl.edu:1094.0] Received message header, size: 8 > [2023-03-10 16:33:07.132076 -0600][Dump ][AsyncSock ] [xrootd-local.unl.edu:1094.0] Received a message of 8 bytes > [2023-03-10 16:33:07.132144 -0600][Debug ][XRootDTransport ] [xrootd-local.unl.edu:1094.0] Authenticated with gsi. > [2023-03-10 16:33:07.132164 -0600][Debug ][PostMaster ] [xrootd-local.unl.edu:1094] Stream 0 connected (IPv4). > [2023-03-10 16:33:07.132201 -0600][Dump ][AsyncSock ] [xrootd-local.unl.edu:1094.0] Wrote a message: kXR_open (file: /store/hello_world.txt, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) (0x2626bb0), 46 bytes > [2023-03-10 16:33:07.132234 -0600][Dump ][AsyncSock ] [xrootd-local.unl.edu:1094.0] Successfully sent message: kXR_open (file: /store/hello_world.txt, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) (0x2626bb0). > [2023-03-10 16:33:07.132249 -0600][Dump ][XRootD ] [xrootd-local.unl.edu:1094] Message kXR_open (file: /store/hello_world.txt, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) has been successfully sent. > [2023-03-10 16:33:07.132258 -0600][Debug ][ExDbgMsg ] [xrootd-local.unl.edu:1094] Moving MsgHandler: 0x262c680 (message: kXR_open (file: /store/hello_world.txt, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) ) from out-queu to in-queue. > [2023-03-10 16:33:07.132269 -0600][Dump ][PostMaster ] [xrootd-local.unl.edu:1094.0] All messages consumed, disable uplink > [2023-03-10 16:33:07.132893 -0600][Dump ][XRootDTransport ] [msg: 0xf4043630] Expecting 21 bytes of message body > [2023-03-10 16:33:07.132937 -0600][Dump ][AsyncSock ] [xrootd-local.unl.edu:1094.0] Received message header for 0xf4043630 size: 8 > [2023-03-10 16:33:07.132961 -0600][Debug ][ExDbgMsg ] [msg: 0xf4043630] Assigned MsgHandler: 0x262c680. > [2023-03-10 16:33:07.132977 -0600][Debug ][ExDbgMsg ] [handler: 0x262c680] Removed MsgHandler: 0x262c680 from the in-queue. > [2023-03-10 16:33:07.133028 -0600][Dump ][AsyncSock ] [xrootd-local.unl.edu:1094.0] Received message 0xf4043630 of 29 bytes > [2023-03-10 16:33:07.133043 -0600][Dump ][PostMaster ] [xrootd-local.unl.edu:1094] Handling received message: 0xf4043630. > [2023-03-10 16:33:07.133218 -0600][Dump ][XRootD ] [xrootd-local.unl.edu:1094] Got kXR_redirect response to message kXR_open (file: /store/hello_world.txt, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ): red-xfer3.unl.edu, port 1094 > [2023-03-10 16:33:07.133258 -0600][Dump ][XRootD ] [xrootd-local.unl.edu:1094] Current server has been assigned as a load-balancer for message kXR_open (file: /store/hello_world.txt, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) > [2023-03-10 16:33:07.133301 -0600][Dump ][Utility ] URL: red-xfer3.unl.edu:1094/ > [2023-03-10 16:33:07.133301 -0600][Dump ][Utility ] Protocol: root > [2023-03-10 16:33:07.133301 -0600][Dump ][Utility ] User Name: > [2023-03-10 16:33:07.133301 -0600][Dump ][Utility ] Password: > [2023-03-10 16:33:07.133301 -0600][Dump ][Utility ] Host Name: red-xfer3.unl.edu > [2023-03-10 16:33:07.133301 -0600][Dump ][Utility ] Port: 1094 > [2023-03-10 16:33:07.133301 -0600][Dump ][Utility ] Path: > [2023-03-10 16:33:07.133332 -0600][Dump ][Utility ] URL: fake://fake:111//fake?xrdcl.requuid=49bd656c-2498-4298-b14d-b3036dc47ad9& > [2023-03-10 16:33:07.133332 -0600][Dump ][Utility ] Protocol: fake > [2023-03-10 16:33:07.133332 -0600][Dump ][Utility ] User Name: > [2023-03-10 16:33:07.133332 -0600][Dump ][Utility ] Password: > [2023-03-10 16:33:07.133332 -0600][Dump ][Utility ] Host Name: fake > [2023-03-10 16:33:07.133332 -0600][Dump ][Utility ] Port: 111 > [2023-03-10 16:33:07.133332 -0600][Dump ][Utility ] Path: /fake > [2023-03-10 16:33:07.133373 -0600][Dump ][Utility ] URL: fake://fake:111//store/hello_world.txt > [2023-03-10 16:33:07.133373 -0600][Dump ][Utility ] Protocol: fake > [2023-03-10 16:33:07.133373 -0600][Dump ][Utility ] User Name: > [2023-03-10 16:33:07.133373 -0600][Dump ][Utility ] Password: > [2023-03-10 16:33:07.133373 -0600][Dump ][Utility ] Host Name: fake > [2023-03-10 16:33:07.133373 -0600][Dump ][Utility ] Port: 111 > [2023-03-10 16:33:07.133373 -0600][Dump ][Utility ] Path: /store/hello_world.txt > [2023-03-10 16:33:07.133415 -0600][Dump ][Utility ] URL: roots://xrootd-local.unl.edu:1094//store/hello_world.txt > [2023-03-10 16:33:07.133415 -0600][Dump ][Utility ] Protocol: roots > [2023-03-10 16:33:07.133415 -0600][Dump ][Utility ] User Name: > [2023-03-10 16:33:07.133415 -0600][Dump ][Utility ] Password: > [2023-03-10 16:33:07.133415 -0600][Dump ][Utility ] Host Name: xrootd-local.unl.edu > [2023-03-10 16:33:07.133415 -0600][Dump ][Utility ] Port: 1094 > [2023-03-10 16:33:07.133415 -0600][Dump ][Utility ] Path: /store/hello_world.txt > [2023-03-10 16:33:07.133443 -0600][Dump ][Utility ] URL: roots://red-xfer3.unl.edu:1094/ > [2023-03-10 16:33:07.133443 -0600][Dump ][Utility ] Protocol: roots > [2023-03-10 16:33:07.133443 -0600][Dump ][Utility ] User Name: > [2023-03-10 16:33:07.133443 -0600][Dump ][Utility ] Password: > [2023-03-10 16:33:07.133443 -0600][Dump ][Utility ] Host Name: red-xfer3.unl.edu > [2023-03-10 16:33:07.133443 -0600][Dump ][Utility ] Port: 1094 > [2023-03-10 16:33:07.133443 -0600][Dump ][Utility ] Path: > [2023-03-10 16:33:07.133479 -0600][Debug ][ExDbgMsg ] [red-xfer3.unl.edu:1094] Retry at server MsgHandler: 0x262c680 (message: kXR_open (file: /store/hello_world.txt, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) ). > [2023-03-10 16:33:07.133502 -0600][Dump ][Utility ] URL: red-xfer3.unl.edu:1094 > [2023-03-10 16:33:07.133502 -0600][Dump ][Utility ] Protocol: root > [2023-03-10 16:33:07.133502 -0600][Dump ][Utility ] User Name: > [2023-03-10 16:33:07.133502 -0600][Dump ][Utility ] Password: > [2023-03-10 16:33:07.133502 -0600][Dump ][Utility ] Host Name: red-xfer3.unl.edu > [2023-03-10 16:33:07.133502 -0600][Dump ][Utility ] Port: 1094 > [2023-03-10 16:33:07.133502 -0600][Dump ][Utility ] Path: > [2023-03-10 16:33:07.133526 -0600][Dump ][Utility ] URL: roots://red-xfer3.unl.edu:1094/ > [2023-03-10 16:33:07.133526 -0600][Dump ][Utility ] Protocol: roots > [2023-03-10 16:33:07.133526 -0600][Dump ][Utility ] User Name: > [2023-03-10 16:33:07.133526 -0600][Dump ][Utility ] Password: > [2023-03-10 16:33:07.133526 -0600][Dump ][Utility ] Host Name: red-xfer3.unl.edu > [2023-03-10 16:33:07.133526 -0600][Dump ][Utility ] Port: 1094 > [2023-03-10 16:33:07.133526 -0600][Dump ][Utility ] Path: > [2023-03-10 16:33:07.133545 -0600][Debug ][PostMaster ] Creating new channel to: roots://red-xfer3.unl.edu:1094/ > [2023-03-10 16:33:07.133569 -0600][Debug ][PostMaster ] [red-xfer3.unl.edu:1094] Stream parameters: Network Stack: IPv4, Connection Window: 120, ConnectionRetry: 5, Stream Error Window: 1800 > [2023-03-10 16:33:07.133586 -0600][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: roots://red-xfer3.unl.edu:1094/" to be run at: [2023-03-10 16:33:22 -0600] > [2023-03-10 16:33:07.133598 -0600][Dump ][PostMaster ] [red-xfer3.unl.edu:1094] Sending message kXR_open (file: /store/hello_world.txt, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) (0x2626bb0) through substream 0 expecting answer at 0 > [2023-03-10 16:33:07.133967 -0600][Debug ][PostMaster ] [red-xfer3.unl.edu:1094] Found 1 address(es): [::ffff:129.93.239.173]:1094 > [2023-03-10 16:33:07.133999 -0600][Debug ][AsyncSock ] [red-xfer3.unl.edu:1094.0] Attempting connection to [::ffff:129.93.239.173]:1094 > [2023-03-10 16:33:07.134039 -0600][Debug ][Poller ] Adding socket 0xf0001e00 to the poller > [2023-03-10 16:33:07.134882 -0600][Debug ][AsyncSock ] [red-xfer3.unl.edu:1094.0] Async connection call returned > [2023-03-10 16:33:07.134937 -0600][Debug ][XRootDTransport ] [red-xfer3.unl.edu:1094.0] Sending out the initial hand shake + kXR_protocol > [2023-03-10 16:33:07.134972 -0600][Dump ][AsyncSock ] [red-xfer3.unl.edu:1094.0] Wrote a message: (0xf4000be0), 44 bytes > [2023-03-10 16:33:07.135482 -0600][Dump ][XRootDTransport ] [msg: 0xf4039090] Expecting 8 bytes of message body > [2023-03-10 16:33:07.135518 -0600][Dump ][AsyncSock ] [red-xfer3.unl.edu:1094.0] Received message header, size: 8 > [2023-03-10 16:33:07.135539 -0600][Dump ][AsyncSock ] [red-xfer3.unl.edu:1094.0] Received a message of 16 bytes > [2023-03-10 16:33:07.135557 -0600][Debug ][XRootDTransport ] [red-xfer3.unl.edu:1094.0] Got the server hand shake response (type: server [], protocol version 511) > [2023-03-10 16:33:07.135584 -0600][Dump ][XRootDTransport ] [msg: 0xf4039090] Expecting 14 bytes of message body > [2023-03-10 16:33:07.135599 -0600][Dump ][AsyncSock ] [red-xfer3.unl.edu:1094.0] Received message header, size: 8 > [2023-03-10 16:33:07.135610 -0600][Dump ][AsyncSock ] [red-xfer3.unl.edu:1094.0] Received a message of 22 bytes > [2023-03-10 16:33:07.135623 -0600][Debug ][XRootDTransport ] [red-xfer3.unl.edu:1094.0] kXR_protocol successful (type: server [], protocol version 511) > [2023-03-10 16:33:07.136169 -0600][Debug ][XRootDTransport ] [red-xfer3.unl.edu:1094.0] Sending out kXR_login request, username: jthiltge, cgi: xrd.cc=us&xrd.tz=-6&xrd.appname=xrdcp&xrd.info=&xrd.hostname=t3.unl.edu&xrd.rn=v5.5.3, dual-stack: true, private IPv4: false, private IPv6: false > [2023-03-10 16:33:07.136187 -0600][Debug ][AsyncSock ] [red-xfer3.unl.edu:1094.0] TLS hand-shake exchange. > [2023-03-10 16:33:07.138392 -0600][Debug ][AsyncSock ] [red-xfer3.unl.edu:1094.0] TLS hand-shake exchange. > [2023-03-10 16:33:07.140367 -0600][Debug ][AsyncSock ] [red-xfer3.unl.edu:1094.0] TLS hand-shake exchange. > [2023-03-10 16:33:07.140474 -0600][Info ][AsyncSock ] [red-xfer3.unl.edu:1094.0] TLS hand-shake done. > [2023-03-10 16:33:07.140518 -0600][Dump ][AsyncSock ] [red-xfer3.unl.edu:1094.0] Wrote a message: (0xf40430f0), 109 bytes > [2023-03-10 16:33:07.141535 -0600][Dump ][XRootDTransport ] [msg: 0xf4000be0] Expecting 75 bytes of message body > [2023-03-10 16:33:07.141560 -0600][Dump ][AsyncSock ] [red-xfer3.unl.edu:1094.0] Received message header, size: 8 > [2023-03-10 16:33:07.142357 -0600][Dump ][AsyncSock ] [red-xfer3.unl.edu:1094.0] Received a message of 83 bytes > [2023-03-10 16:33:07.142389 -0600][Debug ][XRootDTransport ] [red-xfer3.unl.edu:1094.0] Logged in, session: a1645e003b0500007c00000084156000 > [2023-03-10 16:33:07.142398 -0600][Debug ][XRootDTransport ] [red-xfer3.unl.edu:1094.0] Authentication is required: &P=gsi,v:10600,c:ssl,ca:ba240aa8.0|f5f0dfc2.0&P=ztn,0:4096: > [2023-03-10 16:33:07.142406 -0600][Debug ][XRootDTransport ] [red-xfer3.unl.edu:1094.0] Sending authentication data > [2023-03-10 16:33:07.142433 -0600][Debug ][XRootDTransport ] [red-xfer3.unl.edu:1094.0] Trying to authenticate using gsi > [2023-03-10 16:33:07.149042 -0600][Dump ][AsyncSock ] [red-xfer3.unl.edu:1094.0] Wrote a message: (0xf4076e30), 136 bytes > [2023-03-10 16:33:07.160986 -0600][Dump ][XRootDTransport ] [msg: 0xf4076610] Expecting 3272 bytes of message body > [2023-03-10 16:33:07.161004 -0600][Dump ][AsyncSock ] [red-xfer3.unl.edu:1094.0] Received message header, size: 8 > [2023-03-10 16:33:07.161042 -0600][Dump ][AsyncSock ] [red-xfer3.unl.edu:1094.0] Received a message of 3280 bytes > [2023-03-10 16:33:07.161053 -0600][Debug ][XRootDTransport ] [red-xfer3.unl.edu:1094.0] Sending more authentication data for gsi > [2023-03-10 16:33:07.171097 -0600][Dump ][AsyncSock ] [red-xfer3.unl.edu:1094.0] Wrote a message: (0xf4063b20), 9932 bytes > [2023-03-10 16:33:07.184936 -0600][Dump ][XRootDTransport ] [msg: 0xf40430f0] Expecting 0 bytes of message body > [2023-03-10 16:33:07.184964 -0600][Dump ][AsyncSock ] [red-xfer3.unl.edu:1094.0] Received message header, size: 8 > [2023-03-10 16:33:07.184979 -0600][Dump ][AsyncSock ] [red-xfer3.unl.edu:1094.0] Received a message of 8 bytes > [2023-03-10 16:33:07.185284 -0600][Debug ][XRootDTransport ] [red-xfer3.unl.edu:1094.0] XrdSecProtect loaded. > [2023-03-10 16:33:07.185302 -0600][Debug ][XRootDTransport ] [red-xfer3.unl.edu:1094.0] Authenticated with gsi. > [2023-03-10 16:33:07.185323 -0600][Debug ][PostMaster ] [red-xfer3.unl.edu:1094] Stream 0 connected (IPv4). > [2023-03-10 16:33:07.185353 -0600][Dump ][AsyncSock ] [red-xfer3.unl.edu:1094.0] Wrote a message: kXR_open (file: /store/hello_world.txt, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) (0x2626bb0), 46 bytes > [2023-03-10 16:33:07.185378 -0600][Dump ][AsyncSock ] [red-xfer3.unl.edu:1094.0] Successfully sent message: kXR_open (file: /store/hello_world.txt, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) (0x2626bb0). > [2023-03-10 16:33:07.185390 -0600][Dump ][XRootD ] [red-xfer3.unl.edu:1094] Message kXR_open (file: /store/hello_world.txt, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) has been successfully sent. > [2023-03-10 16:33:07.185398 -0600][Debug ][ExDbgMsg ] [red-xfer3.unl.edu:1094] Moving MsgHandler: 0x262c680 (message: kXR_open (file: /store/hello_world.txt, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) ) from out-queu to in-queue. > [2023-03-10 16:33:07.185407 -0600][Dump ][PostMaster ] [red-xfer3.unl.edu:1094.0] All messages consumed, disable uplink > [2023-03-10 16:33:07.186155 -0600][Dump ][XRootDTransport ] [msg: 0xf4067db0] Expecting 87 bytes of message body > [2023-03-10 16:33:07.186182 -0600][Dump ][AsyncSock ] [red-xfer3.unl.edu:1094.0] Received message header for 0xf4067db0 size: 8 > [2023-03-10 16:33:07.186198 -0600][Debug ][ExDbgMsg ] [msg: 0xf4067db0] Assigned MsgHandler: 0x262c680. > [2023-03-10 16:33:07.186212 -0600][Debug ][ExDbgMsg ] [handler: 0x262c680] Removed MsgHandler: 0x262c680 from the in-queue. > [2023-03-10 16:33:07.186640 -0600][Dump ][AsyncSock ] [red-xfer3.unl.edu:1094.0] Received message 0xf4067db0 of 95 bytes > [2023-03-10 16:33:07.186665 -0600][Dump ][PostMaster ] [red-xfer3.unl.edu:1094] Handling received message: 0xf4067db0. > [2023-03-10 16:33:07.186820 -0600][Dump ][XRootD ] [red-xfer3.unl.edu:1094] Got a kXR_ok response to request kXR_open (file: /store/hello_world.txt, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) > [2023-03-10 16:33:07.186848 -0600][Debug ][ExDbgMsg ] [red-xfer3.unl.edu:1094] Calling MsgHandler: 0x262c680 (message: kXR_open (file: /store/hello_world.txt, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) ) with status: [SUCCESS] . > [2023-03-10 16:33:07.186859 -0600][Dump ][XRootD ] [red-xfer3.unl.edu:1094] Parsing the response to kXR_open (file: /store/hello_world.txt, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) as OpenInfo > [2023-03-10 16:33:07.186868 -0600][Dump ][XRootD ] [red-xfer3.unl.edu:1094] Parsing StatInfo in response to kXR_open (file: /store/hello_world.txt, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) > [2023-03-10 16:33:07.186909 -0600][Debug ][File ] ***@***.***://xrootd-local.unl.edu:1094//store/hello_world.txt?xrdcl.requuid=49bd656c-2498-4298-b14d-b3036dc47ad9] Open has returned with status [SUCCESS] > [2023-03-10 16:33:07.186940 -0600][Debug ][File ] ***@***.***://xrootd-local.unl.edu:1094//store/hello_world.txt?xrdcl.requuid=49bd656c-2498-4298-b14d-b3036dc47ad9] successfully opened at red-xfer3.unl.edu:1094, handle: 0x0, session id: 1 > [2023-03-10 16:33:07.186976 -0600][Debug ][XRootD ] Redirect trace-back: > [2023-03-10 16:33:07.186976 -0600][Debug ][XRootD ] 0. Redirected from: roots://xrootd-local.unl.edu:1094//store/hello_world.txt to: roots://red-xfer3.unl.edu:1094/ > [2023-03-10 16:33:07.186988 -0600][Debug ][ExDbgMsg ] [red-xfer3.unl.edu:1094] Destroying MsgHandler: 0x262c680. > [2023-03-10 16:33:07.187022 -0600][Dump ][Utility ] URL: roots://red-xfer3.unl.edu:1094//store/hello_world.txt?xrdcl.requuid=49bd656c-2498-4298-b14d-b3036dc47ad9 > [2023-03-10 16:33:07.187022 -0600][Dump ][Utility ] Protocol: roots > [2023-03-10 16:33:07.187022 -0600][Dump ][Utility ] User Name: > [2023-03-10 16:33:07.187022 -0600][Dump ][Utility ] Password: > [2023-03-10 16:33:07.187022 -0600][Dump ][Utility ] Host Name: red-xfer3.unl.edu > [2023-03-10 16:33:07.187022 -0600][Dump ][Utility ] Port: 1094 > [2023-03-10 16:33:07.187022 -0600][Dump ][Utility ] Path: /store/hello_world.txt > [2023-03-10 16:33:07.187064 -0600][Debug ][File ] ***@***.***://xrootd-local.unl.edu:1094//store/hello_world.txt?xrdcl.requuid=49bd656c-2498-4298-b14d-b3036dc47ad9] Sending a read command for handle 0x0 to red-xfer3.unl.edu:1094 > [2023-03-10 16:33:07.187093 -0600][Dump ][XRootD ] [red-xfer3.unl.edu:1094] Sending message kXR_read (handle: 0x00000000, offset: 0, size: 13) > [2023-03-10 16:33:07.187107 -0600][Debug ][ExDbgMsg ] [red-xfer3.unl.edu:1094] MsgHandler created: 0x262de10 (message: kXR_read (handle: 0x00000000, offset: 0, size: 13) ). > [2023-03-10 16:33:07.187122 -0600][Dump ][PostMaster ] [red-xfer3.unl.edu:1094] Sending message kXR_read (handle: 0x00000000, offset: 0, size: 13) (0x262da30) through substream 0 expecting answer at 0 > [2023-03-10 16:33:07.187161 -0600][Dump ][AsyncSock ] [red-xfer3.unl.edu:1094.0] Wrote a message: kXR_read (handle: 0x00000000, offset: 0, size: 13) (0x262da30), 32 bytes > [2023-03-10 16:33:07.187189 -0600][Dump ][AsyncSock ] [red-xfer3.unl.edu:1094.0] Successfully sent message: kXR_read (handle: 0x00000000, offset: 0, size: 13) (0x262da30). > [2023-03-10 16:33:07.187201 -0600][Dump ][XRootD ] [red-xfer3.unl.edu:1094] Message kXR_read (handle: 0x00000000, offset: 0, size: 13) has been successfully sent. > [2023-03-10 16:33:07.187209 -0600][Debug ][ExDbgMsg ] [red-xfer3.unl.edu:1094] Moving MsgHandler: 0x262de10 (message: kXR_read (handle: 0x00000000, offset: 0, size: 13) ) from out-queu to in-queue. > [2023-03-10 16:33:07.187218 -0600][Dump ][PostMaster ] [red-xfer3.unl.edu:1094.0] All messages consumed, disable uplink > [2023-03-10 16:33:07.187560 -0600][Dump ][XRootDTransport ] [msg: 0xf4076940] Expecting 13 bytes of message body > [2023-03-10 16:33:07.187583 -0600][Dump ][AsyncSock ] [red-xfer3.unl.edu:1094.0] Received message header for 0xf4076940 size: 8 > [2023-03-10 16:33:07.187596 -0600][Debug ][ExDbgMsg ] [msg: 0xf4076940] Assigned MsgHandler: 0x262de10. > [2023-03-10 16:33:07.187604 -0600][Debug ][ExDbgMsg ] [handler: 0x262de10] Removed MsgHandler: 0x262de10 from the in-queue. > [2023-03-10 16:33:07.187612 -0600][Dump ][AsyncSock ] [red-xfer3.unl.edu:1094.0] Will use the raw handler to read body of message 0xf4076940 > [2023-03-10 16:33:07.187639 -0600][Dump ][AsyncSock ] [red-xfer3.unl.edu:1094.0] Received message 0xf4076940 of 21 bytes > [2023-03-10 16:33:07.187651 -0600][Dump ][PostMaster ] [red-xfer3.unl.edu:1094] Handling received message: 0xf4076940. > [2023-03-10 16:33:07.187827 -0600][Dump ][XRootD ] [red-xfer3.unl.edu:1094] Got a kXR_ok response to request kXR_read (handle: 0x00000000, offset: 0, size: 13) > [2023-03-10 16:33:07.187853 -0600][Debug ][ExDbgMsg ] [red-xfer3.unl.edu:1094] Calling MsgHandler: 0x262de10 (message: kXR_read (handle: 0x00000000, offset: 0, size: 13) ) with status: [SUCCESS] . > [2023-03-10 16:33:07.187871 -0600][Dump ][XRootD ] [red-xfer3.unl.edu:1094] Parsing the response to kXR_read (handle: 0x00000000, offset: 0, size: 13) as ChunkInfo > [2023-03-10 16:33:07.187883 -0600][Dump ][File ] ***@***.***://xrootd-local.unl.edu:1094//store/hello_world.txt?xrdcl.requuid=49bd656c-2498-4298-b14d-b3036dc47ad9] Got state response for message kXR_read (handle: 0x00000000, offset: 0, size: 13) > [2023-03-10 16:33:07.187901 -0600][Debug ][ExDbgMsg ] [red-xfer3.unl.edu:1094] Destroying MsgHandler: 0x262de10. > hello world! > [2023-03-10 16:33:07.187952 -0600][Debug ][File ] ***@***.***://xrootd-local.unl.edu:1094//store/hello_world.txt?xrdcl.requuid=49bd656c-2498-4298-b14d-b3036dc47ad9] Sending a close command for handle 0x0 to red-xfer3.unl.edu:1094 > [2023-03-10 16:33:07.187979 -0600][Dump ][XRootD ] [red-xfer3.unl.edu:1094] Sending message kXR_close (handle: 0x00000000) > [2023-03-10 16:33:07.187992 -0600][Debug ][ExDbgMsg ] [red-xfer3.unl.edu:1094] MsgHandler created: 0x262c680 (message: kXR_close (handle: 0x00000000) ). > [2023-03-10 16:33:07.188005 -0600][Dump ][PostMaster ] [red-xfer3.unl.edu:1094] Sending message kXR_close (handle: 0x00000000) (0x26265d0) through substream 0 expecting answer at 0 > [2023-03-10 16:33:07.188043 -0600][Dump ][AsyncSock ] [red-xfer3.unl.edu:1094.0] Wrote a message: kXR_close (handle: 0x00000000) (0x26265d0), 24 bytes > [2023-03-10 16:33:07.188071 -0600][Dump ][AsyncSock ] [red-xfer3.unl.edu:1094.0] Successfully sent message: kXR_close (handle: 0x00000000) (0x26265d0). > [2023-03-10 16:33:07.188083 -0600][Dump ][XRootD ] [red-xfer3.unl.edu:1094] Message kXR_close (handle: 0x00000000) has been successfully sent. > [2023-03-10 16:33:07.188091 -0600][Debug ][ExDbgMsg ] [red-xfer3.unl.edu:1094] Moving MsgHandler: 0x262c680 (message: kXR_close (handle: 0x00000000) ) from out-queu to in-queue. > [2023-03-10 16:33:07.188099 -0600][Dump ][PostMaster ] [red-xfer3.unl.edu:1094.0] All messages consumed, disable uplink > [2023-03-10 16:33:07.188478 -0600][Dump ][XRootDTransport ] [msg: 0xf40767f0] Expecting 0 bytes of message body > [2023-03-10 16:33:07.188501 -0600][Dump ][AsyncSock ] [red-xfer3.unl.edu:1094.0] Received message header for 0xf40767f0 size: 8 > [2023-03-10 16:33:07.188515 -0600][Debug ][ExDbgMsg ] [msg: 0xf40767f0] Assigned MsgHandler: 0x262c680. > [2023-03-10 16:33:07.188528 -0600][Debug ][ExDbgMsg ] [handler: 0x262c680] Removed MsgHandler: 0x262c680 from the in-queue. > [2023-03-10 16:33:07.188536 -0600][Dump ][AsyncSock ] [red-xfer3.unl.edu:1094.0] Received message 0xf40767f0 of 8 bytes > [2023-03-10 16:33:07.188545 -0600][Dump ][PostMaster ] [red-xfer3.unl.edu:1094] Handling received message: 0xf40767f0. > [2023-03-10 16:33:07.188638 -0600][Dump ][XRootD ] [red-xfer3.unl.edu:1094] Got a kXR_ok response to request kXR_close (handle: 0x00000000) > [2023-03-10 16:33:07.188663 -0600][Debug ][ExDbgMsg ] [red-xfer3.unl.edu:1094] Calling MsgHandler: 0x262c680 (message: kXR_close (handle: 0x00000000) ) with status: [SUCCESS] . > [2023-03-10 16:33:07.188683 -0600][Debug ][File ] ***@***.***://xrootd-local.unl.edu:1094//store/hello_world.txt?xrdcl.requuid=49bd656c-2498-4298-b14d-b3036dc47ad9] Close returned from red-xfer3.unl.edu:1094 with: [SUCCESS] > [2023-03-10 16:33:07.188693 -0600][Dump ][File ] ***@***.***://xrootd-local.unl.edu:1094//store/hello_world.txt?xrdcl.requuid=49bd656c-2498-4298-b14d-b3036dc47ad9] Items in the fly 0, queued for recovery 0 > [2023-03-10 16:33:07.188708 -0600][Debug ][ExDbgMsg ] [red-xfer3.unl.edu:1094] Destroying MsgHandler: 0x262c680. > [2023-03-10 16:33:07.189233 -0600][Debug ][JobMgr ] Stopping the job manager... > [2023-03-10 16:33:07.189252 -0600][Dump ][JobMgr ] Stopping worker #0... > [2023-03-10 16:33:07.189416 -0600][Dump ][JobMgr ] Worker #0 stopped > [2023-03-10 16:33:07.189429 -0600][Dump ][JobMgr ] Stopping worker #1... > [2023-03-10 16:33:07.189468 -0600][Dump ][JobMgr ] Worker #1 stopped > [2023-03-10 16:33:07.189478 -0600][Dump ][JobMgr ] Stopping worker #2... > [2023-03-10 16:33:07.189511 -0600][Dump ][JobMgr ] Worker #2 stopped > [2023-03-10 16:33:07.189521 -0600][Debug ][JobMgr ] Job manager stopped > [2023-03-10 16:33:07.189529 -0600][Debug ][TaskMgr ] Stopping the task manager... > [2023-03-10 16:33:07.189602 -0600][Debug ][TaskMgr ] Task manager stopped > [2023-03-10 16:33:07.189612 -0600][Debug ][Poller ] Stopping the poller... > [2023-03-10 16:33:07.189673 -0600][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: roots://red-xfer3.unl.edu:1094/" > [2023-03-10 16:33:07.189699 -0600][Debug ][AsyncSock ] [red-xfer3.unl.edu:1094.0] Closing the socket > [2023-03-10 16:33:07.189745 -0600][Debug ][Poller ] <[::ffff:129.93.239.166]:8496><--><[::ffff:129.93.239.173]:1094> Removing socket from the poller > [2023-03-10 16:33:07.189824 -0600][Debug ][PostMaster ] [red-xfer3.unl.edu:1094] Destroying stream > [2023-03-10 16:33:07.189846 -0600][Debug ][AsyncSock ] [red-xfer3.unl.edu:1094.0] Closing the socket > [2023-03-10 16:33:07.189863 -0600][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: roots://xrootd-local.unl.edu:1094/" > [2023-03-10 16:33:07.189873 -0600][Debug ][AsyncSock ] [xrootd-local.unl.edu:1094.0] Closing the socket > [2023-03-10 16:33:07.189882 -0600][Debug ][Poller ] <[::ffff:129.93.239.166]:19674><--><[::ffff:129.93.239.164]:1094> Removing socket from the poller > [2023-03-10 16:33:07.189998 -0600][Debug ][PostMaster ] [xrootd-local.unl.edu:1094] Destroying stream > [2023-03-10 16:33:07.190014 -0600][Debug ][AsyncSock ] [xrootd-local.unl.edu:1094.0] Closing the socket > ``` > </details> > > -- > Reply to this email directly or view it on GitHub: > https://github.com/xrootd/xrootd/issues/1952#issuecomment-1464603394 > You are receiving this because you commented. > > Message ID: ***@***.***> -- Reply to this email directly or view it on GitHub: https://github.com/xrootd/xrootd/issues/1952#issuecomment-1464636481 You are receiving this because you are subscribed to this thread. Message ID: <[log in to unmask]> ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-DEV list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1