XROOTD-L Archives

Support use of xrootd by HEP experiments

XROOTD-L@LISTSERV.SLAC.STANFORD.EDU

Options: Use Forum View

Use Monospaced Font
Show Text Part by Default
Show All Mail Headers

Message: [<< First] [< Prev] [Next >] [Last >>]
Topic: [<< First] [< Prev] [Next >] [Last >>]
Author: [<< First] [< Prev] [Next >] [Last >>]

Print Reply
Subject:
From:
Doug Benjamin <[log in to unmask]>
Reply To:
Support use of xrootd by HEP experiments <[log in to unmask]>
Date:
Mon, 29 Nov 2021 02:28:28 +0000
Content-Type:
text/plain
Parts/Attachments:
text/plain (1 lines)
Hello,



  We are trying to get the OSG standalone xrootd server to work in multiuser mode.  In order to server files via https (davs) from Lustre storage at BNL.



For testing I am using my DN file that is mapped to the local user usatlas1 (this is the same account that owns the lustre areas)



When I try to use davs protocol, I get this behavior on the client side (using gfal commands) -



[spar0106]> gfal-copy file:///usatlas/u/benjamin/davs-test-file-spar0106 davs://xrootd04.usatlas.bnl.gov:1094//hpcdisk/test/davs-test-file-spar0106

Copying file:///usatlas/u/benjamin/davs-test-file-spar0106   [FAILED]  after 0s                                                                                                                                                                                                                                                                                                                                                                                             

gfal-copy error: 1 (Operation not permitted) - DESTINATION OVERWRITE   HTTP 403 : Permission refused 

[spar0106]> gfal-copy file:///usatlas/u/benjamin/davs-test-file-spar0106 davs://xrootd04.usatlas.bnl.gov//hpcdisk/test/davs-test-file-spar0106

Copying file:///usatlas/u/benjamin/davs-test-file-spar0106   [FAILED]  after 0s                                                                                                                                                                                                                                                                                                                                                                                             

gfal-copy error: 112 (Host is down) - DESTINATION OVERWRITE   Result Could not connect to server after 1 attempts



Here is what xrootd.log looks like - 



211128 21:21:32 65596 Xrd_Inet: Accepted connection on port 1094 from [log in to unmask]

211128 21:21:32 65666 Xrd_Sched: running main accept inq=0

211128 21:21:32 65596 Xrd_ProtLoad: matched port 1094 protocol http

211128 21:21:32 65596 anon.0:34@spar0106 Xrd_Poll: FD 34 attached to poller 0; num=1

211128 21:21:32 65596 XrootdBridge: bdouglas.21:34@spar0106 login as bdouglas

211128 21:21:32 65596 multiuser_UserSentry: Failed to lookup UID for username bdouglas Success

211128 21:21:32 65596 ofs_stat: bdouglas.21:34@spar0106 Unable to locate /hpcdisk/test/davs-test-file-spar0106; permission denied

211128 21:21:32 65596 bdouglas.21:34@spar0106 Xrootd_Response: sending err 3010: Unable to locate /hpcdisk/test/davs-test-file-spar0106; permission denied

211128 21:21:32 65596 XrootdXeq: bdouglas.21:34@spar0106 disc 0:00:00 (send failure)

211128 21:21:32 65596 multiuser_UserSentry: Anonymous client; no user set, cannot change FS UIDs

211128 21:21:32 65596 bdouglas.21:34@spar0106 Xrd_Poll: Poller 0 removing FD 34

211128 21:21:32 65596 bdouglas.21:34@spar0106 Xrd_Poll: FD 34 detached from poller 0; num=0

211128 21:21:32 65666 Xrd_Inet: Accepted connection on port 1094 from [log in to unmask]

211128 21:21:32 65666 Xrd_ProtLoad: matched port 1094 protocol http

211128 21:21:32 65666 anon.0:35@spar0106 Xrd_Poll: FD 35 attached to poller 0; num=1

211128 21:21:32 65597 Xrd_Sched: running main accept inq=0

211128 21:21:32 65666 XrootdBridge: bdouglas.22:35@spar0106 login as bdouglas

211128 21:21:32 65666 multiuser_UserSentry: Failed to lookup UID for username bdouglas Success

211128 21:21:32 65666 ofs_stat: bdouglas.22:35@spar0106 Unable to locate /hpcdisk/test/davs-test-file-spar0106; permission denied

211128 21:21:32 65666 bdouglas.22:35@spar0106 Xrootd_Response: sending err 3010: Unable to locate /hpcdisk/test/davs-test-file-spar0106; permission denied

211128 21:21:32 65666 XrootdXeq: bdouglas.22:35@spar0106 disc 0:00:00 (send failure)

211128 21:21:32 65666 multiuser_UserSentry: Anonymous client; no user set, cannot change FS UIDs

211128 21:21:32 65666 bdouglas.22:35@spar0106 Xrd_Poll: Poller 0 removing FD 35

211128 21:21:32 65666 bdouglas.22:35@spar0106 Xrd_Poll: FD 35 detached from poller 0; num=0

211128 21:21:32 65597 Xrd_Inet: Accepted connection on port 1094 from [log in to unmask]

211128 21:21:32 65597 Xrd_ProtLoad: matched port 1094 protocol http

211128 21:21:32 65597 anon.0:34@spar0106 Xrd_Poll: FD 34 attached to poller 0; num=1

211128 21:21:32 67061 Xrd_Sched: running main accept inq=0

211128 21:21:32 65597 XrootdBridge: bdouglas.23:34@spar0106 login as bdouglas

211128 21:21:32 65597 multiuser_UserSentry: Failed to lookup UID for username bdouglas Success

211128 21:21:32 65597 ofs_stat: bdouglas.23:34@spar0106 Unable to locate /hpcdisk/test/davs-test-file-spar0106; permission denied

211128 21:21:32 65597 bdouglas.23:34@spar0106 Xrootd_Response: sending err 3010: Unable to locate /hpcdisk/test/davs-test-file-spar0106; permission denied

211128 21:21:32 65597 XrootdXeq: bdouglas.23:34@spar0106 disc 0:00:00 (send failure)

211128 21:21:32 65597 multiuser_UserSentry: Anonymous client; no user set, cannot change FS UIDs

211128 21:21:32 65597 bdouglas.23:34@spar0106 Xrd_Poll: Poller 0 removing FD 34

211128 21:21:32 65597 bdouglas.23:34@spar0106 Xrd_Poll: FD 34 detached from poller 0; num=0

211128 21:21:32 67061 Xrd_Inet: Accepted connection on port 1094 from [log in to unmask]

211128 21:21:32 67604 Xrd_Sched: running main accept inq=0

211128 21:21:32 67061 Xrd_ProtLoad: matched port 1094 protocol http

211128 21:21:32 67061 anon.0:35@spar0106 Xrd_Poll: FD 35 attached to poller 0; num=1

211128 21:21:32 67061 XrootdBridge: bdouglas.24:35@spar0106 login as bdouglas

211128 21:21:32 67061 multiuser_UserSentry: Failed to lookup UID for username bdouglas Success

211128 21:21:32 67061 ofs_stat: bdouglas.24:35@spar0106 Unable to locate /hpcdisk/test/davs-test-file-spar0106; permission denied

211128 21:21:32 67061 bdouglas.24:35@spar0106 Xrootd_Response: sending err 3010: Unable to locate /hpcdisk/test/davs-test-file-spar0106; permission denied

211128 21:21:32 67061 XrootdXeq: bdouglas.24:35@spar0106 disc 0:00:00 (send failure)

211128 21:21:32 67061 multiuser_UserSentry: Anonymous client; no user set, cannot change FS UIDs

211128 21:21:32 67061 bdouglas.24:35@spar0106 Xrd_Poll: Poller 0 removing FD 35

211128 21:21:32 67061 bdouglas.24:35@spar0106 Xrd_Poll: FD 35 detached from poller 0; num=0

211128 21:21:32 67604 Xrd_Inet: Accepted connection on port 1094 from [log in to unmask]

211128 21:21:32 65616 Xrd_Sched: running main accept inq=0

211128 21:21:32 67604 Xrd_ProtLoad: matched port 1094 protocol http

211128 21:21:32 67604 anon.0:34@spar0106 Xrd_Poll: FD 34 attached to poller 0; num=1

211128 21:21:32 67604 XrootdBridge: bdouglas.25:34@spar0106 login as bdouglas

211128 21:21:32 67604 multiuser_UserSentry: Failed to lookup UID for username bdouglas Success

211128 21:21:32 67604 ofs_stat: bdouglas.25:34@spar0106 Unable to locate /hpcdisk/test/davs-test-file-spar0106; permission denied

211128 21:21:32 67604 bdouglas.25:34@spar0106 Xrootd_Response: sending err 3010: Unable to locate /hpcdisk/test/davs-test-file-spar0106; permission denied

211128 21:21:32 67604 XrootdXeq: bdouglas.25:34@spar0106 disc 0:00:00 (send failure)

211128 21:21:32 67604 multiuser_UserSentry: Anonymous client; no user set, cannot change FS UIDs

211128 21:21:32 67604 bdouglas.25:34@spar0106 Xrd_Poll: Poller 0 removing FD 34

211128 21:21:32 67604 bdouglas.25:34@spar0106 Xrd_Poll: FD 34 detached from poller 0; num=0

211128 21:21:32 65616 Xrd_Inet: Accepted connection on port 1094 from [log in to unmask]

211128 21:21:32 65596 Xrd_Sched: running main accept inq=0

211128 21:21:32 65616 Xrd_ProtLoad: matched port 1094 protocol http

211128 21:21:32 65616 anon.0:35@spar0106 Xrd_Poll: FD 35 attached to poller 0; num=1

211128 21:21:32 65616 XrootdBridge: bdouglas.26:35@spar0106 login as bdouglas

211128 21:21:32 65616 multiuser_UserSentry: Failed to lookup UID for username bdouglas Success

211128 21:21:32 65616 ofs_stat: bdouglas.26:35@spar0106 Unable to locate /hpcdisk/test/davs-test-file-spar0106; permission denied

211128 21:21:32 65616 bdouglas.26:35@spar0106 Xrootd_Response: sending err 3010: Unable to locate /hpcdisk/test/davs-test-file-spar0106; permission denied

211128 21:21:32 65616 XrootdXeq: bdouglas.26:35@spar0106 disc 0:00:00 (send failure)

211128 21:21:32 65616 multiuser_UserSentry: Anonymous client; no user set, cannot change FS UIDs

211128 21:21:32 65616 bdouglas.26:35@spar0106 Xrd_Poll: Poller 0 removing FD 35

211128 21:21:32 65616 bdouglas.26:35@spar0106 Xrd_Poll: FD 35 detached from poller 0; num=0



----- While if I use the root:// I am able to copy the file -



[spar0106]> gfal-copy file:///usatlas/u/benjamin/davs-test-file-spar0106 root://xrootd04.usatlas.bnl.gov//hpcdisk/test/root-davs-test-file-spar0106

Unable to load sec.protocol plugin libXrdSecztn.so

Copying file:///usatlas/u/benjamin/davs-test-file-spar0106   [DONE]  after 0s                                                                                                                                                                                                                                                                                                                                                                                               



And the xrootd log file looks like - 



211128 21:22:13 65596 Xrd_Inet: Accepted connection on port 1094 from [log in to unmask]

211128 21:22:13 67604 Xrd_Sched: running main accept inq=0

211128 21:22:13 65596 Xrd_ProtLoad: matched port 1094 protocol xroot

211128 21:22:13 65596 anon.0:34@spar0106 Xrd_Poll: FD 34 attached to poller 0; num=1

211128 21:22:13 65596 benjamin.31372:34@spar0106 Xrootd_Protocol: more auth requested; sz=3049

211128 21:22:14 65596 benjamin.31372:34@spar0106 Xrootd_Protocol: more auth requested; sz=6559

211128 21:22:14 65596 XrootdXeq: benjamin.31372:34@spar0106 pub IPv4 login as usatlas1

211128 21:22:14 65596 multiuser_UserSentry: Switching FS uid for user usatlas1

211128 21:22:14 65596 multiuser_UserSentry: Switching FS uid for user usatlas1

211128 21:22:14 65596 ofs_stat: benjamin.31372:34@spar0106 Unable to locate /hpcdisk/test/root-davs-test-file-spar0106; no such file or directory

211128 21:22:14 65596 benjamin.31372:34@spar0106 Xrootd_Response: sending err 3011: Unable to locate /hpcdisk/test/root-davs-test-file-spar0106; no such file or directory

211128 21:22:14 65596 multiuser_UserSentry: Switching FS uid for user usatlas1

211128 21:22:14 65596 ofs_stat: benjamin.31372:34@spar0106 Unable to locate /hpcdisk/test/root-davs-test-file-spar0106; no such file or directory

211128 21:22:14 65596 benjamin.31372:34@spar0106 Xrootd_Response: sending err 3011: Unable to locate /hpcdisk/test/root-davs-test-file-spar0106; no such file or directory

211128 21:22:14 65596 multiuser_UserSentry: Switching FS uid for user usatlas1

211128 21:22:14 65596 multiuser_UserSentry: Switching FS uid for user usatlas1

211128 21:22:14 65596 multiuser_Open: Will not create checksum

211128 21:22:14 65596 XrootdXeq: benjamin.31372:34@spar0106 disc 0:00:01

211128 21:22:14 65596 multiuser_UserSentry: Switching FS uid for user usatlas1

211128 21:22:14 65596 benjamin.31372:34@spar0106 Xrd_Poll: Poller 0 removing FD 34

211128 21:22:14 65596 benjamin.31372:34@spar0106 Xrd_Poll: FD 34 detached from poller 0; num=0



----  



What configuration files should we provide to help debug this issues ?



Regards,

Doug Benjamin







########################################################################

Use REPLY-ALL to reply to list



To unsubscribe from the XROOTD-L list, click the following link:

https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1


ATOM RSS1 RSS2