It has been a long time when I put this thing out – hopefully I still remembers this.
I think the issue may be related to the use of "ofs.forward all”. it was there to ensure that the deletion request are “broadcast” from the redirector to all data servers (for example, a directory may exist on several data servers. Without this forwarding, old xrd and early versions of xrdfs will only delete the directory on one of them). So your xrdfs didn’t actually talk to the data servers, but only to the redirector. This may explain all the things you saw.
One way to verify is the use xrdfs to delete a file that doesn’t exist in your cluster. Xrdfs may not give your an error because of the broadcasting.
regards,
--
Wei Yang | [log in to unmask] | 650-926-3338(O)
From: hds1 <[log in to unmask]>
Reply-To: xrootd/xrootd <[log in to unmask]>
Date: Friday, April 13, 2018 at 5:52 AM
To: xrootd/xrootd <[log in to unmask]>
Cc: Subscribed <[log in to unmask]>
Subject: Re: [xrootd/xrootd] xrdfs 4.8.2 authorization problem (#687)
just experimented a bit more. The issue seems to be that the redirector also needs the Authentication settings. So it looks that xrdcp exchanges auth with the data servers, whereas xrdfs exchanges it with the redirector.
With the xrootd.cf below it works as expected (probably the auth section within the data.server role could be removed then)
This is reproducable. When deleting the global auth settings the "rm" is possible again for all users.
xrootd.cf:
xrd.timeout hail 30 idle 0 kill 3 read 5
all.export /xrootd
set xrdr=REDIRECTOR
set inventory=/var/log/xrootd/inventory
all.manager $(xrdr):3121
cms.allow host *.iup.uni-bremen.de
xrootd.seclib /usr/lib/libXrdSec.so
sec.protocol unix
acc.authdb /etc/xrootd/Authfile
acc.authrefresh 60
ofs.authorize
if $(xrdr) && named cns
all.export $(inventory)
xrd.port 1095
else if $(xrdr)
all.role manager
oss.defaults rw
ofs.forward all
xrd.port 1094
else
all.role server
ofs.notify closew create mkdir mv rm rmdir trunc | /usr/bin/XrdCnsd -d -D 2 -i 90 -b $(xrdr):1095:$(inventory)
ofs.notifymsg create $TID create $FMODE $LFN?$CGI
ofs.notifymsg closew $TID closew $LFN $FSIZE
xrootd.seclib /usr/lib/libXrdSec.so
sec.protocol unix
acc.authdb /etc/xrootd/Authfile
acc.authrefresh 60
ofs.authorize
fi
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly,
view it on GitHub <https://github.com/xrootd/xrootd/issues/687#issuecomment-381125652>, or
mute the thread <https://github.com/notifications/unsubscribe-auth/AE9TA_cynxqZwpIwT6AEeaNIN3n9COAMks5toJ-CgaJpZM4TTQcg>.
{"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":{"external_key":"github/xrootd/xrootd","title":"xrootd/xrootd","subtitle":"GitHub repository","main_image_url":"https://cloud.githubusercontent.com/assets/143418/17495839/a5054eac-5d88-11e6-95fc-7290892c7bb5.png","avatar_image_url":"https://cloud.githubusercontent.com/assets/143418/15842166/7c72db34-2c0b-11e6-9aed-b52498112777.png","action":{"name":"Open in GitHub","url":"https://github.com/xrootd/xrootd"}},"updates":{"snippets":[{"icon":"PERSON","message":"@hds1 in #687: just experimented a bit more. The issue seems to be that the redirector also needs the Authentication settings. So it looks that xrdcp exchanges auth with the data servers, whereas xrdfs exchanges it with the redirector.\r\nWith the xrootd.cf below it works as expected (probably the auth section within the data.server role could be removed then)\r\nThis is reproducable. When deleting the global auth settings the \"rm\" is possible again for all users.\r\n\r\nxrootd.cf:\r\nxrd.timeout hail 30 idle 0 kill 3 read 5\r\nall.export /xrootd\r\nset xrdr=REDIRECTOR\r\nset inventory=/var/log/xrootd/inventory\r\nall.manager $(xrdr):3121\r\ncms.allow host *.iup.uni-bremen.de\r\n\r\nxrootd.seclib /usr/lib/libXrdSec.so\r\nsec.protocol unix\r\nacc.authdb /etc/xrootd/Authfile\r\nacc.authrefresh 60\r\nofs.authorize\r\n\r\nif $(xrdr) \u0026\u0026 named cns\r\n all.export $(inventory)\r\n xrd.port 1095\r\nelse if $(xrdr)\r\n all.role manager\r\n oss.defaults rw\r\n ofs.forward all\r\n xrd.port 1094\r\nelse\r\n all.role server\r\n ofs.notify closew create mkdir mv rm rmdir trunc | /usr/bin/XrdCnsd -d -D 2 -i 90 -b $(xrdr):1095:$(inventory)\r\n ofs.notifymsg create $TID create $FMODE $LFN?$CGI\r\n ofs.notifymsg closew $TID closew $LFN $FSIZE\r\n\r\n xrootd.seclib /usr/lib/libXrdSec.so\r\n sec.protocol unix\r\n acc.authdb /etc/xrootd/Authfile\r\n acc.authrefresh 60\r\n ofs.authorize\r\nfi"}],"action":{"name":"View Issue","url":"https://github.com/xrootd/xrootd/issues/687#issuecomment-381125652"}}}
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.
{"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":{"external_key":"github/xrootd/xrootd","title":"xrootd/xrootd","subtitle":"GitHub repository","main_image_url":"https://cloud.githubusercontent.com/assets/143418/17495839/a5054eac-5d88-11e6-95fc-7290892c7bb5.png","avatar_image_url":"https://cloud.githubusercontent.com/assets/143418/15842166/7c72db34-2c0b-11e6-9aed-b52498112777.png","action":{"name":"Open in GitHub","url":"https://github.com/xrootd/xrootd"}},"updates":{"snippets":[{"icon":"PERSON","message":"@wyang007 in #687: It has been a long time when I put this thing out – hopefully I still remembers this.\r\n\r\nI think the issue may be related to the use of \"ofs.forward all”. it was there to ensure that the deletion request are “broadcast” from the redirector to all data servers (for example, a directory may exist on several data servers. Without this forwarding, old xrd and early versions of xrdfs will only delete the directory on one of them). So your xrdfs didn’t actually talk to the data servers, but only to the redirector. This may explain all the things you saw. \r\n\r\nOne way to verify is the use xrdfs to delete a file that doesn’t exist in your cluster. Xrdfs may not give your an error because of the broadcasting. \r\n\r\nregards,\r\n--\r\nWei Yang | [log in to unmask] | 650-926-3338(O) \r\n\r\n\r\n\r\n\r\n\r\n\r\nFrom: hds1 \[log in to unmask]\u003e\r\nReply-To: xrootd/xrootd \[log in to unmask]\u003e\r\nDate: Friday, April 13, 2018 at 5:52 AM\r\nTo: xrootd/xrootd \[log in to unmask]\u003e\r\nCc: Subscribed \[log in to unmask]\u003e\r\nSubject: Re: [xrootd/xrootd] xrdfs 4.8.2 authorization problem (#687)\r\n\r\n\r\njust experimented a bit more. The issue seems to be that the redirector also needs the Authentication settings. So it looks that xrdcp exchanges auth with the data servers, whereas xrdfs exchanges it with the redirector.\r\nWith the xrootd.cf below it works as expected (probably the auth section within the data.server role could be removed then)\r\nThis is reproducable. When deleting the global auth settings the \"rm\" is possible again for all users.\r\nxrootd.cf:\r\nxrd.timeout hail 30 idle 0 kill 3 read 5\r\nall.export /xrootd\r\nset xrdr=REDIRECTOR\r\nset inventory=/var/log/xrootd/inventory\r\nall.manager $(xrdr):3121\r\ncms.allow host *.iup.uni-bremen.de\r\nxrootd.seclib /usr/lib/libXrdSec.so\r\nsec.protocol unix\r\nacc.authdb /etc/xrootd/Authfile\r\nacc.authrefresh 60\r\nofs.authorize\r\nif $(xrdr) \u0026\u0026 named cns\r\nall.export $(inventory)\r\nxrd.port 1095\r\nelse if $(xrdr)\r\nall.role manager\r\noss.defaults rw\r\nofs.forward all\r\nxrd.port 1094\r\nelse\r\nall.role server\r\nofs.notify closew create mkdir mv rm rmdir trunc | /usr/bin/XrdCnsd -d -D 2 -i 90 -b $(xrdr):1095:$(inventory)\r\nofs.notifymsg create $TID create $FMODE $LFN?$CGI\r\nofs.notifymsg closew $TID closew $LFN $FSIZE\r\n xrootd.seclib /usr/lib/libXrdSec.so\r\n sec.protocol unix\r\n acc.authdb /etc/xrootd/Authfile\r\n acc.authrefresh 60\r\n ofs.authorize\r\nfi\r\n—\r\nYou are receiving this because you are subscribed to this thread.\r\nReply to this email directly, \r\nview it on GitHub \u003chttps://github.com/xrootd/xrootd/issues/687#issuecomment-381125652\u003e, or \r\nmute the thread \u003chttps://github.com/notifications/unsubscribe-auth/AE9TA_cynxqZwpIwT6AEeaNIN3n9COAMks5toJ-CgaJpZM4TTQcg\u003e.\r\n\r\n\r\n{\"api_version\":\"1.0\",\"publisher\":{\"api_key\":\"05dde50f1d1a384dd78767c55493e4bb\",\"name\":\"GitHub\"},\"entity\":{\"external_key\":\"github/xrootd/xrootd\",\"title\":\"xrootd/xrootd\",\"subtitle\":\"GitHub repository\",\"main_image_url\":\"https://cloud.githubusercontent.com/assets/143418/17495839/a5054eac-5d88-11e6-95fc-7290892c7bb5.png\",\"avatar_image_url\":\"https://cloud.githubusercontent.com/assets/143418/15842166/7c72db34-2c0b-11e6-9aed-b52498112777.png\",\"action\":{\"name\":\"Open in GitHub\",\"url\":\"https://github.com/xrootd/xrootd\"}},\"updates\":{\"snippets\":[{\"icon\":\"PERSON\",\"message\":\"@hds1 in #687: just experimented a bit more. The issue seems to be that the redirector also needs the Authentication settings. So it looks that xrdcp exchanges auth with the data servers, whereas xrdfs exchanges it with the redirector.\\r\\nWith the xrootd.cf below it works as expected (probably the auth section within the data.server role could be removed then)\\r\\nThis is reproducable. When deleting the global auth settings the \\\"rm\\\" is possible again for all users.\\r\\n\\r\\nxrootd.cf:\\r\\nxrd.timeout hail 30 idle 0 kill 3 read 5\\r\\nall.export /xrootd\\r\\nset xrdr=REDIRECTOR\\r\\nset inventory=/var/log/xrootd/inventory\\r\\nall.manager $(xrdr):3121\\r\\ncms.allow host *.iup.uni-bremen.de\\r\\n\\r\\nxrootd.seclib /usr/lib/libXrdSec.so\\r\\nsec.protocol unix\\r\\nacc.authdb /etc/xrootd/Authfile\\r\\nacc.authrefresh 60\\r\\nofs.authorize\\r\\n\\r\\nif $(xrdr) \\u0026\\u0026 named cns\\r\\n all.export $(inventory)\\r\\n xrd.port 1095\\r\\nelse if $(xrdr)\\r\\n all.role manager\\r\\n oss.defaults rw\\r\\n ofs.forward all\\r\\n xrd.port 1094\\r\\nelse\\r\\n all.role server\\r\\n ofs.notify closew create mkdir mv rm rmdir trunc | /usr/bin/XrdCnsd -d -D 2 -i 90 -b $(xrdr):1095:$(inventory)\\r\\n ofs.notifymsg create $TID create $FMODE $LFN?$CGI\\r\\n ofs.notifymsg closew $TID closew $LFN $FSIZE\\r\\n\\r\\n xrootd.seclib /usr/lib/libXrdSec.so\\r\\n sec.protocol unix\\r\\n acc.authdb /etc/xrootd/Authfile\\r\\n acc.authrefresh 60\\r\\n ofs.authorize\\r\\nfi\"}],\"action\":{\"name\":\"View Issue\",\"url\":\"https://github.com/xrootd/xrootd/issues/687#issuecomment-381125652\"}}}\r\n"}],"action":{"name":"View Issue","url":"https://github.com/xrootd/xrootd/issues/687#issuecomment-381193171"}}}
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
