I believe the issue was that the sitename given in all.sitename wasn't matching between the nodes.
The working config is:
set redirector = elephant108.heprc.uvic.ca
all.manager $(redirector):1095
all.export /
# The names used here must match what is going to be passed as server_type
if $redirector
all.role manager
http.desthttps yes
else if named s3_proxy
# Proxy setup
all.role server
ofs.osslib libXrdPss.so
pss.origin http://elephant102.heprc.uvic.ca:9000
pss.localroot /bucket
setenv AWS_ACCESS_KEY_ID < /etc/xrootd/access_key
setenv AWS_SECRET_ACCESS_KEY < /etc/xrootd/secret_key
else if named standalone
all.role server
oss.localroot /data
fi
# TLS Setup
xrd.tls /etc/grid-security/xrd/xrdcert.pem /etc/grid-security/xrd/xrdkey.pem
xrd.tlsca certdir /etc/grid-security/certificates
xrootd.tls all
if $redirector
# Pass
else
xrootd.seclib libXrdSec.so
sec.protocol gsi -dlgpxy:request \
-exppxy:=creds \
-ca:verify \
-crl:try \
-gmapopt:nomap \
-vomsat:require \
-vomsfun:default
fi
xrootd.seclib libXrdSec.so
http.secxtractor libXrdHttpVOMS.so
if exec xrootd
xrd.protocol http libXrdHttp.so
fi
# Authorization configuration
ofs.authorize
acc.audit deny grant
acc.authdb /etc/xrootd/Authfile
# TPC Setup
xrootd.chksum adler32
ofs.tpc fcreds ?gsi =X509_USER_PROXY
ofs.tpc require client gsi
ofs.tpc scan stderr
ofs.tpc echo
ofs.tpc ttl 300 600
ofs.tpc xfr 100
ofs.tpc autorm
ofs.tpc pgm /etc/xrootd/tpc.sh
if $redirector
ofs.tpc redirect elephant101.heprc.uvic.ca:1094
fi
http.exthandler xrdtpc libXrdHttpTPC.so
http.header2cgi Authorization authz
# Macaroon setup
http.exthandler xrdmacaroons libXrdMacaroons.so
macaroons.secretkey /etc/xrootd/macaroon-secret
all.sitename test-sitename
ofs.authlib libXrdMacaroons.so
The non working config was attached a few emails back and is:
set redirector = elephant108.heprc.uvic.ca
all.manager $(redirector):1095
all.export /
if $redirector
all.role manager
http.desthttps yes
else
all.role server
oss.localroot /data
fi
# TLS Setup
xrd.tls /etc/grid-security/xrd/xrdcert.pem /etc/grid-security/xrd/xrdkey.pem
xrd.tlsca certdir /etc/grid-security/certificates
xrootd.tls all
if $redirector
# Pass
else
xrootd.seclib libXrdSec.so
sec.protocol gsi -dlgpxy:request \
-exppxy:=creds \
-ca:verify \
-crl:try \
-gmapopt:nomap \
-vomsat:require \
-vomsfun:default
fi
xrootd.seclib libXrdSec.so
http.secxtractor libXrdHttpVOMS.so
if exec xrootd
xrd.protocol http libXrdHttp.so
fi
# Authorization configuration
ofs.authorize
acc.audit deny grant
acc.authdb /etc/xrootd/Authfile
# TPC Setup
xrootd.chksum adler32
ofs.tpc fcreds ?gsi =X509_USER_PROXY
ofs.tpc require client gsi
ofs.tpc scan stderr
ofs.tpc echo
ofs.tpc ttl 300 600
ofs.tpc xfr 100
ofs.tpc autorm
ofs.tpc pgm /etc/xrootd/tpc.sh
ofs.tpc redirect elephant101.heprc.uvic.ca:1094
http.exthandler xrdtpc libXrdHttpTPC.so
http.header2cgi Authorization authz
# Macaroon setup
http.exthandler xrdmacaroons libXrdMacaroons.so
macaroons.secretkey /etc/xrootd/macaroon-secret
all.sitename elephant108.heprc.uvic.ca
ofs.authlib libXrdMacaroons.so
________________________________
From: Yang, Wei <[log in to unmask]>
Sent: Tuesday, February 14, 2023 12:31:13 PM
To: Marcus Lee; [log in to unmask]
Subject: Re: TPC with macaroons authentication
Can I take a look at your config file ?
--
Wei
________________________________________
From: Marcus Lee <[log in to unmask]>
Sent: Tuesday, February 14, 2023 10:12 AM
To: Yang, Wei; [log in to unmask]
Subject: Re: TPC with macaroons authentication
1) It works without any issues if I request directly to the server
2) They do already
Thanks,
________________________________
From: Yang, Wei <[log in to unmask]>
Sent: Monday, February 13, 2023 7:32:04 PM
To: Marcus Lee; [log in to unmask]
Subject: Re: TPC with macaroons authentication
I had a redirector setup a long time ago but not at this moment. Questions
1) what if you request TPC directly against the server behind the director. Does it work?
2) do the macaroon secret the same on both redirector and servers (this is required)
--
Wei
________________________________________
From: [log in to unmask] <[log in to unmask]> on behalf of Marcus Lee <[log in to unmask]>
Sent: Monday, February 13, 2023 5:01 PM
To: [log in to unmask]
Subject: TPC with macaroons authentication
Hello,
I am trying to configure xrootd to use macaroons as the bearer token for TPC over https and place the server behind a redirector. The configuration I have works so far with TPC over https with no redirector and TPClite over the root protocol with delegated credentials.
When I try to do TPC through a redirector node with bearer tokens I get an error saying that the token was redirected and now for the wrong server.
The error I get from gfal is:
TRANSFER ERROR: Copy failed (3rd push). Last attempt: Transfer failure: rejected PUT: 403 Forbidden; redirections
And in the log I get:
macarons_Access: Macaroon is for incorrect location elephant108.heprc.uvic.ca
The ofs configuration reference says that this would happen when redirecting to an xrootd instance that isn't on the same machine and suggests using ofs.tpc redirect to redirect the TPC request. I've tried using that directive and specifying a server to redirect to but this doesn't fix the issue.
The documentation also suggests that I can provide more cgi information to the url to open the file at the redirection target but I'm not sure what to do with that.
Has anyone gotten this to work and are there any resources on how to do this?
I've attached the redirector's configuration.
Thanks
________________________________
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1
