Hi Albert, You are not hallucinating. OSG has not yet made the change to the SciToken library to use the ztn token in the absence of the token in the cgi. I will talk to them on Thursday about that. So, everything you showed me is consistent. Andy On Tue, 29 Mar 2022, Albert Rossi wrote: > Me again. > > I just did an experiment with a simple read using a bearer token. > > built xrootd tip of master from yesterday: > > commit b5f279d616c49b65664ca06b98c52ec13fcc26fe (HEAD -> master, origin/master, origin/HEAD) > Author: Michal Simon <[log in to unmask]> > Date: Mon Mar 28 14:52:46 2022 +0200 > > [XrdCl] xrdcp --server: report IP stack to stderr. > > * IPv4: "!-!IPv4" > * IPv6: "!-!IPv6" > > > Get the Cilogon/fermilab token: > > [arossi@fndcatemp1 ~]$ htgettoken -a fermicloud543.fnal.gov -i fermilab > Attempting to get token from https://fermicloud543.fnal.gov:8200 ... succeeded > Storing bearer token in /run/user/8773/bt_u8773 > > [arossi@fndcatemp1 ~]$ httokendecode > { > "wlcg.ver": "1.0", > "aud": "https://wlcg.cern.ch/jwt/v1/any", > "sub": "[log in to unmask]", > "nbf": 1648568096, > "scope": "storage.create:/fermilab/users/arossi compute.create compute.read compute.cancel compute.modify storage.read:/fermilab/users/arossi", > "iss": "https://cilogon.org/fermilab", > "exp": 1648578901, > "iat": 1648568101, > "wlcg.groups": [ > "/fermilab" > ], > "jti": "https://cilogon.org/oauth2/14fc3800d21ae572deed19b2df2a3f4?type=accessToken&ts=1648568101030&version=v2.0&lifetime=10800000" > } > > [arossi@fndcatemp1 ~]$ cat $XDG_RUNTIME_DIR/bt_u8773 > eyJ0eXAiOiJKV1QiLCJraWQiOiJCODYzNDk1MUZEMUUzMTVEQUY3NUM5NEFFQ0YwMzY2OCIsImFsZyI6IlJTMjU2In0.eyJ3bGNnLnZlciI6IjEuMCIsImF1ZCI6Imh0dHBzOi8vd2xjZy5jZXJuLmNoL2p3dC92MS9hbnkiLCJzdWIiOiJhcm9zc2lAZm5hbC5nb3YiLCJuYmYiOjE2NDg1NjgwOTYsInNjb3BlIjoic3RvcmFnZS5jcmVhdGU6L2Zlcm1pbGFiL3VzZXJzL2Fyb3NzaSBjb21wdXRlLmNyZWF0ZSBjb21wdXRlLnJlYWQgY29tcHV0ZS5jYW5jZWwgY29tcHV0ZS5tb2RpZnkgc3RvcmFnZS5yZWFkOi9mZXJtaWxhYi91c2Vycy9hcm9zc2kiLCJpc3MiOiJodHRwczovL2NpbG9nb24ub3JnL2Zlcm1pbGFiIiwiZXhwIjoxNjQ4NTc4OTAxLCJpYXQiOjE2NDg1NjgxMDEsIndsY2cuZ3JvdXBzIjpbIi9mZXJtaWxhYiJdLCJqdGkiOiJodHRwczovL2NpbG9nb24ub3JnL29hdXRoMi8xNGZjMzgwMGQyMWFlNTcyZGVlZDE5YjJkZjJhM2Y0P3R5cGU9YWNjZXNzVG9rZW4mdHM9MTY0ODU2ODEwMTAzMCZ2ZXJzaW9uPXYyLjAmbGlmZXRpbWU9MTA4MDAwMDAifQ.MwW7gI65R6-tEEoJNqTokE0LHQo6q4FalWvtPxkITflFXqFrCl9D5bO4n_WstuewbkY-uG2ofhJ-6qe38SoGwASyZ35ZmqMictbrXyF9cBPxlFSR7XIsih2VyRMYp8W6LKFmHA1QA8yAxtHgsPm-got2r6RvtkD84F92ZuQ5qlJ-fsg4sR1HA5uc5gjKJWNED1d1iOcgatFT5UGGaIk713631TZGJ51nlFjS4LKwJ03Jqy9YlqCXotBY97rUXgPN_yDKImuqG tizBNY3yozBAg_grCry2Q-Gn1QAFJ3KootLDctU3aNFDnm8TGF2k4lcrvLE4FUsna7vzL0ehsveig > > > 1. ZTN OFF: > > [arossi@fndcatemp1 ~]$ xrdcp5x -f xroots://fndcatemp1.fnal.gov:1094//fermilab/users/arossi/data_1b /dev/null > [0B/0B][100%][==================================================][0B/s] > Run: [ERROR] Server responded with an error: [3010] Unable to open /fermilab/users/arossi/data_1b; permission denied (source) > > [arossi@fndcatemp1 ~]$ xrdcp5x -f xroots://fndcatemp1.fnal.gov:1094//fermilab/users/arossi/data_1b?authz=Bearer%20`cat $XDG_RUNTIME_DIR/bt_u8773` /dev/null > [1B/1B][100%][==================================================][1B/s] > > > 1. ZTN ON: > > [arossi@fndcatemp1 ~]$ xrdcp5x -f xroots://fndcatemp1.fnal.gov:1094//fermilab/users/arossi/data_1b /dev/null > [0B/0B][100%][==================================================][0B/s] > Run: [ERROR] Server responded with an error: [3010] Unable to open /fermilab/users/arossi/data_1b; permission denied (source) > > [arossi@fndcatemp1 ~]$ xrdcp5x -f xroots://fndcatemp1.fnal.gov:1094//fermilab/users/arossi/data_1b?authz=Bearer%20`cat $XDG_RUNTIME_DIR/bt_u8773` /dev/null > [1B/1B][100%][==================================================][1B/s] > > > Sanity check: ZTN is being enforced, and the client is using the token in the known location. If I remove the runtime dir env variable and put the token in an unknown location, authentication fails: > > > [arossi@fndcatemp1 ~]$ mv /run/user/8773/bt_u8773 . > [arossi@fndcatemp1 ~]$ export XDG_RUNTIME_DIR= > [arossi@fndcatemp1 ~]$ xrdcp5x -f xroots://fndcatemp1.fnal.gov:1094//fermilab/users/arossi/data_1b?authz=Bearer%20`cat bt_u8773` /dev/null > [0B/0B][100%][==================================================][0B/s] > Run: [FATAL] Auth failed: No protocols left to try (source) > > > > So there is no difference between the two scenarios in terms of downstream authorization on open (i.e., the server is not falling back to the ZTN token for authorization). > > Is there a special switch that needs to be flipped to make this happen? > > Or did I totally misunderstand what has been going on concerning fallback? > > > Cheers, Al > > ________________________________________________ > Albert L. Rossi > Senior Software Developer > Scientific Computing Division, Scientific Data Services, Distributed Data Development > FCC 229A > Mail Station 369 (FCC 2W) > Fermi National Accelerator Laboratory > Batavia, IL 60510 > (630) 840-3023 > > ________________________________ > From: Albert Rossi <[log in to unmask]> > Sent: Tuesday, March 29, 2022 7:25 AM > To: Andrew Hanushevsky <[log in to unmask]> > Cc: xrootd-dev <[log in to unmask]> > Subject: Re: ZTN and TPC > > Hi Andy, > > I understand the points you are making below. So it is my fault for not making my concerns clearer. > > dCache can indeed accept scitokens for authorization without authentication. That is what I fixed. But that is not the problem. > > I was just trying to look at this from the standpoint of the client, and what Brian was originally worried about -- unless I've misunderstood that. > > So, let me re-ask the question in another way. > > I am an xrootd client. I want to do TPC. > > But my user does not want to expose a token on the path query as a CGI element. After the changes you have made/are making, authorization can fall back to the ZTN token, provided that token has expressed subject and claims as well as issuer and audience. > > But the server I am talking to does not have ZTN turned on; it only has enabled scitoken authorization. > > There is a token at XDG_RUNTIME_DIR in the env in which I initiate the transfer. > > Am I going to succeed in getting authorized? > > Al > ________________________________________________ > Albert L. Rossi > Senior Software Developer > Scientific Computing Division, Scientific Data Services, Distributed Data Development > FCC 229A > Mail Station 369 (FCC 2W) > Fermi National Accelerator Laboratory > Batavia, IL 60510 > (630) 840-3023 > > ________________________________ > From: Andrew Hanushevsky <[log in to unmask]> > Sent: Tuesday, March 29, 2022 1:17 AM > To: Albert Rossi <[log in to unmask]> > Cc: xrootd-dev <[log in to unmask]> > Subject: Re: ZTN and TPC > > Hi Albert, > > Lots of questions here. OK, so let me frst say what is going on here. > > a) When enabled, ztn simply asks the connecting client (irrespective of > what it is going to do) to provide a valid token. This token may or may > not be used for future authorization purposes. That decision is totally > independent. Why? Because at the point we ask for a ztn token all we want > to know is that the client has or can obtain a valid token. That is all we > care about. So, the following points must keep this in mind. > > On Fri, 25 Mar 2022, Albert Rossi wrote: > >> However, that is not the question I have. What I am writing about here >> has to do with ZTN in this equation. If your ZTN module is loaded, how >> does it know to allow the third-party client to get a "pass", since that >> client does not have any JWT token? > Totally independent decision points. Consider this: > a) Client logs in and at that point we have no idea what the purpose is > but we ask for a ztn token. > b) Client supplies one and we check if it is valid, if so, client is > allowed to proceed. > c) Client wants to do a TPC. Perfectly acceptable. Does the ztn token have > anything to do with that? Not necessarily. In fact, we really don't know > and wish Brian would weigh on this but alas no word. > >> Or does it still get the ZTN token even though it does not provide a >> token for authorization to the source server? > Again, please reread the beginning statement. When we ask for a zrtn token > we only wish to ascertain the client's ability to get one. It has nothing > to do with any future autrhorization. To the extend that the authorization > module may use the zn token is up to that module. We really don't care. > >> Or do you have to turn ZTN off with TPC? > > Absolutely not! That is not what the primary purpose of ztn is. I think > you are really confusing the purpose of ztn and what redezvous TPC is > trying to accomplish. The problem here is that unlike xrootd where we can > easily suppress authorization when we know this is a rendezvous TPC dCache > appears not to be able to do this. So, it needs some kind of token in > addition to the rendezvous token to move forward. The question is which > token are we talking about. My asnwer is without Brian's weigh in I don't > know and there is silence at his end. So we are both out of luck. > >> I am asking these questions because I have not figured out, for dCache, >> how to (a) specify ZTN as an authentication protocol, but (b) allow a >> specifically third-party connection not to have to present a ZTN token. >> At authentication time, it does not seem to me the server knows enough >> about the client to be able to distinguish what it is. >> Or does it? > No it does not matter. The problem here is that any server may ask for a > ztn token and you better be prepared to supply one or have another > authentication protocol you can fall back to. I would suggest the simplest > approach here. In the presence of a TPC should the target server ask for a > token then you supply the ztn token should you have it. If you do not then > you will need to fallback on some other authentication protocol that the > target server supports. If there is no other protcol then whole thing > simply fails and you live with that. > > Andy > > > Some guidance here > would be very helpful, > > Thanks, Al >> >> ________________________________________________ >> Albert L. Rossi >> Senior Software Developer >> Scientific Computing Division, Scientific Data Services, Distributed Data Development >> FCC 229A >> Mail Station 369 (FCC 2W) >> Fermi National Accelerator Laboratory >> Batavia, IL 60510 >> (630) 840-3023 >> >> >> ######################################################################## >> Use REPLY-ALL to reply to list >> >> To unsubscribe from the XROOTD-DEV list, click the following link: >> https://urldefense.proofpoint.com/v2/url?u=https-3A__listserv.slac.stanford.edu_cgi-2Dbin_wa-3FSUBED1-3DXROOTD-2DDEV-26A-3D1&d=DwIBAg&c=gRgGjJ3BkIsb5y6s49QqsA&r=60rQ0HHqHmEY1P6VSdyuTQ&m=9aahSq-wsU59ZSWZ00xk_zy5ZFU6hyg63E0HPoGzJQ8F6TWVj47l3nCukhbHNHEw&s=_VQWTuyj544srHCnttohyT1-ZjVHIbgM2r3_V-1H_Oo&e= >> > ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-DEV list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1