 |
 |
Hi Fabrizio, If we want to use macaroons for TPC, I think we need to allow any authenticated user to generate a macaroon. No? As far as I can tell, this is what dCache does when its macaroons are enabled. Brian > On Jun 20, 2018, at 3:34 AM, Fabrizio Furano <[log in to unmask]> wrote: > > Hi, > > slightly OT, may I know what are the actual desiderata for > a macaroon-based system ? > > As I was mentioning, DPM has the macaroons code in the > frontend (disabled by default), yet we consider it experimental. > > Which authentication/authorization criteria may have to > be applied to generate a macaroon? What does dCache do for example? > > Thanks > Fabrizio > > > > On 17.06.18 06:01, Brian Bockelman wrote: >> Hi, >> >> The package has moved to testing: >> - Fedora 28: https://bodhi.fedoraproject.org/updates/FEDORA-2018-9e2640da1e >> - EPEL >> 7: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2018-3b68fe0e41 >> >> If folks could please take some time to review and test it, it'll help >> reduce the 2 week waiting period. >> >> Regardless, on the timescale of the next Xrootd release, libmacaroons >> should be present on at least a subset of the production platforms. >> >> Brian >> >>> On Jun 14, 2018, at 9:23 PM, Brian Bockelman <[log in to unmask] >>> <mailto:[log in to unmask]>> wrote: >>> >>> Hi, >>> >>> I posted the proposed sources here: >>> >>> https://bugzilla.redhat.com/show_bug.cgi?id=1591556 >>> >>> If there's anyone who can pickup the review and push it forward, it >>> would be greatly appreciated. >>> >>> Brian >>> >>>> On Jun 14, 2018, at 10:15 AM, Andrew Hanushevsky >>>> <[log in to unmask] <mailto:[log in to unmask]>> wrote: >>>> >>>> Yes, I think that Michal is blessed as of now (as long as he doesn't >>>> make any mistakes :-). I suppose another reason to host it in the >>>> xroot github repo under, say, xrootd-macaroons. >>>> >>>> Andy >>>> >>>> On Thu, 14 Jun 2018, Fabrizio Furano wrote: >>>> >>>>> Are you a blessed EPEL packager ? >>>>> You need one to bless your package. Should not be difficult,as it's a >>>>> plain library. >>>>> >>>>> I think Michal may have this kind of power. >>>>> >>>>> About Debian I don't know personally. So far we got the key components >>>>> into it by courtesy (someone told me it was Mattias). I'd say that EPEL >>>>> comes first. >>>>> >>>>> f >>>>> >>>>> On 14.06.18 16:57, Brian Bockelman wrote: >>>>>> Hi, >>>>>> >>>>>> I can probably take care of this for EPEL, but I really have no >>>>>> background in Debian. >>>>>> >>>>>> Assuming we could get at least that far, could we get things into >>>>>> the main repo? >>>>>> >>>>>> Andy, your mention of "stock repos" was a bit ambiguous. Do you >>>>>> consider EPEL a stock repo for RHEL-like platforms? >>>>>> >>>>>> Brian >>>>>> >>>>>>> On Jun 14, 2018, at 7:15 AM, Andrew Hanushevsky >>>>>>> <[log in to unmask] <mailto:[log in to unmask]>> wrote: >>>>>>> >>>>>>> Hi Fabrizio, >>>>>>> >>>>>>> I totally agree. We will not enbed libmacroons source into the >>>>>>> xroot repo simply to avoid this kind of mess (for us and everyone >>>>>>> else :-). So, I urge that you or whomever you lean on, repackge >>>>>>> libmacaroons into an EPEL-style package that we can use without >>>>>>> ever looking at the source. >>>>>>> >>>>>>> Andy >>>>>>> >>>>>>> >>>>>>> On Thu, 14 Jun 2018, Fabrizio Furano wrote: >>>>>>> >>>>>>>> Hi Andy, >>>>>>>> >>>>>>>> yes, development style is one thing, and so far the xrootd project >>>>>>>> has been very clear on that. >>>>>>>> >>>>>>>> In this case I had DPM in mind, and its WebDAV frontend >>>>>>>> already embedded the source of libmacaroons quite some time ago. >>>>>>>> Now we are talking about embedding the code of libmacaroons also >>>>>>>> in xrootd. >>>>>>>> >>>>>>>> The result would be to have potentially two different versions of >>>>>>>> the same code in the same DPM system, coming from two different >>>>>>>> frontends. >>>>>>>> >>>>>>>> It will work (I believe), yet the clean solution IMO is to properly >>>>>>>> package libmacaroons into EPEL+debian, so that noone needs to >>>>>>>> embed the code anymore. >>>>>>>> >>>>>>>> f >>>>>>>> >>>>>>>> >>>>>>>> On 06/14/2018 12:01 PM, Andrew Hanushevsky wrote: >>>>>>>>> Hi Fabrizio, >>>>>>>>> >>>>>>>>> I totally agree that having two source builds are a pain and >>>>>>>>> should be avoided. I'm almost tempted to say should be prohibited. >>>>>>>>> In the xroot case, it should be possible to develop a plugin by >>>>>>>>> installing the rpms you are dependent on and restricting >>>>>>>>> yourself to whatever public headers come with it. That's how >>>>>>>>> it's done with all commonly used third party >>>>>>>>> packages. If can't develop that way then something is >>>>>>>>> fundamentally wrong and should be corrected. Doing it that way, >>>>>>>>> avoids the >>>>>>>>> exact problem you mention. So, I guess it comes down to how we >>>>>>>>> enforce that kind of development style. Do you agree? >>>>>>>>> >>>>>>>>> Andy >>>>>>>>> >>>>>>>>> On Thu, 14 Jun 2018, Fabrizio Furano wrote: >>>>>>>>> >>>>>>>>>> Hi Andy, >>>>>>>>>> >>>>>>>>>> yes, that would be clean, and personally I would prefer to >>>>>>>>>> have libmacaroons in the stock repos. >>>>>>>>>> >>>>>>>>>> The annoyance I'd like to avoid is that so far DPM builds >>>>>>>>>> libmacaroons inside the source tree as an internal dep, >>>>>>>>>> exactly as you describe, for using it inside Apache. It's there >>>>>>>>>> since one year and a half... >>>>>>>>>> >>>>>>>>>> It would be quite annoying to have two versions of the same lib >>>>>>>>>> built inside the source tree of two different frontends: >>>>>>>>>> mod_lcgdm_dav >>>>>>>>>> and xrootd >>>>>>>>>> >>>>>>>>>> This is why I'd prefer to investigate if we can organize an >>>>>>>>>> "official" >>>>>>>>>> publishing of libmacaroons, at least to epel and debian. This thing >>>>>>>>>> has already been done in the past, and would represent to me the >>>>>>>>>> cleanest scenario possible, or at least it would give the >>>>>>>>>> possibility >>>>>>>>>> of having a clean situation. >>>>>>>>>> >>>>>>>>>> ... thoughts? >>>>>>>>>> >>>>>>>>>> Cheers >>>>>>>>>> Fabrizio >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 14.06.18 09:55, Andrew Hanushevsky wrote: >>>>>>>>>>> In general, we do not add components to the main repo that >>>>>>>>>>> depend on >>>>>>>>>>> third party libraries that are not available in a stock >>>>>>>>>>> system. The >>>>>>>>>>> reasons for this should be obvious. The Macroon component is >>>>>>>>>>> only one of >>>>>>>>>>> several components that people are developing with third party >>>>>>>>>>> dependencies. So we know we need to solve this problem. Our >>>>>>>>>>> current >>>>>>>>>>> thinking is to setup additional projects in the main xroot >>>>>>>>>>> repo to host >>>>>>>>>>> such developments and include them into our standard build >>>>>>>>>>> pipeline. >>>>>>>>>>> Doing this should solve the dependency issue as well as making it >>>>>>>>>>> trivial to assign proper ownership so we aren't in the loop in >>>>>>>>>>> terms of >>>>>>>>>>> updates and whatever. However, we would package them and sites >>>>>>>>>>> could >>>>>>>>>>> install whatever they want via rpm. That way, those who wish >>>>>>>>>>> to use a >>>>>>>>>>> feature know ahead of time that there will be additional >>>>>>>>>>> libraries that >>>>>>>>>>> will be installed that we have no control over and may >>>>>>>>>>> conflict with >>>>>>>>>>> whatever they already have installed. >>>>>>>>>>> >>>>>>>>>>> This seems the like the cleanest approach to this issue and avoids >>>>>>>>>>> leaving dead code in the main repo as these components come >>>>>>>>>>> into and >>>>>>>>>>> fall out of favor. Michal and I have to work on this as we >>>>>>>>>>> don't it in >>>>>>>>>>> place but should have a workable solution relatively soon. >>>>>>>>>>> >>>>>>>>>>> Andy >>>>>>>>>>> >>>>>>>>>>> On Wed, 13 Jun 2018, Fabrizio Furano wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi Brian, >>>>>>>>>>>> >>>>>>>>>>>> true, macaroons are implemented by DPM and dcache. Having >>>>>>>>>>>> them also in >>>>>>>>>>>> xrootd >>>>>>>>>>>> would be great. >>>>>>>>>>>> >>>>>>>>>>>> About the distribution, my personal preference is to have >>>>>>>>>>>> these things >>>>>>>>>>>> handy to >>>>>>>>>>>> install and clean on the source tree, so I'd love to see >>>>>>>>>>>> macaroons and >>>>>>>>>>>> scitokens >>>>>>>>>>>> in the Xrd codebase, respecting all the strict project rules >>>>>>>>>>>> about >>>>>>>>>>>> makefiles (e.g. turning off the component) and deps. >>>>>>>>>>>> >>>>>>>>>>>> What did you do for libmacaroons? Since it's not distributed, >>>>>>>>>>>> we build >>>>>>>>>>>> it with DPM so far. >>>>>>>>>>>> Are you doing the same? >>>>>>>>>>>> Maybe we want to find a volunteer packager that submits it to >>>>>>>>>>>> epel/debian, etc. >>>>>>>>>>>> and only then refine the details of a build with Xrd ? >>>>>>>>>>>> >>>>>>>>>>>> Fabrizio >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 06/13/2018 04:08 PM, Brian Bockelman wrote: >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> I have an initial (but functioning) support for Macaroons in >>>>>>>>>>>>> Xrootd: >>>>>>>>>>>>> >>>>>>>>>>>>> https://github.com/bbockelm/xrootd-macaroons >>>>>>>>>>>>> >>>>>>>>>>>>> Macaroons are a symmetric-key-based token format. It allows an >>>>>>>>>>>>> entity with access to the secret key generate a bearer token >>>>>>>>>>>>> embedding one more "caveats" (rules for usage, such as >>>>>>>>>>>>> restrictions >>>>>>>>>>>>> on path, expiration time, or valid operations). The bearer >>>>>>>>>>>>> of the token can add additional caveats (reducing >>>>>>>>>>>>> permission) but >>>>>>>>>>>>> doesn't have the ability to remove them. >>>>>>>>>>>>> >>>>>>>>>>>>> Anyone else with the secret key (such as the same host or >>>>>>>>>>>>> another >>>>>>>>>>>>> host in the cluster) can then verify the token as apply the >>>>>>>>>>>>> authorization rules in the caveats. >>>>>>>>>>>>> >>>>>>>>>>>>> Why is this useful? A few cases: >>>>>>>>>>>>> >>>>>>>>>>>>> 1. Delegating fine-grained access: Rucio generates a token >>>>>>>>>>>>> to access >>>>>>>>>>>>> a single file at a site (using its X509 client cert to >>>>>>>>>>>>> authenticate), then sends the token to an end-user. Hence, the >>>>>>>>>>>>> end-user doesn't need to authenticate with the site (no user >>>>>>>>>>>>> X509 necessary) in order to download files. The VO (ATLAS, via >>>>>>>>>>>>> Rucio) can manage the fine-grained rights they delegate to the >>>>>>>>>>>>> user -- even if the storage is run by the site. >>>>>>>>>>>>> >>>>>>>>>>>>> 2. Enabling third-party-copy: If client C wants server A to >>>>>>>>>>>>> download >>>>>>>>>>>>> from server B, the client can contact server B for a >>>>>>>>>>>>> fine-grained access token and provide it to server A as part >>>>>>>>>>>>> of the >>>>>>>>>>>>> HTTP COPY request. >>>>>>>>>>>>> - I suspect I'll get the first FTS3-based transfers working >>>>>>>>>>>>> today. >>>>>>>>>>>>> >>>>>>>>>>>>> IIUC, Macaroons are implemented by dCache and DPM. I >>>>>>>>>>>>> utilized the >>>>>>>>>>>>> same caveat formats and API for requesting new tokens, but >>>>>>>>>>>>> generally one would not expect tokens to be reusable across >>>>>>>>>>>>> different >>>>>>>>>>>>> implementations or site installations. >>>>>>>>>>>>> >>>>>>>>>>>>> Currently, the token is generated via a XrdHttp plugin but >>>>>>>>>>>>> should be >>>>>>>>>>>>> usable throughout the authorization framework; however, you >>>>>>>>>>>>> need an encrypted channel between client and server to use >>>>>>>>>>>>> it safely. >>>>>>>>>>>>> >>>>>>>>>>>>> The obvious question is "how does this differ from SciTokens"? >>>>>>>>>>>>> - SciTokens are based on asymmetric (public key) >>>>>>>>>>>>> cryptography whereas >>>>>>>>>>>>> Macaroons are symmetric key. This means anyone can verify >>>>>>>>>>>>> a SciToken but only the issuing entity can verify a Macaroon. >>>>>>>>>>>>> - In the SciTokens model, the VO issues the authorization, >>>>>>>>>>>>> which may >>>>>>>>>>>>> be valid across many sites. The macaroon is specific to >>>>>>>>>>>>> the site and requires an interaction with the site to generate. >>>>>>>>>>>>> - Macaroons can be attenuated (made weaker) by the end-user. >>>>>>>>>>>>> I can >>>>>>>>>>>>> take a powerful macaroon and add additional limitations >>>>>>>>>>>>> without contacting a third-party, then safely give the limited >>>>>>>>>>>>> macaroon to another person if I desire. SciTokens can only be >>>>>>>>>>>>> attenuated by contacting the VO to exchange the powerful one >>>>>>>>>>>>> for a >>>>>>>>>>>>> new one (hence, the VO always generates the token). >>>>>>>>>>>>> >>>>>>>>>>>>> Just as one uses a mix of asymmetric and symmetric cryptography >>>>>>>>>>>>> throughout the course of the day, I see SciTokens and Macaroons >>>>>>>>>>>>> as complementary approaches, each enabling some distinct and >>>>>>>>>>>>> some >>>>>>>>>>>>> common use cases. >>>>>>>>>>>>> >>>>>>>>>>>>> So - one natural question for me is whether this better >>>>>>>>>>>>> lives inside >>>>>>>>>>>>> the xrootd repository or standalone. Unlike SciTokens, the >>>>>>>>>>>>> dependency stack for Macaroons is more manageable (a direct >>>>>>>>>>>>> dependency on libmacaroons - >>>>>>>>>>>>> https://github.com/rescrv/libmacaroons >>>>>>>>>>>>> - and an indirect dependency on libbsd). This means >>>>>>>>>>>>> upstreaming of >>>>>>>>>>>>> the macaroons code is more approachable than the SciTokens >>>>>>>>>>>>> code. Additionally, since the code includes both an issuer >>>>>>>>>>>>> and a >>>>>>>>>>>>> verifier, Macaroons are more immediately usable - no >>>>>>>>>>>>> non-Xrootd services setup. >>>>>>>>>>>>> >>>>>>>>>>>>> Accordingly, I'm leaning to converting this into a module >>>>>>>>>>>>> inside the >>>>>>>>>>>>> xrootd repository. >>>>>>>>>>>>> >>>>>>>>>>>>> Thoughts? >>>>>>>>>>>>> >>>>>>>>>>>>> Brian >>>>>>>>>>>>> >>>>>>>>>>>>> -------------------------------------------------------------------------------------------------------------------------------- >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Use REPLY-ALL to reply to list >>>>>>>>>>>>> >>>>>>>>>>>>> To unsubscribe from the XROOTD-DEV list, click the following >>>>>>>>>>>>> link: >>>>>>>>>>>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1 >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> ######################################################################## >>>>>>>>>>>> Use REPLY-ALL to reply to list >>>>>>>>>>>> >>>>>>>>>>>> To unsubscribe from the XROOTD-DEV list, click the following >>>>>>>>>>>> link: >>>>>>>>>>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1 >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> ######################################################################## >>>>>>>>>>> Use REPLY-ALL to reply to list >>>>>>>>>>> >>>>>>>>>>> To unsubscribe from the XROOTD-DEV list, click the following link: >>>>>>>>>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>> >>>>> >>>>> >>>>> >>> >>> >>> ------------------------------------------------------------------------ >>> >>> Use REPLY-ALL to reply to list >>> >>> To unsubscribe from the XROOTD-DEV list, click the following link: >>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1 >>> >> >> >> ------------------------------------------------------------------------ >> >> Use REPLY-ALL to reply to list >> >> To unsubscribe from the XROOTD-DEV list, click the following link: >> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1 >> > > ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-DEV list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1