 |
 |
Hi, slightly OT, may I know what are the actual desiderata for a macaroon-based system ? As I was mentioning, DPM has the macaroons code in the frontend (disabled by default), yet we consider it experimental. Which authentication/authorization criteria may have to be applied to generate a macaroon? What does dCache do for example? Thanks Fabrizio On 17.06.18 06:01, Brian Bockelman wrote: > Hi, > > The package has moved to testing: > - Fedora 28: https://bodhi.fedoraproject.org/updates/FEDORA-2018-9e2640da1e > - EPEL > 7: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2018-3b68fe0e41 > > If folks could please take some time to review and test it, it'll help > reduce the 2 week waiting period. > > Regardless, on the timescale of the next Xrootd release, libmacaroons > should be present on at least a subset of the production platforms. > > Brian > >> On Jun 14, 2018, at 9:23 PM, Brian Bockelman <[log in to unmask] >> <mailto:[log in to unmask]>> wrote: >> >> Hi, >> >> I posted the proposed sources here: >> >> https://bugzilla.redhat.com/show_bug.cgi?id=1591556 >> >> If there's anyone who can pickup the review and push it forward, it >> would be greatly appreciated. >> >> Brian >> >>> On Jun 14, 2018, at 10:15 AM, Andrew Hanushevsky >>> <[log in to unmask] <mailto:[log in to unmask]>> wrote: >>> >>> Yes, I think that Michal is blessed as of now (as long as he doesn't >>> make any mistakes :-). I suppose another reason to host it in the >>> xroot github repo under, say, xrootd-macaroons. >>> >>> Andy >>> >>> On Thu, 14 Jun 2018, Fabrizio Furano wrote: >>> >>>> Are you a blessed EPEL packager ? >>>> You need one to bless your package. Should not be difficult,as it's a >>>> plain library. >>>> >>>> I think Michal may have this kind of power. >>>> >>>> About Debian I don't know personally. So far we got the key components >>>> into it by courtesy (someone told me it was Mattias). I'd say that EPEL >>>> comes first. >>>> >>>> f >>>> >>>> On 14.06.18 16:57, Brian Bockelman wrote: >>>>> Hi, >>>>> >>>>> I can probably take care of this for EPEL, but I really have no >>>>> background in Debian. >>>>> >>>>> Assuming we could get at least that far, could we get things into >>>>> the main repo? >>>>> >>>>> Andy, your mention of "stock repos" was a bit ambiguous. Do you >>>>> consider EPEL a stock repo for RHEL-like platforms? >>>>> >>>>> Brian >>>>> >>>>>> On Jun 14, 2018, at 7:15 AM, Andrew Hanushevsky >>>>>> <[log in to unmask] <mailto:[log in to unmask]>> wrote: >>>>>> >>>>>> Hi Fabrizio, >>>>>> >>>>>> I totally agree. We will not enbed libmacroons source into the >>>>>> xroot repo simply to avoid this kind of mess (for us and everyone >>>>>> else :-). So, I urge that you or whomever you lean on, repackge >>>>>> libmacaroons into an EPEL-style package that we can use without >>>>>> ever looking at the source. >>>>>> >>>>>> Andy >>>>>> >>>>>> >>>>>> On Thu, 14 Jun 2018, Fabrizio Furano wrote: >>>>>> >>>>>>> Hi Andy, >>>>>>> >>>>>>> yes, development style is one thing, and so far the xrootd project >>>>>>> has been very clear on that. >>>>>>> >>>>>>> In this case I had DPM in mind, and its WebDAV frontend >>>>>>> already embedded the source of libmacaroons quite some time ago. >>>>>>> Now we are talking about embedding the code of libmacaroons also >>>>>>> in xrootd. >>>>>>> >>>>>>> The result would be to have potentially two different versions of >>>>>>> the same code in the same DPM system, coming from two different >>>>>>> frontends. >>>>>>> >>>>>>> It will work (I believe), yet the clean solution IMO is to properly >>>>>>> package libmacaroons into EPEL+debian, so that noone needs to >>>>>>> embed the code anymore. >>>>>>> >>>>>>> f >>>>>>> >>>>>>> >>>>>>> On 06/14/2018 12:01 PM, Andrew Hanushevsky wrote: >>>>>>>> Hi Fabrizio, >>>>>>>> >>>>>>>> I totally agree that having two source builds are a pain and >>>>>>>> should be avoided. I'm almost tempted to say should be prohibited. >>>>>>>> In the xroot case, it should be possible to develop a plugin by >>>>>>>> installing the rpms you are dependent on and restricting >>>>>>>> yourself to whatever public headers come with it. That's how >>>>>>>> it's done with all commonly used third party >>>>>>>> packages. If can't develop that way then something is >>>>>>>> fundamentally wrong and should be corrected. Doing it that way, >>>>>>>> avoids the >>>>>>>> exact problem you mention. So, I guess it comes down to how we >>>>>>>> enforce that kind of development style. Do you agree? >>>>>>>> >>>>>>>> Andy >>>>>>>> >>>>>>>> On Thu, 14 Jun 2018, Fabrizio Furano wrote: >>>>>>>> >>>>>>>>> Hi Andy, >>>>>>>>> >>>>>>>>> yes, that would be clean, and personally I would prefer to >>>>>>>>> have libmacaroons in the stock repos. >>>>>>>>> >>>>>>>>> The annoyance I'd like to avoid is that so far DPM builds >>>>>>>>> libmacaroons inside the source tree as an internal dep, >>>>>>>>> exactly as you describe, for using it inside Apache. It's there >>>>>>>>> since one year and a half... >>>>>>>>> >>>>>>>>> It would be quite annoying to have two versions of the same lib >>>>>>>>> built inside the source tree of two different frontends: >>>>>>>>> mod_lcgdm_dav >>>>>>>>> and xrootd >>>>>>>>> >>>>>>>>> This is why I'd prefer to investigate if we can organize an >>>>>>>>> "official" >>>>>>>>> publishing of libmacaroons, at least to epel and debian. This thing >>>>>>>>> has already been done in the past, and would represent to me the >>>>>>>>> cleanest scenario possible, or at least it would give the >>>>>>>>> possibility >>>>>>>>> of having a clean situation. >>>>>>>>> >>>>>>>>> ... thoughts? >>>>>>>>> >>>>>>>>> Cheers >>>>>>>>> Fabrizio >>>>>>>>> >>>>>>>>> >>>>>>>>> On 14.06.18 09:55, Andrew Hanushevsky wrote: >>>>>>>>>> In general, we do not add components to the main repo that >>>>>>>>>> depend on >>>>>>>>>> third party libraries that are not available in a stock >>>>>>>>>> system. The >>>>>>>>>> reasons for this should be obvious. The Macroon component is >>>>>>>>>> only one of >>>>>>>>>> several components that people are developing with third party >>>>>>>>>> dependencies. So we know we need to solve this problem. Our >>>>>>>>>> current >>>>>>>>>> thinking is to setup additional projects in the main xroot >>>>>>>>>> repo to host >>>>>>>>>> such developments and include them into our standard build >>>>>>>>>> pipeline. >>>>>>>>>> Doing this should solve the dependency issue as well as making it >>>>>>>>>> trivial to assign proper ownership so we aren't in the loop in >>>>>>>>>> terms of >>>>>>>>>> updates and whatever. However, we would package them and sites >>>>>>>>>> could >>>>>>>>>> install whatever they want via rpm. That way, those who wish >>>>>>>>>> to use a >>>>>>>>>> feature know ahead of time that there will be additional >>>>>>>>>> libraries that >>>>>>>>>> will be installed that we have no control over and may >>>>>>>>>> conflict with >>>>>>>>>> whatever they already have installed. >>>>>>>>>> >>>>>>>>>> This seems the like the cleanest approach to this issue and avoids >>>>>>>>>> leaving dead code in the main repo as these components come >>>>>>>>>> into and >>>>>>>>>> fall out of favor. Michal and I have to work on this as we >>>>>>>>>> don't it in >>>>>>>>>> place but should have a workable solution relatively soon. >>>>>>>>>> >>>>>>>>>> Andy >>>>>>>>>> >>>>>>>>>> On Wed, 13 Jun 2018, Fabrizio Furano wrote: >>>>>>>>>> >>>>>>>>>>> Hi Brian, >>>>>>>>>>> >>>>>>>>>>> true, macaroons are implemented by DPM and dcache. Having >>>>>>>>>>> them also in >>>>>>>>>>> xrootd >>>>>>>>>>> would be great. >>>>>>>>>>> >>>>>>>>>>> About the distribution, my personal preference is to have >>>>>>>>>>> these things >>>>>>>>>>> handy to >>>>>>>>>>> install and clean on the source tree, so I'd love to see >>>>>>>>>>> macaroons and >>>>>>>>>>> scitokens >>>>>>>>>>> in the Xrd codebase, respecting all the strict project rules >>>>>>>>>>> about >>>>>>>>>>> makefiles (e.g. turning off the component) and deps. >>>>>>>>>>> >>>>>>>>>>> What did you do for libmacaroons? Since it's not distributed, >>>>>>>>>>> we build >>>>>>>>>>> it with DPM so far. >>>>>>>>>>> Are you doing the same? >>>>>>>>>>> Maybe we want to find a volunteer packager that submits it to >>>>>>>>>>> epel/debian, etc. >>>>>>>>>>> and only then refine the details of a build with Xrd ? >>>>>>>>>>> >>>>>>>>>>> Fabrizio >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 06/13/2018 04:08 PM, Brian Bockelman wrote: >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> I have an initial (but functioning) support for Macaroons in >>>>>>>>>>>> Xrootd: >>>>>>>>>>>> >>>>>>>>>>>> https://github.com/bbockelm/xrootd-macaroons >>>>>>>>>>>> >>>>>>>>>>>> Macaroons are a symmetric-key-based token format. It allows an >>>>>>>>>>>> entity with access to the secret key generate a bearer token >>>>>>>>>>>> embedding one more "caveats" (rules for usage, such as >>>>>>>>>>>> restrictions >>>>>>>>>>>> on path, expiration time, or valid operations). The bearer >>>>>>>>>>>> of the token can add additional caveats (reducing >>>>>>>>>>>> permission) but >>>>>>>>>>>> doesn't have the ability to remove them. >>>>>>>>>>>> >>>>>>>>>>>> Anyone else with the secret key (such as the same host or >>>>>>>>>>>> another >>>>>>>>>>>> host in the cluster) can then verify the token as apply the >>>>>>>>>>>> authorization rules in the caveats. >>>>>>>>>>>> >>>>>>>>>>>> Why is this useful? A few cases: >>>>>>>>>>>> >>>>>>>>>>>> 1. Delegating fine-grained access: Rucio generates a token >>>>>>>>>>>> to access >>>>>>>>>>>> a single file at a site (using its X509 client cert to >>>>>>>>>>>> authenticate), then sends the token to an end-user. Hence, the >>>>>>>>>>>> end-user doesn't need to authenticate with the site (no user >>>>>>>>>>>> X509 necessary) in order to download files. The VO (ATLAS, via >>>>>>>>>>>> Rucio) can manage the fine-grained rights they delegate to the >>>>>>>>>>>> user -- even if the storage is run by the site. >>>>>>>>>>>> >>>>>>>>>>>> 2. Enabling third-party-copy: If client C wants server A to >>>>>>>>>>>> download >>>>>>>>>>>> from server B, the client can contact server B for a >>>>>>>>>>>> fine-grained access token and provide it to server A as part >>>>>>>>>>>> of the >>>>>>>>>>>> HTTP COPY request. >>>>>>>>>>>> - I suspect I'll get the first FTS3-based transfers working >>>>>>>>>>>> today. >>>>>>>>>>>> >>>>>>>>>>>> IIUC, Macaroons are implemented by dCache and DPM. I >>>>>>>>>>>> utilized the >>>>>>>>>>>> same caveat formats and API for requesting new tokens, but >>>>>>>>>>>> generally one would not expect tokens to be reusable across >>>>>>>>>>>> different >>>>>>>>>>>> implementations or site installations. >>>>>>>>>>>> >>>>>>>>>>>> Currently, the token is generated via a XrdHttp plugin but >>>>>>>>>>>> should be >>>>>>>>>>>> usable throughout the authorization framework; however, you >>>>>>>>>>>> need an encrypted channel between client and server to use >>>>>>>>>>>> it safely. >>>>>>>>>>>> >>>>>>>>>>>> The obvious question is "how does this differ from SciTokens"? >>>>>>>>>>>> - SciTokens are based on asymmetric (public key) >>>>>>>>>>>> cryptography whereas >>>>>>>>>>>> Macaroons are symmetric key. This means anyone can verify >>>>>>>>>>>> a SciToken but only the issuing entity can verify a Macaroon. >>>>>>>>>>>> - In the SciTokens model, the VO issues the authorization, >>>>>>>>>>>> which may >>>>>>>>>>>> be valid across many sites. The macaroon is specific to >>>>>>>>>>>> the site and requires an interaction with the site to generate. >>>>>>>>>>>> - Macaroons can be attenuated (made weaker) by the end-user. >>>>>>>>>>>> I can >>>>>>>>>>>> take a powerful macaroon and add additional limitations >>>>>>>>>>>> without contacting a third-party, then safely give the limited >>>>>>>>>>>> macaroon to another person if I desire. SciTokens can only be >>>>>>>>>>>> attenuated by contacting the VO to exchange the powerful one >>>>>>>>>>>> for a >>>>>>>>>>>> new one (hence, the VO always generates the token). >>>>>>>>>>>> >>>>>>>>>>>> Just as one uses a mix of asymmetric and symmetric cryptography >>>>>>>>>>>> throughout the course of the day, I see SciTokens and Macaroons >>>>>>>>>>>> as complementary approaches, each enabling some distinct and >>>>>>>>>>>> some >>>>>>>>>>>> common use cases. >>>>>>>>>>>> >>>>>>>>>>>> So - one natural question for me is whether this better >>>>>>>>>>>> lives inside >>>>>>>>>>>> the xrootd repository or standalone. Unlike SciTokens, the >>>>>>>>>>>> dependency stack for Macaroons is more manageable (a direct >>>>>>>>>>>> dependency on libmacaroons - >>>>>>>>>>>> https://github.com/rescrv/libmacaroons >>>>>>>>>>>> - and an indirect dependency on libbsd). This means >>>>>>>>>>>> upstreaming of >>>>>>>>>>>> the macaroons code is more approachable than the SciTokens >>>>>>>>>>>> code. Additionally, since the code includes both an issuer >>>>>>>>>>>> and a >>>>>>>>>>>> verifier, Macaroons are more immediately usable - no >>>>>>>>>>>> non-Xrootd services setup. >>>>>>>>>>>> >>>>>>>>>>>> Accordingly, I'm leaning to converting this into a module >>>>>>>>>>>> inside the >>>>>>>>>>>> xrootd repository. >>>>>>>>>>>> >>>>>>>>>>>> Thoughts? >>>>>>>>>>>> >>>>>>>>>>>> Brian >>>>>>>>>>>> >>>>>>>>>>>> -------------------------------------------------------------------------------------------------------------------------------- >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Use REPLY-ALL to reply to list >>>>>>>>>>>> >>>>>>>>>>>> To unsubscribe from the XROOTD-DEV list, click the following >>>>>>>>>>>> link: >>>>>>>>>>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1 >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> ######################################################################## >>>>>>>>>>> Use REPLY-ALL to reply to list >>>>>>>>>>> >>>>>>>>>>> To unsubscribe from the XROOTD-DEV list, click the following >>>>>>>>>>> link: >>>>>>>>>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1 >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> ######################################################################## >>>>>>>>>> Use REPLY-ALL to reply to list >>>>>>>>>> >>>>>>>>>> To unsubscribe from the XROOTD-DEV list, click the following link: >>>>>>>>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1 >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>> >>>>> >>>> >>>> >>>> >> >> >> ------------------------------------------------------------------------ >> >> Use REPLY-ALL to reply to list >> >> To unsubscribe from the XROOTD-DEV list, click the following link: >> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1 >> > > > ------------------------------------------------------------------------ > > Use REPLY-ALL to reply to list > > To unsubscribe from the XROOTD-DEV list, click the following link: > https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1 > ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-DEV list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1