 |
 |
Hi, I posted the proposed sources here: https://bugzilla.redhat.com/show_bug.cgi?id=1591556 <https://bugzilla.redhat.com/show_bug.cgi?id=1591556> If there's anyone who can pickup the review and push it forward, it would be greatly appreciated. Brian > On Jun 14, 2018, at 10:15 AM, Andrew Hanushevsky <[log in to unmask]> wrote: > > Yes, I think that Michal is blessed as of now (as long as he doesn't make any mistakes :-). I suppose another reason to host it in the xroot github repo under, say, xrootd-macaroons. > > Andy > > On Thu, 14 Jun 2018, Fabrizio Furano wrote: > >> Are you a blessed EPEL packager ? >> You need one to bless your package. Should not be difficult,as it's a >> plain library. >> >> I think Michal may have this kind of power. >> >> About Debian I don't know personally. So far we got the key components >> into it by courtesy (someone told me it was Mattias). I'd say that EPEL >> comes first. >> >> f >> >> On 14.06.18 16:57, Brian Bockelman wrote: >>> Hi, >>> >>> I can probably take care of this for EPEL, but I really have no background in Debian. >>> >>> Assuming we could get at least that far, could we get things into the main repo? >>> >>> Andy, your mention of "stock repos" was a bit ambiguous. Do you consider EPEL a stock repo for RHEL-like platforms? >>> >>> Brian >>> >>>> On Jun 14, 2018, at 7:15 AM, Andrew Hanushevsky <[log in to unmask]> wrote: >>>> >>>> Hi Fabrizio, >>>> >>>> I totally agree. We will not enbed libmacroons source into the xroot repo simply to avoid this kind of mess (for us and everyone else :-). So, I urge that you or whomever you lean on, repackge libmacaroons into an EPEL-style package that we can use without ever looking at the source. >>>> >>>> Andy >>>> >>>> >>>> On Thu, 14 Jun 2018, Fabrizio Furano wrote: >>>> >>>>> Hi Andy, >>>>> >>>>> yes, development style is one thing, and so far the xrootd project >>>>> has been very clear on that. >>>>> >>>>> In this case I had DPM in mind, and its WebDAV frontend >>>>> already embedded the source of libmacaroons quite some time ago. >>>>> Now we are talking about embedding the code of libmacaroons also in xrootd. >>>>> >>>>> The result would be to have potentially two different versions of >>>>> the same code in the same DPM system, coming from two different >>>>> frontends. >>>>> >>>>> It will work (I believe), yet the clean solution IMO is to properly >>>>> package libmacaroons into EPEL+debian, so that noone needs to embed the code anymore. >>>>> >>>>> f >>>>> >>>>> >>>>> On 06/14/2018 12:01 PM, Andrew Hanushevsky wrote: >>>>>> Hi Fabrizio, >>>>>> >>>>>> I totally agree that having two source builds are a pain and should be avoided. I'm almost tempted to say should be prohibited. >>>>>> In the xroot case, it should be possible to develop a plugin by installing the rpms you are dependent on and restricting >>>>>> yourself to whatever public headers come with it. That's how it's done with all commonly used third party >>>>>> packages. If can't develop that way then something is fundamentally wrong and should be corrected. Doing it that way, avoids the >>>>>> exact problem you mention. So, I guess it comes down to how we enforce that kind of development style. Do you agree? >>>>>> >>>>>> Andy >>>>>> >>>>>> On Thu, 14 Jun 2018, Fabrizio Furano wrote: >>>>>> >>>>>>> Hi Andy, >>>>>>> >>>>>>> yes, that would be clean, and personally I would prefer to >>>>>>> have libmacaroons in the stock repos. >>>>>>> >>>>>>> The annoyance I'd like to avoid is that so far DPM builds >>>>>>> libmacaroons inside the source tree as an internal dep, >>>>>>> exactly as you describe, for using it inside Apache. It's there >>>>>>> since one year and a half... >>>>>>> >>>>>>> It would be quite annoying to have two versions of the same lib >>>>>>> built inside the source tree of two different frontends: mod_lcgdm_dav >>>>>>> and xrootd >>>>>>> >>>>>>> This is why I'd prefer to investigate if we can organize an "official" >>>>>>> publishing of libmacaroons, at least to epel and debian. This thing >>>>>>> has already been done in the past, and would represent to me the >>>>>>> cleanest scenario possible, or at least it would give the possibility >>>>>>> of having a clean situation. >>>>>>> >>>>>>> ... thoughts? >>>>>>> >>>>>>> Cheers >>>>>>> Fabrizio >>>>>>> >>>>>>> >>>>>>> On 14.06.18 09:55, Andrew Hanushevsky wrote: >>>>>>>> In general, we do not add components to the main repo that depend on >>>>>>>> third party libraries that are not available in a stock system. The >>>>>>>> reasons for this should be obvious. The Macroon component is only one of >>>>>>>> several components that people are developing with third party >>>>>>>> dependencies. So we know we need to solve this problem. Our current >>>>>>>> thinking is to setup additional projects in the main xroot repo to host >>>>>>>> such developments and include them into our standard build pipeline. >>>>>>>> Doing this should solve the dependency issue as well as making it >>>>>>>> trivial to assign proper ownership so we aren't in the loop in terms of >>>>>>>> updates and whatever. However, we would package them and sites could >>>>>>>> install whatever they want via rpm. That way, those who wish to use a >>>>>>>> feature know ahead of time that there will be additional libraries that >>>>>>>> will be installed that we have no control over and may conflict with >>>>>>>> whatever they already have installed. >>>>>>>> >>>>>>>> This seems the like the cleanest approach to this issue and avoids >>>>>>>> leaving dead code in the main repo as these components come into and >>>>>>>> fall out of favor. Michal and I have to work on this as we don't it in >>>>>>>> place but should have a workable solution relatively soon. >>>>>>>> >>>>>>>> Andy >>>>>>>> >>>>>>>> On Wed, 13 Jun 2018, Fabrizio Furano wrote: >>>>>>>> >>>>>>>>> Hi Brian, >>>>>>>>> >>>>>>>>> true, macaroons are implemented by DPM and dcache. Having them also in >>>>>>>>> xrootd >>>>>>>>> would be great. >>>>>>>>> >>>>>>>>> About the distribution, my personal preference is to have these things >>>>>>>>> handy to >>>>>>>>> install and clean on the source tree, so I'd love to see macaroons and >>>>>>>>> scitokens >>>>>>>>> in the Xrd codebase, respecting all the strict project rules about >>>>>>>>> makefiles (e.g. turning off the component) and deps. >>>>>>>>> >>>>>>>>> What did you do for libmacaroons? Since it's not distributed, we build >>>>>>>>> it with DPM so far. >>>>>>>>> Are you doing the same? >>>>>>>>> Maybe we want to find a volunteer packager that submits it to >>>>>>>>> epel/debian, etc. >>>>>>>>> and only then refine the details of a build with Xrd ? >>>>>>>>> >>>>>>>>> Fabrizio >>>>>>>>> >>>>>>>>> >>>>>>>>> On 06/13/2018 04:08 PM, Brian Bockelman wrote: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> I have an initial (but functioning) support for Macaroons in Xrootd: >>>>>>>>>> >>>>>>>>>> https://github.com/bbockelm/xrootd-macaroons >>>>>>>>>> >>>>>>>>>> Macaroons are a symmetric-key-based token format. It allows an >>>>>>>>>> entity with access to the secret key generate a bearer token >>>>>>>>>> embedding one more "caveats" (rules for usage, such as restrictions >>>>>>>>>> on path, expiration time, or valid operations). The bearer >>>>>>>>>> of the token can add additional caveats (reducing permission) but >>>>>>>>>> doesn't have the ability to remove them. >>>>>>>>>> >>>>>>>>>> Anyone else with the secret key (such as the same host or another >>>>>>>>>> host in the cluster) can then verify the token as apply the >>>>>>>>>> authorization rules in the caveats. >>>>>>>>>> >>>>>>>>>> Why is this useful? A few cases: >>>>>>>>>> >>>>>>>>>> 1. Delegating fine-grained access: Rucio generates a token to access >>>>>>>>>> a single file at a site (using its X509 client cert to >>>>>>>>>> authenticate), then sends the token to an end-user. Hence, the >>>>>>>>>> end-user doesn't need to authenticate with the site (no user >>>>>>>>>> X509 necessary) in order to download files. The VO (ATLAS, via >>>>>>>>>> Rucio) can manage the fine-grained rights they delegate to the >>>>>>>>>> user -- even if the storage is run by the site. >>>>>>>>>> >>>>>>>>>> 2. Enabling third-party-copy: If client C wants server A to download >>>>>>>>>> from server B, the client can contact server B for a >>>>>>>>>> fine-grained access token and provide it to server A as part of the >>>>>>>>>> HTTP COPY request. >>>>>>>>>> - I suspect I'll get the first FTS3-based transfers working today. >>>>>>>>>> >>>>>>>>>> IIUC, Macaroons are implemented by dCache and DPM. I utilized the >>>>>>>>>> same caveat formats and API for requesting new tokens, but >>>>>>>>>> generally one would not expect tokens to be reusable across different >>>>>>>>>> implementations or site installations. >>>>>>>>>> >>>>>>>>>> Currently, the token is generated via a XrdHttp plugin but should be >>>>>>>>>> usable throughout the authorization framework; however, you >>>>>>>>>> need an encrypted channel between client and server to use it safely. >>>>>>>>>> >>>>>>>>>> The obvious question is "how does this differ from SciTokens"? >>>>>>>>>> - SciTokens are based on asymmetric (public key) cryptography whereas >>>>>>>>>> Macaroons are symmetric key. This means anyone can verify >>>>>>>>>> a SciToken but only the issuing entity can verify a Macaroon. >>>>>>>>>> - In the SciTokens model, the VO issues the authorization, which may >>>>>>>>>> be valid across many sites. The macaroon is specific to >>>>>>>>>> the site and requires an interaction with the site to generate. >>>>>>>>>> - Macaroons can be attenuated (made weaker) by the end-user. I can >>>>>>>>>> take a powerful macaroon and add additional limitations >>>>>>>>>> without contacting a third-party, then safely give the limited >>>>>>>>>> macaroon to another person if I desire. SciTokens can only be >>>>>>>>>> attenuated by contacting the VO to exchange the powerful one for a >>>>>>>>>> new one (hence, the VO always generates the token). >>>>>>>>>> >>>>>>>>>> Just as one uses a mix of asymmetric and symmetric cryptography >>>>>>>>>> throughout the course of the day, I see SciTokens and Macaroons >>>>>>>>>> as complementary approaches, each enabling some distinct and some >>>>>>>>>> common use cases. >>>>>>>>>> >>>>>>>>>> So - one natural question for me is whether this better lives inside >>>>>>>>>> the xrootd repository or standalone. Unlike SciTokens, the >>>>>>>>>> dependency stack for Macaroons is more manageable (a direct >>>>>>>>>> dependency on libmacaroons - https://github.com/rescrv/libmacaroons >>>>>>>>>> - and an indirect dependency on libbsd). This means upstreaming of >>>>>>>>>> the macaroons code is more approachable than the SciTokens >>>>>>>>>> code. Additionally, since the code includes both an issuer and a >>>>>>>>>> verifier, Macaroons are more immediately usable - no >>>>>>>>>> non-Xrootd services setup. >>>>>>>>>> >>>>>>>>>> Accordingly, I'm leaning to converting this into a module inside the >>>>>>>>>> xrootd repository. >>>>>>>>>> >>>>>>>>>> Thoughts? >>>>>>>>>> >>>>>>>>>> Brian >>>>>>>>>> >>>>>>>>>> -------------------------------------------------------------------------------------------------------------------------------- >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Use REPLY-ALL to reply to list >>>>>>>>>> >>>>>>>>>> To unsubscribe from the XROOTD-DEV list, click the following link: >>>>>>>>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1 >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> ######################################################################## >>>>>>>>> Use REPLY-ALL to reply to list >>>>>>>>> >>>>>>>>> To unsubscribe from the XROOTD-DEV list, click the following link: >>>>>>>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1 >>>>>>>>> >>>>>>>> >>>>>>>> ######################################################################## >>>>>>>> Use REPLY-ALL to reply to list >>>>>>>> >>>>>>>> To unsubscribe from the XROOTD-DEV list, click the following link: >>>>>>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1 >>>>>>> >>>>>>> >>>>>>> >>>>> >>> >> >> >> ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-DEV list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1