 |
 |
Hi Fabrizio, I totally agree. We will not enbed libmacroons source into the xroot repo simply to avoid this kind of mess (for us and everyone else :-). So, I urge that you or whomever you lean on, repackge libmacaroons into an EPEL-style package that we can use without ever looking at the source. Andy On Thu, 14 Jun 2018, Fabrizio Furano wrote: > Hi Andy, > > yes, development style is one thing, and so far the xrootd project > has been very clear on that. > > In this case I had DPM in mind, and its WebDAV frontend > already embedded the source of libmacaroons quite some time ago. > Now we are talking about embedding the code of libmacaroons also in xrootd. > > The result would be to have potentially two different versions of > the same code in the same DPM system, coming from two different > frontends. > > It will work (I believe), yet the clean solution IMO is to properly > package libmacaroons into EPEL+debian, so that noone needs to embed the code anymore. > > f > > > On 06/14/2018 12:01 PM, Andrew Hanushevsky wrote: >> Hi Fabrizio, >> >> I totally agree that having two source builds are a pain and should be avoided. I'm almost tempted to say should be prohibited. >> In the xroot case, it should be possible to develop a plugin by installing the rpms you are dependent on and restricting >> yourself to whatever public headers come with it. That's how it's done with all commonly used third party >> packages. If can't develop that way then something is fundamentally wrong and should be corrected. Doing it that way, avoids the >> exact problem you mention. So, I guess it comes down to how we enforce that kind of development style. Do you agree? >> >> Andy >> >> On Thu, 14 Jun 2018, Fabrizio Furano wrote: >> >>> Hi Andy, >>> >>> yes, that would be clean, and personally I would prefer to >>> have libmacaroons in the stock repos. >>> >>> The annoyance I'd like to avoid is that so far DPM builds >>> libmacaroons inside the source tree as an internal dep, >>> exactly as you describe, for using it inside Apache. It's there >>> since one year and a half... >>> >>> It would be quite annoying to have two versions of the same lib >>> built inside the source tree of two different frontends: mod_lcgdm_dav >>> and xrootd >>> >>> This is why I'd prefer to investigate if we can organize an "official" >>> publishing of libmacaroons, at least to epel and debian. This thing >>> has already been done in the past, and would represent to me the >>> cleanest scenario possible, or at least it would give the possibility >>> of having a clean situation. >>> >>> ... thoughts? >>> >>> Cheers >>> Fabrizio >>> >>> >>> On 14.06.18 09:55, Andrew Hanushevsky wrote: >>>> In general, we do not add components to the main repo that depend on >>>> third party libraries that are not available in a stock system. The >>>> reasons for this should be obvious. The Macroon component is only one of >>>> several components that people are developing with third party >>>> dependencies. So we know we need to solve this problem. Our current >>>> thinking is to setup additional projects in the main xroot repo to host >>>> such developments and include them into our standard build pipeline. >>>> Doing this should solve the dependency issue as well as making it >>>> trivial to assign proper ownership so we aren't in the loop in terms of >>>> updates and whatever. However, we would package them and sites could >>>> install whatever they want via rpm. That way, those who wish to use a >>>> feature know ahead of time that there will be additional libraries that >>>> will be installed that we have no control over and may conflict with >>>> whatever they already have installed. >>>> >>>> This seems the like the cleanest approach to this issue and avoids >>>> leaving dead code in the main repo as these components come into and >>>> fall out of favor. Michal and I have to work on this as we don't it in >>>> place but should have a workable solution relatively soon. >>>> >>>> Andy >>>> >>>> On Wed, 13 Jun 2018, Fabrizio Furano wrote: >>>> >>>>> Hi Brian, >>>>> >>>>> true, macaroons are implemented by DPM and dcache. Having them also in >>>>> xrootd >>>>> would be great. >>>>> >>>>> About the distribution, my personal preference is to have these things >>>>> handy to >>>>> install and clean on the source tree, so I'd love to see macaroons and >>>>> scitokens >>>>> in the Xrd codebase, respecting all the strict project rules about >>>>> makefiles (e.g. turning off the component) and deps. >>>>> >>>>> What did you do for libmacaroons? Since it's not distributed, we build >>>>> it with DPM so far. >>>>> Are you doing the same? >>>>> Maybe we want to find a volunteer packager that submits it to >>>>> epel/debian, etc. >>>>> and only then refine the details of a build with Xrd ? >>>>> >>>>> Fabrizio >>>>> >>>>> >>>>> On 06/13/2018 04:08 PM, Brian Bockelman wrote: >>>>>> Hi, >>>>>> >>>>>> I have an initial (but functioning) support for Macaroons in Xrootd: >>>>>> >>>>>> https://github.com/bbockelm/xrootd-macaroons >>>>>> >>>>>> Macaroons are a symmetric-key-based token format. It allows an >>>>>> entity with access to the secret key generate a bearer token >>>>>> embedding one more "caveats" (rules for usage, such as restrictions >>>>>> on path, expiration time, or valid operations). The bearer >>>>>> of the token can add additional caveats (reducing permission) but >>>>>> doesn't have the ability to remove them. >>>>>> >>>>>> Anyone else with the secret key (such as the same host or another >>>>>> host in the cluster) can then verify the token as apply the >>>>>> authorization rules in the caveats. >>>>>> >>>>>> Why is this useful? A few cases: >>>>>> >>>>>> 1. Delegating fine-grained access: Rucio generates a token to access >>>>>> a single file at a site (using its X509 client cert to >>>>>> authenticate), then sends the token to an end-user. Hence, the >>>>>> end-user doesn't need to authenticate with the site (no user >>>>>> X509 necessary) in order to download files. The VO (ATLAS, via >>>>>> Rucio) can manage the fine-grained rights they delegate to the >>>>>> user -- even if the storage is run by the site. >>>>>> >>>>>> 2. Enabling third-party-copy: If client C wants server A to download >>>>>> from server B, the client can contact server B for a >>>>>> fine-grained access token and provide it to server A as part of the >>>>>> HTTP COPY request. >>>>>> - I suspect I'll get the first FTS3-based transfers working today. >>>>>> >>>>>> IIUC, Macaroons are implemented by dCache and DPM. I utilized the >>>>>> same caveat formats and API for requesting new tokens, but >>>>>> generally one would not expect tokens to be reusable across different >>>>>> implementations or site installations. >>>>>> >>>>>> Currently, the token is generated via a XrdHttp plugin but should be >>>>>> usable throughout the authorization framework; however, you >>>>>> need an encrypted channel between client and server to use it safely. >>>>>> >>>>>> The obvious question is "how does this differ from SciTokens"? >>>>>> - SciTokens are based on asymmetric (public key) cryptography whereas >>>>>> Macaroons are symmetric key. This means anyone can verify >>>>>> a SciToken but only the issuing entity can verify a Macaroon. >>>>>> - In the SciTokens model, the VO issues the authorization, which may >>>>>> be valid across many sites. The macaroon is specific to >>>>>> the site and requires an interaction with the site to generate. >>>>>> - Macaroons can be attenuated (made weaker) by the end-user. I can >>>>>> take a powerful macaroon and add additional limitations >>>>>> without contacting a third-party, then safely give the limited >>>>>> macaroon to another person if I desire. SciTokens can only be >>>>>> attenuated by contacting the VO to exchange the powerful one for a >>>>>> new one (hence, the VO always generates the token). >>>>>> >>>>>> Just as one uses a mix of asymmetric and symmetric cryptography >>>>>> throughout the course of the day, I see SciTokens and Macaroons >>>>>> as complementary approaches, each enabling some distinct and some >>>>>> common use cases. >>>>>> >>>>>> So - one natural question for me is whether this better lives inside >>>>>> the xrootd repository or standalone. Unlike SciTokens, the >>>>>> dependency stack for Macaroons is more manageable (a direct >>>>>> dependency on libmacaroons - https://github.com/rescrv/libmacaroons >>>>>> - and an indirect dependency on libbsd). This means upstreaming of >>>>>> the macaroons code is more approachable than the SciTokens >>>>>> code. Additionally, since the code includes both an issuer and a >>>>>> verifier, Macaroons are more immediately usable - no >>>>>> non-Xrootd services setup. >>>>>> >>>>>> Accordingly, I'm leaning to converting this into a module inside the >>>>>> xrootd repository. >>>>>> >>>>>> Thoughts? >>>>>> >>>>>> Brian >>>>>> >>>>>> -------------------------------------------------------------------------------------------------------------------------------- >>>>>> >>>>>> >>>>>> >>>>>> Use REPLY-ALL to reply to list >>>>>> >>>>>> To unsubscribe from the XROOTD-DEV list, click the following link: >>>>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1 >>>>>> >>>>> >>>>> >>>>> ######################################################################## >>>>> Use REPLY-ALL to reply to list >>>>> >>>>> To unsubscribe from the XROOTD-DEV list, click the following link: >>>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1 >>>>> >>>> >>>> ######################################################################## >>>> Use REPLY-ALL to reply to list >>>> >>>> To unsubscribe from the XROOTD-DEV list, click the following link: >>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1 >>> >>> >>> > > ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-DEV list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1