 |
 |
Hi Andy, yes, development style is one thing, and so far the xrootd project has been very clear on that. In this case I had DPM in mind, and its WebDAV frontend already embedded the source of libmacaroons quite some time ago. Now we are talking about embedding the code of libmacaroons also in xrootd. The result would be to have potentially two different versions of the same code in the same DPM system, coming from two different frontends. It will work (I believe), yet the clean solution IMO is to properly package libmacaroons into EPEL+debian, so that noone needs to embed the code anymore. f On 06/14/2018 12:01 PM, Andrew Hanushevsky wrote: > Hi Fabrizio, > > I totally agree that having two source builds are a pain and should be avoided. I'm almost tempted to say should be prohibited. > In the xroot case, it should be possible to develop a plugin by installing the rpms you are dependent on and restricting > yourself to whatever public headers come with it. That's how it's done with all commonly used third party > packages. If can't develop that way then something is fundamentally wrong and should be corrected. Doing it that way, avoids the > exact problem you mention. So, I guess it comes down to how we enforce that kind of development style. Do you agree? > > Andy > > On Thu, 14 Jun 2018, Fabrizio Furano wrote: > >> Hi Andy, >> >> yes, that would be clean, and personally I would prefer to >> have libmacaroons in the stock repos. >> >> The annoyance I'd like to avoid is that so far DPM builds >> libmacaroons inside the source tree as an internal dep, >> exactly as you describe, for using it inside Apache. It's there >> since one year and a half... >> >> It would be quite annoying to have two versions of the same lib >> built inside the source tree of two different frontends: mod_lcgdm_dav >> and xrootd >> >> This is why I'd prefer to investigate if we can organize an "official" >> publishing of libmacaroons, at least to epel and debian. This thing >> has already been done in the past, and would represent to me the >> cleanest scenario possible, or at least it would give the possibility >> of having a clean situation. >> >> ... thoughts? >> >> Cheers >> Fabrizio >> >> >> On 14.06.18 09:55, Andrew Hanushevsky wrote: >>> In general, we do not add components to the main repo that depend on >>> third party libraries that are not available in a stock system. The >>> reasons for this should be obvious. The Macroon component is only one of >>> several components that people are developing with third party >>> dependencies. So we know we need to solve this problem. Our current >>> thinking is to setup additional projects in the main xroot repo to host >>> such developments and include them into our standard build pipeline. >>> Doing this should solve the dependency issue as well as making it >>> trivial to assign proper ownership so we aren't in the loop in terms of >>> updates and whatever. However, we would package them and sites could >>> install whatever they want via rpm. That way, those who wish to use a >>> feature know ahead of time that there will be additional libraries that >>> will be installed that we have no control over and may conflict with >>> whatever they already have installed. >>> >>> This seems the like the cleanest approach to this issue and avoids >>> leaving dead code in the main repo as these components come into and >>> fall out of favor. Michal and I have to work on this as we don't it in >>> place but should have a workable solution relatively soon. >>> >>> Andy >>> >>> On Wed, 13 Jun 2018, Fabrizio Furano wrote: >>> >>>> Hi Brian, >>>> >>>> true, macaroons are implemented by DPM and dcache. Having them also in >>>> xrootd >>>> would be great. >>>> >>>> About the distribution, my personal preference is to have these things >>>> handy to >>>> install and clean on the source tree, so I'd love to see macaroons and >>>> scitokens >>>> in the Xrd codebase, respecting all the strict project rules about >>>> makefiles (e.g. turning off the component) and deps. >>>> >>>> What did you do for libmacaroons? Since it's not distributed, we build >>>> it with DPM so far. >>>> Are you doing the same? >>>> Maybe we want to find a volunteer packager that submits it to >>>> epel/debian, etc. >>>> and only then refine the details of a build with Xrd ? >>>> >>>> Fabrizio >>>> >>>> >>>> On 06/13/2018 04:08 PM, Brian Bockelman wrote: >>>>> Hi, >>>>> >>>>> I have an initial (but functioning) support for Macaroons in Xrootd: >>>>> >>>>> https://github.com/bbockelm/xrootd-macaroons >>>>> >>>>> Macaroons are a symmetric-key-based token format. It allows an >>>>> entity with access to the secret key generate a bearer token >>>>> embedding one more "caveats" (rules for usage, such as restrictions >>>>> on path, expiration time, or valid operations). The bearer >>>>> of the token can add additional caveats (reducing permission) but >>>>> doesn't have the ability to remove them. >>>>> >>>>> Anyone else with the secret key (such as the same host or another >>>>> host in the cluster) can then verify the token as apply the >>>>> authorization rules in the caveats. >>>>> >>>>> Why is this useful? A few cases: >>>>> >>>>> 1. Delegating fine-grained access: Rucio generates a token to access >>>>> a single file at a site (using its X509 client cert to >>>>> authenticate), then sends the token to an end-user. Hence, the >>>>> end-user doesn't need to authenticate with the site (no user >>>>> X509 necessary) in order to download files. The VO (ATLAS, via >>>>> Rucio) can manage the fine-grained rights they delegate to the >>>>> user -- even if the storage is run by the site. >>>>> >>>>> 2. Enabling third-party-copy: If client C wants server A to download >>>>> from server B, the client can contact server B for a >>>>> fine-grained access token and provide it to server A as part of the >>>>> HTTP COPY request. >>>>> - I suspect I'll get the first FTS3-based transfers working today. >>>>> >>>>> IIUC, Macaroons are implemented by dCache and DPM. I utilized the >>>>> same caveat formats and API for requesting new tokens, but >>>>> generally one would not expect tokens to be reusable across different >>>>> implementations or site installations. >>>>> >>>>> Currently, the token is generated via a XrdHttp plugin but should be >>>>> usable throughout the authorization framework; however, you >>>>> need an encrypted channel between client and server to use it safely. >>>>> >>>>> The obvious question is "how does this differ from SciTokens"? >>>>> - SciTokens are based on asymmetric (public key) cryptography whereas >>>>> Macaroons are symmetric key. This means anyone can verify >>>>> a SciToken but only the issuing entity can verify a Macaroon. >>>>> - In the SciTokens model, the VO issues the authorization, which may >>>>> be valid across many sites. The macaroon is specific to >>>>> the site and requires an interaction with the site to generate. >>>>> - Macaroons can be attenuated (made weaker) by the end-user. I can >>>>> take a powerful macaroon and add additional limitations >>>>> without contacting a third-party, then safely give the limited >>>>> macaroon to another person if I desire. SciTokens can only be >>>>> attenuated by contacting the VO to exchange the powerful one for a >>>>> new one (hence, the VO always generates the token). >>>>> >>>>> Just as one uses a mix of asymmetric and symmetric cryptography >>>>> throughout the course of the day, I see SciTokens and Macaroons >>>>> as complementary approaches, each enabling some distinct and some >>>>> common use cases. >>>>> >>>>> So - one natural question for me is whether this better lives inside >>>>> the xrootd repository or standalone. Unlike SciTokens, the >>>>> dependency stack for Macaroons is more manageable (a direct >>>>> dependency on libmacaroons - https://github.com/rescrv/libmacaroons >>>>> - and an indirect dependency on libbsd). This means upstreaming of >>>>> the macaroons code is more approachable than the SciTokens >>>>> code. Additionally, since the code includes both an issuer and a >>>>> verifier, Macaroons are more immediately usable - no >>>>> non-Xrootd services setup. >>>>> >>>>> Accordingly, I'm leaning to converting this into a module inside the >>>>> xrootd repository. >>>>> >>>>> Thoughts? >>>>> >>>>> Brian >>>>> >>>>> -------------------------------------------------------------------------------------------------------------------------------- >>>>> >>>>> >>>>> >>>>> Use REPLY-ALL to reply to list >>>>> >>>>> To unsubscribe from the XROOTD-DEV list, click the following link: >>>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1 >>>>> >>>> >>>> >>>> ######################################################################## >>>> Use REPLY-ALL to reply to list >>>> >>>> To unsubscribe from the XROOTD-DEV list, click the following link: >>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1 >>>> >>> >>> ######################################################################## >>> Use REPLY-ALL to reply to list >>> >>> To unsubscribe from the XROOTD-DEV list, click the following link: >>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1 >> >> >> ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-DEV list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1