 |
 |
Yup, little problem, I am not a packager :-D so we have to find one. On 14.06.18 14:15, Andrew Hanushevsky wrote: > Hi Fabrizio, > > I totally agree. We will not enbed libmacroons source into the xroot > repo simply to avoid this kind of mess (for us and everyone else :-). > So, I urge that you or whomever you lean on, repackge libmacaroons into > an EPEL-style package that we can use without ever looking at the source. > > Andy > > > On Thu, 14 Jun 2018, Fabrizio Furano wrote: > >> Hi Andy, >> >> yes, development style is one thing, and so far the xrootd project >> has been very clear on that. >> >> In this case I had DPM in mind, and its WebDAV frontend >> already embedded the source of libmacaroons quite some time ago. >> Now we are talking about embedding the code of libmacaroons also in >> xrootd. >> >> The result would be to have potentially two different versions of >> the same code in the same DPM system, coming from two different >> frontends. >> >> It will work (I believe), yet the clean solution IMO is to properly >> package libmacaroons into EPEL+debian, so that noone needs to embed >> the code anymore. >> >> f >> >> >> On 06/14/2018 12:01 PM, Andrew Hanushevsky wrote: >>> Hi Fabrizio, >>> >>> I totally agree that having two source builds are a pain and should >>> be avoided. I'm almost tempted to say should be prohibited. >>> In the xroot case, it should be possible to develop a plugin by >>> installing the rpms you are dependent on and restricting >>> yourself to whatever public headers come with it. That's how it's >>> done with all commonly used third party >>> packages. If can't develop that way then something is fundamentally >>> wrong and should be corrected. Doing it that way, avoids the >>> exact problem you mention. So, I guess it comes down to how we >>> enforce that kind of development style. Do you agree? >>> >>> Andy >>> >>> On Thu, 14 Jun 2018, Fabrizio Furano wrote: >>> >>>> Hi Andy, >>>> >>>> yes, that would be clean, and personally I would prefer to >>>> have libmacaroons in the stock repos. >>>> >>>> The annoyance I'd like to avoid is that so far DPM builds >>>> libmacaroons inside the source tree as an internal dep, >>>> exactly as you describe, for using it inside Apache. It's there >>>> since one year and a half... >>>> >>>> It would be quite annoying to have two versions of the same lib >>>> built inside the source tree of two different frontends: mod_lcgdm_dav >>>> and xrootd >>>> >>>> This is why I'd prefer to investigate if we can organize an "official" >>>> publishing of libmacaroons, at least to epel and debian. This thing >>>> has already been done in the past, and would represent to me the >>>> cleanest scenario possible, or at least it would give the possibility >>>> of having a clean situation. >>>> >>>> ... thoughts? >>>> >>>> Cheers >>>> Fabrizio >>>> >>>> >>>> On 14.06.18 09:55, Andrew Hanushevsky wrote: >>>>> In general, we do not add components to the main repo that depend on >>>>> third party libraries that are not available in a stock system. The >>>>> reasons for this should be obvious. The Macroon component is only >>>>> one of >>>>> several components that people are developing with third party >>>>> dependencies. So we know we need to solve this problem. Our current >>>>> thinking is to setup additional projects in the main xroot repo to >>>>> host >>>>> such developments and include them into our standard build pipeline. >>>>> Doing this should solve the dependency issue as well as making it >>>>> trivial to assign proper ownership so we aren't in the loop in >>>>> terms of >>>>> updates and whatever. However, we would package them and sites could >>>>> install whatever they want via rpm. That way, those who wish to use a >>>>> feature know ahead of time that there will be additional libraries >>>>> that >>>>> will be installed that we have no control over and may conflict with >>>>> whatever they already have installed. >>>>> >>>>> This seems the like the cleanest approach to this issue and avoids >>>>> leaving dead code in the main repo as these components come into and >>>>> fall out of favor. Michal and I have to work on this as we don't it in >>>>> place but should have a workable solution relatively soon. >>>>> >>>>> Andy >>>>> >>>>> On Wed, 13 Jun 2018, Fabrizio Furano wrote: >>>>> >>>>>> Hi Brian, >>>>>> >>>>>> true, macaroons are implemented by DPM and dcache. Having them >>>>>> also in >>>>>> xrootd >>>>>> would be great. >>>>>> >>>>>> About the distribution, my personal preference is to have these >>>>>> things >>>>>> handy to >>>>>> install and clean on the source tree, so I'd love to see macaroons >>>>>> and >>>>>> scitokens >>>>>> in the Xrd codebase, respecting all the strict project rules about >>>>>> makefiles (e.g. turning off the component) and deps. >>>>>> >>>>>> What did you do for libmacaroons? Since it's not distributed, we >>>>>> build >>>>>> it with DPM so far. >>>>>> Are you doing the same? >>>>>> Maybe we want to find a volunteer packager that submits it to >>>>>> epel/debian, etc. >>>>>> and only then refine the details of a build with Xrd ? >>>>>> >>>>>> Fabrizio >>>>>> >>>>>> >>>>>> On 06/13/2018 04:08 PM, Brian Bockelman wrote: >>>>>>> Hi, >>>>>>> >>>>>>> I have an initial (but functioning) support for Macaroons in Xrootd: >>>>>>> >>>>>>> https://github.com/bbockelm/xrootd-macaroons >>>>>>> >>>>>>> Macaroons are a symmetric-key-based token format. It allows an >>>>>>> entity with access to the secret key generate a bearer token >>>>>>> embedding one more "caveats" (rules for usage, such as restrictions >>>>>>> on path, expiration time, or valid operations). The bearer >>>>>>> of the token can add additional caveats (reducing permission) but >>>>>>> doesn't have the ability to remove them. >>>>>>> >>>>>>> Anyone else with the secret key (such as the same host or another >>>>>>> host in the cluster) can then verify the token as apply the >>>>>>> authorization rules in the caveats. >>>>>>> >>>>>>> Why is this useful? A few cases: >>>>>>> >>>>>>> 1. Delegating fine-grained access: Rucio generates a token to >>>>>>> access >>>>>>> a single file at a site (using its X509 client cert to >>>>>>> authenticate), then sends the token to an end-user. Hence, the >>>>>>> end-user doesn't need to authenticate with the site (no user >>>>>>> X509 necessary) in order to download files. The VO (ATLAS, via >>>>>>> Rucio) can manage the fine-grained rights they delegate to the >>>>>>> user -- even if the storage is run by the site. >>>>>>> >>>>>>> 2. Enabling third-party-copy: If client C wants server A to >>>>>>> download >>>>>>> from server B, the client can contact server B for a >>>>>>> fine-grained access token and provide it to server A as part of the >>>>>>> HTTP COPY request. >>>>>>> - I suspect I'll get the first FTS3-based transfers working today. >>>>>>> >>>>>>> IIUC, Macaroons are implemented by dCache and DPM. I utilized the >>>>>>> same caveat formats and API for requesting new tokens, but >>>>>>> generally one would not expect tokens to be reusable across >>>>>>> different >>>>>>> implementations or site installations. >>>>>>> >>>>>>> Currently, the token is generated via a XrdHttp plugin but should be >>>>>>> usable throughout the authorization framework; however, you >>>>>>> need an encrypted channel between client and server to use it >>>>>>> safely. >>>>>>> >>>>>>> The obvious question is "how does this differ from SciTokens"? >>>>>>> - SciTokens are based on asymmetric (public key) cryptography >>>>>>> whereas >>>>>>> Macaroons are symmetric key. This means anyone can verify >>>>>>> a SciToken but only the issuing entity can verify a Macaroon. >>>>>>> - In the SciTokens model, the VO issues the authorization, which may >>>>>>> be valid across many sites. The macaroon is specific to >>>>>>> the site and requires an interaction with the site to generate. >>>>>>> - Macaroons can be attenuated (made weaker) by the end-user. I can >>>>>>> take a powerful macaroon and add additional limitations >>>>>>> without contacting a third-party, then safely give the limited >>>>>>> macaroon to another person if I desire. SciTokens can only be >>>>>>> attenuated by contacting the VO to exchange the powerful one for a >>>>>>> new one (hence, the VO always generates the token). >>>>>>> >>>>>>> Just as one uses a mix of asymmetric and symmetric cryptography >>>>>>> throughout the course of the day, I see SciTokens and Macaroons >>>>>>> as complementary approaches, each enabling some distinct and some >>>>>>> common use cases. >>>>>>> >>>>>>> So - one natural question for me is whether this better lives inside >>>>>>> the xrootd repository or standalone. Unlike SciTokens, the >>>>>>> dependency stack for Macaroons is more manageable (a direct >>>>>>> dependency on libmacaroons - https://github.com/rescrv/libmacaroons >>>>>>> - and an indirect dependency on libbsd). This means upstreaming of >>>>>>> the macaroons code is more approachable than the SciTokens >>>>>>> code. Additionally, since the code includes both an issuer and a >>>>>>> verifier, Macaroons are more immediately usable - no >>>>>>> non-Xrootd services setup. >>>>>>> >>>>>>> Accordingly, I'm leaning to converting this into a module inside the >>>>>>> xrootd repository. >>>>>>> >>>>>>> Thoughts? >>>>>>> >>>>>>> Brian >>>>>>> >>>>>>> -------------------------------------------------------------------------------------------------------------------------------- >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> Use REPLY-ALL to reply to list >>>>>>> >>>>>>> To unsubscribe from the XROOTD-DEV list, click the following link: >>>>>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1 >>>>>>> >>>>>> >>>>>> >>>>>> ######################################################################## >>>>>> >>>>>> Use REPLY-ALL to reply to list >>>>>> >>>>>> To unsubscribe from the XROOTD-DEV list, click the following link: >>>>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1 >>>>>> >>>>> >>>>> ######################################################################## >>>>> >>>>> Use REPLY-ALL to reply to list >>>>> >>>>> To unsubscribe from the XROOTD-DEV list, click the following link: >>>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1 >>>> >>>> >>>> >> >> > > ######################################################################## > Use REPLY-ALL to reply to list > > To unsubscribe from the XROOTD-DEV list, click the following link: > https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1 ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-DEV list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1