Hi,
slightly OT, may I know what are the actual desiderata for
a macaroon-based system ?
As I was mentioning, DPM has the macaroons code in the
frontend (disabled by default), yet we consider it experimental.
Which authentication/authorization criteria may have to
be applied to generate a macaroon? What does dCache do for example?
Thanks
Fabrizio
On 17.06.18 06:01, Brian Bockelman wrote:
> Hi,
>
> The package has moved to testing:
> - Fedora 28: https://bodhi.fedoraproject.org/updates/FEDORA-2018-9e2640da1e
> - EPEL
> 7: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2018-3b68fe0e41
>
> If folks could please take some time to review and test it, it'll help
> reduce the 2 week waiting period.
>
> Regardless, on the timescale of the next Xrootd release, libmacaroons
> should be present on at least a subset of the production platforms.
>
> Brian
>
>> On Jun 14, 2018, at 9:23 PM, Brian Bockelman <[log in to unmask]
>> <mailto:[log in to unmask]>> wrote:
>>
>> Hi,
>>
>> I posted the proposed sources here:
>>
>> https://bugzilla.redhat.com/show_bug.cgi?id=1591556
>>
>> If there's anyone who can pickup the review and push it forward, it
>> would be greatly appreciated.
>>
>> Brian
>>
>>> On Jun 14, 2018, at 10:15 AM, Andrew Hanushevsky
>>> <[log in to unmask] <mailto:[log in to unmask]>> wrote:
>>>
>>> Yes, I think that Michal is blessed as of now (as long as he doesn't
>>> make any mistakes :-). I suppose another reason to host it in the
>>> xroot github repo under, say, xrootd-macaroons.
>>>
>>> Andy
>>>
>>> On Thu, 14 Jun 2018, Fabrizio Furano wrote:
>>>
>>>> Are you a blessed EPEL packager ?
>>>> You need one to bless your package. Should not be difficult,as it's a
>>>> plain library.
>>>>
>>>> I think Michal may have this kind of power.
>>>>
>>>> About Debian I don't know personally. So far we got the key components
>>>> into it by courtesy (someone told me it was Mattias). I'd say that EPEL
>>>> comes first.
>>>>
>>>> f
>>>>
>>>> On 14.06.18 16:57, Brian Bockelman wrote:
>>>>> Hi,
>>>>>
>>>>> I can probably take care of this for EPEL, but I really have no
>>>>> background in Debian.
>>>>>
>>>>> Assuming we could get at least that far, could we get things into
>>>>> the main repo?
>>>>>
>>>>> Andy, your mention of "stock repos" was a bit ambiguous. Do you
>>>>> consider EPEL a stock repo for RHEL-like platforms?
>>>>>
>>>>> Brian
>>>>>
>>>>>> On Jun 14, 2018, at 7:15 AM, Andrew Hanushevsky
>>>>>> <[log in to unmask] <mailto:[log in to unmask]>> wrote:
>>>>>>
>>>>>> Hi Fabrizio,
>>>>>>
>>>>>> I totally agree. We will not enbed libmacroons source into the
>>>>>> xroot repo simply to avoid this kind of mess (for us and everyone
>>>>>> else :-). So, I urge that you or whomever you lean on, repackge
>>>>>> libmacaroons into an EPEL-style package that we can use without
>>>>>> ever looking at the source.
>>>>>>
>>>>>> Andy
>>>>>>
>>>>>>
>>>>>> On Thu, 14 Jun 2018, Fabrizio Furano wrote:
>>>>>>
>>>>>>> Hi Andy,
>>>>>>>
>>>>>>> yes, development style is one thing, and so far the xrootd project
>>>>>>> has been very clear on that.
>>>>>>>
>>>>>>> In this case I had DPM in mind, and its WebDAV frontend
>>>>>>> already embedded the source of libmacaroons quite some time ago.
>>>>>>> Now we are talking about embedding the code of libmacaroons also
>>>>>>> in xrootd.
>>>>>>>
>>>>>>> The result would be to have potentially two different versions of
>>>>>>> the same code in the same DPM system, coming from two different
>>>>>>> frontends.
>>>>>>>
>>>>>>> It will work (I believe), yet the clean solution IMO is to properly
>>>>>>> package libmacaroons into EPEL+debian, so that noone needs to
>>>>>>> embed the code anymore.
>>>>>>>
>>>>>>> f
>>>>>>>
>>>>>>>
>>>>>>> On 06/14/2018 12:01 PM, Andrew Hanushevsky wrote:
>>>>>>>> Hi Fabrizio,
>>>>>>>>
>>>>>>>> I totally agree that having two source builds are a pain and
>>>>>>>> should be avoided. I'm almost tempted to say should be prohibited.
>>>>>>>> In the xroot case, it should be possible to develop a plugin by
>>>>>>>> installing the rpms you are dependent on and restricting
>>>>>>>> yourself to whatever public headers come with it. That's how
>>>>>>>> it's done with all commonly used third party
>>>>>>>> packages. If can't develop that way then something is
>>>>>>>> fundamentally wrong and should be corrected. Doing it that way,
>>>>>>>> avoids the
>>>>>>>> exact problem you mention. So, I guess it comes down to how we
>>>>>>>> enforce that kind of development style. Do you agree?
>>>>>>>>
>>>>>>>> Andy
>>>>>>>>
>>>>>>>> On Thu, 14 Jun 2018, Fabrizio Furano wrote:
>>>>>>>>
>>>>>>>>> Hi Andy,
>>>>>>>>>
>>>>>>>>> yes, that would be clean, and personally I would prefer to
>>>>>>>>> have libmacaroons in the stock repos.
>>>>>>>>>
>>>>>>>>> The annoyance I'd like to avoid is that so far DPM builds
>>>>>>>>> libmacaroons inside the source tree as an internal dep,
>>>>>>>>> exactly as you describe, for using it inside Apache. It's there
>>>>>>>>> since one year and a half...
>>>>>>>>>
>>>>>>>>> It would be quite annoying to have two versions of the same lib
>>>>>>>>> built inside the source tree of two different frontends:
>>>>>>>>> mod_lcgdm_dav
>>>>>>>>> and xrootd
>>>>>>>>>
>>>>>>>>> This is why I'd prefer to investigate if we can organize an
>>>>>>>>> "official"
>>>>>>>>> publishing of libmacaroons, at least to epel and debian. This thing
>>>>>>>>> has already been done in the past, and would represent to me the
>>>>>>>>> cleanest scenario possible, or at least it would give the
>>>>>>>>> possibility
>>>>>>>>> of having a clean situation.
>>>>>>>>>
>>>>>>>>> ... thoughts?
>>>>>>>>>
>>>>>>>>> Cheers
>>>>>>>>> Fabrizio
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 14.06.18 09:55, Andrew Hanushevsky wrote:
>>>>>>>>>> In general, we do not add components to the main repo that
>>>>>>>>>> depend on
>>>>>>>>>> third party libraries that are not available in a stock
>>>>>>>>>> system. The
>>>>>>>>>> reasons for this should be obvious. The Macroon component is
>>>>>>>>>> only one of
>>>>>>>>>> several components that people are developing with third party
>>>>>>>>>> dependencies. So we know we need to solve this problem. Our
>>>>>>>>>> current
>>>>>>>>>> thinking is to setup additional projects in the main xroot
>>>>>>>>>> repo to host
>>>>>>>>>> such developments and include them into our standard build
>>>>>>>>>> pipeline.
>>>>>>>>>> Doing this should solve the dependency issue as well as making it
>>>>>>>>>> trivial to assign proper ownership so we aren't in the loop in
>>>>>>>>>> terms of
>>>>>>>>>> updates and whatever. However, we would package them and sites
>>>>>>>>>> could
>>>>>>>>>> install whatever they want via rpm. That way, those who wish
>>>>>>>>>> to use a
>>>>>>>>>> feature know ahead of time that there will be additional
>>>>>>>>>> libraries that
>>>>>>>>>> will be installed that we have no control over and may
>>>>>>>>>> conflict with
>>>>>>>>>> whatever they already have installed.
>>>>>>>>>>
>>>>>>>>>> This seems the like the cleanest approach to this issue and avoids
>>>>>>>>>> leaving dead code in the main repo as these components come
>>>>>>>>>> into and
>>>>>>>>>> fall out of favor. Michal and I have to work on this as we
>>>>>>>>>> don't it in
>>>>>>>>>> place but should have a workable solution relatively soon.
>>>>>>>>>>
>>>>>>>>>> Andy
>>>>>>>>>>
>>>>>>>>>> On Wed, 13 Jun 2018, Fabrizio Furano wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Brian,
>>>>>>>>>>>
>>>>>>>>>>> true, macaroons are implemented by DPM and dcache. Having
>>>>>>>>>>> them also in
>>>>>>>>>>> xrootd
>>>>>>>>>>> would be great.
>>>>>>>>>>>
>>>>>>>>>>> About the distribution, my personal preference is to have
>>>>>>>>>>> these things
>>>>>>>>>>> handy to
>>>>>>>>>>> install and clean on the source tree, so I'd love to see
>>>>>>>>>>> macaroons and
>>>>>>>>>>> scitokens
>>>>>>>>>>> in the Xrd codebase, respecting all the strict project rules
>>>>>>>>>>> about
>>>>>>>>>>> makefiles (e.g. turning off the component) and deps.
>>>>>>>>>>>
>>>>>>>>>>> What did you do for libmacaroons? Since it's not distributed,
>>>>>>>>>>> we build
>>>>>>>>>>> it with DPM so far.
>>>>>>>>>>> Are you doing the same?
>>>>>>>>>>> Maybe we want to find a volunteer packager that submits it to
>>>>>>>>>>> epel/debian, etc.
>>>>>>>>>>> and only then refine the details of a build with Xrd ?
>>>>>>>>>>>
>>>>>>>>>>> Fabrizio
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 06/13/2018 04:08 PM, Brian Bockelman wrote:
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> I have an initial (but functioning) support for Macaroons in
>>>>>>>>>>>> Xrootd:
>>>>>>>>>>>>
>>>>>>>>>>>> https://github.com/bbockelm/xrootd-macaroons
>>>>>>>>>>>>
>>>>>>>>>>>> Macaroons are a symmetric-key-based token format. It allows an
>>>>>>>>>>>> entity with access to the secret key generate a bearer token
>>>>>>>>>>>> embedding one more "caveats" (rules for usage, such as
>>>>>>>>>>>> restrictions
>>>>>>>>>>>> on path, expiration time, or valid operations). The bearer
>>>>>>>>>>>> of the token can add additional caveats (reducing
>>>>>>>>>>>> permission) but
>>>>>>>>>>>> doesn't have the ability to remove them.
>>>>>>>>>>>>
>>>>>>>>>>>> Anyone else with the secret key (such as the same host or
>>>>>>>>>>>> another
>>>>>>>>>>>> host in the cluster) can then verify the token as apply the
>>>>>>>>>>>> authorization rules in the caveats.
>>>>>>>>>>>>
>>>>>>>>>>>> Why is this useful? A few cases:
>>>>>>>>>>>>
>>>>>>>>>>>> 1. Delegating fine-grained access: Rucio generates a token
>>>>>>>>>>>> to access
>>>>>>>>>>>> a single file at a site (using its X509 client cert to
>>>>>>>>>>>> authenticate), then sends the token to an end-user. Hence, the
>>>>>>>>>>>> end-user doesn't need to authenticate with the site (no user
>>>>>>>>>>>> X509 necessary) in order to download files. The VO (ATLAS, via
>>>>>>>>>>>> Rucio) can manage the fine-grained rights they delegate to the
>>>>>>>>>>>> user -- even if the storage is run by the site.
>>>>>>>>>>>>
>>>>>>>>>>>> 2. Enabling third-party-copy: If client C wants server A to
>>>>>>>>>>>> download
>>>>>>>>>>>> from server B, the client can contact server B for a
>>>>>>>>>>>> fine-grained access token and provide it to server A as part
>>>>>>>>>>>> of the
>>>>>>>>>>>> HTTP COPY request.
>>>>>>>>>>>> - I suspect I'll get the first FTS3-based transfers working
>>>>>>>>>>>> today.
>>>>>>>>>>>>
>>>>>>>>>>>> IIUC, Macaroons are implemented by dCache and DPM. I
>>>>>>>>>>>> utilized the
>>>>>>>>>>>> same caveat formats and API for requesting new tokens, but
>>>>>>>>>>>> generally one would not expect tokens to be reusable across
>>>>>>>>>>>> different
>>>>>>>>>>>> implementations or site installations.
>>>>>>>>>>>>
>>>>>>>>>>>> Currently, the token is generated via a XrdHttp plugin but
>>>>>>>>>>>> should be
>>>>>>>>>>>> usable throughout the authorization framework; however, you
>>>>>>>>>>>> need an encrypted channel between client and server to use
>>>>>>>>>>>> it safely.
>>>>>>>>>>>>
>>>>>>>>>>>> The obvious question is "how does this differ from SciTokens"?
>>>>>>>>>>>> - SciTokens are based on asymmetric (public key)
>>>>>>>>>>>> cryptography whereas
>>>>>>>>>>>> Macaroons are symmetric key. This means anyone can verify
>>>>>>>>>>>> a SciToken but only the issuing entity can verify a Macaroon.
>>>>>>>>>>>> - In the SciTokens model, the VO issues the authorization,
>>>>>>>>>>>> which may
>>>>>>>>>>>> be valid across many sites. The macaroon is specific to
>>>>>>>>>>>> the site and requires an interaction with the site to generate.
>>>>>>>>>>>> - Macaroons can be attenuated (made weaker) by the end-user.
>>>>>>>>>>>> I can
>>>>>>>>>>>> take a powerful macaroon and add additional limitations
>>>>>>>>>>>> without contacting a third-party, then safely give the limited
>>>>>>>>>>>> macaroon to another person if I desire. SciTokens can only be
>>>>>>>>>>>> attenuated by contacting the VO to exchange the powerful one
>>>>>>>>>>>> for a
>>>>>>>>>>>> new one (hence, the VO always generates the token).
>>>>>>>>>>>>
>>>>>>>>>>>> Just as one uses a mix of asymmetric and symmetric cryptography
>>>>>>>>>>>> throughout the course of the day, I see SciTokens and Macaroons
>>>>>>>>>>>> as complementary approaches, each enabling some distinct and
>>>>>>>>>>>> some
>>>>>>>>>>>> common use cases.
>>>>>>>>>>>>
>>>>>>>>>>>> So - one natural question for me is whether this better
>>>>>>>>>>>> lives inside
>>>>>>>>>>>> the xrootd repository or standalone. Unlike SciTokens, the
>>>>>>>>>>>> dependency stack for Macaroons is more manageable (a direct
>>>>>>>>>>>> dependency on libmacaroons -
>>>>>>>>>>>> https://github.com/rescrv/libmacaroons
>>>>>>>>>>>> - and an indirect dependency on libbsd). This means
>>>>>>>>>>>> upstreaming of
>>>>>>>>>>>> the macaroons code is more approachable than the SciTokens
>>>>>>>>>>>> code. Additionally, since the code includes both an issuer
>>>>>>>>>>>> and a
>>>>>>>>>>>> verifier, Macaroons are more immediately usable - no
>>>>>>>>>>>> non-Xrootd services setup.
>>>>>>>>>>>>
>>>>>>>>>>>> Accordingly, I'm leaning to converting this into a module
>>>>>>>>>>>> inside the
>>>>>>>>>>>> xrootd repository.
>>>>>>>>>>>>
>>>>>>>>>>>> Thoughts?
>>>>>>>>>>>>
>>>>>>>>>>>> Brian
>>>>>>>>>>>>
>>>>>>>>>>>> --------------------------------------------------------------------------------------------------------------------------------
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Use REPLY-ALL to reply to list
>>>>>>>>>>>>
>>>>>>>>>>>> To unsubscribe from the XROOTD-DEV list, click the following
>>>>>>>>>>>> link:
>>>>>>>>>>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> ########################################################################
>>>>>>>>>>> Use REPLY-ALL to reply to list
>>>>>>>>>>>
>>>>>>>>>>> To unsubscribe from the XROOTD-DEV list, click the following
>>>>>>>>>>> link:
>>>>>>>>>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ########################################################################
>>>>>>>>>> Use REPLY-ALL to reply to list
>>>>>>>>>>
>>>>>>>>>> To unsubscribe from the XROOTD-DEV list, click the following link:
>>>>>>>>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>
>>>>
>>>>
>>>>
>>
>>
>> ------------------------------------------------------------------------
>>
>> Use REPLY-ALL to reply to list
>>
>> To unsubscribe from the XROOTD-DEV list, click the following link:
>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
>>
>
>
> ------------------------------------------------------------------------
>
> Use REPLY-ALL to reply to list
>
> To unsubscribe from the XROOTD-DEV list, click the following link:
> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
>
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
|
