Hi Frossie,
I think the qserv monthlies are closer in spirit to the whole-stack
bi-annuals than they are to the whole-stack weeklies? These are in some
sense "external" releases for the qserv-only audience (such as it is),
and we would like to have them published on the eups distrib server with
easy-to-tell-someone names (i.e. not containing git SHAs).
Operationally, these are the checkpoints at which we would have new
folks interested in exploring qserv get on the bus. They are also the
points at which we bootstrap new devs, in terms of all our package
dependencies (the qserv team as yet uses the eups-centric dev model, and
not the lsstsw one, generally.) And this is the stride with which we
currently update our externally visible published documentation,
summarize our git log into a more user-centric changelog, etc. This has
worked well for us because it matches our sprint schedule as well.
I don't think we (qserv) currently have anything like the whole-stack
weeklies -- the monthlies provide a rapid-enough stride for users, and
devs operate with latest-monthly base + manually upgraded
distrib-published dependency packages (when necessary) on top, plus
latest qserv sources on top. Devs generally resync their stack for
dependencies after each monthly release. Since we have so few
dependencies, we usually have only one or two, if any, dependent
packages that ever need to be manually updated between the monthly
releases (e.g. xrootd, db, log, or boost).
Hope that provides some insight into how we view/use/do these.
Suggestions for how to improve/streamline are of course always welcomed!
thanks,
--FritzM.
On 11/04/2015 10:07 PM, Becla, Jacek wrote:
> Frossie,
>
> We typically care about two most recent monthly releases, no reason
> to keep very old ones, if we could keep the last few that’d be enough.
>
> —
>
> I believe we all kind of like real tags instead of master+g{ref} for the few
> packages we care most about, like qserv, qserv_testdata, xrootd, db, log,
> dax_*. Could we possibly tag just these?
>
> ---
>
> Yes, we do switch between releases using eups, like:
>
> eups distrib install —tag 2015_09 qserv_distrib
>
> I think that is important to us!
>
> ---
>
> Jenkins testing does not help us for final validation because we want
> to run integration tests, compilation is not enough. Hopefully that will
> get better with container-based technology.
>
> —
>
> Same tags for both (dax_ and qserv_) would be fine.
>
> Thanks
> Jacek
>
>
>
>> On Nov 4, 2015, at 8:57 PM, Frossie Economou <[log in to unmask]> wrote:
>>
>> Okay so for those who care about the release process:
>>
>> These are the points where the documented QServ process diverges with where I am going with the stack release process for engineering releases (dailies/weeklies/monthlies). Now this does NOT mean the QServ process has to change, but obviously the more similar they are the easier it is, and the more serendipitous improvements will happen. So I’m listing them here so that you folks can tell me what is what for a reason, or for no particular signing.
>>
>> If someone can get me an answer to #4 I can do the release « new style » otherwise I’ll just follow the script as is for this one.
>>
>> 1- Cryptographic signing. Jacek covered that on hitchat, it doesn’t seem like a hard requirement. My process uses the Github API to make the tags, so at least there is some protection against spoofing. But since there’s no real intent for auth, looks like I can that this the same as the stack weeklies etc.
>>
>> 2- The qserv-install.sh - whatever this is meant to catch, we should roll it in as a Jenkins test or something - by the time a release is getting tagged, we should have faith it is a good one.
>>
>> 3- This is the big one. So for the weeklies I follow a different process from the « public » 6-monthly releases. The public releases do get a build from the tag ref and so do end up with an eups package version like v11.0. For the weeklies I don’t do that; I apply *git* tags but the eups package version remains master+g{ref} or whatever. There are two reasons for this: (a) it leaves me the freedom to delete a git tag to avoid excess tag noise on the repos (e.g. who cares about the weekly from a year ago?) and (b) it is part of weaning the release process away from eups. Now the git tag exists, so one can ALWAYS submit this as a ref to the CI system and get a build reproducing that particular release; you just can’t do it solely from within eups. So if you guys really do switch between monthlies using eups you should tell me to treat these as official releases. If you’re happy as long as you have a gitref tagged qserv_2015_10 because you can always build from that, I can treat your monthlies like my weeklies, which simplifies things.
>>
>> 4- Git tags applied: I have a heuristic based on Github team names - e.g., repos belonging to Data Management team in the LSST org get the stack tags. That way, if someone adds a repo I don’t have to care - the tagger will magically find it. So right now there is a Database team that has both qserv and dax_ repos in it. The question is, do you want the same git tag for both (in which case do you want qserv.2015.10 or db.2015.20) or do you want different tags for dax, in which case we should make a new team and stuff the dax repos in it?
>>
>> 5- Docs. So you guys have a sphinx site. Fab, that’ll make it super easy to drop it into the new doc system that hosts sphinx on readthedocs - you can see a small example here http://sqr-001.lsst.codes/en/master/ - the idea is to continuously deploy the docs so the release step should be very lightweight and doable via API calls. (My holy grail is to do a release with no local checkout involved).
>>
>> --
>> Frossie Economou
>> Science Quality and Reliability Engineering (SQuaRE)
>> Large Synoptic Survey Telescope
>>
>> ########################################################################
>> Use REPLY-ALL to reply to list
>>
>> To unsubscribe from the QSERV-L list, click the following link:
>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
>
> ########################################################################
> Use REPLY-ALL to reply to list
>
> To unsubscribe from the QSERV-L list, click the following link:
> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
