Hi Andy,
On Fri, Feb 18, 2005 at 02:32:15AM -0800, Andrew Hanushevsky wrote:
> All of this points out to a packaging problem we have. The only way we
> really test releases is to create what we call a development release. That,
> unfortunately, makes it available to everyone else -- even before we can
> certify it as being materially correct. I do know we've had some
> development releases that should have never seen the light of day, but
> unfortuantely the process lets them out. We are trying to get a new
> process in to place that will *never* cut a release unless we know that it
> will actually work on a reasonablly sized system.
Releasing something as a "development" release for testing is independent
from demonstrating that there are no bugs which prevent it from working
on some canonical testbed system. (And this has nothing to do with
packaging, either.)
The thing we can add here is obtain and use the requested small testbed at
SLAC plus define some set of standard functionality tests which are done
on the development releases. The fact that we do or don't do such tests
shouldn't however prevent the release of clearly labeled "development"
versions to get feedback in the open-source sense. This will in any case
be necessary as the testbed system by definition cannot test all of the
configurations people might use out in the world.
To summarize, what I would like to see happen is:
1) we develop a set of basic (and relatively automated) system
functionality tests
2) development releases are made as they are now
3) when each development release comes out, we use the system
functionality tests on the testbed at SLAC (and perhaps post the
results as a followup to the release announcement)
This implies that #3 is easy to do, of course. In the open source style,
people can wait for #3 to happen or not before trying to use it.
How does that sound?
Pete
-------------------------------------------------------------------------
Peter Elmer E-mail: [log in to unmask] Phone: +41 (22) 767-4644
Address: CERN Division PPE, Bat. 32 2C-14, CH-1211 Geneva 23, Switzerland
-------------------------------------------------------------------------
