[GRASS-dev] Re: testing in 6.5 before backporting to 6.4

The lack of manpower makes it even more important that all developers test any changes before committing--including backporting. I agree with Glynn that moving all dev work to 7 would simplify things. But it is also important that all developers only commit code that passes a WFM (works for me) validation test. This will catch errors and typos that are easy to fix by the coder at the time of submitting but which are very difficult to track down by someone else later. Only code that the developer can be sure works on his/her local machine should be committed. This kind of test is simple enough to do. If doing a WFM validation on code changes seems to take too much time, not doing it eats up much, much more of time in the rest of the dev and user community. Even with a WFM test, there will always be the possibility that a code change can cause unforeseen problems in other parts of the code base or may not work on another platform. These are problematic enough to identify and debug without having to also deal with errors that should have been caught by the developer with a minimum of checking. We simply don't have the manpower for the hours needed to debug simple errors that should have been caught at the time of committing. This should apply to the dev branch as well as the release branch for a couple of reasons. First, many people use the dev branch on a regular basis. Making a change that breaks a vital function by not testing before committing, impacts the work of many people. Yes there is a risk to using a dev version, but we also should follow practices that attempt to limit as much as possible introducing easily avoidable bugs into a program that is used by so many people globally. Secondly, if all code has at least passed a WFM test, then bugs that are encountered in a dev branch are easier to track down.

Along the same lines, I don't think that developers should be committing 'unfinished' code that knowingly does not work but is linked to an interface element. When a user encounters a button or pull-down that does nothing, does something incorrectly, or crashes a module it is very frustrating. The user does not know if it is a bug, a platform issue, or a problem with their own system. This starts a chain of emails and tests in the user and dev community to try and solve the issue. When this is done intentionally, it needlessly eats into the very limited developer resources. Sometimes there are good reasons why it is necessary to commit unfinished code; in such cases it should be well-commented as such (so that others can help complete it if necessary or at least avoid duplication) and there should be no associated interface element that could confuse and frustrate users. It is not enough to put in the manual that x function does not yet work. Just don't add the button for x function to the interface in the first place. Again, this is quite easy to do for GRASS development.

Michael
______________________________
C. Michael Barton
Director, Center for Social Dynamics & Complexity
Professor of Anthropology, School of Human Evolution & Social Change
Arizona State University
Tempe, AZ 85287-2402
USA

voice: 480-965-6262 (SHESC), 480-727-9746 (CSDC)
fax: 480-965-7671(SHESC), 480-727-0709 (CSDC)
www: http://csdc.asu.edu, http://shesc.asu.edu
    http://www.public.asu.edu/~cmbarton

On Jun 15, 2011, at 9:00 AM, grass-dev-request@lists.osgeo.org wrote:

Date: Wed, 15 Jun 2011 08:51:57 +0200
From: Markus Metz <markus.metz.giswork@googlemail.com>
Subject: Re: [GRASS-dev] Re: testing in 6.5 before backporting to 6.4
To: Martin Landa <landa.martin@gmail.com>
Cc: Helena Mitasova <hmitaso@ncsu.edu>, grass-dev
       <grass-dev@lists.osgeo.org>
Message-ID: <BANLkTin8zPoX8MiM6fGdeG0guvUOEdd5_Q@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1

On Tue, Jun 14, 2011 at 5:35 PM, Martin Landa <landa.martin@gmail.com> wrote:

Hi,

2011/6/14 Markus Metz <markus.metz.giswork@googlemail.com>:

A number of changes to 6.5 and 7.0 went in untested, which is IMHO not
a good idea, but being development branches, maybe sometimes
excusable. Any changes to the release branch should however be tested
before submitted. For most changes, the kind of test is pretty much
straightforward, and, if omitted and the change introduces a bug,
places a burden on users and other developers to figure out what went
wrong and why, wasting other people's time and leaving a not so good
impression of a releasebranch.

I agree, the real problem is the manpower which we have, or better to
say we don't have. Taking special care about three branches - trunk
(development), devbr6 (test-bed) and relbr64 (only tested commits) is
too much for us. With current manpower and our requirements, we could
end up with frozen relbr64 for years. Other issue is G7, when it could
be released? One year, two years? At the end the situation forces us
to keep relbr64 alive, not only bugfixes. One example is progressive
wxGUI. In the case we would just backport bugfixes the wxGUI would
remain three more years without improvements (counting 6.4.0 and
7.0.0). Almost impossible to maintain (two different code bases).
After that most of the users start to use 7.0 (since 6.5 shouldn't be
far away from 6.4 branch). Your point is very good, but unfortunately
the number of people actively contributing to the project (GRASS) is
so low that it is going to be unrealistic. Nice to be a solid rock,
also think about releasing policy. To have old-solid release is nice,
at the end the most of user starts to use dev versions. Probably that
could be a reason, why such solid stable releases haven't so many
reported bugs :wink:

I'm not against new features, I have myself introduced a number of new
features from 6.4.0 to 6.4.2. What I wanted to say is that I fully
agree with Helena and Hamish about the need for testing, that we may
need to think about a way to highly encourage testing before
backporting to 6.4, and that testing before submitting changes is IMHO
generally a good idea.

If you say that taking special care about three different branches is
too much for us, that means that changes really need to be tested for
7 as as well before submitting. As I said, in most cases testing can
be quickly done by the person introducing the changes, because that
person should know best what to test for.

Markus M

Hi all,

2011/6/17 Michael Barton <michael.barton@asu.edu>:

The lack of manpower makes it even more important that all developers test any changes before committing--including backporting.

I guess there is no one who is against testing. The subjective point
of view is "how much" the committed changes are tested. It depends on
the contributor, what he/she is committing (core libs, modules, GUI),
release life cycle (speaking about stable releases), etc. There is no
measurement, I believe everyone is going her/his best. Even someone
can have different impression. The important issue is to have working
test suites (modules - thanks Soeren for good work, GUI, etc).

Martin

--
Martin Landa <landa.martin gmail.com> * http://geo.fsv.cvut.cz/~landa

No disagreement from me on this.

MIchael
______________________________
C. Michael Barton
Director, Center for Social Dynamics & Complexity
Professor of Anthropology, School of Human Evolution & Social Change
Arizona State University
Tempe, AZ 85287-2402
USA

voice: 480-965-6262 (SHESC), 480-727-9746 (CSDC)
fax: 480-965-7671(SHESC), 480-727-0709 (CSDC)
www: http://csdc.asu.edu, http://shesc.asu.edu
    http://www.public.asu.edu/~cmbarton

On Jun 17, 2011, at 1:27 PM, Martin Landa wrote:

Hi all,

2011/6/17 Michael Barton <michael.barton@asu.edu>:

The lack of manpower makes it even more important that all developers test any changes before committing--including backporting.

I guess there is no one who is against testing. The subjective point
of view is "how much" the committed changes are tested. It depends on
the contributor, what he/she is committing (core libs, modules, GUI),
release life cycle (speaking about stable releases), etc. There is no
measurement, I believe everyone is going her/his best. Even someone
can have different impression. The important issue is to have working
test suites (modules - thanks Soeren for good work, GUI, etc).

Martin

--
Martin Landa <landa.martin gmail.com> * Studijní program Geodézie a kartografie – GeoWikiCZ