[GRASS5] GRASS development model Was: A private conversation

mlennert · February 19, 2001, 2:10pm

First of all I'm glad that this discussion is happening in the open, since I
believe that it concerns everyone.

I have to agree with Markus on what he said about Grass5 being a beta version.
Noone has ever claimed something else and to me beta has always meant "use at
your own risk".

I personally use Grass in my everyday work, and I am very happy with the CVS
system which allows me to very quickly implement changes when they interest
me. But I also know that I won't use the very latest version for critical
work. I wait for a week or so to see if there are no serious problems (such as
the make clean in beta10). I will then happily test that version (which
generally gives me nice new features), and because of CVS I can very easily
get corrections or improvements to any module seperately. And, yes, sometimes
it can happen that someone just committed a change before I do my update. But
the probability of this being fatal is quite low, and in any case, I'm not
obliged to update the entire Grass tree, but can limit to the modules I know I
want to update.

Now if users are defined as being people who want to use the newest features
but in a stable environment without any risk, ideally with the software
packaged in the easiest form possible, I can understand Richard's point of
view, and I agree, as I think everyone does, that there has to be a feature
freeze at one point. But I do believe that the force of OpenSource is just
that: no market pressure to oblige the development team to bring out the next
version. It is exactly this pressure that creates the bugs !

As the development team is very creative, there will always and constantly be
new ideas. And since, as far as I understand, most developpers are also users,
they want them implemented. I often want them implemented, and am very happy
they are. As I've said, I'm using Grass5beta11 (CVS) on a daily basis and
would not want to miss features such as the click-closing of a monitor (which
gives not only more comfort, but for me more stability) and the latest
improvements Radim has brought into ps.map. I know I'm using a beta version,
and that's ok, I don't care as long I can do the work I want to do. But as I
said above, I use it at my own risk.

Software development, and open source especially, is always subject to debate
between the "cathedral and the bazar". Yes, sometimes, it helps to have
someone say stop, let's do it now, but generally I think it is the dynamic
process of ongoing development as a result of usage that makes the best
programs. And who defines when is the right moment to say stop, and on what
criteria ? Is the new socket code important enough to wait, yes or no. Richard
asks Markus to be the one to decide. Well, he has and he has decided that it
is important enough. I respect this decision.

Moritz

----------------------------------------
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo@geog.uni-hannover.de with
subject 'unsubscribe grass5'

neteler · February 19, 2001, 3:06pm

On Mon, Feb 19, 2001 at 03:10:05PM +0100, mlennert@club.worldonline.be wrote:
[... useful comments from Moritz]

But I do believe that the force of OpenSource is just
that: no market pressure to oblige the development team to bring out the next
version. It is exactly this pressure that creates the bugs !

I just want to stress this point: beta10 and beta11 have been published
under such pressure as

- "why don't you release the stable version?" (because the BUGS file
is not empty yet) and
- "why does it take so much time to get a new version?" opposite to
- "why again a new beta something?"

We can't help everybody.

But we have learned from beta10 release:

- change of Makefile system was required to prevent the kill of
/usr/local/bin [implemented, error free now]
- have a pre-test phase

and from beta11 release:
- don't apply changes during pre-test phase [will do for next release]
- have a two-fold pre-test phase including binary package test [will do for
next release]

Everybody should accept that we try to learn. And: We are glad to receive
comments on the development model (therefore I have put this (again) into
public).

GRASS is not proprietary software and follows the Open Source development
idea. This *is* different to other products. And, if you consider the
"history" of GRASS 5: Fixing bugs takes often less than 24h. This seems to
be somewhat faster that the bug releases of proprietary software products.

Please believe in us that we still have some potential to learn and to
improve the GRASS development model. However, more testers are required and
more programmers who can take over some specific tasks (e.g.: need a software
engineer to clean up the G3D library and modules).

Most of us are using GRASS in daily production. You can be sure that
here at university teaching GRASS with GRASS being broken would be
impossible! However, several bugs have been found by my students and fixed
by team members. That's a sort of users' contribution which is very helpful.
GRASS is no banana-product (ripes at customer's office), we definitly
want to publish the stable version as soon as possible. But it *must* be
stable - the last beta's have shown severe problems. So the announcement
of GRASS 5.0.0 (stable) would have become a desaster.

I expect the sockets thing to be the last to be finished for 5.0.0. Hope
no other major bugs are left. The new compression seems to be running
perfectly (as it's in use for some time now).

Those being involved are working hard to "finish" the 5.0.0 and to stabilize
the new features most users don't want to miss.

Regards

Markus Neteler

PS: Volunteers wanted to
    - migrate all vector modules to new 5.1 vector lib
    - set up new automake system for 5.1
    - build shared libraries
    - ...
    -> Bereich Geographie – Naturwissenschaftliche Fakultät – Leibniz Universität Hannover

----------------------------------------
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo@geog.uni-hannover.de with
subject 'unsubscribe grass5'

rshepard · February 19, 2001, 4:58pm

On Mon, 19 Feb 2001 mlennert@club.worldonline.be wrote:

Software development, and open source especially, is always subject to debate
between the "cathedral and the bazar". Yes, sometimes, it helps to have
someone say stop, let's do it now, but generally I think it is the dynamic
process of ongoing development as a result of usage that makes the best
programs. And who defines when is the right moment to say stop, and on what
criteria ? Is the new socket code important enough to wait, yes or no. Richard
asks Markus to be the one to decide. Well, he has and he has decided that it
is important enough. I respect this decision.

Moritz,

Stop and think for a minute: where would the linux kernel be if it was
developed as GRASS-5 is being developed? Ask youself if you are for or
against Linus making the final decisions regarding what's in the kernel, at
what version, and when it's time for a release.

There is a compelling counter argument for each point you make. But, I no
longer care. What I intended to be a constructive dialog (dia == 2) has
become the general ego-generated flame. I'm both disappointed and disgusted.

Rich

Dr. Richard B. Shepard, President

                       Applied Ecosystem Services, Inc. (TM)
              Making environmentally-responsible mining happen. (SM)
                       --------------------------------
            2404 SW 22nd Street | Troutdale, OR 97060-1247 | U.S.A.
+ 1 503-667-4517 (voice) | + 1 503-667-8863 (fax) | rshepard@appl-ecosys.com

----------------------------------------
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo@geog.uni-hannover.de with
subject 'unsubscribe grass5'

Bernhard_Reiter · February 19, 2001, 5:23pm

Rich,

we would all like to hear about your concerns regarding the
development process. When I read through the replies I could see on
this mailinglist, your opinion was treated in a polite manner.

On Mon, Feb 19, 2001 at 08:58:00AM -0800, Rich Shepard wrote:

On Mon, 19 Feb 2001 mlennert@club.worldonline.be wrote:

> Software development, and open source especially, is always
> subject to debate between the "cathedral and the bazar". Yes,
> sometimes, it helps to have someone say stop, let's do it now,
> but generally I think it is the dynamic process of ongoing
> development as a result of usage that makes the best programs.
> And who defines when is the right moment to say stop, and on
> what criteria ?

Stop and think for a minute: where would the linux kernel be if it was
developed as GRASS-5 is being developed? Ask youself if you are for or
against Linus making the final decisions regarding what's in the kernel, at
what version, and when it's time for a release.

Well GRASS is almost developed like the Linux kernel, were do you
see the difference? It is Markus who does the final decision based
on the arguments made by reputated developers and the discussion on
this mailinglist. As far as I know this equals Linus' way of
deciding things.

And you are right in the direction of Moritz that it is a lot better
to have someone who decides. GRASS project leader is Markus and he
has full control over what goes into a release.
(Maybe this is a missunderstanding about how CVS is used?
CVS actually adds more control to the project leader.)

There is a compelling counter argument for each point you make. But, I no
longer care. What I intended to be a constructive dialog (dia == 2) has
become the general ego-generated flame. I'm both disappointed and disgusted.

Please do not feel offended. I believe that it was nobodies'
intention to flame you.

Bernhard

--
Professional Service around Free Software (intevation.net)
The FreeGIS Project (freegis.org)
Association for a Free Informational Infrastructure (ffii.org)
FSF Europe (fsfeurope.org)

bruce_byars · February 19, 2001, 6:05pm

Ok,

Everyone has had some valid points. IF *anyone* wants to
talk in a rational (not emotional) manner about things, feel
free to give me a call.

Bruce

tel: 254-710-6814
Bruce_Byars@baylor.edu

----------------------------------------
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo@geog.uni-hannover.de with
subject 'unsubscribe grass5'

neteler · February 19, 2001, 7:14pm

On Mon, Feb 19, 2001 at 08:58:00AM -0800, Rich Shepard wrote:
[...]

What I intended to be a constructive dialog (dia == 2) has
become the general ego-generated flame. I'm both disappointed and disgusted.

I have been writing personally to Rich to calm down this discussion.
In fact I should be more careful in future to cc to the grass5
list, means to get the o.k. from the opposite side of the email line.
The reasons for me were the "anarchy" Rich sees in current development model
which sounded rather offending to me.

Well...

My recent suggestion to Rich is to prepare a *detailed* list of things to
be improved in GRASS development model. Rich is right that the last
two releases weren't lucky. Therefore I have prepared a step-by-step
"ruleset":

http://freegis.org/cgi-bin/viewcvs.cgi/~checkout~/grass/documents/release_rules.txt

which still needs to be improved. Like that stupid problems shouldn't happen
any more. We definitly need to improve the pre-testing phase (hi volunteers,
we need more testers).

Hopefully we can come back to a pieceful and rational discussion here on
GRASS development model.

Regards

Markus Neteler

----------------------------------------
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo@geog.uni-hannover.de with
subject 'unsubscribe grass5'

Malcolm_Blue1 · February 20, 2001, 2:30am

Markus,

I had a quick look at the release rules that you identified. One thing that stood out (from my experience) is that in the pre-release testing and fixes only critical bugs should be addressed. These are bugs that stop the system, or major parts of the system from functioning correctly. At that point the updates should be limited to only those fixes - not to feature enhancements. Someone, in this case you or your delegate(s), should be given the responsibility of identifying the critical bugs, verifying the fixes and applying the updates to the source tree.

Overall, I think the development model being used is a good one. There were a huge number of improvements made over the last few months. In an open development model it is obvious that user needs and wishes can be quickly met. As we’ve seen with some of the key changes in GRASS some significant changes can be made in a short time. This is a huge testament to the developers.

As I said above, the only thing that I (try to) do differently in my development projects is to stop feature creep once we’ve gone into pre-release testing. This usually upsets some people, but I think that in the end it results in a better product. This is easier in the more structured development groups that I have usually worked with, but I think can be applied here as well.

Rich mentioned the Linux development model, but I don’t think that he described it completely enough. As he said, Linus controlled the changes to the kernal, but all of the other Linux “system” changes were made by a much larger circle and these were then packaged together into what we think of as Linux (all of the different flavors of Linux). The kernel development is (now) split into two cycles. Once a stable kernel is reached, it is used for distributions, while additional development continues on for the development kernel. Other packages continue on in development mode - but have to work with the stable kernel. Using this model, maybe some of the “key” pieces of grass should each be updated by only one person - who would have to maintain a stable version of that piece. You (Markus) would have to decide who that person was for each piece. While this may sound limiting, if it was applied in the pre-release/bug-fixing stage it might be manageable. Hopefully, it won’t curb the enthusiasm of the developers or overtax the people maintaining the “key” code.

(Those of us who have used Linux since the early/mid 90’s know that it was not a smooth process in the Linux world. Hindsight may have smoothed those curves, but it took a long time to get from the experimental stage - I think it was release .95 I first tried - to the current 2.4 release. And it took a lot of work. Early releases of Linux did not usually work as well as grass does today, and since version 5 is the first open source version of grass, that says a lot about the work you (Markus) have done to organize the development process.)

To date, I have been routing my changes through other developers and have had extremely good response. Especially with core functionality, I would rather have someone reject my suggestions than break something that works for other users. I hope that others feel the same way.

I am very impressed with the improvements in grass over the last year or so (on Linux and on Windows/Cygwon). A year ago I factored grass into my business plan as a future possibility. Today, I consider grass to be a viable alternative to commercial GIS. While I’m not using it on current projects, I will consider it as an option on future projects. I also plan on contributing when I can.

Markus: I’ll take a look around the Web and see if I can find information on how some of the other open source initiatives manage the testing/release cycle. There should be existing models that can be applied here to make a good system better. If I find anything of use I’ll send a link or comments to you.

Also, I don’t think that Rich should feel he was flamed. Although there were a few defensive comments, I thinks that most of the points raised were valid and most responses were on the mark. I expect that everyone wants to improve the situation. As a newcomer to the grass development group, I hope that I can help. As an experienced developer, I know that you can expect to hear negative comments quicker and louder than positive comments. It’s when people stop commenting that you should really get worried.

Adding my .02 CAN,

Malcolm

Markus Neteler wrote:

[...]

What I intended to be a constructive dialog (dia == 2) has
become the general ego-generated flame. I'm both disappointed and disgusted.


I have been writing personally to Rich to calm down this discussion.
In fact I should be more careful in future to cc to the grass5
list, means to get the o.k. from the opposite side of the email line.
The reasons for me were the "anarchy" Rich sees in current development model
which sounded rather offending to me. 

Well...

My recent suggestion to Rich is to prepare a *detailed* list of things to 
be improved in GRASS development model. Rich is right that the last
two releases weren't lucky. Therefore I have prepared a step-by-step
"ruleset":

[http://freegis.org/cgi-bin/viewcvs.cgi/~checkout~/grass/documents/release_rules.txt](http://freegis.org/cgi-bin/viewcvs.cgi/~checkout~/grass/documents/release_rules.txt)

which still needs to be improved. Like that stupid problems shouldn't happen
any more. We definitly need to improve the pre-testing phase (hi volunteers,
we need more testers).

Hopefully we can come back to a pieceful and rational discussion here on
GRASS development model.

Regards

 Markus Neteler

---------------------------------------- 
If you want to unsubscribe from GRASS Development Team mailing list write to:
[minordomo@geog.uni-hannover.de](mailto:minordomo@geog.uni-hannover.de) with
subject 'unsubscribe grass5'

---------------------------------------- If you want to unsubscribe from GRASS Development Team mailing list write to: minordomo@geog.uni-hannover.de with subject ‘unsubscribe grass5’

neteler · February 20, 2001, 9:43am

Hi Malcolm,

[please always post in ASCII, not in HTML... ]

On Mon, Feb 19, 2001 at 10:30:55PM -0400, Malcolm Blue wrote:

I had a quick look at the release rules that you identified. One thing
that stood out (from my experience) is that in the pre-release testing and
fixes only critical bugs should be addressed. These are bugs that stop the
system, or major parts of the system from functioning correctly. At that
point the updates should be limited to only those fixes - not to feature
enhancements. Someone, in this case you or your delegate(s), should be
given the responsibility of identifying the critical bugs, verifying the
fixes and applying the updates to the source tree.

I agree, this was what we have to learn from the beta11 release.

Overall, I think the development model being used is a good one. There
were a huge number of improvements made over the last few months. In an
open development model it is obvious that user needs and wishes can be
quickly met. As we've seen with some of the key changes in GRASS some
significant changes can be made in a short time. This is a huge testament
to the developers.<br>

As I said above, the only thing that I (try to) do differently in my
development projects is to stop feature creep once we've gone into
pre-release testing. This usually upsets some people, but I think that in
the end it results in a better product. This is easier in the more
structured development groups that I have usually worked with, but I think
can be applied here as well.<br>

Here I agree as well. Maybe myself I try "feature creep" as well which
should be strictly avoided. We simply need some discipline

Rich mentioned the Linux development model, but I don't think that he
described it completely enough. As he said, Linus controlled the changes
to the kernal, but all of the other Linux "system" changes were made by a
much larger circle and these were then packaged together into what we
think of as Linux (all of the different flavors of Linux). The kernel
development is (now) split into two cycles. Once a stable kernel is
reached, it is used for distributions, while additional development
continues on for the development kernel. Other packages continue on in
development mode - but have to work with the stable kernel. Using this
model, maybe some of the "key" pieces of grass should each be updated by
only one person - who would have to maintain a stable version of that
piece. You (Markus) would have to decide who that person was for each
piece. While this may sound limiting, if it was applied in the
pre-release/bug-fixing stage it might be manageable. Hopefully, it won't
curb the enthusiasm of the developers or overtax the people maintaining
the "key" code.

We need some "release managers" for the several binary distributions.
And we'll start with branching asap. This will need some time to learn
how to branch with CVS, but should lead to a clear structure.

(Those of us who have used Linux since the early/mid 90's know that it was
not a smooth process in the Linux world. Hindsight may have smoothed those
curves, but it took a long time to get from the experimental stage - I
think it was release .95 I first tried - to the current 2.4 release. And
it took a lot of work. Early releases of Linux did not usually work as
well as grass does today, and since version 5 is the first open source
version of grass, that says a lot about the work you (Markus) have done to
organize the development process.)

BTW: I have been starting to try Linux with 0.13, stopped, came back
with 0.95 and removed Windows when 0.99p1 was out. Of course I was real
novice that time.

To date, I have been routing my changes through other developers and have
had extremely good response. Especially with core functionality, I would
rather have someone reject my suggestions than break something that works
for other users. I hope that others feel the same way.
I am very impressed with the improvements in grass over the last year or so
(on Linux and on Windows/Cygwon). A year ago I factored grass into my
business plan as a future possibility. Today, I consider grass to be a
viable alternative to commercial GIS.  While I'm not using it on
current projects, I will consider it as an option on future projects. 
I also plan on contributing when I can.

Thanks, Malcolm. We need your input!

Markus:   I'll take a look around the Web and see if I can find
information on how some of the other open source initiatives manage the
testing/release cycle.  There should be existing models that can be
applied here to make a good system better.  If I find anything of use
I'll send a link or comments to you.

Yes, great. Please post to the list for discussion.

Also,  I don't think that Rich should feel he was flamed. 
Although there were a few defensive comments, I thinks that most of the
points raised were valid and most responses were on the mark. I expect
that everyone wants to improve the situation.  As a newcomer to the
grass development group, I hope that I can help.  As an experienced
developer, I know that you can expect to hear negative comments quicker
and louder than positive comments.  It's when people stop commenting
that you should really get worried.<br>

Adding my .02 CAN,
Malcolm

We need this development model discussion and should continue it from time
to time.

Kind regards

Markus Neteler

----------------------------------------
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo@geog.uni-hannover.de with
subject 'unsubscribe grass5'