[GRASS5] Some news: KerGIS

GRASS GRASS developer
1

Hello,

Some may wonder, after the discussions held in the late november, if I
have made some vaporware announces and if actually something is done.

Short answer: YES.

Long answer follows...

0. Introduction

In the end of November 2003 I have decided that it was more easy to
start from scratch from the last CERL public release of the public
domain GRASS than to start to reorganize and fix the code in the current
GNU GRASS.

The reasons are the following:

  - CERL GRASS was developped by a very small core team, focusing on
  general programs and functions, and having explicit goals. That is:
  this code was consistent. So it is far more easy to evolve a
  consistent, "small" code, than to try to fix code gathered from
  distinct sources, not having the same goals nor skills, and
  developping for customed applications with no overview in mind.

  - to make the code evolve, one needs an overview. The same premises
  lead to the same conclusion: more easy to gain this overview with
  the smallest set of consistent code.

1. State after 2 months (calendar) -- 1 month (real coding time)

My premises were correct. Indeed the code is consistent. "Old" i.e. pre
C standardization, yes ---and this is a pain by itself. But the project
conducted by Michael Shapiro and Dave Gerdes _was_ conducted, and they
have anticipated a great deal about things that were not obvious to
many by the time.

The fact that GRASS (and indeed, until very recently _this_ GRASS) is
still here is due to this far-sighted leading. It's normal that after
more than ten years, the same code is no more far-sighted... but this
can be in no way attributed to its creators, but to _us_ who have made
nothing of this stuff.

There are places ---mainly with the vector code--- where some
disorganization is found. But this is because all was under construction
or under reorganization. There are a lot of TODO, and some duplication
of code that have conducted Dave Gerdes to plan to strenghten the code
(creation of the digitizer library, reorganization of the digitizing
stuff with incorporation of LTPlus and so on).

  => I have fixed these while fixing the code and put the libraries
  where they belong and created, for example, the digitizer library.

So after this time, and after fixing the things explained in details
after (in short, standard C and POSIX, fix the code ---so there
is no more warnings from the compiler--- first reorganization), _all_
the libraries of the not X code, all the programs from general, display,
fonts, vector are UP and RUNNING, fixed and compile with:

COMPILE_FLAGS = -O -fno-builtin -Wall -Wstrict-prototypes -Wmissing-prototypes -Werror

except one (newly created file) defining the new function G_read_line
which replaces the limited and unsafe G_gets, with dynamic allocation of
memory to absorb a not limited line and handling signals, since I use a
non local goto and gcc issues a non pertinent warning [gcc is a great
tool, it's just unfortunate that one can not specify to not stop on
"may" errors].

Here is the list:

src/libes/btree
src/libes/ibtree
src/libes/linkm
src/libes/gis
src/libes/display
src/libes/dig_atts
src/libes/digit
src/libes/vector
src/libes/vask
src/libes/transform
src/libes/segment
src/libes/rowio
src/libes/raster
src/libes/lock
src/libes/imagery
src/libes/icon
src/libes/proj
src/libes/coorcnv
src/libes/bitmap
src/libes/D
src/libes/digitizer
src/libes/build
src/display/devices/lib
src/display/devices/XDRIVER
src/display/devices/monitorcap
src/fonts/for_grass
src/display/d.3d
src/display/d.ask
src/display/d.colormode
src/display/d.colors
src/display/d.colortable
src/display/d.display
src/display/d.erase
src/display/d.font
src/display/d.frame
src/display/d.geodesic
src/display/d.graph
src/display/d.grid
src/display/d.his
src/display/d.histogram
src/display/d.icons
src/display/d.label
src/display/d.labels
src/display/d.legend
src/display/d.mapgraph
src/display/d.measure
src/display/d.menu
src/display/d.mon
src/display/d.p.labels
src/display/d.points
src/display/d.profile
src/display/d.rast
src/display/d.rast.arrow
src/display/d.rast.edit
src/display/d.rast.num
src/display/d.rast.zoom
src/display/d.rgb
src/display/d.rhumbline
src/display/d.save
src/display/d.scale
src/display/d.sites
src/display/d.text
src/display/d.title
src/display/d.vect
src/display/d.what.rast
src/display/d.what.vect
src/display/d.where
src/display/d.zoom
src/general/g.access
src/general/g.ask
src/general/g.filename
src/general/g.findfile
src/general/g.gisenv
src/general/g.help
src/general/g.manual
src/general/g.mapsets
src/general/g.region
src/general/g.menu.region
src/general/g.tempfile
src/general/g.setproj
src/general/g.version
src/general/gis
src/general/manage
src/vector/v.alabel
src/vector/v.apply.census
src/vector/v.area
src/vector/v.build
src/vector/v.cadlabel
src/vector/v.clean
src/vector/v.cutter
src/vector/v.digit
src/vector/v.extract
src/vector/v.in.arc
src/vector/v.in.ascii
src/vector/v.in.dlg
src/vector/v.in.dxf
src/vector/v.in.tig
src/vector/v.in.transects
src/vector/v.mkgrid
src/vector/v.out.arc
src/vector/v.out.dxf
src/vector/v.out.moss
src/vector/v.patch
src/vector/v.proj
src/vector/v.prune
src/vector/v.spag
src/vector/v.stats
src/vector/v.support
src/vector/v.to.rast
src/vector/v.to.sites
src/vector/v.transform

2. Fixing: Goals and Method

I want the code to compile smoothly on any standard C and POSIX
platform. This is a portability goal, but a consistency one too: the
standard C prototypes are the correct way to ensure that the functions
are used the way they are designed to be [and I have actually found
programs where this was not the case!]

The implication are simple: support of TERMIOS and that's all, no
support for the deprecated and not portable setruid(3) and setrgid(3).

The method was the following.

First, since the amount of work is huge, I have started by discarding
all the code that was obsolete or useless. By principe, I have kept all
the CERL code, and review first the contrib sources. The result is that
I have discarded everything in the contrib sources ---limited use,
licence problems, poor coding--- but general programs found in
the SCS.

I have only kept devices that I plan to use: i.e. PostScript for
hardcopy, X for display.

This was the first simplification.

While fixing the code I have made further ones, deleting junks
(including a SPARC core dump file!), backup, saved,
duplicates, unused (and useless) files and kept only 1 version, the
latest one (if there are problems in the latest ones, this needs to be
fixed but it is useless to keep all the others).

If I found a program whose utility was doubtful AND poorly coded I have
saved it in my unlimited storage device: /dev/null.

Further more, I have kept only the "cmd" version, since the huge
majority of the programs already used the parser. So the files of the
program live now directly under its subdirectory and not in the
<program>/cmd. If the "interactive" version was totally different from
the "cli" one, i have created a new program: example (sole one at the
moment) g.region, where the "interactive" version is a menu driven one
=> creation of g.menu.region.

              THERE IS NO MORE MAKELINKS DANCE.

If I found _inside a file_, unused portions of code I have deleted them
if they were useless, or resurrected portions that were usefull ones.

Example: v.digit
  -> There were functions handy to identify (debug) one particular
  element (and I was planning to modify the code to support this)
    => resurrected
  -> There was one SCS extension usefull to everybody (Snap to line)
    => resurrected
  Note: the original version do the Zoom is more handy and powerful
  than the present ones (can widen the window i.e. see the whole map
  with the portion presently displayed and can define a new zone)
    => this "original" version of v.digit is more handy than the
    present one.

In mode details

2.A Algorithms

At the present no changes are attempted in this area so no bugs are
introduced while trying to suppress the old ones.

2.C Consistency

To ease the maintainance, there must be some style and some naming
conventions: I have modified the names so that a library is identified
clearly by a conventional prefix (gis had G_, I have created DZ_ for the
digitizer libray, B_ for the build one and so on).

General functions put in userland programs have been extracted from the
programs and put back in their libraries.

A first attempt has been made to avoid circular dependices between
libraries, and for example certain functions in GISLIB using VASKLIB
have been extracted and renammed and put back in VASKLIB, the goal
being, too, to avoid the ABAB dance when linking libraries with
reciprocal dependencies.

Headers have been created and standardized too with a common naming
scheme and so on.

Libraries and headers have been gathered in the correct places i.e. in
one place.

A common style is defined (beginning) for the name of files, headers,
the use of general functions (DEBUGF) and so on.

2.O Optimizations

"Early optimizations are the root of all evil".

Since I wanted to start by fixing the old code, I have, initially,
refrained from doing optimizations.
But I have finally decided to do the obvious ones (so obvious that these
should not imply new bugs).

The main optimizations are macros replacing functions simply calling
another function (without sharing static variables not accessible from
extern), and the suppression of GIS functions simply mimicking (with
limitations...) standard C ones (G_*alloc, string manipulations
functions). [not to mention in the case of the memory allocation that
some day one will want to "improve" them by doing a custom management,
and that programs were happily mixing G_*alloc with *alloc and free...]

  AFTER THE FIRST TESTS, THESE SIMPLE OPTIMIZATIONS HAVE ALREADY
  LED TO SPEED IMPROVMENTS

2.S Security

G_gets has been replaced with the secure (newly created) G_read_line

But the code remains unsafe, since there are a lot of sprintf spread all
around that don't limit nor check the amount of data they put in the
buffer
  => TODO

3. Future directions

I will continue to fix the code in this order:
paint (PostScript)
raster
sites
imagery
misc

and finally xgrass.

In the mean time (since the work is mainly boring, I need to have some
more exciting coding stuff) I will start developping new applications
in the DB field (where I keep strictly nothing).

Things to be fixed:
  XDRIVER: only handle 8 bits depth -> improved, with GLX support

The combination is standard C, POSIX, X, GLX and Motif for the X
interface.

The goals are always:

0. Consistency, primitives not policies
1. Portability
2. Accuracy
3. Efficiency
4. Scalability

4. The name of the game

KerGIS

5. Licence

At the moment, BSD style one. The first public release will be done when
the code is fixed and the source tree reorganized.

To be continued...
--
Thierry Laronde (Alceste) <tlaronde@polynum.org>
Key fingerprint = 0FF7 E906 FBAF FE95 FD89 250D 52B1 AE95 6006 F40C

2

On Sat, Jan 24, 2004 at 03:08:14PM +0100, Thierry Laronde wrote:

Some may wonder, after the discussions held in the late november, if I
have made some vaporware announces and if actually something is done.

Short answer: YES.

Thierry,

thanks for the heads up.

You are taking an interesting more radical approach.
I'll maintain the notion that was not possible in 1999,
but the time might be right for it now.

I would tend to protect the freedom of the code better
at least chosing LGPL as a license,
but this does not seem to be set in stone
so it is mood to have a longer argument about it as this point.

I am curious how KerGIS will work out
and wish the best
as the Free Software GIS community and GRASS can only win.

  Bernhard

3

Hello Thierry

On Sat, 24 Jan 2004, Thierry Laronde wrote:

[...]

2.S Security

G_gets has been replaced with the secure (newly created) G_read_line

But the code remains unsafe, since there are a lot of sprintf spread all
around that don't limit nor check the amount of data they put in the
buffer
  => TODO

Do you have an opinion on the G_asprintf() by Eric Miller that we use in
GRASS 5.7?:

http://grass.itc.it/pipermail/grass5/2002-May/005324.html

http://freegis.org/cgi-bin/viewcvs.cgi/grass51/lib/gis/asprintf.c?rev=1.1&content-type=text/vnd.viewcvs-markup

Also did you have a chance to have a good look at g.setproj (I see you
have kept it; I wouldn't)? I always thought it seems to be quite badly
written and had been done separately from v.proj and src/libes/proj,
which apparently were done by SCS, and had been done quite well and
consistently. I think the original idea was to create PROJ_INFO /
PROJ_UNITS manually with a text editor and g.setproj (perhaps originally
called m.setproj?) was added by somebody else.
As you have obviously been looking at the old code more than I have I am
just interested if you have formed an opinion on the evolution of the proj
library.

I look forward to hearing more about KerGIS.

Paul

4

Hello Paul,

On Sat, Jan 24, 2004 at 05:37:57PM +0000, Paul Kelly wrote:

> 2.S Security
>
> G_gets has been replaced with the secure (newly created) G_read_line
>
> But the code remains unsafe, since there are a lot of sprintf spread all
> around that don't limit nor check the amount of data they put in the
> buffer
> => TODO
>

Do you have an opinion on the G_asprintf() by Eric Miller that we use in
GRASS 5.7?:

http://grass.itc.it/pipermail/grass5/2002-May/005324.html

http://freegis.org/cgi-bin/viewcvs.cgi/grass51/lib/gis/asprintf.c?rev=1.1&content-type=text/vnd.viewcvs-markup

No, no opinion since I have absolutely not looked of what has been done
(on a more general basis, when I have fixed the code and the program
was not working as expected I have debug it my way and not try to look
at what had been done previously to correct the thing).

Also did you have a chance to have a good look at g.setproj (I see you
have kept it; I wouldn't)? I always thought it seems to be quite badly
written and had been done separately from v.proj and src/libes/proj,
which apparently were done by SCS, and had been done quite well and
consistently. I think the original idea was to create PROJ_INFO /
PROJ_UNITS manually with a text editor and g.setproj (perhaps originally
called m.setproj?) was added by somebody else.

As I have explained, _by principle_ I have kept all the programs
distributed by CERL throwing away duplicates, junk, multiple versions,
things that are useless to support now (hardware specific display
drivers, paint drivers, print management ---will be done with the system
tools) and, for programs for which I have some doubt if I will keep them
finally or not, I have been conservative UNLESS when I tried to fix
the code (C syntactic correctness) I stumble on errors that made
clear that the coder did not understand what he was doing.

Since I don't know if g.setproj will survive (and I know that there are
programs that I have fixed whose lifetime in KerGIS will be short) but I
had the average problems to fix it, it is still here.

There is another reason for this conservative approach.

For example, with the current v.digit in GNU GRASS, I have never been
able to digitize a non connected line, which leads to the chicken and
egg problem: if a line must be connected, I do you draw the first one?
BUT the original version of v.digit doesn't suffer from this: v.digit
issues a warning that the nodes are not connected, but the line is
created.

What I mean is that to have the complete overview, and to understand
what was the big picture and how things were meant to work together, we
need to see the original bare version. Relying on "features" of the
present programs, where people have blindy modified some things that may
have broken the whole, is dangerous.

For the PROJ library, I must say that I had a rough time with it at the
beginning. Since to track, generally, the uninitialized variables one
must read the code and the code must be reasonnably readable, I have
launched indent and on that particular library it was a disaster due to
the way it is coded.
I distate heartily the coding style and particularily the way the
pre-processor is used, but I must say that, from the strict C standard
point of view, this was a library that caused very little problems.

So my conclusions are that, even if I do not like the style, the
programmer understood what he was doing (and for contributions not
originating from CERL this is an assumption that do not generally
holds...).

For g.setproj and v.proj they were both developped by the same guy from
SCS: M.L.Holko. So... we will have to wait to see the whole picture.

Cheers,
--
Thierry Laronde (Alceste) <tlaronde@polynum.org>
Key fingerprint = 0FF7 E906 FBAF FE95 FD89 250D 52B1 AE95 6006 F40C

5

Hello Bernhard,

On Sat, Jan 24, 2004 at 06:00:06PM +0100, Bernhard Reiter wrote:

Thierry,

thanks for the heads up.

You are taking an interesting more radical approach.
I'll maintain the notion that was not possible in 1999,
but the time might be right for it now.

I would tend to protect the freedom of the code better
at least chosing LGPL as a license,
but this does not seem to be set in stone
so it is mood to have a longer argument about it as this point.

I am curious how KerGIS will work out
and wish the best
as the Free Software GIS community and GRASS can only win.

Thanks. And for the curiosity about how KerGIS will work out, and what
it will be, the short answer again is: it will be what people will
make it.

But this will not be made randomly: a consistent error is
almost a truth, since if you have consistently made the very same error,
a simple translation can make the whole becomes the truth. So, for
people who are wondering, KerGIS will be on the cathedral side, and not
on the bazaar one...

Cheers,
--
Thierry Laronde (Alceste) <tlaronde@polynum.org>
Key fingerprint = 0FF7 E906 FBAF FE95 FD89 250D 52B1 AE95 6006 F40C

6

On Sun, Jan 25, 2004 at 12:57:19PM +0100, Thierry Laronde wrote:

For example, with the current v.digit in GNU GRASS,

It is missleading to call GRASS like this.
GRASS is not part of the GNU project.
It is Free Software just happens to use the GNU GPL.

7

On Sun, Jan 25, 2004 at 03:27:01PM +0100, Bernhard Reiter wrote:

On Sun, Jan 25, 2004 at 12:57:19PM +0100, Thierry Laronde wrote:
> For example, with the current v.digit in GNU GRASS,

It is missleading to call GRASS like this.
GRASS is not part of the GNU project.
It is Free Software just happens to use the GNU GPL.

Correct. It was just a lapsus. I have written GNU when I thought GPL. I,
for one, should have known better...

--
Thierry Laronde (Alceste) <tlaronde@polynum.org>
Key fingerprint = 0FF7 E906 FBAF FE95 FD89 250D 52B1 AE95 6006 F40C

8

What I mean is that to have the complete overview, and to understand
what was the big picture and how things were meant to work together,
we need to see the original bare version. Relying on "features" of the
present programs, where people have blindy modified some things that
may have broken the whole, is dangerous.

It would be most useful to those of us who are new to working on GRASS
and whom only really understand the underlying code in a superficial
manner if someone who did understand such things could throw together a
block diagram of the different components and how they all were/are
meant to fit together. It might help to reduce future code fragmentation
and make it easier for new people contribute to KerGIS/GRASS in a useful
and correct way.

thanks,
Hamish

9

On Saturday 24 January 2004 15:08, Thierry Laronde wrote:

In the mean time (since the work is mainly boring, I need to have some
more exciting coding stuff) I will start developping new applications
in the DB field (where I keep strictly nothing).

Can you concretize 'DB field'?

4. The name of the game

KerGIS

Much better than YAG, but what does 'Ker' mean?

And from your original plans :

6) Scalability: YAG must be able to run on clusters
        -> All non strictly interactive tools MUST run in batch mode
        -> Client/server architecture
        -> Multithreading (not a priority, but must be thought about)

Do you think that to make the code thread save may be postponed?

Radim

10

On Mon, Jan 26, 2004 at 07:32:11PM +1300, Hamish wrote:

> What I mean is that to have the complete overview, and to understand
> what was the big picture and how things were meant to work together,
> we need to see the original bare version. Relying on "features" of the
> present programs, where people have blindy modified some things that
> may have broken the whole, is dangerous.

It would be most useful to those of us who are new to working on GRASS
and whom only really understand the underlying code in a superficial
manner if someone who did understand such things could throw together a
block diagram of the different components and how they all were/are
meant to fit together. It might help to reduce future code fragmentation
and make it easier for new people contribute to KerGIS/GRASS in a useful
and correct way.

The documentation is part of the project, and it is one of the most
important.

BTW, I will say it once and never repeat it after: I will certainly put
things and say things rather roughly when I analyze and criticize the
way things have happened. It is in no way a criticism on the individuals
it is just the constatation that with any project, despite the quality
of the persons, if the contributions are not planned and coordonnated,
the time spent is lost.

In no way I will proclam that I'm omniscient and "error free". No: I
make errors and will continue till the end. I'm simply trying to learn,
that is to make new ones.

Cheers,
--
Thierry Laronde (Alceste) <tlaronde@polynum.org>
Key fingerprint = 0FF7 E906 FBAF FE95 FD89 250D 52B1 AE95 6006 F40C

11

On Mon, Jan 26, 2004 at 11:02:45AM +0100, Radim Blazek wrote:

On Saturday 24 January 2004 15:08, Thierry Laronde wrote:
> In the mean time (since the work is mainly boring, I need to have some
> more exciting coding stuff) I will start developping new applications
> in the DB field (where I keep strictly nothing).

Can you concretize 'DB field'?

I will simply sketch them since everything is not curved in stone at the
moment. This is also a demonstration of the need of the overview (the
big picture).

In the following when I speak about DB, it is meant in the common sense
nowadays, i.e "data handled via RDBMs". The "GIS Data Base" will be
called simply the GIS Data Repository: GDR.

I will take the example of the vector, since this is the one I use
commonly.

At the moment, the embryo of the DB is, indeed, the attributes files.
The link between geometrical data and textual one was made via the
attributes (labels), these attributes being translated to text via
categories if somebody wanted.

So the DB was a one enregistrement with one field one.

A typical use ---I think--- now, is to put labels distinct for every
element in a layer (incrementally for example) and to link this label
with a field in a DB.

Consequences and future directions:
  - The attributes are doomed to disappear as well as the categories

  - Since the index used tend (and it's logical) to identify uniquely
  an element, the future vector format MUST support directly an
  identifier, and the format must be made so that one can specify the
  size of the identifers in power of two tetrabytes (I follow Knuth
  terminology, tetrabyte == 4 bytes, and a byte is 8 bits), so that
  the format will be able to handle tremendous amount of data [NOTE:
  not all the bits in the identifiants will be used for the
  identification, since for example, if the vector format will be a
  topological one, the some bits in the most significant
  byte will be used to code the orientation of the element].

  - KerGIS/GRASS is not a vectorial package per se, so the attributes
  commonly found in CADs (vectorial formats coding inside the
  attributes) will not be supported. The vectorial format will support
  the geometrical definitions, incorporate the topological information
  now found in the dig_plus (in fact I'm thinking of a special
  topological format ---but I will not say anything more for the
  moment since it is not the priority), the mean of the
  "support/build" operation being precisely to generate a fast
  searching index between geometrical elements and textuals attributes
  (in this view, if someone wants to specify particular visual aspects
  (color and so on) of elements, these will be placed in a table in a
  DB).

  => Geometrical operations don't have to deal with attributes (you
  SELECT elements on an attribute basis XOR geometrical basis [the
  combination of the two is the combination of two distinct
  selections], and in the DB you need only the identifiant of the
  geometrical element NOT its geometrical definition (the geometrical
  extensions of some RDBMs will not be used since it's, in my mind,
  both from an efficiency and a logical point of view, an original
  sin).

KegGIS shall support multiple distinct DB sources for a single layer.
These sources will be specified, in an URI style
(method:location_of_table#name_of_field_holding_the_identifiants) in a
newly created file in the GDR/$LOCATION/$MAPSET.

To be able to import data from different sources, a TEXT file, with
ASCII commands (the language will be specified via a
LANG=<ISO_DEFINITION> so that we can handle languages in 7, 8 bits,
or wyde chars for the text fields) with the following characteristics:

  - No loss of precision is allowed: the format will handle (since
  it's text, it's easy) infinite (to the extent of the place on mass
  storage devices...) precision: the 1000 digits precision allowed by
  PostgreSQL will still be here;
  - No loss of features is allowed: functions will be described too;
  - The format must be easy to read and easy to scan.

  => The reasons for the TEXT format are:
    - The import/export modules will have not n x n possibilities,
    but 2n : FOO->KerGIS_DB_FORMAT, KerGIS_DB_FORMAT->BLA
    -> hence the "no loss" constraints [this is not the case for
    example at the moment(say pg.in.dbf, but others are touched);
    - The use of CVS will be eased, since it's easier with text
    format and it's more efficient (for diffs);
    - allowing simple editing, debugging, text manipulations;

A corresponding binary format will be designed which will be accessed
via the common scheme (-KerGIS DB Format will simply be another type).

=> This proper format will not have to be tremendously customizable or
efficient. It will be a fallback for simple applications or people not
having other RDBMs at disposal. The "functions" will not be supported at
least at the beginning.

a $GDR/$LOCATION/$MAPSET/db directory will be created.

> 4. The name of the game
>
> KerGIS

Much better than YAG, but what does 'Ker' mean?

Ker <-> core: the essentials

Kernel GIS i.e. must allow the developpement of applications linking
directly to "system calls" from the "graphical kernel".

Ker in a "mathematical" sense meaning that for every GIS application
ker(application) = KerGIS i.e. where others have one billion different
functions this can be done with a single well designed system
call in KerGIS (the mathematical correctness of the description is not
meant to be questionned too far; it's word play).

And from your original plans :
> 6) Scalability: YAG must be able to run on clusters
> -> All non strictly interactive tools MUST run in batch mode
> -> Client/server architecture
> -> Multithreading (not a priority, but must be thought about)

Do you think that to make the code thread save may be postponed?

As long as the base is not orthogonal, with the different parts
(analysis, device access, presentation layer and so on) not
clearly identified and insulated, this must not be attempted.

Once the reorganization is achieved AND the big picture is gained, this
will allow too the parallelizing of efforts.
--
Thierry Laronde (Alceste) <tlaronde@polynum.org>
Key fingerprint = 0FF7 E906 FBAF FE95 FD89 250D 52B1 AE95 6006 F40C

12

On Mon, Jan 26, 2004 at 02:16:00PM +0100, Thierry Laronde wrote:

BTW, I will say it once and never repeat it after: I will certainly put
things and say things rather roughly when I analyze and criticize the
way things have happened. It is in no way a criticism on the individuals
it is just the constatation that with any project, despite the quality
of the persons, if the contributions are not planned and coordonnated,
the time spent is lost.

Honest criticising will be no problem.
But you have to be careful of tying this to the process
(and thus decisions of persons).

You are looking at a certain state of GRASS
and you have several assumptions about the process.
I don't share all of them, especially your last concertation
is something I don't share to some parts.

Somehow you probably would not try KerGRASS now
if GRASS would not have been continued the way it did in 1999.
I believe that without GPL GRASS, we would not have seen KerGIS.
It is true that you or anybody could have started KerGIS earlier,
but nobody did. Now that GPL GRASS grew green enough to show the
potential and the advantages of Free Software are more known,
a radical approach to do a new Free Software GIS is not bad.
But this is what KerGIS is, this is why you rightfully gave
it a different name.

If you make general statements that are beyond the technology,
you might provoke counter arguments.

You can say: Technical this is in a bad state know.
You cannot say: the process was wrong, this is why the state is so,
without getting into general arguments.

In no way I will proclam that I'm omniscient and "error free". No: I
make errors and will continue till the end. I'm simply trying to learn,
that is to make new ones.

I'd be interested in how you judge the potential of the other
Free Software GIS approaches. PostGIS in a way also starts
to be a GIS coming from the database side of things.
Thuban trying to be an interactive geodata viewer,
also becomes more and more a GIS as people refine the mean of GIS.
This probably would be a dissussion to be held on the FreeGIS list.

  Bernhard

13

Hello Thierry

On Sun, 25 Jan 2004, Thierry Laronde wrote:

On Sat, Jan 24, 2004 at 06:00:06PM +0100, Bernhard Reiter wrote:
>

[...]

> You are taking an interesting more radical approach.
> I'll maintain the notion that was not possible in 1999,
> but the time might be right for it now.
>

Thanks. And for the curiosity about how KerGIS will work out, and what
it will be, the short answer again is: it will be what people will
make it.

But this will not be made randomly: a consistent error is
almost a truth, since if you have consistently made the very same error,
a simple translation can make the whole becomes the truth. So, for
people who are wondering, KerGIS will be on the cathedral side, and not
on the bazaar one...

I was interested when you mentioned the cathedral and the bazaar, and it
prompted me to read a little bit (a few pages) of this paper:
http://www.catb.org/~esr/writings/cathedral-bazaar/cathedral-bazaar/
While reading it and thinking how the bazaar approach resulted in quality
software in the case of Linux I thought about the similarities between
GRASS and Linux, which Bernhard has mentioned a few times, and why the
bazaar approach has not worked for GRASS and therefore you are trying the
cathedral approach.

It seems to me that the bazaar approach only works when you have good
quality control, and for a few years after the CERL support of GRASS ended
there was inadequate quality control or central authority and bugs got in.
And it is very hard to track them apart from some comments in the source
files, since this is before the introduction of the CVS server in 1999. Of
course many more bugs have been introduced since 1999 but they have been
much easier to track and revert with CVS in use.

But to counteract the bugs that were introduced when GRASS was 'drifting'
for a few years it would indeed seem that the only logical approach is to
start with the last CERL release, i.e. what you are doing. But once you
get it organised and tidied up and make a release, then providing you are
still interested in looking after it and providing quality control over
the improvements, the bazaar approach might once again become appropriate.

So I hope it works out well and I wish you the best of luck.

Incidentally, talking about when GRASS was drifting with no clear
technical leadership reminds me of a counter to one of Bernhard's pro-GPL
arguments for GRASS: that without the GPL, people would take all of GRASS
and make it into a proprietary product and not give anything back to the
GRASS community. I take it the example of this is Blackland GRASS; is this
correct? I would argue that because there was no clear technical
leadership or strong Open Source Free software GIS lobby at the time, that
it was not even an obvious course of action for people creating a
proprietary GRASS to give back some of their developments. (Of course I
do not know what was really going on then as I was not involved; I can
just get an idea by reading the old mailing list archives.)

I argue that with the more organised state that GRASS is in now, the pressure
on such a proprietary developer to give back developments to the GRASS
community would be much higher than it was when Blacklands GRASS was
developed. Partially I base this idea on discussions I have seen on the
GDAL list (GDAL has the BSD-style licence) where developers of proprietary
software are using GDAL and feel grateful for it and are putting pressure
on their bosses to release part of their product as Open Source.

Just thought I'd mention this as I haven't seen it discussed before. BTW
I'm not sure what the best licence for GRASS is and don't have a strong
opinion as long as it is some kind of open source that researchers can
play around with.

Paul

14

On Mon, Jan 26, 2004 at 04:11:44PM +0100, Bernhard Reiter wrote:

On Mon, Jan 26, 2004 at 02:16:00PM +0100, Thierry Laronde wrote:

> BTW, I will say it once and never repeat it after: I will certainly put
> things and say things rather roughly when I analyze and criticize the
> way things have happened. It is in no way a criticism on the individuals
> it is just the constatation that with any project, despite the quality
> of the persons, if the contributions are not planned and coordonnated,
> the time spent is lost.

Honest criticising will be no problem.
But you have to be careful of tying this to the process
(and thus decisions of persons).

To explain more:

1) If GPL GRASS had not existed, resurrected by Markus Neteler almost
alone some years ago, Public Domain GRASS would have been lost, and I,
as others, would never have the opportunity to derive something from it;

2) If I have not seen something working by working with GPL GRASS, I
will have not attempted what I'm doing.

=> So the work done is not lost on this basis but must be capitalized to
learn how to make things work better.

The "errors" made on the process are the errors that, to some extents,
_we_ ---in the libre software world--- all have made: to believe that
opening the sources will attract more developers and that people using
them will spontaneously try to contribute.

=> That's wrong. The english term "free" has the desastrous meaning
"gratis" when working on libre software have never been gratis for the
developers.
Now a libre software must work on the assumption that it will be not
given "gratis": the user that don't contribute code will contribute time
spent _for his own sake_ to learn how to use the software, and the only
users will have dedicated lists where they will be able to exchange and
help themselves [a reasonnably clear documentation shall be provided for
the beginners; it is in no way a design goal to have ununderstandable
documentation just to throw users away]. If they don't want to help
themselves and to work a little for their own sake, they can:

  -stop using the software: it is in no way an obligation...
  -pay someone for doing what they do not want to do: laziness is a
  sin and the punition has to come...

Libre software is a school of energy, will and skills. Before libre
software, even if you had the will and the skills, if you do not have
the money you were blocked. Now, if you have the skills and the will
everything is at your reach. And if you have the will, you can gain
the skills by learning, and participating to such a huge project as
KerGIS/GRASS is highly beneficial [I have learned a lot simply by
fixing the code].

The second error is trying to be "kind" with contributions, that is to
think that accepting badly thought pieces of code will allow to keep
wanabees contributors and that being too direct will disgust people at
first attempt.

That's wrong. For the sake of everybody, the goal must be an improvement
of the quality of the code. A non optimal contribution is a lost, and a
non contribution is a status quo. In this case non contribution is
better since it is a gain compared to the loss.

The management has nothing to do with insults, sarcasms and so on (being
noted that it is sometimes difficult for non english native speakers to
translate accurately what they thought in the pseudo-english that we
use). I have heard remarks on my code sometimes, and I was not angry
against the one who criticized (if he were right) but against myself: I
was upset because of my "amour-propre". This has simply led me to show
that I could do better than that...

So, we have made errors. That's not unforgivable if we do not continue.
If we continue these will be faults.

> In no way I will proclam that I'm omniscient and "error free". No: I
> make errors and will continue till the end. I'm simply trying to learn,
> that is to make new ones.

I'd be interested in how you judge the potential of the other
Free Software GIS approaches.

By intuition and by experiment, I do not believe this is the correct
way. At least, this is a way I do not want to follow.

There are enough people who already make the very same things. Let the
evolving GRASS still be what it is: a topological GIS. Do not make the
same errors as other, make our own :wink:

--
Thierry Laronde (Alceste) <tlaronde@polynum.org>
Key fingerprint = 0FF7 E906 FBAF FE95 FD89 250D 52B1 AE95 6006 F40C

15

Hello Paul,

On Mon, Jan 26, 2004 at 04:52:45PM +0000, Paul Kelly wrote:

I was interested when you mentioned the cathedral and the bazaar, and it
prompted me to read a little bit (a few pages) of this paper:
http://www.catb.org/~esr/writings/cathedral-bazaar/cathedral-bazaar/
While reading it and thinking how the bazaar approach resulted in quality
software in the case of Linux I thought about the similarities between
GRASS and Linux, which Bernhard has mentioned a few times, and why the
bazaar approach has not worked for GRASS and therefore you are trying the
cathedral approach.

The problem with Eric S. Raymond's paper is that this plea for the
bazaar is based on a word play. The bazaar is not on the management side
it's the opportunity to accept improvements or new ideas coming from
a "non-authoritative" source.

The main driving force that _can_ be behind libre software is the
interconnection, that is the possibility to gather energies and skills
spread all around the earth and to combine good ideas that will be lost
in an classical university/enterprise environment where there are the
ones who have the right to speak and the other ones who have the right
to listen. But this is not a random incorporation: the sources are
random (good skills or ideas can come from a non expected side), but the
process is not.
This means too that the Internet is the definitive back-bone of the
libre software and that we must be very carefull with the laws being
made at the moment.

And note that if there is a bazaar this is a "bazaar of cathedrals".
When the sources were closed, it was impossible to anybody to start a
big project, since it meant start from scratch. With open sources, you
can with a reasonable amount of work construct your own cathedral if you
don't agree with the directions taken by others (the fork processes).

All the succesfull leading project are cathedrals (strictly driven ones)
: FSF is not a bazaar (it's very difficult to enter by the way), Linux
was not a bazaar (Linus Torvalds have endured a huge amount of
criticisms due to his refusals to incorporate some functionnalities),
4.4BSD Lite2 was not a bazaar (cf the Release Engineering paragraphs in
"The Design and Implementation of the 4.4BSD Operating System" by
McKusick, Bostic, Karels, Quarterman ---Addison-Wesley :

  The CSRG [Computer System Research Group at the University of
  California at Berkeley] was always a small group of software
  developers. This resource limitation required careful
  software-engineering management. Careful coordination was needed not
  only of the CSRG personnel, but also of members of the general
  community who contributed to the development of the system.[...]
  [...]
  This process illustrates an _advantage_ of having only a few
  principal developers: The developers all knew the whole system
  thoroughly enough to be able to coordinate their own work with that
  of other people to produce a coherent final system. Companies with
  large development organizations find this result difficult to
  duplicate.

).

[..]
I argue that with the more organised state that GRASS is in now, the pressure
on such a proprietary developer to give back developments to the GRASS
community would be much higher than it was when Blacklands GRASS was
developed. Partially I base this idea on discussions I have seen on the
GDAL list (GDAL has the BSD-style licence) where developers of proprietary
software are using GDAL and feel grateful for it and are putting pressure
on their bosses to release part of their product as Open Source.

Just thought I'd mention this as I haven't seen it discussed before. BTW
I'm not sure what the best licence for GRASS is and don't have a strong
opinion as long as it is some kind of open source that researchers can
play around with.

I have the sentiment that to prevent a not fair use of libre software
the following is mandatory:

- Obligation of advertisement: one can not hide that one uses software
he has not developed -> this advertises the sources hence the project;
this moderates the prices [if one wants to make money he has to
justify about an added value] and this gives too an incitative to
work as a core developer [if you claim to be a good developer whose
work is derived from this one, it could be strange that you do not
contribute to the main project...]

- Allow to make money with it [and for this you must have the
opportunity to not give away the code you have developed]

- BUT the strict interdiction to put patents on derived work (I'm
fanatic about that). And this can be achieved (I have discussed about
this with RMS some times ago and he didn't think there was a easy way to
do it) with this kind of disposition I have found in a crypto software,
that is saying that if anybody derived a work from this library and put
patents on the derived work, ipso facto and immediately the provider of
the library will gain the patents for its own use.

As long as the software doesn't focus on the USER INTERFACE, where most
of the added value can be done, this will allow individuals,
universities and enterprises to contribute to the core while protecting
their business interests if any.
--
Thierry Laronde (Alceste) <tlaronde@polynum.org>
Key fingerprint = 0FF7 E906 FBAF FE95 FD89 250D 52B1 AE95 6006 F40C

16

Thierry Laronde wrote:

The second error is trying to be "kind" with contributions, that is to
think that accepting badly thought pieces of code will allow to keep
wanabees contributors and that being too direct will disgust people at
first attempt.

I haven't been paying much attention to this thread, but I would just
like to voice my agreement with the above.

--
Glynn Clements <glynn.clements@virgin.net>

17

Subject says it all. I will follow my planning and expect to have the
whole (without X/Motif) fixed in 1 month (calendar).

Cheers,
--
Thierry Laronde (Alceste) <tlaronde@polynum.org>
Key fingerprint = 0FF7 E906 FBAF FE95 FD89 250D 52B1 AE95 6006 F40C