[GRASS5] Re: KerGIS

GRASS GRASS developer
,
1

I've been reading the current news on KerGIS this morning and I'm glad to
see the initiative regarding quality control and the addtional resources to
the underdeveloped and overworked GRASS developement pool. I've been wanting
to utilize GRASS (as well as contribute) for a couple of years now and (and
I'm not trying to gripe too much here and I'm not criticizing) it seems that
the goals are constantly shifting as well as feature creep regarding the
architecture. With all the development that has gone on in the last couple
of years, GRASS still cannot handle some of the most basic of vector and
database operations to satisfaction in a production environment. I think
GRASS can be an awsome GIS tool (and in lots of respects it *is* -- raster
and satellite processing mostly).

I read the news regarding KerGIS with great enthusiasm as I've wished for
such an event to happen for a couple of years. I think it just what the
doctor ordered for the code. No more gmake5, etc. Fixing and simplify and
remove optimizations! Yes, yes, yes... I've been thinking about doing this
for a couple fo years now, but since I've got to continue to produce and
can't spend the time to do anything other create small supporting packages,
I've not had the chance. As it stands now, I'm not sure where to contribute.
There are *at least* three code streams for GRASS (5.0, 5.3 and 5.7 -- not
to mention 4.3) This seems to be the opposite of fix and simplify. And none
of them seem to include the capabilites I need. In order for 5.7 to work, I
need the code for 5.3. Why? Shouldn't all that new code simply go into 5.3?

If I could get everthing to compile on my BSD platform, I would use it
there. If I could get everything to compile on cygwin, I'd use it there. But
they don't. There's just too many switches and options for a simple (for
example)

./configure
make --with-mysql
make install

to work (at least for the 5.0.3 source code). For the 5.7 source code, you
need 5.3 source code and the cygwin "weekly" binaries for 5.7 aren't availab
le. I've not seen a project with so much connection between code streams.
Why?

Almost all of my work is with vectors and the attributes, lots of attributes
of polygons (adjacency lists, unions, intersects, area calculations, and
attributes -- think FRAGSTATS woulodn't it be nice to have a version for
GRASS for people to build on). For my projects, I'm constantly moving files
between GRASS and Arc/Info and ArcView for projecting files across all sorts
of projections, datums and coordinate systems and I have yet to be able to
accomplish this in GRASS simply because I can't gdal/GRASS to work resulting
from some sort of goofy circular dependencies). My Database is MySQL and my
stats package is R (which seems to have be best model for development I've
seen so far). These are two great packages and the development process is
solid almost militant in nature. MySQL releases a couple of times per year
as does R and it seems GRASS has been limping along in the meantime. We need
help. Isn't there an institution that promotes the development of GRASS
where people can work on it full time? It seems that the overworked
developers are spread pretty thin and the project needs a small army of
people pushing on.

Okay, this seems a bit long winded so I'll shup now. And thanks for hearing
my pleas.

Thanks for the good work,
Jeff.

---
Jeff D. Hamann
Forest Informatics, Inc.
PO Box 1421
Corvallis, Oregon USA 97339-1421
(office) 541-754-1428
(cell) 541-740-5988
jeff.hamann@forestinformatics.com
www.forestinformatics.com

2

On Mon, Jan 26, 2004 at 12:00:39PM -0800, Jeff D. Hamann wrote:

I've been reading the current news on KerGIS this morning and I'm glad to
see the initiative regarding quality control and the addtional resources to
the underdeveloped and overworked GRASS developement pool. I've been wanting
to utilize GRASS (as well as contribute) for a couple of years now and (and
I'm not trying to gripe too much here and I'm not criticizing) it seems that
the goals are constantly shifting as well as feature creep regarding the
architecture. With all the development that has gone on in the last couple
of years, GRASS still cannot handle some of the most basic of vector and
database operations to satisfaction in a production environment. I think
GRASS can be an awsome GIS tool (and in lots of respects it *is* -- raster
and satellite processing mostly).

Did you ever try GRASS 5.7? If you refer to 4.x, 5.0-5.3: yes, vector
management is poor. But 5.7 is quite different. We would be glad to
hear comments why you (dis)like 5.7.

I read the news regarding KerGIS with great enthusiasm as I've wished for
such an event to happen for a couple of years. I think it just what the
doctor ordered for the code. No more gmake5, etc. Fixing and simplify and
remove optimizations! Yes, yes, yes...

Again 5.7: No more Gmakefile. Since 2001/2002 (can't remember precisely).

I've been thinking about doing this
for a couple fo years now, but since I've got to continue to produce and
can't spend the time to do anything other create small supporting packages,
I've not had the chance. As it stands now, I'm not sure where to contribute.
There are *at least* three code streams for GRASS (5.0, 5.3 and 5.7 -- not
to mention 4.3) This seems to be the opposite of fix and simplify.

Several developers prefer to work on 5.3 - even if things should have
been already merged into 5.7.

And none
of them seem to include the capabilites I need. In order for 5.7 to work, I
need the code for 5.3. Why? Shouldn't all that new code simply go into 5.3?

No. Because then people would complain that backward compatibility is not there.
Indeed 5.3 should go into 5.7 while undergoing a code cleanup. But there
is not too much interest in that IMHO.
As long as the developers base for 5.7 is so small, we will have to live
with that odd "merge" problem. It's way to much work for "2.5 people" to
also merge 5.3 changes into 5.7 in case the code were replicated there.

If I could get everthing to compile on my BSD platform, I would use it
there.

5.7 was reported to compile successfully on FreeBSD. Did you try?
Then: I didn't see any error reports for BSD.

If I could get everything to compile on cygwin, I'd use it there. But
they don't.

You could simply download precompiled version from the grass.itc.it web
site (Linux, Cygwin, MacOSX soon).

There's just too many switches and options for a simple (for
example)

./configure
make --with-mysql
make install

to work (at least for the 5.0.3 source code). For the 5.7 source code, you
need 5.3 source code and the cygwin "weekly" binaries for 5.7 aren't availab
le. I've not seen a project with so much connection between code streams.
Why?

Because there aren't enough people to sort out this problem.
If it helped, we could merge the code into 5.7. But then I expect
a major desaster: 5.3 will receive changes and some developers
regularly ignore to update the relevant files in 5.7 as well.
I try to hunt for those files, but it's not that easy.

Almost all of my work is with vectors and the attributes, lots of attributes
of polygons (adjacency lists, unions, intersects, area calculations, and
attributes -- think FRAGSTATS woulodn't it be nice to have a version for
GRASS for people to build on). For my projects, I'm constantly moving files
between GRASS and Arc/Info and ArcView for projecting files across all sorts
of projections, datums and coordinate systems and I have yet to be able to
accomplish this in GRASS simply because I can't gdal/GRASS to work resulting
from some sort of goofy circular dependencies).

We are using gdal/GRASS daily - what's the problem on which platform?

My Database is MySQL and my
stats package is R (which seems to have be best model for development I've
seen so far).

Yes, because R has x times so many developers.

These are two great packages and the development process is
solid almost militant in nature. MySQL releases a couple of times per year
as does R and it seems GRASS has been limping along in the meantime. We need
help. Isn't there an institution that promotes the development of GRASS
where people can work on it full time? It seems that the overworked
developers are spread pretty thin and the project needs a small army of
people pushing on.

So you are welcome to participate. There are plenty of things to do.

Okay, this seems a bit long winded so I'll shup now. And thanks for hearing
my pleas.

Thanks for the good work,
Jeff.

Markus

3

> : Jeff
  : Markus

------------

> As it stands now, I'm not sure where to contribute.

Fixing things, or at least complaining about things, you need as you
need them is a good start I think. Over time this gets the
non-structural problems taken care of, and I think most GRASS's bugs
(from the user's perspective) are non-structural.

> There are *at least* three code streams for GRASS (5.0, 5.3 and 5.7
> -- not to mention 4.3) This seems to be the opposite of fix and
> simplify.

I think at this point no one is directly working on 5.0, it's only old
business. So there are really two active branches. Actually only 1.5
branches really as 5.7 is only about 1.8 megabytes of code-- most of it
is automatically linked directly from the live 5.3 CVS and only exists
in one place. From the end user's perspective though, there is a tyranny
of choice.

Several developers prefer to work on 5.3 - even if things should have
been already merged into 5.7.

some feedback re. new business:
I don't really use vector or database support so using 5.7 isn't
critical for me. I do however need to use site data heavily, and the
simplicity & bash script-ability of the site_lists format (on the raw
ascii file level) makes me dependant on 5.3. If it wasn't for that I'd
use 5.7 in a minute. As I don't really work on the vector modules, any
improvements to the display or raster modules automatically get fed into
5.7 so I don't feel at all guilty about 'neglecting' it. I just have to
remember to update the man pages and ps.map and fight the urge to
clean up the sites modules.
None of this is structural work of course, just details.
I think the structural work for 5.3->5.7 is a full time job for someone.

Maybe I should look to see what I really need that couldn't be done
without anything more than s.in.ascii and s.out.ascii? Looking through
some scripts, not much actually! I seem to be somewhat addicted to
creating site_lists files manually with sed &/or Matlab though.

(s.univar was today's outside need, but that was to analyze some cell
stats on a raster map converted with r.to.sites; what a cludge! we need
a stats library! [again, I offer to help populate functions if someone
else lays down the framework; even though it should really be done by a
better coder/statistician than I])

> And none of them seem to include the capabilites I need.

They never will unless people (repetitively) explain what is missing &
what they need.

> In order for 5.7 to work, I need the code for 5.3. Why? Shouldn't
> all that new code simply go into 5.3?

The idea, as I understand it, is for the 5.3 code branch (which includes
all the old 5.0 and 4.3- legacy code) to be retired almost as soon as it
is released. It has features (ie datum support) too important to be put
off until the official 5.7.0 release, yet too radical to go into the
stable 5.0 series. 5.7 is the opportunity for a vast clean up and
simplification of the GRASS code base-- we only transfer to it what is
actually used.

While we wait for that, I don't think it's too much to do as an end user
to rebuild 5.7 locally 3 or 4 times a year. You can write a script to
automate it or at least use as cut-and-paste notes for next time.. it's
a chore, but not 'hard'.

> If I could get everything to compile on cygwin, I'd use it there.
> But they don't.

What doesn't? Is there a bug report? (does r.terraflow compile, btw?)

> There's just too many switches and options for a simple (for
> example)
>
> ./configure
> make --with-mysql
> make install

Most of the "switches" (that I have to use, anyway) are really paths
pointing to where libraries/includes are. It'll never compile on say BSD
or even on both the two most popular Linux distributions without such
hints.

> I've not seen a project with so much connection between code
> streams. Why?

Because there aren't enough people to sort out this problem.
If it helped, we could merge the code into 5.7. But then I expect
a major desaster: 5.3 will receive changes and some developers
regularly ignore to update the relevant files in 5.7 as well.
I try to hunt for those files, but it's not that easy.

Yes, it would be a huge pain to have to check in everything twice into
divergent code bases. The problem goes away as soon as 5.3 moves into
bug-fix-only mode though. It would be nice if we had a countdown to
5.3.0 todo-list on a webpage somewhere.

> Almost all of my work is with vectors and the attributes, lots of
> attributes of polygons (adjacency lists, unions, intersects, area
> calculations, and attributes -- think FRAGSTATS woulodn't it be nice
> to have a version for GRASS for people to build on).

So what's missing from 5.7?? I don't understand. What does 5.3 have that
5.7 is missing from for you? (besides simplicity of end-user building)

> For my
> projects, I'm constantly moving files between GRASS and Arc/Info and
> ArcView for projecting files across all sorts of projections, datums
> and coordinate systems and I have yet to be able to accomplish this
> in GRASS simply because I can't gdal/GRASS to work resulting from
> some sort of goofy circular dependencies).

So do you have problems with *both* PROJ.4 *and* GDAL/OGR then? I was
under the impression that the proj stuff was working really well in both
5.3 and 5.7...

What are the problems? Are there bug reports filed for them?
These are the sort of things that need to be fixed ASAP, IMO.

> We need help. Isn't there an institution that promotes the
> development of GRASS where people can work on it full time? It seems
> that the overworked developers are spread pretty thin and the
> project needs a small army of people pushing on.

If there are specific things you need fixed ASAP to make you more
productive so as to make your company more money, *especially* with
respect to PROJ.4 or GDAL, there are a number of people reading this
list who code for cash, probably for cheaper than an ESRI module, and it
might help break the log jam for others as well, which means more
support, which means more users, .. more support, etc.

> And thanks for hearing my pleas.

ditto

regards & US$0.013,
Hamish

4

On Tue, Jan 27, 2004 at 10:59:12PM +1300, Hamish wrote:

> > : Jeff
> : Markus
------------

[...]

As I don't really work on the vector modules, any
improvements to the display or raster modules automatically get fed into
5.7 so I don't feel at all guilty about 'neglecting' it.

This is actually not completely true.

E.g. (recent changes)

2004-01-24 02:53 <hidden>

        * XDRIVER/XDRIVER24/Graph_Set.c: Erase window at startup (prevents
        "transparent" window when starting without selecting).

... I had to merge. It's not clear to me why 5.7 is really *ignored*
by some developers. If 5.7 files (there are some due to necessary
changes) are not sync'ed, it will become more and more difficult to
maintain it. Why should we trace down bugs again if they are already
solved? A waste of time. Syncing these (few) files in 5.7 cannot be
so difficult. A bit frustrating at least for me.

[...]

> > And none of them seem to include the capabilites I need.

They never will unless people (repetitively) explain what is missing &
what they need.

Agreed.

[...]

> > There's just too many switches and options for a simple (for
> > example)
> >
> > ./configure
> > make --with-mysql
> > make install

Most of the "switches" (that I have to use, anyway) are really paths
pointing to where libraries/includes are. It'll never compile on say BSD
or even on both the two most popular Linux distributions without such
hints.

Hint: Here are configure scripts:

5.0:
http://grass.itc.it/grass5/source/conf_scripts/
cygwin_generic/ 09-May-2003 17:28 -
cygwin_xserver/ 09-May-2003 17:28 -
linux/ 09-May-2003 17:30 -

5.7:
http://grass.itc.it/grass51/source/conf_scripts/
cygwin_xserver/ 08-Jan-2004 15:15 -
freebsd/ 12-Jan-2004 14:25 -
linux/ 08-Jan-2004 15:12 -
macosx/ 08-Jan-2004 15:13 -

5.7 also includes a ./debian/ directory. Instructions here:
http://mpa.itc.it/markus/grass57/debian/grass57deb/

> > I've not seen a project with so much connection between code
> > streams. Why?
>
> Because there aren't enough people to sort out this problem.
> If it helped, we could merge the code into 5.7. But then I expect
> a major desaster: 5.3 will receive changes and some developers
> regularly ignore to update the relevant files in 5.7 as well.
> I try to hunt for those files, but it's not that easy.

Yes, it would be a huge pain to have to check in everything twice into
divergent code bases. The problem goes away as soon as 5.3 moves into
bug-fix-only mode though. It would be nice if we had a countdown to
5.3.0 todo-list on a webpage somewhere.

Such TODO list have been regularly ignored in the past.

[...]

If there are specific things you need fixed ASAP to make you more
productive so as to make your company more money, *especially* with
respect to PROJ.4 or GDAL, there are a number of people reading this
list who code for cash, probably for cheaper than an ESRI module, and it
might help break the log jam for others as well, which means more
support, which means more users, .. more support, etc.

Agreed.

[...]

Markus

5

Markus Neteler wrote:

> As I don't really work on the vector modules, any
> improvements to the display or raster modules automatically get fed into
> 5.7 so I don't feel at all guilty about 'neglecting' it.

This is actually not completely true.

E.g. (recent changes)

2004-01-24 02:53 <hidden>

        * XDRIVER/XDRIVER24/Graph_Set.c: Erase window at startup (prevents
        "transparent" window when starting without selecting).

... I had to merge. It's not clear to me why 5.7 is really *ignored*
by some developers.

Because it's a sufficiently radical departure from the existing code
base that it's effectively a separate project, and not everyone has
the time to work on two projects.

Maybe you should be asking whether it was really necessary fo 5.7 to
have its own version of Graph_set.c, given that the differences amount
to:

1. A different cursor.
2. The default background being white instead of black.

Does this really justify forking that file (and also Serve_Xevent.c,
solely for point 2)?

How many of the files which have been forked in 5.7 are really
necessary? I'm not counting cases where, all other things being equal,
the change would have been an improvement; because all other things
aren't equal.

Minimising divergence between the two code bases ought to carry *some*
weight (I'm not arguing how much weight, only that it should be
non-zero), and changes should only be made if they provide "enough"
benefit.

--
Glynn Clements <glynn.clements@virgin.net>

6

On Tue, Jan 27, 2004 at 01:15:29PM +0000, Glynn Clements wrote:

Markus Neteler wrote:

> > As I don't really work on the vector modules, any
> > improvements to the display or raster modules automatically get fed into
> > 5.7 so I don't feel at all guilty about 'neglecting' it.
>
> This is actually not completely true.
>
> E.g. (recent changes)
>
> 2004-01-24 02:53 <hidden>
>
> * XDRIVER/XDRIVER24/Graph_Set.c: Erase window at startup (prevents
> "transparent" window when starting without selecting).
>
> ... I had to merge. It's not clear to me why 5.7 is really *ignored*
> by some developers.

Because it's a sufficiently radical departure from the existing code
base that it's effectively a separate project, and not everyone has
the time to work on two projects.

Maybe you should be asking whether it was really necessary fo 5.7 to
have its own version of Graph_set.c, given that the differences amount
to:

1. A different cursor.
2. The default background being white instead of black.

Does this really justify forking that file (and also Serve_Xevent.c,
solely for point 2)?

In my opinion there is no fork needed. I would appreciate to
see the 5.7 changes in 5.3.

How many of the files which have been forked in 5.7 are really
necessary? I'm not counting cases where, all other things being equal,
the change would have been an improvement; because all other things
aren't equal.

Several (most?) of the forks in libes/gis/, XDRIVER etc. could be
merged from 5.7 into 5.3. But this requires that someone else than
me checks it.
E.g: If 5.3 users prefer the black X-monitor over the white X-monitor
     in 5.7, we cannot merge. The same applies to the improved cursor
     in 5.7.

We'll certainly not replace the improvements in 5.7 with old code
from 5.3.

Minimising divergence between the two code bases ought to carry *some*
weight (I'm not arguing how much weight, only that it should be
non-zero), and changes should only be made if they provide "enough"
benefit.

Minimising divergence requires that 5.3 developers look at 5.7.
Unless they aren't willing to do that, the minimization of divergences
will have to wait, unfortunately.

Markus

7

On Tue, Jan 27, 2004 at 01:15:29PM +0000, Glynn Clements wrote:

Markus Neteler wrote:

> ... I had to merge. It's not clear to me why 5.7 is really *ignored*
> by some developers.

Because it's a sufficiently radical departure from the existing code
base that it's effectively a separate project, and not everyone has
the time to work on two projects.

That is too strongly worded, I'd say.

The situation was that we wanted to have people
make radical changes without interrupting the growing
trust old users started to gain in GRASS 5 development.
So Radim started out in agreement with many people
and did a good job. In fact he did not get enough feedback on his work.

When I see how many people now start to test 5.7 once in a while,
I would call that strategy a success.

Maybe you should be asking whether it was really necessary fo 5.7 to
have its own version of Graph_set.c

Those technical detail questions and feedbak certainly
is one that Radim asked for many times.

Minimising divergence between the two code bases ought to carry *some*
weight (I'm not arguing how much weight, only that it should be
non-zero), and changes should only be made if they provide "enough"
benefit.

It already carries _some_ weight.
I trust the 5.7 developers to make reasonable judgements.
However if you find places that can be improved
I guess they'd like feedback. :slight_smile:
  
  Bernhard

8

Several (most?) of the forks in libes/gis/, XDRIVER etc. could be
merged from 5.7 into 5.3. But this requires that someone else than
me checks it.
E.g: If 5.3 users prefer the black X-monitor over the white X-monitor
     in 5.7, we cannot merge.

I don't mean to argue your point but I thought DEFAULT_FG_COLOR and
DEFAULT_BG_COLOR were introduced to both 5.3 & 5.7 to deal with this
specific case..?

e.g.
grass/src/display/d.leg.thin/main.c
Revision 1.21, Tue Oct 21 13:01:24 2003 UTC by markus
Branch: MAIN
Changes since 1.20: +3 -3 lines
Diff to previous 1.20

use of DEFAULT_FG_COLOR/DEFAULT_BG_COLOR for G5.7 compliance

Most of the 5.3->5.7 changes aren't as easy to mitigate of course.

Hamish

9

On Sun, Feb 01, 2004 at 12:29:33AM +1300, Hamish wrote:

> Several (most?) of the forks in libes/gis/, XDRIVER etc. could be
> merged from 5.7 into 5.3. But this requires that someone else than
> me checks it.
> E.g: If 5.3 users prefer the black X-monitor over the white X-monitor
> in 5.7, we cannot merge.

I don't mean to argue your point but I thought DEFAULT_FG_COLOR and
DEFAULT_BG_COLOR were introduced to both 5.3 & 5.7 to deal with this
specific case..?

Right. But XDRIVER also comes with an improved mouse cursor. If I recall
correctly, it's the same file.

e.g.
grass/src/display/d.leg.thin/main.c
Revision 1.21, Tue Oct 21 13:01:24 2003 UTC by markus
Branch: MAIN
Changes since 1.20: +3 -3 lines
Diff to previous 1.20

use of DEFAULT_FG_COLOR/DEFAULT_BG_COLOR for G5.7 compliance

A few display modules may still need that update, I didn't
check them all.

Most of the 5.3->5.7 changes aren't as easy to mitigate of course.

Sure, but that's not needed (at time). At least a minimization of
changes to be achieved by merging back above mentioned improvements.
But this requires that (some) 5.3 developers look at these selected
5.7 changes and merge them back. Then I'll clean 5.7 accordingly (or
they do it).

Markus

10

Hamish wrote:

> Several (most?) of the forks in libes/gis/, XDRIVER etc. could be
> merged from 5.7 into 5.3. But this requires that someone else than
> me checks it.
> E.g: If 5.3 users prefer the black X-monitor over the white X-monitor
> in 5.7, we cannot merge.

I don't mean to argue your point but I thought DEFAULT_FG_COLOR and
DEFAULT_BG_COLOR were introduced to both 5.3 & 5.7 to deal with this
specific case..?

e.g.
grass/src/display/d.leg.thin/main.c
Revision 1.21, Tue Oct 21 13:01:24 2003 UTC by markus
Branch: MAIN
Changes since 1.20: +3 -3 lines
Diff to previous 1.20

use of DEFAULT_FG_COLOR/DEFAULT_BG_COLOR for G5.7 compliance

1. Neither XDRIVER nor d.erase use these macros. d.erase could be
changed easily enough, but doing it for XDRIVER would be more work (we
would need to add code to parse a string to an R/G/B triple).

2. Such an option should probably be a run-time setting rather than a
compile-time one.

--
Glynn Clements <glynn.clements@virgin.net>

11

> use of DEFAULT_FG_COLOR/DEFAULT_BG_COLOR for G5.7 compliance

1. Neither XDRIVER nor d.erase use these macros. d.erase could be
changed easily enough, but doing it for XDRIVER would be more work (we
would need to add code to parse a string to an R/G/B triple).

d.erase is now updated, including support for color=R:G:B.
I've left XDRIVER alone as I'm not very familiar with it.

I didn't update grass51/display/d.erase/main.c, as it can now be removed??

Hamish

12

On Sun, Feb 01, 2004 at 08:18:30PM +1300, Hamish wrote:

> > use of DEFAULT_FG_COLOR/DEFAULT_BG_COLOR for G5.7 compliance
>
> 1. Neither XDRIVER nor d.erase use these macros. d.erase could be
> changed easily enough, but doing it for XDRIVER would be more work (we
> would need to add code to parse a string to an R/G/B triple).

d.erase is now updated, including support for color=R:G:B.

Nice!

I've left XDRIVER alone as I'm not very familiar with it.

Nor me unfortunately.

I didn't update grass51/display/d.erase/main.c, as it can now be removed??

If that's the only difference, please remove from 5.7.

Markus

13

Hamish wrote:

> > use of DEFAULT_FG_COLOR/DEFAULT_BG_COLOR for G5.7 compliance
>
> 1. Neither XDRIVER nor d.erase use these macros. d.erase could be
> changed easily enough, but doing it for XDRIVER would be more work (we
> would need to add code to parse a string to an R/G/B triple).

d.erase is now updated, including support for color=R:G:B.

Should Derase() be changing the driver's colour table?

I've left XDRIVER alone as I'm not very familiar with it.

I can add code to handle R:G:B easily enough, but the default settings
of DEFAULT_FG_COLOR and DEFAULT_BG_COLOR are "white" and "black".

Actually, I could add code to handle named colours easily enough (with
D_translate_color() plus the code which is already in the driver to
select standard colours). However:

1. Using D_translate_color() requires libdisplay, but I'm not sure
that display drivers should be using libdisplay (currently, they don't
even use libraster).

2. Implementing a D_translate_color() clone would require the two to
be kept in sync.

Basically, I'm wondering where the definitions of standard colours
(i.e. their names, indices and R/G/B values) should reside; maybe it
belongs in libraster or even libgis?

--
Glynn Clements <glynn.clements@virgin.net>

14

> > > use of DEFAULT_FG_COLOR/DEFAULT_BG_COLOR for G5.7 compliance
> >
> > 1. Neither XDRIVER nor d.erase use these macros. d.erase could be
> > changed easily enough,

...

> d.erase is now updated, including support for color=R:G:B.

Should Derase() be changing the driver's colour table?

Well, it isn't ideal, but I don't think it hurts anything. The part of
the table[*] that is modified is off the end of the chart (MAXCOLORS + 1)
..thus ~extending the color table more than "changing" it. If
something unrelated is calling that color number, it would be a bug
anyway..

[*] here I mean the GRASS standard colors' table, not anything a driver
uses.

src/include/colors.h seems extensible; I don't see anything that uses
MAXCOLORS to set the size of a memory array or anything.

That's not to say something funky won't go on after you call
R_reset_color(), I don't really understand the translation to the
drivers' color tables, but by the look of the comments in colors.h
it would be ok -- and it seems to work ok...

----
NB:
raster/r.out.png/pngfunc.h redefines it to 256 for no apparent reason.

src.contrib/CERL/SGI/panel/9.6/src/D.apps/cam.c redefines it to 4096,
then uses that for an alloc().

src.contrib/CERL/SGI/panel/9.6/src/D.apps/ep.c ditto.

src.contrib/GMSL/g3d/src3d/raster/r3.showdspf.openGL/togif.c =>
int[256].

but in those four cases MAXCOLORS isn't colors.h's one and should
probably be changed to something else anyway.
----

I'm happy to entertain better solutions, but AFAICT it works properly
now (including non-X drivers) and isn't evil.

?
Hamish

15

Hamish wrote:

> > > > use of DEFAULT_FG_COLOR/DEFAULT_BG_COLOR for G5.7 compliance
> > >
> > > 1. Neither XDRIVER nor d.erase use these macros. d.erase could be
> > > changed easily enough,
...
> > d.erase is now updated, including support for color=R:G:B.
>
> Should Derase() be changing the driver's colour table?

Well, it isn't ideal, but I don't think it hurts anything. The part of
the table[*] that is modified is off the end of the chart (MAXCOLORS + 1)
..thus ~extending the color table more than "changing" it. If
something unrelated is calling that color number, it would be a bug
anyway..

I think that you are confusing the two colour tables.

The standard colour table is initialised to the colours specified in
color.h, and cannot be changed. R_standard_color() selects a colour
from the standard colour table.

The other colour table is also initialised to the same set of colours.
However, the entries may be changed using R_reset_color() and
R_reset_colors(). R_color() selects colours from this table, as do the
R_raster* functions.

Note: on indexed-colour displays, the driver can be put into
"floating" colour mode (e.g. with "d.colormode float"). In that
situation, the colour table is effectively the hardware palette.

These settings persist, so one client may set the colour table, while
another client uses it. Having said that:

1. The d.* programs normally explicitly set any colours which they
will be using, so the prior state of the colour table is irrelevant in
those cases.

2. While there may be cases where the colour table is required to
remain unchanged between commands, it seems unlikely that it would be
necessary for the colour table to remain unchanged by Derase(). I
would assume that a sequence of "erase, set colour table, draw" would
be more likely than "set colour table, erase, draw".

IOW, the change is probably safe, although it isn't a foregone
conclusion.

[*] here I mean the GRASS standard colors' table, not anything a driver
uses.

src/include/colors.h seems extensible; I don't see anything that uses
MAXCOLORS to set the size of a memory array or anything.

That's not to say something funky won't go on after you call
R_reset_color(), I don't really understand the translation to the
drivers' color tables,

That's OK; the process is rather convoluted. I generally need to
refresh my memory whenever I do anything in that area.

NB:
raster/r.out.png/pngfunc.h redefines it to 256 for no apparent reason.

src.contrib/CERL/SGI/panel/9.6/src/D.apps/cam.c redefines it to 4096,
then uses that for an alloc().

src.contrib/CERL/SGI/panel/9.6/src/D.apps/ep.c ditto.

src.contrib/GMSL/g3d/src3d/raster/r3.showdspf.openGL/togif.c =>
int[256].

but in those four cases MAXCOLORS isn't colors.h's one

If they don't use colors.h, they aren't actually *redefining* it; it's
just defining another macro which happens to have the same name.
MAXCOLORS is a fairly generic name; I doubt that the authors of those
programs were even aware of the MAXCOLORS in colors.h.

and should probably be changed to something else anyway.

If anything, colors.h should change. The more widely-used a header is,
the greater the obligation for it not to use "common" names,
especially for preprocessor macros.

That applies even more to the colour names (RED, BLUE etc), and
especially to FLOAT. [I'm wondering if AQUA could eventually cause
problems on a Mac.]

--
Glynn Clements <glynn.clements@virgin.net>

16

> > > > > use of DEFAULT_FG_COLOR/DEFAULT_BG_COLOR for G5.7 compliance
> > > >
> > > > 1. Neither XDRIVER nor d.erase use these macros. d.erase could
> > > > be changed easily enough,
> ...
> > > d.erase is now updated, including support for color=R:G:B.
> >
> > Should Derase() be changing the driver's colour table?
>
> Well, it isn't ideal, but I don't think it hurts anything. The part
> of the table[*] that is modified is off the end of the chart
> (MAXCOLORS + 1)..thus ~extending the color table more than
> "changing" it. If something unrelated is calling that color number,
> it would be a bug anyway..

I think that you are confusing the two colour tables.

The standard colour table is initialised to the colours specified in
color.h, and cannot be changed. R_standard_color() selects a colour
from the standard colour table.

The other colour table is also initialised to the same set of colours.

ok,

However, the entries may be changed using R_reset_color() and
R_reset_colors(). R_color() selects colours from this table, as do the
R_raster* functions.

What I am concerned with is instead of resetting the 14th of 13 colors,
as intended, I've actually reset color 14 of say 256 or 65536, which
even if it still works will lead to further confusion in the future..

thanks for the insight,
Hamish

17

Hamish wrote:

> > > > > > use of DEFAULT_FG_COLOR/DEFAULT_BG_COLOR for G5.7 compliance
> > > > >
> > > > > 1. Neither XDRIVER nor d.erase use these macros. d.erase could
> > > > > be changed easily enough,
> > ...
> > > > d.erase is now updated, including support for color=R:G:B.
> > >
> > > Should Derase() be changing the driver's colour table?
> >
> > Well, it isn't ideal, but I don't think it hurts anything. The part
> > of the table[*] that is modified is off the end of the chart
> > (MAXCOLORS + 1)..thus ~extending the color table more than
> > "changing" it. If something unrelated is calling that color number,
> > it would be a bug anyway..
>
> I think that you are confusing the two colour tables.
>
> The standard colour table is initialised to the colours specified in
> color.h, and cannot be changed. R_standard_color() selects a colour
> from the standard colour table.
>
> The other colour table is also initialised to the same set of colours.

Also, I've just noticed that it is reset to match the standard colour
table by R_color_table_fixed(). Similarly, the hardware palette is
similarly reset by R_color_table_float(). These two functions are used
by d.colormode.

> However, the entries may be changed using R_reset_color() and
> R_reset_colors(). R_color() selects colours from this table, as do the
> R_raster* functions.

What I am concerned with is instead of resetting the 14th of 13 colors,
as intended,

Actually, the standard colour table has 15 entries (0 to 14); I have
no idea why, as only entries 1 to 13 are used, and there is no way for
clients to modify entries.

I've actually reset color 14 of say 256 or 65536,

Yep; although, that is better than resetting any of colours 1 to 13,
given the behaviour of R_color_table_{fixed,float}, and the comment in
the d.colormode manpage:

BUGS
       It is strongly recommended that the user erase the graph­
       ics monitor screen (e.g., by running d.erase) immediately
       after changing the mode between fixed and float.

Presumably there is a reason why, R_color_table_{fixed,float}
initialise colours 1 to 13, so we probably wouldn't want to modify any
of those on the next operation.

--
Glynn Clements <glynn.clements@virgin.net>

18

> > > > > d.erase is now updated, including support for color=R:G:B.
> > > > Should Derase() be changing the driver's colour table?

...

Also, I've just noticed that it is reset to match the standard colour
table by R_color_table_fixed(). Similarly, the hardware palette is
similarly reset by R_color_table_float(). These two functions are used
by d.colormode.

[...]

comment in the d.colormode manpage:
BUGS
       It is strongly recommended that the user erase the graph­
       ics monitor screen (e.g., by running d.erase) immediately
       after changing the mode between fixed and float.

Shall we change d.colormode to do this automatically?
Or at least have it issue a warning?

> > However, the entries may be changed using R_reset_color() and
> > R_reset_colors(). R_color() selects colours from this table, as do
> > the R_raster* functions.
>
> What I am concerned with is instead of resetting the 14th of 13
> colors, as intended,

Actually, the standard colour table has 15 entries (0 to 14); I have
no idea why, as only entries 1 to 13 are used, and there is no way for
clients to modify entries.

> I've actually reset color 14 of say 256 or 65536,

Yep; although, that is better than resetting any of colours 1 to 13,
given the behaviour of R_color_table_{fixed,float}, and the comment in
the d.colormode manpage

So for picking an arbitrary number, maybe 14 and 15 aren't so bad after
all, as it may not be a good idea to assume the color table will be >256,
e.g. for use on a 4-bit display. Better not to use 255 anyway.

(15 is used when there are both custom foreground and background colors)

... I've got a 1-bit NCD X-terminal here (ncolors=2), and d.erase works
correctly there: color=127:127:127 -> black, color=128:128:128 -> white.

Is the color table always going to be 256*256*256 long, or is it created
at run-time based on the color depth with colors pulled from 0-1,0-1,0-1
and translated on the fly?

shrug
I guess we should check for color blips on a smooth color transition
with d.legend or a map made with r.cost, but otherwise I guess I'll leave
it as is.

Hamish

19

Hamish wrote:

> > > > > > d.erase is now updated, including support for color=R:G:B.
> > > > > Should Derase() be changing the driver's colour table?
...
> Also, I've just noticed that it is reset to match the standard colour
> table by R_color_table_fixed(). Similarly, the hardware palette is
> similarly reset by R_color_table_float(). These two functions are used
> by d.colormode.
[...]
> comment in the d.colormode manpage:
> BUGS
> It is strongly recommended that the user erase the graph­
> ics monitor screen (e.g., by running d.erase) immediately
> after changing the mode between fixed and float.

Shall we change d.colormode to do this automatically?

No; that would mean that there was no way to change the mode without
erasing the screen.

Or at least have it issue a warning?

Probably not. Most users won't ever use d.colormode; presumably those
who do will know what they're doing.

In any case, I'm not sure how relevant d.colormode really is nowadays.
Have you tried using a 256-colour display recently? Anything more
fanciful than an xterm will probably start by popping up a pretty
splash screen which consumes the entire colourmap on the spot.

I'm not even sure that floating colour mode still works correctly; I
did as much testing as I could when I re-wrote XDRIVER's colour
handling (back in May 2001), but there may well be cases which have
been overlooked. I doubt that it gets much "real-world" testing these
days.

> > > However, the entries may be changed using R_reset_color() and
> > > R_reset_colors(). R_color() selects colours from this table, as do
> > > the R_raster* functions.
> >
> > What I am concerned with is instead of resetting the 14th of 13
> > colors, as intended,
>
> Actually, the standard colour table has 15 entries (0 to 14); I have
> no idea why, as only entries 1 to 13 are used, and there is no way for
> clients to modify entries.
>
> > I've actually reset color 14 of say 256 or 65536,
>
> Yep; although, that is better than resetting any of colours 1 to 13,
> given the behaviour of R_color_table_{fixed,float}, and the comment in
> the d.colormode manpage

So for picking an arbitrary number, maybe 14 and 15 aren't so bad after
all, as it may not be a good idea to assume the color table will be >256,
e.g. for use on a 4-bit display. Better not to use 255 anyway.

Well, XDRIVER itself should actually work on a 1-bit display, although
I'm not sure that you would be able to do anything useful with it.

BTW, in fixed colour mode, the number of colours in the ("other", as
opposed to "standard") colour table isn't limited by the display
depth. You could still define 1000 colours on a 1-bit display, but
they would all be either black or white (whichever is closest to the
specified R/G/B triple).

(15 is used when there are both custom foreground and background colors)

... I've got a 1-bit NCD X-terminal here (ncolors=2), and d.erase works
correctly there: color=127:127:127 -> black, color=128:128:128 -> white.

Is the color table always going to be 256*256*256 long, or is it created
at run-time based on the color depth with colors pulled from 0-1,0-1,0-1
and translated on the fly?

The standard colour table always has 15 entries, of which 1-13 are
explicitly set (0 and 14 will always be black).

The fixed colour table is unlimited[1] in both directions (negative
numbers are allowed).

The floating colour table is limited by the hardware palette (and is
only applicable on displays which actually have a hardware palette,
i.e. PseudoColor and GrayScale visuals).

BTW, there is also another level of colour translation within XDRIVER,
but that shouldn't be at all visible to clients.

[1] Well, it's limited by memory: drawing 24-bit rasters by setting a
16-million-entry palette definitely works; after I extended XDRIVER to
correctly report 2^24 colours on a 24-bit display, I had to change
D_set_colors() not to try to actually define that many colours, as
doing so results in XDRIVER using >64Mb of memory.

shrug
I guess we should check for color blips on a smooth color transition
with d.legend or a map made with r.cost, but otherwise I guess I'll leave
it as is.

On the subject of smooth colour transitions:

For integer rasters, if there are fewer categories than the number of
monitor colours (as reported by R_get_num_colors()), that number of
colours will be defined (using R_reset_colors()).

If there are more categories than the monitor has colours,
D_set_colors() allocates the largest colour cube which will fit within
the reported number of colours (up to a maximum of 32^3 (32768
colors), to limit the driver's memory usage).

For floating-point maps, G_color_range() reports a range of -255^3 to
255^3, which will always exceed the value reported by
R_get_num_colors(), and so will again result in the use of a colour
cube.

Given the nature of raster colour tables, a colour cube isn't really
an appropriate mechanism. It will typically result in colours being
allocated to regions of the colour cube which are never used, at the
expense of regions which are used. [Ironically, this means that
floating-point rasters will usually have worse colour resolution than
integer rasters.]

E.g. most of the standard colour tables (in the sense of "r.colors
color=...") only use highly saturated colours, which form a path along
the edges around the middle of the colour cube. Distributing the
available colours along the path corresponding to the colour table
(whether evenly, or biased according to the map's histogram) would
provide a much better fit.

--
Glynn Clements <glynn.clements@virgin.net>