[GRASSLIST:9441] Re: Rasters not displaying from old (Windows) Gr ass Projects

GRASS GRASS user
1

Just a guess, but rasters may need converting from DOS format (CR+LF at
ends of lines) to Unix (LF only). Try using -a as a parameter to unzip
in Linux, or running dos2unix on the raster files.

Chris

Chris, I tried the -a switch on gunzip, but that isn't supported on Ubuntu.

Glynn, the filesizes look fine - all rasters are several MB (normal sized).

Do I run dos2unix on every single file in every directory in my location? Or
could I tar the whole location and run that tarfile through dos2unix? I
tried that for fun, but I get an'Error exit delayed from previous errors'
when dos2unix runs on my tarfile. Does it make sense to run a dos -> unix
ascii convertor on Grass rasters? Aren't they in binary format?

Here's another clue - When to display any raster, I get the following error
message:

GRASS 6.1.cvs (Canso):~/Projects/Canso/PERMANENT > Monitor <x0>: Premature
EOF

I thought maybe I would try to tar the location, ftp it to another Linux
workstation running Grass 5.3, and see if I could display the rasters there.
All rasters display fine. So I have the situation where:

Cygwin Grass 6.1cvs ---> Ubuntu Grass 6.1cvs = rasters broken, vectors OK
Cygwin Grass 6.1cvs ---> Red Hat Grass 5.3 = rasters OK, 6.1 vectors don't
display on 5.3 (which is normal)

I really need a way of getting all my Cygwin work running OK on Ubuntu,
because I've passed the point of no return in terms of getting this data
worked up do this level - I definitely can't
do it all over again.

~ Eric.

2

Patton, Eric wrote:

>Just a guess, but rasters may need converting from DOS format (CR+LF at
>ends of lines) to Unix (LF only). Try using -a as a parameter to unzip
>in Linux, or running dos2unix on the raster files.

>Chris

Chris, I tried the -a switch on gunzip, but that isn't supported on Ubuntu.

Glynn, the filesizes look fine - all rasters are several MB (normal sized).

I meant to compare the exact file sizes. If the files have undergone
EOL conversion, the Linux versions will typically be a few bytes
smaller than the Windows versions.

Do I run dos2unix on every single file in every directory in my location? Or
could I tar the whole location and run that tarfile through dos2unix? I
tried that for fun, but I get an'Error exit delayed from previous errors'
when dos2unix runs on my tarfile. Does it make sense to run a dos -> unix
ascii convertor on Grass rasters? Aren't they in binary format?

They are in binary format. You definitely shouldn't be running
dos2unix on them. My concern is that the process used to transfer them
from Windows to Linux may have performed the translation implicitly.

I'm not sure exactly how the data was transferred, but there are
several tools which may do EOL conversions:

1. The Cygwin DLL does EOL conversion by default, unless either:

a) The application specifically opens the file in "binary" mode (the
O_BINARY flag, which is a Cygwin extension; Unix programs won't use
this flag unless they have been specifically "ported" to Cygwin).

b) The CYWGIN environment variable contains the "binmode" option.

c) The "device" from which the file was read was mounted in binary
mode. Cygwin's "mount" command will tell you whether a device is in
text or binary mode.

2. FTP transfers may do EOL conversion. An FTP session normally starts
in "text" mode (EOL conversions are performed on all transfers). With
a traditional Unix command-line client, you need to use the "binary"
command to select binary transfers (without EOL conversions).
Graphical clients usually have a a checkbox or radio buttons to choose
between text and binary transfers.

3. Archiving utilities (ZIP, RAR) may perform EOL conversions.

--
Glynn Clements <glynn@gclements.plus.com>

3

Dear Eric,

Since your rasters work in 5.3 there clearly isn't a problem with DOS format files: my guess was wrong. (Incidentally, zip/unzip are different from gzip/gunzip. They are available in Ubuntu.)

I would avoid using GRASS 6.1.cvs unless it is important to you to have some feature not available in a stable release. I am amazed how many GRASS users seem to use the development version which is much more likely to contain bugs. Can you manage with 6.0.1?

Chris

PS If you want to send me a tar or zip file of some of your maps I can check they load with 6.0.1.

Patton, Eric wrote:

Just a guess, but rasters may need converting from DOS format (CR+LF at
ends of lines) to Unix (LF only). Try using -a as a parameter to unzip
in Linux, or running dos2unix on the raster files.

Chris

Chris, I tried the -a switch on gunzip, but that isn't supported on Ubuntu.

Glynn, the filesizes look fine - all rasters are several MB (normal sized).

Do I run dos2unix on every single file in every directory in my location? Or
could I tar the whole location and run that tarfile through dos2unix? I
tried that for fun, but I get an'Error exit delayed from previous errors'
when dos2unix runs on my tarfile. Does it make sense to run a dos -> unix
ascii convertor on Grass rasters? Aren't they in binary format?

Here's another clue - When to display any raster, I get the following error
message:

GRASS 6.1.cvs (Canso):~/Projects/Canso/PERMANENT > Monitor <x0>: Premature
EOF

I thought maybe I would try to tar the location, ftp it to another Linux
workstation running Grass 5.3, and see if I could display the rasters there.
All rasters display fine. So I have the situation where:

Cygwin Grass 6.1cvs ---> Ubuntu Grass 6.1cvs = rasters broken, vectors OK
Cygwin Grass 6.1cvs ---> Red Hat Grass 5.3 = rasters OK, 6.1 vectors don't
display on 5.3 (which is normal)

I really need a way of getting all my Cygwin work running OK on Ubuntu,
because I've passed the point of no return in terms of getting this data
worked up do this level - I definitely can't do it all over again.

~ Eric.

4

Chris George wrote:

I would avoid using GRASS 6.1.cvs unless it is important to you to have
some feature not available in a stable release. I am amazed how many
GRASS users seem to use the development version which is much more
likely to contain bugs.

IMHO, the development version is no more or less likely to contain
bugs than a "stable" release.

It isn't as if the releases are the result of a concerted effort to
fix existing bugs without introducing new ones (as is the case for
some projects).

As development proceeds, old bugs get fixed and new bugs get
introduced. The overall level of reliability will remain largely
constant. Beyond that, newer versions will have features which older
versions lacked (other than the fact that some features were removed
in the 5.x -> 6.x transition).

--
Glynn Clements <glynn@gclements.plus.com>

5

Glynn Clements wrote:

IMHO, the development version is no more or less likely to contain
bugs than a "stable" release.

It isn't as if the releases are the result of a concerted effort to
fix existing bugs without introducing new ones (as is the case for
some projects).

In which case GRASS is an unusual project. I would hope that maintainers did make an effort to thoroughly test stable releases, and restrict the work to bug fixes and bringing the documentation up to date in the period leading up to such a release. Then users have a choice between having something reliable and being able to try the latest enhancements.

As development proceeds, old bugs get fixed and new bugs get
introduced. The overall level of reliability will remain largely
constant. Beyond that, newer versions will have features which older
versions lacked (other than the fact that some features were removed
in the 5.x -> 6.x transition).

So why have releases like 6.0.0 or 6.0.1? If all releases are equal in status they are all just the cvs version at a particular date.

If GRASS hopes to keep a body of conservative users, whose data and time are valuable, and for whom reliability is more important than the latest enhancements, it needs to provide them with a reliable baseline system.
More radical users, more willing to take risks, or more interested in experimenting, or who need a new feature, can take the risks if they wish.

Please note I am trying not to make any value judgement between my terms "conservative" and "radical". I merely want to stress that both kinds of user exist. (Even, at different times, within the same person.) I also think, if GRASS is (or is to be) successful the conservative group will be much the larger. There is a danger with open source projects that the enthusiastic few, who tend to be radicals, dominate and undervalue the needs of the conservatives.

Chris

6

On ¶ro, 2005-12-14 at 14:41 +0800, Chris George wrote:

Glynn Clements wrote:

> IMHO, the development version is no more or less likely to contain
> bugs than a "stable" release.
>
> It isn't as if the releases are the result of a concerted effort to
> fix existing bugs without introducing new ones (as is the case for
> some projects).

In which case GRASS is an unusual project. I would hope that
maintainers did make an effort to thoroughly test stable releases,

I wish it was that easy. The truth is that Grass has limited man power.
Developers are doing what they can, but they don't have that much time.
They, usually, write software because they need it, they use it and
publish as is for the other folks to use it. This is cool. They are not
paid for it, with some exceptions and excluding what they gain being
Grass whizzes. They fix reported bugs, improve documentation - mostly
just because they are kind enough to care. Or if the bug/bad feature has
impact on their own work.

Grass needs more testing, that's true. And more programmers to fix
existing bugs and bad features. This needs more people (comitted users).
Those people who committ to Grass can't do it all alone.

I'm permanently surprised with the fact so many people use Grass, but so
few of them care to even report bugs. The ususall attitude of Grass
users _I know_ is - "Ouch, a bug. I'm sure devs will fix it." And they
are surprised it is not when the new release comes out. I have really
found many bugs in Grass which simply I can't believe were never noticed
before I did (sure I have also found several bugs which are not bugs at
all but that's another story :wink: ).

Say, what can you do for your software? :slight_smile:

and
restrict the work to bug fixes and bringing the documentation up to date
  in the period leading up to such a release. Then users have a choice
between having something reliable and being able to try the latest
enhancements.

> As development proceeds, old bugs get fixed and new bugs get
> introduced. The overall level of reliability will remain largely
> constant. Beyond that, newer versions will have features which older
> versions lacked (other than the fact that some features were removed
> in the 5.x -> 6.x transition).

So why have releases like 6.0.0 or 6.0.1? If all releases are equal in
status they are all just the cvs version at a particular date.

It is not really _that_ bad. There is the most advanced, but also the
most bug prone line, which is now the 6.1, because changes happen only
there. In the same time it always has the most detected bugs fixed, but
also the most of new ones, obviously.

There is also the stable 6.0.x branch which originates from the last
major release, the 6.0. The latest 6.0.x release, 6.0.1, contains only
bug fixes and documentation fixes, ported from the 6.1 line. There won't
be new functionality in 6.0.x branch. Hopefully, 6.0.2 will be released
with more fixes ported from 6.1. However, it is true that until then
actually you would be on a safer side using 6.0.x cvs snapshots than the
6.0.1.

If GRASS hopes to keep a body of conservative users, whose data and time
are valuable, and for whom reliability is more important than the latest
enhancements, it needs to provide them with a reliable baseline system.

For me it looks like this is achieved. 6.0.x stable, 6.1 "unstable"
("" because I use it for production and feel pain when have to switch to
6.0.1 occasionaly on workstations maintained by my more conservative
colleague :slight_smile: ). Some day 6.1 will be released and become the stable
line, 7.0 will take it's place. Fair enough for me.

More radical users, more willing to take risks, or more interested in
experimenting, or who need a new feature, can take the risks if they wish.

Sure. But those brave ones are highly desired to test, and if they are
programmers, to help fixing bugs introduced within the development.
Other words - if you have a spare hour and care for Grass to be your GIS
software of choice, give 6.1 CVS a try and get back with any fixes, bug
reports or suggestions :). Same if you find any bug still breeding on
6.0.x.

Please note I am trying not to make any value judgement between my terms
"conservative" and "radical". I merely want to stress that both kinds
of user exist. (Even, at different times, within the same person.) I
also think, if GRASS is (or is to be) successful the conservative group
will be much the larger. There is a danger with open source projects
that the enthusiastic few, who tend to be radicals, dominate and
undervalue the needs of the conservatives.

Your's conservatively radical,
Maciek

7

On Wed, Dec 14, 2005 at 02:41:24PM +0800, Chris George wrote:

Glynn Clements wrote:

>IMHO, the development version is no more or less likely to contain
>bugs than a "stable" release.
>
>It isn't as if the releases are the result of a concerted effort to
>fix existing bugs without introducing new ones (as is the case for
>some projects).

In which case GRASS is an unusual project.

That's true. It *is* unusual. But IMHO in a good sense.

I would hope that
maintainers did make an effort to thoroughly test stable releases, and
restrict the work to bug fixes and bringing the documentation up to date
in the period leading up to such a release.

I think we are exactly trying to do that. You may want to take a look
at http://grass.itc.it/devel/grasscvscommit.php
-> Archive
    -> http://grass.itc.it/pipermail/grass-commit/2005-December/subject.html
       see all changes concerning "description.html"

Then users have a choice
between having something reliable and being able to try the latest
enhancements.

Which holds true for GRASS. Many new features of GRASS 6.1-CVS are
not backported to GRASS 6.0.x.

>As development proceeds, old bugs get fixed and new bugs get
>introduced. The overall level of reliability will remain largely
>constant. Beyond that, newer versions will have features which older
>versions lacked (other than the fact that some features were removed
>in the 5.x -> 6.x transition).

So why have releases like 6.0.0 or 6.0.1? If all releases are equal in
status they are all just the cvs version at a particular date.

Please read the announcement:

http://grass.itc.it/announces/announce_grass601.html
-> What's new in GRASS 6.0.1

(Almost) Everything is documented.

If GRASS hopes to keep a body of conservative users, whose data and time
are valuable, and for whom reliability is more important than the latest
enhancements, it needs to provide them with a reliable baseline system.
More radical users, more willing to take risks, or more interested in
experimenting, or who need a new feature, can take the risks if they wish.

Fully agreed. And in fact we have all versions available:
http://grass.itc.it/download/
# 6.1-CVS
# 6.0.1
# GRASS 5.7 [old] Download GRASS 5.7
# GRASS 5.3 [old] Download GRASS 5.3
# GRASS 5.0 [old] Download GRASS 5.0
# GRASS 4.3 [old] Download GRASS 4.3
# GRASS 4.2/4.2.1 [merged into 4.3, old]
# GRASS 4.1 [old] Download GRASS 4.1
# GRASS 4.0 [old] Download GRASS 4.0

I would be happy to receive also GRASS 3.0 from someone (unfortunately
I don't have a copy) to document the code evolution.

It is left to the user to decide which version s/he uses.

Please note I am trying not to make any value judgement between my terms
"conservative" and "radical". I merely want to stress that both kinds
of user exist. (Even, at different times, within the same person.)

And the development team tries to serve both groups by providing
stable and unstable releases. But AFAIK there is no software, even
when it is called "stable" that is bug free. Since GRASS is modular,
it's unlikely that the entire system is non-functional. But maybe
a certain functionality does not work as expected. In this case the
users are requested to report.

If you (would have) look(ed) into the CVS commit archive or the
ChangeLog files, you can find lots of "merged from head" comments:

http://www.google.com/search?q=merge+from+head&as_sitesearch=grass.itc.it
-> 318 hits

http://www.google.com/search?q=merged+from+head&as_sitesearch=grass.itc.it
-> 110 hits

All those are backports from the development version into the stable
version, to fix later known bugs. Subsequently .x releases are published
(6.0.2 for example in the near future).

I also think, if GRASS is (or is to be) successful the conservative group
will be much the larger. There is a danger with open source projects
that the enthusiastic few, who tend to be radicals, dominate and
undervalue the needs of the conservatives.

Chris

This is your opinion.

Markus

--
Markus Neteler <neteler itc it> http://mpa.itc.it
ITC-irst - Centro per la Ricerca Scientifica e Tecnologica
MPBA - Predictive Models for Biol. & Environ. Data Analysis
Via Sommarive, 18 - 38050 Povo (Trento), Italy

8

Markus,

Thanks for your detailed responses. I should say, first, that I am enormously impressed by the GRASS software and by the efforts of the people who continuously maintain it and provide so much help to other users.

What started my comments were the remarks from someone else to the effect that one might as well use the latest cvs version as the latest stable one. I expected that a "conservative" user should be advised to use the latest stable release. Your remarks have confirmed this, thanks.

Concerning another message in this thread, no, I have no evidence that the CVS version contains more bugs. I don't use the CVS version, so I could collect no such evidence. But I am only talking about degrees of risk. Most open-source projects adopt the same strategy of "stable" and "development" versions and will accept that the development one is more likely to contain bugs, especially if one means so far undetected ones. In absolute terms it might contains less, in that a discovered bug has been fixed in the development version but not yet backported. But in general use of the development version is more risky.

I should add that I did not intend that my remark "There is a danger with open source projects that the enthusiastic few, who tend to be radicals, dominate and undervalue the needs of the conservatives" should be taken as applying in a derogatory sense to GRASS. Again, the evidence below suggests on the contrary that the danger is consciously avoided.

Thanks again to the efforts of you and the rest of the GRASS team.

Chris

Markus Neteler wrote:

On Wed, Dec 14, 2005 at 02:41:24PM +0800, Chris George wrote:

Glynn Clements wrote:

IMHO, the development version is no more or less likely to contain
bugs than a "stable" release.

It isn't as if the releases are the result of a concerted effort to
fix existing bugs without introducing new ones (as is the case for
some projects).

In which case GRASS is an unusual project.

That's true. It *is* unusual. But IMHO in a good sense.

I would hope that maintainers did make an effort to thoroughly test stable releases, and restrict the work to bug fixes and bringing the documentation up to date in the period leading up to such a release.

I think we are exactly trying to do that. You may want to take a look
at http://grass.itc.it/devel/grasscvscommit.php
-> Archive -> http://grass.itc.it/pipermail/grass-commit/2005-December/subject.html
       see all changes concerning "description.html"

Then users have a choice between having something reliable and being able to try the latest enhancements.

Which holds true for GRASS. Many new features of GRASS 6.1-CVS are
not backported to GRASS 6.0.x.

As development proceeds, old bugs get fixed and new bugs get
introduced. The overall level of reliability will remain largely
constant. Beyond that, newer versions will have features which older
versions lacked (other than the fact that some features were removed
in the 5.x -> 6.x transition).

So why have releases like 6.0.0 or 6.0.1? If all releases are equal in status they are all just the cvs version at a particular date.

Please read the announcement:

http://grass.itc.it/announces/announce_grass601.html
-> What's new in GRASS 6.0.1

(Almost) Everything is documented.

If GRASS hopes to keep a body of conservative users, whose data and time are valuable, and for whom reliability is more important than the latest enhancements, it needs to provide them with a reliable baseline system.
More radical users, more willing to take risks, or more interested in experimenting, or who need a new feature, can take the risks if they wish.

Fully agreed. And in fact we have all versions available:
http://grass.itc.it/download/
# 6.1-CVS
# 6.0.1 # GRASS 5.7 [old] Download GRASS 5.7
# GRASS 5.3 [old] Download GRASS 5.3
# GRASS 5.0 [old] Download GRASS 5.0
# GRASS 4.3 [old] Download GRASS 4.3
# GRASS 4.2/4.2.1 [merged into 4.3, old]
# GRASS 4.1 [old] Download GRASS 4.1
# GRASS 4.0 [old] Download GRASS 4.0

I would be happy to receive also GRASS 3.0 from someone (unfortunately
I don't have a copy) to document the code evolution.

It is left to the user to decide which version s/he uses.

Please note I am trying not to make any value judgement between my terms "conservative" and "radical". I merely want to stress that both kinds of user exist. (Even, at different times, within the same person.)

And the development team tries to serve both groups by providing
stable and unstable releases. But AFAIK there is no software, even
when it is called "stable" that is bug free. Since GRASS is modular,
it's unlikely that the entire system is non-functional. But maybe
a certain functionality does not work as expected. In this case the
users are requested to report.

If you (would have) look(ed) into the CVS commit archive or the
ChangeLog files, you can find lots of "merged from head" comments:

http://www.google.com/search?q=merge+from+head&as_sitesearch=grass.itc.it
-> 318 hits

http://www.google.com/search?q=merged+from+head&as_sitesearch=grass.itc.it
-> 110 hits

All those are backports from the development version into the stable
version, to fix later known bugs. Subsequently .x releases are published
(6.0.2 for example in the near future).

I also think, if GRASS is (or is to be) successful the conservative group will be much the larger. There is a danger with open source projects that the enthusiastic few, who tend to be radicals, dominate and undervalue the needs of the conservatives.

Chris

This is your opinion.

Markus

9

> Fully agreed. And in fact we have all versions available:
> http://grass.itc.it/download/
> # 6.1-CVS
> # 6.0.1
> # GRASS 5.7 [old] Download GRASS 5.7

GRASS 5.4.0 ! (legacy stable)

> # GRASS 5.3 [old] Download GRASS 5.3
> # GRASS 5.0 [old] Download GRASS 5.0
> # GRASS 4.3 [old] Download GRASS 4.3
> # GRASS 4.2/4.2.1 [merged into 4.3, old]
> # GRASS 4.1 [old] Download GRASS 4.1
> # GRASS 4.0 [old] Download GRASS 4.0

10

On Sat, Dec 17, 2005 at 12:08:24AM +1300, Hamish wrote:

> > Fully agreed. And in fact we have all versions available:
> > http://grass.itc.it/download/
> > # 6.1-CVS
> > # 6.0.1
> > # GRASS 5.7 [old] Download GRASS 5.7

GRASS 5.4.0 ! (legacy stable)

Right. Added.

> > # GRASS 5.3 [old] Download GRASS 5.3
> > # GRASS 5.0 [old] Download GRASS 5.0
> > # GRASS 4.3 [old] Download GRASS 4.3
> > # GRASS 4.2/4.2.1 [merged into 4.3, old]
> > # GRASS 4.1 [old] Download GRASS 4.1
> > # GRASS 4.0 [old] Download GRASS 4.0

11

Chris George wrote:

Thanks for your detailed responses. I should say, first, that I am
enormously impressed by the GRASS software and by the efforts of the
people who continuously maintain it and provide so much help to other
users.

What started my comments were the remarks from someone else to the
effect that one might as well use the latest cvs version as the latest
stable one. I expected that a "conservative" user should be advised to
  use the latest stable release. Your remarks have confirmed this, thanks.

Concerning another message in this thread, no, I have no evidence that
the CVS version contains more bugs. I don't use the CVS version, so I
could collect no such evidence. But I am only talking about degrees of
risk. Most open-source projects adopt the same strategy of "stable" and
"development" versions and will accept that the development one is more
likely to contain bugs, especially if one means so far undetected ones.
  In absolute terms it might contains less, in that a discovered bug has
been fixed in the development version but not yet backported. But in
general use of the development version is more risky.

A few things to bear in mind regarding the nature of bugs and
debugging:

1. The size of the GRASS code base is huge. By some measures, 5.0.x
was on a par with XFree86 (6.x has had a lot of unmaintained code
removed), but both the user base and developer base are much smaller.
Bugs can lurk for years without being discovered (I've found a couple
of bugs in 6.x which have existed since 4.x, and quite a few which
were introduced in the 4.x -> 5.0 transition).

2. A lot of the heaviest or most demanding users use the CVS mainline
because of the additional features, so it potentially gets more
thorough testing than the stable releases.

3. Bugs are often introduced in bursts of active development then
removed on a more continuous basis. For any given module, the version
from just before a substantial update may be the most reliable.

--
Glynn Clements <glynn@gclements.plus.com>