[GRASS5] GRASS beta11 and LZW...

Eric_G_Miller · January 18, 2001, 4:27am

I know I'm becoming a regular pest about this...

Since the "beta11" (which was originally going to be simply a "make" fix
for "beta10") has been substantially longer in coming and includes a
number of changes, I request that we re-revert to using the new libz FP
raster compression for this release.

Additional Reasoning:

1) Hopefully this will be the *last* beta before we publish and announce
GRASS 5.0. This *required* change, should see some more testing.
Initial testing has shown it works fine as a drop-in replacement.

2) We have to get that patented stuff out of GRASS. Institutions such as
Baylor, Hannover and Intevation have a slight liability risk in hosting
this code (though I suspect we're well below UNISYS's radar, and I'm
not sure if Germany recognizes this patent???). Nonetheless,
distribution of GRASS would surely be hampered by inclusion of patented
algorithms.

Okay, I've said my bit...

--
Eric G. Miller <egm2@jps.net>

----------------------------------------
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo@geog.uni-hannover.de with
subject 'unsubscribe grass5'

Jan-Oliver_Wagner1 · January 18, 2001, 6:51am

On Wed, Jan 17, 2001 at 08:27:43PM -0800, Eric G . Miller wrote:

I know I'm becoming a regular pest about this...

this special pest wipes out evil, so its good

Since the "beta11" (which was originally going to be simply a "make" fix
for "beta10") has been substantially longer in coming and includes a
number of changes, I request that we re-revert to using the new libz FP
raster compression for this release.

I strongly vote for dropping lzw right now and not switch back
to it. 5.0 must be released without, because 5.0 will be broadly
recognized when announced.

Jan

--
Jan-Oliver Wagner http://intevation.de/~jan/

Intevation GmbH http://intevation.de/
FreeGIS http://freegis.org/

----------------------------------------
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo@geog.uni-hannover.de with
subject 'unsubscribe grass5'

Bernhard_Reiter · January 18, 2001, 11:31am

On Wed, Jan 17, 2001 at 08:27:43PM -0800, Eric G . Miller wrote:

Since the "beta11" (which was originally going to be simply a "make" fix
for "beta10") has been substantially longer in coming and includes a
number of changes, I request that we re-revert to using the new libz FP
raster compression for this release.

I agree!

2) We have to get that patented stuff out of GRASS. Institutions such as
Baylor, Hannover and Intevation have a slight liability risk in hosting
this code (though I suspect we're well below UNISYS's radar, and I'm
not sure if Germany recognizes this patent???).

I do not think Germany recognizes it.
( BTW: I am part of the FFII and the eurolinux Alliance to fight software
patents in Europe. Some already exist, but some only have a half
solid legal status.)

Bernhard

--
Professional Service around Free Software (intevation.net)
The FreeGIS Project (freegis.org)
Association for a Free Informational Infrastructure (ffii.org)
FSF Europe (www.fsfeurope.org)

neteler · January 18, 2001, 4:50pm

Hi all,

it's done

On Wed, Jan 17, 2001 at 08:27:43PM -0800, Eric G . Miller wrote:

I know I'm becoming a regular pest about this...

It was the last time

Since the "beta11" (which was originally going to be simply a "make" fix
for "beta10") has been substantially longer in coming and includes a
number of changes, I request that we re-revert to using the new libz FP
raster compression for this release.

[...]

I have deactivated LZW here in CVS:
src/libes/g3d/fpcompress.c
src/libes/gis/put_row.c
src/libes/gis/get_row.c

This requires a full re-compile of GRASS.

But wait!!!
1. First, you have to decompress all raster files.
   Use the script here:
   Bereich Geographie – Naturwissenschaftliche Fakultät – Leibniz Universität Hannover
     fp_uncompress.sh

Perform this job on *all* locations. Ok, it's boring but required.

2. Then re-compile GRASS to activate flate/zlib compression.

3. re-compress all raster maps:
Bereich Geographie – Naturwissenschaftliche Fakultät – Leibniz Universität Hannover
fp_recompress.sh

Other option: Use the r.lzw2z written by Eric Miller:
http://www.jps.net/egm2/r.lzw2z.tgz

(Eric, please update the web server not to display this file but to
open the "save file" dialog, otherwise only lynx can download it
lynx -source http://www.jps.net/egm2/r.lzw2z.tgz > r.lzw2z.tgz
- a little bit inconvenient for thousand of users interested in r.lzw2z...
Or please rename to r.lzw2z.tar.gz)

r.lzw2z: the data is read/decoded/recoded to a temp file, then reread
and if successful transferred back to the original.

Generally I think it would be o.k. to add it to GRASS since it does the
removal of LZW. But I am not a laywer.

Our motto: Welcome to the patent-free GRASS 5 beta11.

Regards

Markus

----------------------------------------
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo@geog.uni-hannover.de with
subject 'unsubscribe grass5'

neteler · January 18, 2001, 5:14pm

Hi again,

On Thu, Jan 18, 2001 at 07:51:20AM +0100, Jan-Oliver Wagner wrote:

On Wed, Jan 17, 2001 at 08:27:43PM -0800, Eric G . Miller wrote:
> Since the "beta11" (which was originally going to be simply a "make" fix
> for "beta10") has been substantially longer in coming and includes a
> number of changes, I request that we re-revert to using the new libz FP
> raster compression for this release.

I strongly vote for dropping lzw right now and not switch back
to it. 5.0 must be released without, because 5.0 will be broadly
recognized when announced.

Eric,

do you agree that we keep the commented #ifdef LZW for beta11, wait for
users' responsed, then remove the LZW code fully for 5.0stable?

I am sure now that beta11 will be the last release before stable!

Markus

----------------------------------------
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo@geog.uni-hannover.de with
subject 'unsubscribe grass5'

Eric_G_Miller · January 19, 2001, 2:00am

On Thu, Jan 18, 2001 at 05:14:21PM +0000, Markus Neteler wrote:

Eric,

do you agree that we keep the commented #ifdef LZW for beta11, wait for
users' responsed, then remove the LZW code fully for 5.0stable?

Yes, fine. It will be trivially easy to remove before 5.0 stable for
most of GRASS. The g3d libs and modules have LZW stuff spread out in
many more places. Mostly, it will be a matter of finding all of these
and renaming them as appropriate (mostly #define's or module options).

--
Eric G. Miller <egm2@jps.net>

----------------------------------------
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo@geog.uni-hannover.de with
subject 'unsubscribe grass5'

Eric_G_Miller · January 19, 2001, 11:26am

On Thu, Jan 18, 2001 at 04:50:33PM +0000, Markus Neteler wrote:

Other option: Use the r.lzw2z written by Eric Miller:
http://www.jps.net/egm2/r.lzw2z.tgz

(Eric, please update the web server not to display this file but to
open the "save file" dialog, otherwise only lynx can download it
lynx -source http://www.jps.net/egm2/r.lzw2z.tgz > r.lzw2z.tgz
- a little bit inconvenient for thousand of users interested in r.lzw2z...
Or please rename to r.lzw2z.tar.gz)

r.lzw2z: the data is read/decoded/recoded to a temp file, then reread
and if successful transferred back to the original.

Generally I think it would be o.k. to add it to GRASS since it does the
removal of LZW. But I am not a laywer.

Our motto: Welcome to the patent-free GRASS 5 beta11.

<g>

Okay, Markus I made the tarball r.lzw2z.tar.gz (I have no control over
the server's Mime/Types, Shift-Left Button prompts for download in
Netscrape!). I've also written a little GRASS style help
file (which is included in the new tarball). The helpfile with link
to the source tarball is:

http://www.jps.net/egm2/r.lzw2z/r.lzw2z.html

or grab the tarball directly with:

wget http://www.jps.net/egm2/r.lzw2z/r.lzw2z.tar.gz

Maybe some people want to mirror this should I not be able to keep it
posted for whatever reason?

--
Eric G. Miller <egm2@jps.net>

----------------------------------------
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo@geog.uni-hannover.de with
subject 'unsubscribe grass5'

neteler · January 19, 2001, 2:05pm

Hi Eric,

On Fri, Jan 19, 2001 at 03:26:01AM -0800, Eric G . Miller wrote:

On Thu, Jan 18, 2001 at 04:50:33PM +0000, Markus Neteler wrote:

[...]

Okay, Markus I made the tarball r.lzw2z.tar.gz (I have no control over
the server's Mime/Types, Shift-Left Button prompts for download in
Netscrape!). I've also written a little GRASS style help
file (which is included in the new tarball). The helpfile with link
to the source tarball is:

http://www.jps.net/egm2/r.lzw2z/r.lzw2z.html

or grab the tarball directly with:

wget http://www.jps.net/egm2/r.lzw2z/r.lzw2z.tar.gz

Both opens well! Thanks for uploading the new doc.
I have added the link to:
http://www.geog.uni-hannover.de/grass/announces/announce_lzw_removal.html

Maybe some people want to mirror this should I not be able to keep it
posted for whatever reason?

Well, shall I take it over here? To get it into the web site mirror-system?
Would be no problem for me.

I suggest to post above announcement when (not if) beta11 is out.
This night David fixed v.in.shape - it doesn't even crash with our
worst-case EDBS/SHAPE maps (congrats and many thanks, David!!)

This morning I have added s.in.dbf developed from pg.in.dbf.
As it was a pain to get a mixed types (int, strings etc) table into
GRASS I have taken the liberty of adding it. Feel free to improve...

Comment on r.in.tiff: Perhaps we write a simple wrapper script to
r.in.gdal and don't use the r.in.tiff code furtherly?

I propose to get out beta11 now.

Markus

----------------------------------------
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo@geog.uni-hannover.de with
subject 'unsubscribe grass5'

neteler · January 19, 2001, 9:06pm

Hi all,

currently I am facing several problems with site lists.
Different lists are not accepted with different sites modules.
It seems to be a weak point at time (can't remember that is
was so weak).

Note: The s.in.dbf I have updated again to write the variable
types somewhat ordered (first int/float, then strings). But
this doesn't change the behaviour of some modules.

Some examples:

1. A map imported with s.in.shape. It is a single line map for this
example:

site_lists/punkte

d.sites punkte
ERROR: ebuf 3569150.00000005 nbuf 5772499.99979246

However: s.info accepts.
s.info punkte
WARNING: Invalid absolute datetime format
NDIM=2, RTYPE = 0, NSTR=2, NDEC=5
Total sites read: 1
Total sites outside region: 0
punkte read.

2. Large tables drive s.info to sleep:
s.info sitesmap2
NDIM=2, RTYPE = -1, NSTR=11, NDEC=29
[nothing happens, CPU=100%, debug output says, that the file is not
fully read, however usually a "Bad format" error would occur]

d.sites sitesmap2
ERROR: ebuf 3570018 nbuf 5766419

s.univar sitesmap2
number of points 0

3. s.proj on sitesmap2
projects 1/3 of this map (result is written), then goto sleep as well.

This is somewhat unusual and confusing. I would be glad if someone of
you
- could experiment a bit with large tables, too
- try the s.in.dbf (doesn't use G_sites functions yet, but the format
should be o.k.)

4. small tables with different types of variables imported with
s.in.dbf are *no* problem. Only large ones...

The line length is below the 4k limit.
I know that the sites format is not a spreadsheet/dbms replacement.
But it should work at least slowly

Ideas would be welcome!

Markus

----------------------------------------
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo@geog.uni-hannover.de with
subject 'unsubscribe grass5'