[GRASS5] Removing LZW

Okay,

I've done a small test of new code to use deflate compression for
floating point rasters versus LZW and it seems to work. The reason, of
course, is to get that patented algorithm out of GRASS. I have a
utility that will do the conversion of LZW rasters. It needs a little
cleaning up, but otherwise seems to work correctly. I haven't yet
looked at the g3d stuff, however, I imagine we can just remove the LZW
option alltogether. The docs seem to indicate it is not used by
default, and doesn't buy much after RLE encoding.

So, the big question I need to ask is. are we ready to make this drastic
change to the GRASS source? It's a fundamental change, and will cause
breakage for almost everything, not to mention all users will have to
run the r.lzw2z module on *all* compressed floating point rasters,
before they will be readable again (backups are strongly recommended).

Arguments PRO:

o LZW is patented, and must go unless UNISYS gives us special
dispensation.

o Better to make this change before GRASS 5 goes "stable", since it
will reduce the number of affected users if the change is made later.

o CELL rasters are never LZW compressed (so shouldn't cause problems
for people upgrading from 4.x who haven't used the betas).

o others ???

Arguments CON:

o Will possibly destabilize GRASS code base for a few weeks as kinks are
worked out.

o Burden of "translating" rasters from LZW to Deflate.

o others ???

I think we can wait until after beta9 goes out, this will give me time
to address the g3d stuff as well as clean up the r.lzw2z utility.

Since it is something of an all or nothing situation, I think it's best
to get consensus and when exactly this should be done, and then notify
everyone of this change (so they can act appropriately).

Thanks,
--
Eric G. Miller <egm2@jps.net>

----------------------------------------
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo@geog.uni-hannover.de with
subject 'unsubscribe grass5'

Well nobody seems interested, so I've uploaded versions of get_row.c and
put_row.c that *can* use the new G_zlib_*() compression routines in
flate.c. By default, they still use the G_lzw_*() compression routines.

If anyone wants to help me test these routines, you should follow these
instructions.

1) DO NOT test on data you aren't willing to have destroyed! If the
routines screw up, there may be no way to recover your data.

2) These routines only affect compressed floating point rasters. Such
rasters will have to be in a format readable after recompilation (see
below).
  
  a) For the data set you are willing to sacrifice, run the r.compress
     command on it, with the '-u' flag, using a version of GRASS that
     still uses LZW compression.
     
     or
     
  b) I have written a utility called r.lzw2z that will translate
  compressed floating point rasters from LZW to DEFLATE. It is
  only lightly tested. The source is available here:
  http://www.jps.net/egm2/r.lzw2z.tgz

3) Modify the files "put_row.c" and "get_row.c" in grass/src/libes/gis
   by commenting out '#define USE_LZW_COMPRESSION'.

4) Edit grass/src/libes/gis/Gmakefile, adding an entry for "flate.o".

5) Edit grass/src/CMD/generic/make.mid adding either "-lz" or "$(ZLIB)"
   behind "GISLIB = $(LIBDIR)/libgis.a". This is needed because all
   modules linking to GISLIB will now have to link to libz.

6) Recompile the libgis.a

7) At the top src dir, run "make install" again to relink all modules
   to new libgis.a. (Note, there may be some modules which this will
   have no effect on and they must be completely recompiled).

8) Test away!

If you receive error such as "ERROR: unable to read row[n]!", first make
sure the module you are using has been recompiled completely. If there
is still a problem, then let me know.

I appreciate all the help I can get testing this out. It is very
important it is well tested before we axe LZW compression. I have yet
to test out similar configuration with G3d files (though, the functions
to do so are in place).

Cheers,
   
--
Eric G. Miller <egm2@jps.net>

----------------------------------------
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo@geog.uni-hannover.de with
subject 'unsubscribe grass5'

Well nobody seems interested, so I've uploaded versions of get_row.c and
put_row.c that *can* use the new G_zlib_*() compression routines in
flate.c. By default, they still use the G_lzw_*() compression routines.

Eric,

I for one am interested, however I am at lost at what I can do to help.

I think it would be a shame to remove the LZW compression.

Is there anyway that I may help? Shall I contact the patent holders? It seems that they should make an exception for open-source software, after all they did create a _STANDARD_.

Best,
--
Jeshua Lacock
Cartographer/Owner
http://SierraMaps.com
http://3dTopoMaps.com
Telephone: (760) 935-4481

---------------------------------------- If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo@geog.uni-hannover.de with
subject 'unsubscribe grass5'

On Tue, Nov 28, 2000 at 01:33:47AM -0800, Jeshua Lacock wrote:

>Well nobody seems interested, so I've uploaded versions of get_row.c and
>put_row.c that *can* use the new G_zlib_*() compression routines in
>flate.c. By default, they still use the G_lzw_*() compression routines.

Erik,
thanks for your efforts, it is apprecitated.
However I am in no position to help you testing right now.

I think it would be a shame to remove the LZW compression.

Is there anyway that I may help? Shall I contact the patent holders?
It seems that they should make an exception for open-source software,
after all they did create a _STANDARD_.

Believe me, this is useless.
Really giving an exception to Free Software would be equal to
completly give up right to the patent, which Unisys will never do.
They are known for not even giving exceptions to hobby proprietory
software.

Regards,
  Bernhard

--
Professional Service around Free Software (intevation.net)
The FreeGIS Project (freegis.org)
Association for a Free Informational Infrastructure (ffii.org)
FSF Europe (www.fsfeurope.org)