[GRASS-user] GRASS export GeoTiff adventure :-)

GRASS GRASS user
,
1

Many questions... Thank you in advance, Nikos :slight_smile:

Importing RGB tiff's (size: 2500, 2500 --- with INTERLEAVE=PIXEL) in
GRASS' database splits them, as expected, in red, green and blue raster
maps. After MASKing out unwanted areas, how should one export the maps
to end-up with tiff files similar (or improved) to the original ones?

1. Using r.composite and exporting with r.out.gdal gives "white" as
nodata. Whenever I set "nodata=0" I don't get a viewable photo

2. r.out.tiff does not provide a "nodata" option. What's the necessity
of r.out.tiff if r.out.gdal can do the same with more options?

3. GeoTiff metadata are lost always (?) while exporting with
r.out.gdal.

4. The createopt="INTERLEAVE=PIXEL" is not respected by r.out.gdal (?)

e.g.
r.out.gdal in=koblenz_2618_5566.red_i type=Byte
out=/home/nik/grassdb/rlp_fgaps/data/orthophotos/koblenz/test/koblenz_2618_5566.red_i2 createopt="INTERLEAVE=PIXEL,TFW=YES"

gdalinfo koblenz_2618_5566.red_i2
[...]
INTERLEAVE=BAND
[...]

5. What is the correct way to import an rgb photo, process, export back
in GeoTiff?

6. Is it better to export the red, green and blue GRASS raster maps
one-by-one and merge them with gdal_translate (or gdal_merge.py)?

7. Exporting a group (the ones that automatically are created for each
imported photo and contain the red, green and blue components
respectively), gives "INTERLEAVE=PIXEL" but "Block=2500x3". Is it
correct to force createopt="BLOCKYSIZE=1" which gives "Block=2500x1"?

8. I read in Markus' blog [1] how to "write out a GRASS raster map to
GeoTIFF -- this format
includes the coordinates within the file's metadata" using
gdal_translate.
Doesn't r.out.gdal do that and if not why?

There is also that tiles provide "Fast image display...". From my small
experience, I can't really judge if TILES accelerate image display.
Sometimes I have the impression that they are slower that STRIPPED
tiffs.

8. There are several compression methods? Does compression affect
significantly reading/displaying an image?

9. Should one prefer another file format?

[1]
http://gfoss.blogspot.com/2008/06/gdal-raster-data-tips-and-tricks.html

2

Nikos Alexandris wrote:

1. Using r.composite and exporting with r.out.gdal gives "white" as
nodata. Whenever I set "nodata=0" I don't get a viewable photo
  

Weird, it might have something to do with exporting a composite, it should work on single bands and groups.

3. GeoTiff metadata are lost always (?) while exporting with
r.out.gdal.
  

Also when checking with gdalinfo?

4. The createopt="INTERLEAVE=PIXEL" is not respected by r.out.gdal (?)
  

I would not change the default settings of gdal until I have good reason to do so.

6. Is it better to export the red, green and blue GRASS raster maps
one-by-one and merge them with gdal_translate (or gdal_merge.py)?
  

If I want a RGB GeoTIFF, I rather export the group.

7. Exporting a group (the ones that automatically are created for each
imported photo and contain the red, green and blue components
respectively), gives "INTERLEAVE=PIXEL" but "Block=2500x3". Is it
correct to force createopt="BLOCKYSIZE=1" which gives "Block=2500x1"?
  

BLOCKXSIZE and BLOCKYSIZE affect the tile size, as far as I understand the gdal documentation. I would not change the defaults. Leaving BLOCKXSIZE on default = 256 and setting BLOCKYSIZE to 1 should produce tiles of size 256x1.

8. There are several compression methods? Does compression affect
significantly reading/displaying an image?
  

I would stay away from tiff internal compression because other software might not support it (LZW and DEFLATE are sometimes not supported). If I want to distribute the tiff, I rather create an archive and send that.

9. Should one prefer another file format?
  

GeoTIFF is widely accepted, but if I want to provide a raster for a particular application, gdal might support the native raster format of that application and then this would be the preferred format.

TFW=YES would only be needed if you export a raster for an application that can not read GeoTIFF internal metadata. In this case I would rather use the native format of that application. Many spatial data viewer make use of gdal and should therefore read GeoTIFF properly.

Maybe there is a problem with your gdal installation? I would first check that simple single band export with default settings works before trying to fix the gdal installation.

I hope that helps,

Markus

3

Markus,
thank you for your precious time. See below some comments.

On Sat, 2008-08-02 at 15:53 +0200, Markus Metz wrote:

Nikos Alexandris wrote:
> 1. Using r.composite and exporting with r.out.gdal gives "white" as
> nodata. Whenever I set "nodata=0" I don't get a viewable photo
>
Weird, it might have something to do with exporting a composite, it
should work on single bands and groups.
> 3. GeoTiff metadata are lost always (?) while exporting with
> r.out.gdal.
>
Also when checking with gdalinfo?

No, I was talking about a known error [1]. How important is this error?

> 4. The createopt="INTERLEAVE=PIXEL" is not respected by r.out.gdal (?)
>
I would not change the default settings of gdal until I have good reason
to do so.

Well, the original photos are "INTERLEAVE=PIXEL" and I suppose, from
what I read, it's a requirement for "limited" viewers (!) [2][3].
Exporting single maps gives "INTERLEAVE=BAND"!

> 6. Is it better to export the red, green and blue GRASS raster maps
> one-by-one and merge them with gdal_translate (or gdal_merge.py)?
>
If I want a RGB GeoTIFF, I rather export the group.

What about the error message "ERROR 6: SetColorTable() not supported for
multi-sample TIFF files." which appears 3 times for a group that
contains 3 maps (red, green and blue in our case)?

> 7. Exporting a group (the ones that automatically are created for each
> imported photo and contain the red, green and blue components
> respectively), gives "INTERLEAVE=PIXEL" but "Block=2500x3". Is it
> correct to force createopt="BLOCKYSIZE=1" which gives "Block=2500x1"?
>
BLOCKXSIZE and BLOCKYSIZE affect the tile size, as far as I understand
the gdal documentation. I would not change the defaults. Leaving
BLOCKXSIZE on default = 256 and setting BLOCKYSIZE to 1 should produce
tiles of size 256x1.

The original photos I have report (gdalinfo) for example:
Driver: GTiff/GeoTIFF
[...]
Size is 2500, 2500
[...]
Image Structure Metadata:
  INTERLEAVE=PIXEL
[...]
Band 1 Block=2500x1 Type=Byte, ColorInterp=Red
Band 2 Block=2500x1 Type=Byte, ColorInterp=Green
Band 3 Block=2500x1 Type=Byte, ColorInterp=Blue

Exporting a composite (created with r.composite) gives:
Driver: GTiff/GeoTIFF
[...]
Size is 2500, 2500
[...]
Pixel Size = (0.800000000000000,-0.800000000000000)
Metadata:
  AREA_OR_POINT=Area
Image Structure Metadata:
  INTERLEAVE=BAND
[...]
Band 1 Block=2500x3 Type=Byte, ColorInterp=Palette

Exporting a group gives "correctly" something like:
[...]
Band 1 Block=2500x1 Type=Byte, ColorInterp=Red
  Metadata:
    COLOR_TABLE_RULES_COUNT=1
    COLOR_TABLE_RULE_RGB_0=0.000000e+00 2.550000e+02 0 0 0 255 255 255
Band 2 Block=2500x1 Type=Byte, ColorInterp=Green
  Metadata:
    COLOR_TABLE_RULES_COUNT=1
    COLOR_TABLE_RULE_RGB_0=0.000000e+00 2.550000e+02 0 0 0 255 255 255
Band 3 Block=2500x1 Type=Byte, ColorInterp=Blue
  Metadata:
    COLOR_TABLE_RULES_COUNT=1
    COLOR_TABLE_RULE_RGB_0=0.000000e+00 2.550000e+02 0 0 0 255 255 255

> 8. There are several compression methods? Does compression affect
> significantly reading/displaying an image?
>
I would stay away from tiff internal compression because other software
might not support it (LZW and DEFLATE are sometimes not supported). If I
want to distribute the tiff, I rather create an archive and send that.

Maciek suggested to compress the overviews to save disk space. Would
this cause a viewing problem in Arc? (Have no access to any commercial
package currently).

> 9. Should one prefer another file format?
>
GeoTIFF is widely accepted, but if I want to provide a raster for a
particular application, gdal might support the native raster format of
that application and then this would be the preferred format.

TFW=YES would only be needed if you export a raster for an application
that can not read GeoTIFF internal metadata. In this case I would rather
use the native format of that application. Many spatial data viewer make
use of gdal and should therefore read GeoTIFF properly.

Maybe there is a problem with your gdal installation? I would first
check that simple single band export with default settings works before
trying to fix the gdal installation.

I hope that helps,

Markus

I will re-check.

[1] "Warning 1: Lost metadata writing to GeoTIFF ... too large to fit in
tag.": The color table metadata are too large. It is recommended to use
a different output format.

More in the manual of r.out.gdal:
http://grass.osgeo.org/grass64/manuals/html64_user/r.out.gdal.html

[2] http://www.gdal.org/frmt_gtiff.html under section "Creation Options"

[3]
http://gfoss.blogspot.com/2008/06/gdal-raster-data-tips-and-tricks.html
under section "Export to (limited) TIFF readers such as ArcView, or
ImageMagick"

4

Nikos Alexandris wrote:

3. GeoTiff metadata are lost always (?) while exporting with
r.out.gdal.
      

Also when checking with gdalinfo?
    

No, I was talking about a known error [1]. How important is this error?
  

Oh, it is the colour table! That caused me headache too. Some viewers don't like built-in colour table, so they should be deleted before using r.out.gdal with r.colors -r. Colour tables are not too big when using BYTE as output type (max 255 entries), but they can become too big (thousands or millions of entries) with any bigger output type. AFAIK r.out.gdal only supports colour table export for BYTE and UINT16. In case of imagery, colour tables are not important anyway. It can be different if I have a raster with painfully developed categories/classes and would really like that other people use the same colour table like me.

4. The createopt="INTERLEAVE=PIXEL" is not respected by r.out.gdal (?)
      

Well, the original photos are "INTERLEAVE=PIXEL" and I suppose, from
what I read, it's a requirement for "limited" viewers (!) [2][3].
Exporting single maps gives "INTERLEAVE=BAND"!
  

Same here, I exported a single map using "INTERLEAVE=PIXEL" and get "INTERLEAVE=BAND"
Then I used gdal_translate -co "INTERLEAVE=PIXEL" and still get "INTERLEAVE=BAND", no warning message, nothing. This is a gdal problem or I missed something in the documentation. Unfortunately, I don't have a "limited" viewer to test it, only open source stuff using gdal and there it works...

6. Is it better to export the red, green and blue GRASS raster maps
one-by-one and merge them with gdal_translate (or gdal_merge.py)?
      

If I want a RGB GeoTIFF, I rather export the group.
    

What about the error message "ERROR 6: SetColorTable() not supported for
multi-sample TIFF files." which appears 3 times for a group that
contains 3 maps (red, green and blue in our case)?
  

See r.out.gdal man page: ... "the message can be ignored." What happened is that r.out.gdal tried to export a colour table, but couldn't, which might be better anyway. Colour tables are not crucial for imagery anyway, and it might be safer not to export a colour table. Actually it would be nice to have this as an option in r.out.gdal: export colortable yes/no

Exporting a group gives "correctly" something like:
[...]
Band 1 Block=2500x1 Type=Byte, ColorInterp=Red
  Metadata:
    COLOR_TABLE_RULES_COUNT=1
    COLOR_TABLE_RULE_RGB_0=0.000000e+00 2.550000e+02 0 0 0 255 255 255
Band 2 Block=2500x1 Type=Byte, ColorInterp=Green
  Metadata:
    COLOR_TABLE_RULES_COUNT=1
    COLOR_TABLE_RULE_RGB_0=0.000000e+00 2.550000e+02 0 0 0 255 255 255
Band 3 Block=2500x1 Type=Byte, ColorInterp=Blue
  Metadata:
    COLOR_TABLE_RULES_COUNT=1
    COLOR_TABLE_RULE_RGB_0=0.000000e+00 2.550000e+02 0 0 0 255 255 255
  

BLOCKXSIZE and BLOCKYSIZE affect tiles, they have no effect if not used with TILES=YES. Default is TILED=NO, files are by default stripped giving blocks of, in your case 2500x1. Translating a stripped file with TILES=YES gives you Block=256x256 (default gdal values). So I think there is nothing wrong with it, it matches the original photos anyway.

Maciek suggested to compress the overviews to save disk space. Would
this cause a viewing problem in Arc? (Have no access to any commercial
package currently).
  

Don't create overviews in this case. And yes, there can well be problems with ESRI products, they are not that good in raster handling. ArcGIS 9.2+ uses gdal (says http://trac.osgeo.org/gdal/wiki/SoftwareUsingGdal), there should not be a problem. ArcView on the other hand I don't know. It might work without colour table, TFW=YES,and a not too fancy data type. Why not using ESRI .hdr labelled (gdal code: EHdr) in this case?

[1] "Warning 1: Lost metadata writing to GeoTIFF ... too large to fit in
tag.": The color table metadata are too large. It is recommended to use
a different output format.

More in the manual of r.out.gdal:
http://grass.osgeo.org/grass64/manuals/html64_user/r.out.gdal.html

[2] http://www.gdal.org/frmt_gtiff.html under section "Creation Options"

[3]
http://gfoss.blogspot.com/2008/06/gdal-raster-data-tips-and-tricks.html
under section "Export to (limited) TIFF readers such as ArcView, or
ImageMagick"

5

Forgot to cc...

Sorry for the sloppy looking up, here is the solution for INTERLEAVE=PIXEL.
It is only supported for mulitband images, exporting or translating a
single band will always give INTERLEAVE=BAND. A look at the gdal source
code told me that this is apparently a gdal feature, not a bug.

Nikos Alexandris wrote:

4. The createopt="INTERLEAVE=PIXEL" is not respected by r.out.gdal (?)
      

Well, the original photos are "INTERLEAVE=PIXEL" and I suppose, from
what I read, it's a requirement for "limited" viewers (!) [2][3].
Exporting single maps gives "INTERLEAVE=BAND"!
  

[...]

Exporting a group gives "correctly" something like:
[...]
  

What does INTERLEAVE say here? Did you try "INTERLEAVE=PIXEL" here?

Band 1 Block=2500x1 Type=Byte, ColorInterp=Red
  Metadata:
    COLOR_TABLE_RULES_COUNT=1
    COLOR_TABLE_RULE_RGB_0=0.000000e+00 2.550000e+02 0 0 0 255 255 255
Band 2 Block=2500x1 Type=Byte, ColorInterp=Green
  Metadata:
    COLOR_TABLE_RULES_COUNT=1
    COLOR_TABLE_RULE_RGB_0=0.000000e+00 2.550000e+02 0 0 0 255 255 255
Band 3 Block=2500x1 Type=Byte, ColorInterp=Blue
  Metadata:
    COLOR_TABLE_RULES_COUNT=1
    COLOR_TABLE_RULE_RGB_0=0.000000e+00 2.550000e+02 0 0 0 255 255 255
  

6

I am glad that this topic is getting some traction again. I have
noticed that r.out.gdal does not always produce images readable in
ESRI products. Heck, sometime the resulting TIFF would not even be
readable by QGIS. This was some time ago, and associated with several
blog postings / trac tickets:

Posts:
1. http://casoilresource.lawr.ucdavis.edu/drupal/node/337
2. http://gfoss.blogspot.com/2008/06/gdal-raster-data-tips-and-tricks.html

Ticket:
3. http://trac.osgeo.org/grass/ticket/73

It would be a good idea to get this fixed in GRASS64 before marching
into GRASS7 as import/export operations are fundamental to GIS work.

Cheers,

Dylan

On Sat, Aug 2, 2008 at 11:43 AM, Markus Metz <markus_metz@gmx.de> wrote:

Nikos Alexandris wrote:

3. GeoTiff metadata are lost always (?) while exporting with
r.out.gdal.

Also when checking with gdalinfo?

No, I was talking about a known error [1]. How important is this error?

Oh, it is the colour table! That caused me headache too. Some viewers don't
like built-in colour table, so they should be deleted before using
r.out.gdal with r.colors -r. Colour tables are not too big when using BYTE
as output type (max 255 entries), but they can become too big (thousands or
millions of entries) with any bigger output type. AFAIK r.out.gdal only
supports colour table export for BYTE and UINT16. In case of imagery, colour
tables are not important anyway. It can be different if I have a raster with
painfully developed categories/classes and would really like that other
people use the same colour table like me.

4. The createopt="INTERLEAVE=PIXEL" is not respected by r.out.gdal (?)

Well, the original photos are "INTERLEAVE=PIXEL" and I suppose, from
what I read, it's a requirement for "limited" viewers (!) [2][3].
Exporting single maps gives "INTERLEAVE=BAND"!

Same here, I exported a single map using "INTERLEAVE=PIXEL" and get
"INTERLEAVE=BAND"
Then I used gdal_translate -co "INTERLEAVE=PIXEL" and still get
"INTERLEAVE=BAND", no warning message, nothing. This is a gdal problem or I
missed something in the documentation. Unfortunately, I don't have a
"limited" viewer to test it, only open source stuff using gdal and there it
works...

6. Is it better to export the red, green and blue GRASS raster maps
one-by-one and merge them with gdal_translate (or gdal_merge.py)?

If I want a RGB GeoTIFF, I rather export the group.

What about the error message "ERROR 6: SetColorTable() not supported for
multi-sample TIFF files." which appears 3 times for a group that
contains 3 maps (red, green and blue in our case)?

See r.out.gdal man page: ... "the message can be ignored." What happened is
that r.out.gdal tried to export a colour table, but couldn't, which might be
better anyway. Colour tables are not crucial for imagery anyway, and it
might be safer not to export a colour table. Actually it would be nice to
have this as an option in r.out.gdal: export colortable yes/no

Exporting a group gives "correctly" something like:
[...]
Band 1 Block=2500x1 Type=Byte, ColorInterp=Red
Metadata:
   COLOR_TABLE_RULES_COUNT=1
   COLOR_TABLE_RULE_RGB_0=0.000000e+00 2.550000e+02 0 0 0 255 255 255
Band 2 Block=2500x1 Type=Byte, ColorInterp=Green
Metadata:
   COLOR_TABLE_RULES_COUNT=1
   COLOR_TABLE_RULE_RGB_0=0.000000e+00 2.550000e+02 0 0 0 255 255 255
Band 3 Block=2500x1 Type=Byte, ColorInterp=Blue
Metadata:
   COLOR_TABLE_RULES_COUNT=1
   COLOR_TABLE_RULE_RGB_0=0.000000e+00 2.550000e+02 0 0 0 255 255 255

BLOCKXSIZE and BLOCKYSIZE affect tiles, they have no effect if not used with
TILES=YES. Default is TILED=NO, files are by default stripped giving blocks
of, in your case 2500x1. Translating a stripped file with TILES=YES gives
you Block=256x256 (default gdal values). So I think there is nothing wrong
with it, it matches the original photos anyway.

Maciek suggested to compress the overviews to save disk space. Would
this cause a viewing problem in Arc? (Have no access to any commercial
package currently).

Don't create overviews in this case. And yes, there can well be problems
with ESRI products, they are not that good in raster handling. ArcGIS 9.2+
uses gdal (says http://trac.osgeo.org/gdal/wiki/SoftwareUsingGdal), there
should not be a problem. ArcView on the other hand I don't know. It might
work without colour table, TFW=YES,and a not too fancy data type. Why not
using ESRI .hdr labelled (gdal code: EHdr) in this case?

[1] "Warning 1: Lost metadata writing to GeoTIFF ... too large to fit in
tag.": The color table metadata are too large. It is recommended to use
a different output format.

More in the manual of r.out.gdal:
http://grass.osgeo.org/grass64/manuals/html64_user/r.out.gdal.html

[2] http://www.gdal.org/frmt_gtiff.html under section "Creation Options"

[3]
http://gfoss.blogspot.com/2008/06/gdal-raster-data-tips-and-tricks.html
under section "Export to (limited) TIFF readers such as ArcView, or
ImageMagick"

_______________________________________________
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

7

Thank you for you patience :slight_smile:

On Sat, 2008-08-02 at 21:14 +0200, Markus Metz wrote:

Forgot to cc...

Sorry for the sloppy looking up, here is the solution for INTERLEAVE=PIXEL.
It is only supported for mulitband images, exporting or translating a
single band will always give INTERLEAVE=BAND. A look at the gdal source
code told me that this is apparently a gdal feature, not a bug.

That's useful info.

Nikos Alexandris wrote:
>>> 4. The createopt="INTERLEAVE=PIXEL" is not respected by r.out.gdal (?)
>>>
> Well, the original photos are "INTERLEAVE=PIXEL" and I suppose, from
> what I read, it's a requirement for "limited" viewers (!) [2][3].
> Exporting single maps gives "INTERLEAVE=BAND"!
>
[...]
> Exporting a group gives "correctly" something like:
> [...]
>
What does INTERLEAVE say here? Did you try "INTERLEAVE=PIXEL" here?

Yes, it's:
Image Structure Metadata:
  INTERLEAVE=PIXEL

> Band 1 Block=2500x1 Type=Byte, ColorInterp=Red
> Metadata:
> COLOR_TABLE_RULES_COUNT=1
> COLOR_TABLE_RULE_RGB_0=0.000000e+00 2.550000e+02 0 0 0 255 255 255
> Band 2 Block=2500x1 Type=Byte, ColorInterp=Green
> Metadata:
> COLOR_TABLE_RULES_COUNT=1
> COLOR_TABLE_RULE_RGB_0=0.000000e+00 2.550000e+02 0 0 0 255 255 255
> Band 3 Block=2500x1 Type=Byte, ColorInterp=Blue
> Metadata:
> COLOR_TABLE_RULES_COUNT=1
> COLOR_TABLE_RULE_RGB_0=0.000000e+00 2.550000e+02 0 0 0 255 255 255
>

8

By the way, what is the "correct" nodata value for a "Byte" image? I
mean, won't I loose information if I give nodata=0 to my RGB aerial
photos? And the default nodata=255 is not exactly convenient I think.

On Sat, 2008-08-02 at 12:17 -0700, Dylan Beaudette wrote:

I am glad that this topic is getting some traction again. I have
noticed that r.out.gdal does not always produce images readable in
ESRI products. Heck, sometime the resulting TIFF would not even be
readable by QGIS. This was some time ago, and associated with several
blog postings / trac tickets:

Posts:
1. http://casoilresource.lawr.ucdavis.edu/drupal/node/337
2. http://gfoss.blogspot.com/2008/06/gdal-raster-data-tips-and-tricks.html

Ticket:
3. http://trac.osgeo.org/grass/ticket/73

It would be a good idea to get this fixed in GRASS64 before marching
into GRASS7 as import/export operations are fundamental to GIS work.

Cheers,

Dylan

[...]

9

[...]

What does INTERLEAVE say here? Did you try "INTERLEAVE=PIXEL" here?
> Band 1 Block=2500x1 Type=Byte, ColorInterp=Red
> Metadata:
> COLOR_TABLE_RULES_COUNT=1
> COLOR_TABLE_RULE_RGB_0=0.000000e+00 2.550000e+02 0 0 0 255 255 255
> Band 2 Block=2500x1 Type=Byte, ColorInterp=Green
> Metadata:
> COLOR_TABLE_RULES_COUNT=1
> COLOR_TABLE_RULE_RGB_0=0.000000e+00 2.550000e+02 0 0 0 255 255 255
> Band 3 Block=2500x1 Type=Byte, ColorInterp=Blue
> Metadata:
> COLOR_TABLE_RULES_COUNT=1
> COLOR_TABLE_RULE_RGB_0=0.000000e+00 2.550000e+02 0 0 0 255 255 255

Sorry, I did not completely answered. It is "INTERLEAVE=PIXEL" and I did
not force it.

10

On 02/08/08 21:17, Dylan Beaudette wrote:

Ticket:
3. http://trac.osgeo.org/grass/ticket/73

It would be a good idea to get this fixed in GRASS64 before marching
into GRASS7 as import/export operations are fundamental to GIS work.

+1

Moritz

11

I think exporting colour tables is one big problem [also see 3, 4]: GeoTIFFs exported from GRASS suddenly became readable after deleting the colour table before exporting (my own fiddling around). IMHO, there should be an option to export colour tables with default to no. There should also be a warning if you really want to export a colour table. For multiband images (exporting a group), exporting colour tables might need to be disabled completely. This is easy to implement in the current version of r.out.gdal. In my experience, Tiffs without a colour table are indeed useful [contrary to 4], because for imagery, most viewers have their own contrast enhancement and colour table manipulation methods. For a raster with labelled categories, a separate text file/table with labels and maybe a suggested colouring scheme needs to be provided anyway. I haven't found yet a raster format that gdal can write where gdal can export category labels from GRASS.

Another issue might be handling of nodata, that is at least a starting point for testing/fixing.

There might be other problems with the readability of the output of r.out.gdal, but maybe sorting out the two problems listed above helps a lot.

Having said that, r.out.gdal and GDAL Utility Programs as they are now are probably unmatched in their compatibility and interoperability [5].

Regards,

Markus

Dylan Beaudette wrote:

I am glad that this topic is getting some traction again. I have
noticed that r.out.gdal does not always produce images readable in
ESRI products. Heck, sometime the resulting TIFF would not even be
readable by QGIS. This was some time ago, and associated with several
blog postings / trac tickets:

Posts:
1. http://casoilresource.lawr.ucdavis.edu/drupal/node/337
2. http://gfoss.blogspot.com/2008/06/gdal-raster-data-tips-and-tricks.html

Ticket:
3. http://trac.osgeo.org/grass/ticket/73
  

Other posts:
4. http://lists.osgeo.org/pipermail/grass-dev/2008-February/035987.html
5. http://lists.osgeo.org/pipermail/grass-user/2008-July/045860.html

It would be a good idea to get this fixed in GRASS64 before marching
into GRASS7 as import/export operations are fundamental to GIS work.

Cheers,

Dylan
  

12

(Continued...)

Dear GRASSers,
how do you really export as GeoTiffs your raster maps to view in OpenEV,
QGIS?

I think I solved my problem by removing color tables (as Markus Metz
suggested), exported groups, nodata=-9999 and, very important, setting
in my loop over all groups a "g.region rast=$i" each time.

Without it I couldn't see anything whenever I have put an out of range
nodata value. I am posting my questions I had before "solving" my
problem anyway. Maybe there is still something interesting to
read/ask/answer.

Thank you, Nikos.

---
Some info:

uname -a
Linux vertical 2.6.24-19-generic #1 SMP Fri Jul 11 21:01:46 UTC 2008
x86_64 GNU/Linux

gdalinfo --version
GDAL 1.5.2, released 2008/05/29

g.version
GRASS 6.4.svn (2008)

For the records: I am not interested to open my images with any other
tool than TheGIMP, QGIS, OpenEV (maybe ImageMagick).

A.
I export the groups (red, green and blue maps for each group) with:
for i in `g.mlist group`; do r.out.gdal $i type=Byte out=$i.tiff; done"

I can see the exported images in QGIS, OpenEV. However, this sets 255 as
"nodata" which is something that I don't want. Whenever I set a nodata
value out of range [1] (for Byte that would be smaller than 0 or bigger
than 255 --- right?) I can't see propely the exported tiffs in either
QGIS or OpenEV. In QGIS nothing appears and in OpenEV the individual
tiff's (loading with openev -h *.tiff) "flash" and dissapear to leave a
black screen.

Is there a way to use r.out.tiff to export groups?

B.
Frank Warmerdam wrote [2] "There is no standard tag in TIFF or GeoTIFF
for describing nodata pixel
values. The only approach you might be able to use is to somehow add
an
alpha channel and mark the nodata regions as alpha=0; however, I'm not
sure how you would go about that."

Is this related to the "nodata" problem?
If yes, has anybody found a solution to it?
Could the "metaopt=" option of r.out.gdal be useful here?

C.
I can't seem to export raster maps with gdal_translate as described in
Markus' blog [3]
(copy-paste from [3] gdal_translate -of
Gtiff /usr/local/share/grassdata/spearfish/PERMANENT/cellhd/soils
soilmap.tif)

D.
What about this work-around [4] that Maciek postd some time ago?

[1] http://www.gdal.org/classGDALRasterBand.html

[2]
http://www.nabble.com/Remove-black-background-of-GeoTiff-td6400610.html

[3]
http://gfoss.blogspot.com/2008/06/gdal-raster-data-tips-and-tricks.html

[4] http://www.mail-archive.com/grassuser@grass.itc.it/msg01634.html

13

Nikos Alexandris wrote:

I think I solved my problem by removing color tables

That doesn't solve the problem, I learned. If you remove the associated
colour table, r.out.gdal writes a completely black colour table.
Applications that read the colour table will consequently show a
completely black image, but the true values are still there. I think
this is a bug in r.out.gdal. If there is no colour table, no colour
table should be written to the output file at all, because only then
would other applications use the real values to display the image. Fixed
after removing lines 208 and 210 in raster/r.out.gdal/main.c and
compiling again, GRASS 6.3.0
The lines read:
line 208
hCT = GDALCreateColorTable(GPI_RGB);
line 210
GDALSetRasterColorTable(hBand, hCT);

I can see the exported images in QGIS, OpenEV. However, this sets 255 as
"nodata" which is something that I don't want. Whenever I set a nodata
value out of range [1] (for Byte that would be smaller than 0 or bigger
than 255 --- right?) I can't see propely the exported tiffs in either
QGIS or OpenEV.

255 is the default nodata value. r.out.gdal does the right thing if it
replaces a specified nodata value that is out of range with 255.

In QGIS nothing appears and in OpenEV the individual
tiff's (loading with openev -h *.tiff) "flash" and dissapear to leave a
black screen.
  

See above, the colour table might be all black. Check with gdalinfo, if
all colour table entries read something like
XXX: 0,0,0,255
with XXX going from 0 to 255 in case of Byte type, this is an all-black
colour table.
If there is no colour table, the real values are used.
The "flash" in openev appears because openev needs a bit of time to
apply the all-black colour table.

More on INTERLEAVE=PIXEL

For single band images, there is no difference between INTERLEAVE=PIXEL
and INTERLEAVE=BAND. TIFF specs from 1992:
ftp.remotesensing.org/pub/libtiff/doc/TIFF6.pdf

ESRI's capability reading GeoTIFF: limited, check
http://webhelp.esri.com/arcgisdesktop/9.2/index.cfm?TopicName=Technical_specifications_for_raster_dataset_formats

Regards,

Markus

14

Markus,
I appreciate it that you clarify the details.

On Sun, 2008-08-03 at 17:22 +0200, Markus Metz wrote:

Nikos Alexandris wrote:
> I think I solved my problem by removing color tables

That doesn't solve the problem, I learned. If you remove the associated
colour table, r.out.gdal writes a completely black colour table.
Applications that read the colour table will consequently show a
completely black image, but the true values are still there. I think
this is a bug in r.out.gdal.
If there is no colour table, no colour
table should be written to the output file at all, because only then
would other applications use the real values to display the image. Fixed
after removing lines 208 and 210 in raster/r.out.gdal/main.c and
compiling again, GRASS 6.3.0
The lines read:
line 208
hCT = GDALCreateColorTable(GPI_RGB);
line 210
GDALSetRasterColorTable(hBand, hCT);

Did you already filed a ticket for that?

> I can see the exported images in QGIS, OpenEV. However, this sets 255 as
> "nodata" which is something that I don't want. Whenever I set a nodata
> value out of range [1] (for Byte that would be smaller than 0 or bigger
> than 255 --- right?) I can't see propely the exported tiffs in either
> QGIS or OpenEV.
255 is the default nodata value. r.out.gdal does the right thing if it
replaces a specified nodata value that is out of range with 255.

So nodata=255 does not mean "loss of information"?
The red, green, blue channels are 8-bit (2 high 8 = 256, values counting
from 0 up to 255).

> In QGIS nothing appears and in OpenEV the individual
> tiff's (loading with openev -h *.tiff) "flash" and dissapear to leave a
> black screen.
>
See above, the colour table might be all black. Check with gdalinfo, if
all colour table entries read something like
XXX: 0,0,0,255
with XXX going from 0 to 255 in case of Byte type, this is an all-black
colour table.

One example report for an image imported with r.in.gdal, MASKed,
r.colors -r, r.out.gdal as a group with nodata=-9999:
Driver: GTiff/GeoTIFF
Files: simmern_2624_5548.tiff
       simmern_2624_5548.tiff.aux.xml
Size is 2500, 2500
Coordinate System is:
PROJCS["Transverse Mercator",
    GEOGCS["bessel",
        DATUM["Deutsches_Hauptdreiecksnetz",
            SPHEROID["Bessel 1841",6377397.155,299.1528128000008,
                AUTHORITY["EPSG","7004"]],
            AUTHORITY["EPSG","6314"]],
        PRIMEM["Greenwich",0],
        UNIT["degree",0.0174532925199433]],
    PROJECTION["Transverse_Mercator"],
    PARAMETER["latitude_of_origin",0],
    PARAMETER["central_meridian",6],
    PARAMETER["scale_factor",1],
    PARAMETER["false_easting",2500000],
    PARAMETER["false_northing",0],
    UNIT["metre",1,
        AUTHORITY["EPSG","9001"]]]
Origin = (2624000.000000000000000,5550000.000000000000000)
Pixel Size = (0.800000000000000,-0.800000000000000)
Metadata:
  AREA_OR_POINT=Area
Image Structure Metadata:
  INTERLEAVE=PIXEL
Corner Coordinates:
Upper Left ( 2624000.000, 5550000.000) ( 7d43'56.88"E, 50d 4'28.10"N)
Lower Left ( 2624000.000, 5548000.000) ( 7d43'54.54"E, 50d 3'23.39"N)
Upper Right ( 2626000.000, 5550000.000) ( 7d45'37.42"E, 50d 4'26.59"N)
Lower Right ( 2626000.000, 5548000.000) ( 7d45'35.05"E, 50d 3'21.88"N)
Center ( 2625000.000, 5549000.000) ( 7d44'45.97"E, 50d 3'54.99"N)
Band 1 Block=2500x1 Type=Byte, ColorInterp=Red
  NoData Value=-9999
  Metadata:
    COLOR_TABLE_RULES_COUNT=5
    COLOR_TABLE_RULE_RGB_0=0.000000e+00 4.640000e+01 255 255 0 0 255 0
    COLOR_TABLE_RULE_RGB_1=4.640000e+01 9.280000e+01 0 255 0 0 255 255
    COLOR_TABLE_RULE_RGB_2=9.280000e+01 1.392000e+02 0 255 255 0 0 255
    COLOR_TABLE_RULE_RGB_3=1.392000e+02 1.856000e+02 0 0 255 255 0 255
    COLOR_TABLE_RULE_RGB_4=1.856000e+02 2.320000e+02 255 0 255 255 0 0
Band 2 Block=2500x1 Type=Byte, ColorInterp=Green
  NoData Value=-9999
  Metadata:
    COLOR_TABLE_RULES_COUNT=5
    COLOR_TABLE_RULE_RGB_0=0.000000e+00 4.680000e+01 255 255 0 0 255 0
    COLOR_TABLE_RULE_RGB_1=4.680000e+01 9.360000e+01 0 255 0 0 255 255
    COLOR_TABLE_RULE_RGB_2=9.360000e+01 1.404000e+02 0 255 255 0 0 255
    COLOR_TABLE_RULE_RGB_3=1.404000e+02 1.872000e+02 0 0 255 255 0 255
    COLOR_TABLE_RULE_RGB_4=1.872000e+02 2.340000e+02 255 0 255 255 0 0
Band 3 Block=2500x1 Type=Byte, ColorInterp=Blue
  NoData Value=-9999
  Metadata:
    COLOR_TABLE_RULES_COUNT=5
    COLOR_TABLE_RULE_RGB_0=3.400000e+01 7.320000e+01 255 255 0 0 255 0
    COLOR_TABLE_RULE_RGB_1=7.320000e+01 1.124000e+02 0 255 0 0 255 255
    COLOR_TABLE_RULE_RGB_2=1.124000e+02 1.516000e+02 0 255 255 0 0 255
    COLOR_TABLE_RULE_RGB_3=1.516000e+02 1.908000e+02 0 0 255 255 0 255
    COLOR_TABLE_RULE_RGB_4=1.908000e+02 2.300000e+02 255 0 255 255 0 0

If there is no colour table, the real values are used.
The "flash" in openev appears because openev needs a bit of time to
apply the all-black colour table.

More on INTERLEAVE=PIXEL

For single band images, there is no difference between INTERLEAVE=PIXEL
and INTERLEAVE=BAND. TIFF specs from 1992:
ftp.remotesensing.org/pub/libtiff/doc/TIFF6.pdf

Thats a good source.

ESRI's capability reading GeoTIFF: limited, check
http://webhelp.esri.com/arcgisdesktop/9.2/index.cfm?TopicName=Technical_specifications_for_raster_dataset_formats

Regards,

Markus

15

Nikos Alexandris wrote:

Markus,
I appreciate it that you clarify the details.
  

It seems I try but fail. Something is still wrong...

If you remove the associated colour table, r.out.gdal writes a completely black colour table. Applications that read the colour table will consequently show a completely black image, but the true values are still there. I think this is a bug in r.out.gdal.
If there is no colour table, no colour table should be written to the output file at all, because only then would other applications use the real values to display the image. Fixed after removing lines 208 and 210 in raster/r.out.gdal/main.c and compiling again, GRASS 6.3.0
The lines read:
line 208
hCT = GDALCreateColorTable(GPI_RGB);
line 210
GDALSetRasterColorTable(hBand, hCT);
    
Did you already filed a ticket for that?
  

Not yet, will do.

So nodata=255 does not mean "loss of information"?
The red, green, blue channels are 8-bit (2 high 8 = 256, values counting
from 0 up to 255).
  

It means that NULL cells in the GRASS raster are replaced with 255 or the specified nodata value.

One example report for an image imported with r.in.gdal, MASKed,
r.colors -r, r.out.gdal as a group with nodata=-9999:
Driver: GTiff/GeoTIFF
Files: simmern_2624_5548.tiff
       simmern_2624_5548.tiff.aux.xml
Size is 2500, 2500
Coordinate System is:
PROJCS["Transverse Mercator",
    GEOGCS["bessel",
        DATUM["Deutsches_Hauptdreiecksnetz",
            SPHEROID["Bessel 1841",6377397.155,299.1528128000008,
                AUTHORITY["EPSG","7004"]],
            AUTHORITY["EPSG","6314"]],
        PRIMEM["Greenwich",0],
        UNIT["degree",0.0174532925199433]],
    PROJECTION["Transverse_Mercator"],
    PARAMETER["latitude_of_origin",0],
    PARAMETER["central_meridian",6],
    PARAMETER["scale_factor",1],
    PARAMETER["false_easting",2500000],
    PARAMETER["false_northing",0],
    UNIT["metre",1,
        AUTHORITY["EPSG","9001"]]]
Origin = (2624000.000000000000000,5550000.000000000000000)
Pixel Size = (0.800000000000000,-0.800000000000000)
Metadata:
  AREA_OR_POINT=Area
Image Structure Metadata:
  INTERLEAVE=PIXEL
Corner Coordinates:
Upper Left ( 2624000.000, 5550000.000) ( 7d43'56.88"E, 50d 4'28.10"N)
Lower Left ( 2624000.000, 5548000.000) ( 7d43'54.54"E, 50d 3'23.39"N)
Upper Right ( 2626000.000, 5550000.000) ( 7d45'37.42"E, 50d 4'26.59"N)
Lower Right ( 2626000.000, 5548000.000) ( 7d45'35.05"E, 50d 3'21.88"N)
Center ( 2625000.000, 5549000.000) ( 7d44'45.97"E, 50d 3'54.99"N)
Band 1 Block=2500x1 Type=Byte, ColorInterp=Red
  NoData Value=-9999
  Metadata:
    COLOR_TABLE_RULES_COUNT=5
    COLOR_TABLE_RULE_RGB_0=0.000000e+00 4.640000e+01 255 255 0 0 255 0
    COLOR_TABLE_RULE_RGB_1=4.640000e+01 9.280000e+01 0 255 0 0 255 255
    COLOR_TABLE_RULE_RGB_2=9.280000e+01 1.392000e+02 0 255 255 0 0 255
    COLOR_TABLE_RULE_RGB_3=1.392000e+02 1.856000e+02 0 0 255 255 0 255
    COLOR_TABLE_RULE_RGB_4=1.856000e+02 2.320000e+02 255 0 255 255 0 0
Band 2 Block=2500x1 Type=Byte, ColorInterp=Green
  NoData Value=-9999
  Metadata:
    COLOR_TABLE_RULES_COUNT=5
    COLOR_TABLE_RULE_RGB_0=0.000000e+00 4.680000e+01 255 255 0 0 255 0
    COLOR_TABLE_RULE_RGB_1=4.680000e+01 9.360000e+01 0 255 0 0 255 255
    COLOR_TABLE_RULE_RGB_2=9.360000e+01 1.404000e+02 0 255 255 0 0 255
    COLOR_TABLE_RULE_RGB_3=1.404000e+02 1.872000e+02 0 0 255 255 0 255
    COLOR_TABLE_RULE_RGB_4=1.872000e+02 2.340000e+02 255 0 255 255 0 0
Band 3 Block=2500x1 Type=Byte, ColorInterp=Blue
  NoData Value=-9999
  Metadata:
    COLOR_TABLE_RULES_COUNT=5
    COLOR_TABLE_RULE_RGB_0=3.400000e+01 7.320000e+01 255 255 0 0 255 0
    COLOR_TABLE_RULE_RGB_1=7.320000e+01 1.124000e+02 0 255 0 0 255 255
    COLOR_TABLE_RULE_RGB_2=1.124000e+02 1.516000e+02 0 255 255 0 0 255
    COLOR_TABLE_RULE_RGB_3=1.516000e+02 1.908000e+02 0 0 255 255 0 255
    COLOR_TABLE_RULE_RGB_4=1.908000e+02 2.300000e+02 255 0 255 255 0 0

I have one last idea. Type is Byte, but NoData Value=-9999, out of range for Byte. Maybe that causes a problem? Can you try with nodata=0, then NULL cells in the GRASS raster should appear black in the GeoTIFF?
If that fails too, maybe it's the COLOR_TABLE_RULE entries. Then try data type = INT16, not UINT16, after removing the colour table with r.colors -r, this should remove both any colour table and these colour table rule entries in the GeoTIFF metadata.

Good luck!

16

Markus Metz wrote:

> I think I solved my problem by removing color tables
That doesn't solve the problem, I learned. If you remove the associated
colour table, r.out.gdal writes a completely black colour table.
Applications that read the colour table will consequently show a
completely black image, but the true values are still there. I think
this is a bug in r.out.gdal. If there is no colour table, no colour
table should be written to the output file at all, because only then
would other applications use the real values to display the image. Fixed
after removing lines 208 and 210 in raster/r.out.gdal/main.c and
compiling again, GRASS 6.3.0

This should be fixed in 6.4 and 7.0 SVN. If a map doesn't have an
explicit colour table, the default rainbow colour table will be used
instead.

--
Glynn Clements <glynn@gclements.plus.com>

17

[...]

I have one last idea. Type is Byte, but NoData Value=-9999, out of range
for Byte. Maybe that causes a problem? Can you try with nodata=0, then
NULL cells in the GRASS raster should appear black in the GeoTIFF?
If that fails too, maybe it's the COLOR_TABLE_RULE entries. Then try
data type = INT16, not UINT16, after removing the colour table with
r.colors -r, this should remove both any colour table and these colour
table rule entries in the GeoTIFF metadata.

Markus,

I did try also with nodata=0 before my first post but the region was set
to a bigger "block" which sort of includes all of my images. And I could
not see anything.

After adding the "g.region rast=$i" (zooming in each single photo) in my
loop I can see images properly.

What is the role of the active region? Maybe nothing... Now I add
nodata=0 and both, g.region rast=block or g.region rast=$i in my export
loop give viewable photos. I can't reproduce this behaviour any more :-?

Have no idea...

18

On Sun, 2008-08-03 at 18:12 +0200, Markus Metz wrote:

> So nodata=255 does not mean "loss of information"?
> The red, green, blue channels are 8-bit (2 high 8 = 256, values
counting
> from 0 up to 255).
>
It means that NULL cells in the GRASS raster are replaced with 255 or
the specified nodata value.

And what happens to the 255 pixels if the NULL cells become 255 after
export?

OK, let me figure this out:

I have an RGB photo with values from 0 to 255.
Import in GRASS splits in red, green, blue with values from 0 to 255
Why should I want to eliminate those 255 values after exporting? This is
what I mean loss of information.

Where am I wrong?

19

Nikos Alexandris wrote:

> > So nodata=255 does not mean "loss of information"?
> > The red, green, blue channels are 8-bit (2 high 8 = 256, values counting
> > from 0 up to 255).
> >
> It means that NULL cells in the GRASS raster are replaced with 255 or
> the specified nodata value.

And what happens to the 255 pixels if the NULL cells become 255 after
export?

Nothing happens to them. You just cannot distinguish between cells
which are 255 and those which are null.

OK, let me figure this out:

I have an RGB photo with values from 0 to 255.
Import in GRASS splits in red, green, blue with values from 0 to 255
Why should I want to eliminate those 255 values after exporting? This is
what I mean loss of information.

If you import data which doesn't contain any nulls, then re-export it,
the nodata= option will typically have no effect, as there aren't any
nulls which need to be converted.

OTOH, if the GRASS raster which you are exporting contains values from
0 to 255 inclusive, and also contains nulls, that's 257 possible
values. If the format to which you are exporting only has 256 possible
values (i.e. 0 to 255), then you can't avoid losing information.

--
Glynn Clements <glynn@gclements.plus.com>

20

On Sun, 2008-08-03 at 19:32 +0100, Glynn Clements wrote:

Nikos Alexandris wrote:

[...]

If you import data which doesn't contain any nulls, then re-export it,
the nodata= option will typically have no effect, as there aren't any
nulls which need to be converted.

OTOH, if the GRASS raster which you are exporting contains values from
0 to 255 inclusive, and also contains nulls, that's 257 possible
values. If the format to which you are exporting only has 256 possible
values (i.e. 0 to 255), then you can't avoid losing information.

Thank you Glynn. It just got stuck to my mind that nodata= will convert
the 255 values in nodata and not the opposite. So I though that my 255's
will go away :slight_smile: