Hi
I've noticed that r.out.gdal creates GeoTIFFs significantly larger than
gdal_translate. Could this be avoided?
Example, in spearfish60:
$ r.out.gdal in=landcover.30m out=landcover.30m.tif type=Byte
$ ls -l landcover.30m.tif
-rw-r--r-- 1 shoofi shoofi 303978 sie 20 11:49 landcover.30m.tif
$ gdal_translate landcover.30m.tif landcover.30m_gt.tif
$ ls -l landcover.30m_gt.tif
-rw-r--r-- 1 shoofi shoofi 300220 sie 20 11:49 landcover.30m_gt.tif
Applying compression makes the difference more distinct:
$ r.out.gdal in=landcover.30m out=landcover.30m.tif type=Byte createopt="COMPRESS=DEFLATE"
$ ls -l landcover.30m.tif
-rw-r--r-- 1 shoofi shoofi 51338 sie 20 11:50 landcover.30m.tif
$ gdal_translate -co "COMPRESS=DEFLATE" landcover.30m.tif landcover.30m_gt.tif
$ ls -l landcover.30m_gt.tif
-rw-r--r-- 1 shoofi shoofi 47567 sie 20 11:51 landcover.30m_gt.tif
(Probably related to [1]. Some more info in a duplicate [2].)
[1]http://trac.osgeo.org/gdal/ticket/1688
[2]http://trac.osgeo.org/gdal/ticket/1689).
Maciek
--
Maciej Sieczka
www.sieczka.org
Maciej Sieczka wrote:
Hi
I've noticed that r.out.gdal creates GeoTIFFs significantly larger than
gdal_translate. Could this be avoided?
Not completely, but file sizes are near identical (see below) in this
version:
http://markus.metz.giswork.googlepages.com/r.out.gdal.conservative.tar.gz
The fix is to first create a new GDAL raster in memory and then copy the
raster to the desired output format, after GDAL API Tutorial
http://www.gdal.org/gdal_tutorial.html
I guess, when GDAL creates a new raster instead of copying a raster, it
reserves a bit of extra space just in case. Maybe this is analogous to
gdalwarp as explained in [1,2].
Example, in spearfish60:
$ r.out.gdal in=landcover.30m out=landcover.30m.tif type=Byte
$ ls -l landcover.30m.tif
-rw-r--r-- 1 shoofi shoofi 303978 sie 20 11:49 landcover.30m.tif
$ gdal_translate landcover.30m.tif landcover.30m_gt.tif
$ ls -l landcover.30m_gt.tif
-rw-r--r-- 1 shoofi shoofi 300220 sie 20 11:49 landcover.30m_gt.tif
Now nearly the same size:
Same commands give sizes 298928 for r.out.gdal and 298914 for
gdal_translate.
I did not export the colortable, thus the smaller file sizes on my system.
Applying compression makes the difference more distinct:
$ r.out.gdal in=landcover.30m out=landcover.30m.tif type=Byte
createopt="COMPRESS=DEFLATE"
$ ls -l landcover.30m.tif
-rw-r--r-- 1 shoofi shoofi 51338 sie 20 11:50 landcover.30m.tif
$ gdal_translate -co "COMPRESS=DEFLATE" landcover.30m.tif
landcover.30m_gt.tif
$ ls -l landcover.30m_gt.tif
-rw-r--r-- 1 shoofi shoofi 47567 sie 20 11:51 landcover.30m_gt.tif
Same commands give sizes 46275 for r.out.gdal and 46261 for gdal_translate.
I did not export the colortable, thus the smaller file sizes on my system.
The size difference is in both cases now 14 bytes instead of about 3000
bytes.
(Probably related to [1]. Some more info in a duplicate [2].)
[1]http://trac.osgeo.org/gdal/ticket/1688
[2]http://trac.osgeo.org/gdal/ticket/1689).
Maciek
Markus
Markus Metz pisze:
Maciej Sieczka wrote:
I've noticed that r.out.gdal creates GeoTIFFs significantly larger than
gdal_translate. Could this be avoided?
Not completely, but file sizes are near identical (see below) in this
version:
http://markus.metz.giswork.googlepages.com/r.out.gdal.conservative.tar.gz
>
The fix is to first create a new GDAL raster in memory and then copy the
raster to the desired output format, after GDAL API Tutorial
http://www.gdal.org/gdal_tutorial.html
>
I guess, when GDAL creates a new raster instead of copying a raster, it
reserves a bit of extra space just in case. Maybe this is analogous to
gdalwarp as explained in [1,2].
Applying compression makes the difference more distinct:
$ r.out.gdal in=landcover.30m out=landcover.30m.tif type=Byte
createopt="COMPRESS=DEFLATE"
$ ls -l landcover.30m.tif
-rw-r--r-- 1 shoofi shoofi 51338 sie 20 11:50 landcover.30m.tif
$ gdal_translate -co "COMPRESS=DEFLATE" landcover.30m.tif
landcover.30m_gt.tif
$ ls -l landcover.30m_gt.tif
-rw-r--r-- 1 shoofi shoofi 47567 sie 20 11:51 landcover.30m_gt.tif
Same commands give sizes 46275 for r.out.gdal and 46261 for gdal_translate.
I did not export the colortable, thus the smaller file sizes on my system.
The size difference is in both cases now 14 bytes instead of about 3000
bytes.
Cool! The size difference due to colortable is a non-issue.
I can't invest time in testing Markus's modifications thorougly, but
this particular improvement would be great to have.
(Probably related to [1]. Some more info in a duplicate [2].)
[1]http://trac.osgeo.org/gdal/ticket/1688
[2]http://trac.osgeo.org/gdal/ticket/1689).
Maciek
--
Maciej Sieczka
www.sieczka.org
Maciej Sieczka wrote:
I've noticed that r.out.gdal creates GeoTIFFs significantly larger than
gdal_translate. Could this be avoided?
Example, in spearfish60:
$ r.out.gdal in=landcover.30m out=landcover.30m.tif type=Byte
$ ls -l landcover.30m.tif
-rw-r--r-- 1 shoofi shoofi 303978 sie 20 11:49 landcover.30m.tif
$ gdal_translate landcover.30m.tif landcover.30m_gt.tif
$ ls -l landcover.30m_gt.tif
-rw-r--r-- 1 shoofi shoofi 300220 sie 20 11:49 landcover.30m_gt.tif
Applying compression makes the difference more distinct:
$ r.out.gdal in=landcover.30m out=landcover.30m.tif type=Byte
createopt="COMPRESS=DEFLATE"
$ ls -l landcover.30m.tif
-rw-r--r-- 1 shoofi shoofi 51338 sie 20 11:50 landcover.30m.tif
$ gdal_translate -co "COMPRESS=DEFLATE" landcover.30m.tif
landcover.30m_gt.tif
$ ls -l landcover.30m_gt.tif
-rw-r--r-- 1 shoofi shoofi 47567 sie 20 11:51 landcover.30m_gt.tif
(Probably related to [1]. Some more info in a duplicate [2].)
[1]http://trac.osgeo.org/gdal/ticket/1688
[2]http://trac.osgeo.org/gdal/ticket/1689).
I wouldn't call 3-4KiB "significantly larger", especially as this
appears to be a fixed overhead, rather than a proportional one.
Have you compared the output from gdalinfo for the images in question?
If the extra is caused by the GRASS colour rules being added as
metadata, we could add a flag to suppress that. OTOH, if it requires
making the code significantly more complex, then it isn't likely to
happen for the sake of a few KiB.
--
Glynn Clements <glynn@gclements.plus.com>
Glynn Clements pisze:
Maciej Sieczka wrote:
I've noticed that r.out.gdal creates GeoTIFFs significantly larger than
gdal_translate. Could this be avoided?
Example, in spearfish60:
$ r.out.gdal in=landcover.30m out=landcover.30m.tif type=Byte
$ ls -l landcover.30m.tif
-rw-r--r-- 1 shoofi shoofi 303978 sie 20 11:49 landcover.30m.tif
$ gdal_translate landcover.30m.tif landcover.30m_gt.tif
$ ls -l landcover.30m_gt.tif
-rw-r--r-- 1 shoofi shoofi 300220 sie 20 11:49 landcover.30m_gt.tif
Applying compression makes the difference more distinct:
$ r.out.gdal in=landcover.30m out=landcover.30m.tif type=Byte createopt="COMPRESS=DEFLATE"
$ ls -l landcover.30m.tif
-rw-r--r-- 1 shoofi shoofi 51338 sie 20 11:50 landcover.30m.tif
$ gdal_translate -co "COMPRESS=DEFLATE" landcover.30m.tif landcover.30m_gt.tif
$ ls -l landcover.30m_gt.tif
-rw-r--r-- 1 shoofi shoofi 47567 sie 20 11:51 landcover.30m_gt.tif
(Probably related to [1]. Some more info in a duplicate [2].)
[1]http://trac.osgeo.org/gdal/ticket/1688
[2]http://trac.osgeo.org/gdal/ticket/1689).
I wouldn't call 3-4KiB "significantly larger", especially as this
appears to be a fixed overhead, rather than a proportional one.
It becomes really significant when compression is involved. Please read
in my original message and in
http://trac.osgeo.org/gdal/ticket/1688#comment:4.
Have you compared the output from gdalinfo for the images in question?
Identical.
If the extra is caused by the GRASS colour rules being added as
metadata, we could add a flag to suppress that. OTOH, if it requires
making the code significantly more complex, then it isn't likely to
happen for the sake of a few KiB.
The metadata is not the culprit for sure.
--
Maciej Sieczka
www.sieczka.org