I have been doing a lot of merging recently with GDAL and I have been following this process
1. gdalwarp -multi -of GTiff -s_srs EPSG:32641 -t_srs EPSG:4326 -r cubic --config GDAL_CACHEMAX 600 -wm 600 -et 0 -srcnodata 0 -dstnodata 0 $SOURCE_FILE.sid $DEST_FILE.4326.tif
Repeat for each source file in the folder
2. ls -1 *.tif >> warpinput.txt
Create an input file from all the warped images
3. gdalbuildvrt -input_file_list warpinput.txt DEST_FILE.4326.vrt
4. time gdalwarp -of GTiff -multi -r cubic --config GDAL_CACHEMAX 600 -wm 600 -et 0 -srcnodata 0 -dstnodata 0 -co TILED=YES -co BLOCKXSIZE=512 -co BLOCKYSIZE=512 -co BIGTIFF=YES -cutline <path to>library/one_degree_cells/4326/e074n37_onedegree.shp DEST_FILE.4326.vrt DEST_FILE.4326.merged.tif
5. gdaladdo -r gauss --config GDAL_CACHEMAX 4096 --config COMPRESS_OVERVIEW JPEG --config JPEG_QUALITY_OVERVIEW 85 --config INTERLEAVE_OVERVIEW PIXEL DEST_FILE.4326.merged.tif 2 4 8 16 32
This has worked fairly well for me so far, your mileage may vary.
Chris Snider
Senior Software Engineer
Intelligent Software Solutions, Inc.
-----Original Message-----
From: Rahkonen Jukka [mailto:jukka.rahkonen@anonymised.com]
Sent: Friday, November 22, 2013 11:19 AM
To: geoserver-users@lists.sourceforge.net
Subject: Re: [Geoserver-users] GeoTIFFs verus JP2
Hi,
I think that the reason is that is is hard/impossible to make an optimal deflate compressed tiff when gdal_merge goes through the circle "open file - add data - close file". There are other alternatives to test:
- You can use a non-optimal deflate compressed tiff as a temporary file - is is lossless and the final result will be identical.
- Use gdalbuildvrt for making an interim .VRT file and convert that to final product instead of using gdal_merge.py. Perhaps GDAL can make an optimal deflate compressed image from the VRT file with one run.
A hint: If you play with big datasets and deflate compressed images GDAL sometimes does not guess when it should jump into BigTIFF. Give the -co bigtiff=yes parameter manually if you feel that it could be needed.
If Even is still sneaking here he can say if the behaviour you see with gdal_merge.py is expected or a bug. Otherwise you can ask it from gdal-dev mailing list.
-Jukka-
________________________________
Jonathan Moules wrote:
On 21 November 2013 21:23, Rahkonen Jukka <jukka.rahkonen@anonymised.com<mailto:jukka.rahkonen@anonymised.com>> wrote:
I would try if it makes difference to save result from gdal_merge first into uncompressed tiff and compress it with deflate in a separate run.
Yep, that worked. Although that's unfeasible for the full dataset; I don't have enough diskspace for an uncompressed version.
Also, doing that (deflate, level 9), the tiled version was about 15% smaller than the untiled version.
Thanks!
Jonathan
Jukka Rahkonen
________________________________
Jonathan Moules<mailto:jonathanmoules@anonymised.com> wrote:
It is not completely surprising. If there are no repeateable patterns in the uncompressed stream, deflate can result in (a bit) larger files. You might want to try with -co INTERLEAVE=BAND added to see if it improves things. But generally DEFLATE is not appropriate for aerial imagery.
The "bit" larger files are typically about double the size of the uncompressed one!
I know deflate isn't as well suited, but I'm experimenting with a whole bunch of different settings to see what sticks.
======
I'm now trying to get the four band into GeoServer (regular GeoTIFF), but it's not having it. I've created it the same way as the 3 band:
gdal_merge -q -o rgbi.tif -of GTiff -co TILED=YES -co BIGTIFF=YES -co COMPRESS=JPEG -co JPEG_QUALITY=25 -co BLOCKXSIZE=512 -co BLOCKYSIZE=512 --optfile tiff_list.txt (low compression for testing). No pyramids yet.
The first thing I notice is that the 4 band is *smaller* (118MB) than the 3 band (164MB).
The gdalinfo for the four band:
Driver: GTiff/GeoTIFF
Files: rgbi.tif
Size is 8000, 24000
Coordinate System is:
PROJCS["OSGB 1936 / British National Grid", [brevity removed] Origin = (419000.000000000000000,240000.000000000000000)
Pixel Size = (0.125000000000000,-0.125000000000000)
Metadata:
AREA_OR_POINT=Area
Image Structure Metadata:
COMPRESSION=JPEG
INTERLEAVE=PIXEL
Corner Coordinates:
Upper Left ( 419000.000, 240000.000) ( 1d43'22.27"W, 52d 3'27.49"N) Lower Left ( 419000.000, 237000.000) ( 1d43'22.87"W, 52d 1'50.38"N) Upper Right ( 420000.000, 240000.000) ( 1d42'29.76"W, 52d 3'27.37"N) Lower Right ( 420000.000, 237000.000) ( 1d42'30.39"W, 52d 1'50.26"N)
Center ( 419500.000, 238500.000) ( 1d42'56.32"W, 52d 2'38.88"N)
Band 1 Block=512x512 Type=Byte, ColorInterp=Red
Mask Flags: PER_DATASET ALPHA
Band 2 Block=512x512 Type=Byte, ColorInterp=Green
Mask Flags: PER_DATASET ALPHA
Band 3 Block=512x512 Type=Byte, ColorInterp=Blue
Mask Flags: PER_DATASET ALPHA
Band 4 Block=512x512 Type=Byte, ColorInterp=Alpha
----
The GeoServer logs contain an error (WMS GetMap request):
2013-11-20 17:56:07,498 ERROR [org.geoserver.ows] -
java.lang.RuntimeException: javax.imageio.IIOException: Unsupported Image Type at com.sun.media.jai.imageioimpl.ImageReadOpImage.computeTile(ImageReadOpImage.java:706)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
at javax.media.jai.OpImage.getTile(OpImage.java:1129)
....
Caused by: javax.imageio.IIOException: Unsupported Image Type at com.sun.imageio.plugins.jpeg.JPEGImageReader.readInternal(Unknown Source) at com.sun.imageio.plugins.jpeg.JPEGImageReader.read(Unknown Source) at it.geosolutions.imageioimpl.plugins.tiff.TIFFJPEGDecompressor.decodeRaw(TIFFJPEGDecompressor.java:278)
Ideally I'm looking to have one source file which can be symbolised as not just RGB, but also as any number of false-colour possibilities.
Any suggestions anyone? I'm stumbling into new (for me) territory here.
Thanks,
Jonathan
This transmission is intended for the named addressee(s) only and may contain sensitive or protectively marked material up to RESTRICTED and should be handled accordingly. Unless you are the named addressee (or authorised to receive it for the addressee) you may not copy or use it, or disclose it to anyone else. If you have received this transmission in error please notify the sender immediately. All email traffic sent to or from us, including without limitation all GCSX traffic, may be subject to recording and/or monitoring in accordance with relevant legislation.
------------------------------------------------------------------------------
Shape the Mobile Experience: Free Subscription Software experts and developers: Be at the forefront of tech innovation.
Intel(R) Software Adrenaline delivers strategic insight and game-changing conversations that shape the rapidly evolving mobile landscape. Sign up now.
http://pubads.g.doubleclick.net/gampad/clk?id=63431311&iu=/4140/ostg.clktrk
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net<mailto:Geoserver-users@anonymised.comrge.net>
https://lists.sourceforge.net/lists/listinfo/geoserver-users
This transmission is intended for the named addressee(s) only and may contain sensitive or protectively marked material up to RESTRICTED and should be handled accordingly. Unless you are the named addressee (or authorised to receive it for the addressee) you may not copy or use it, or disclose it to anyone else. If you have received this transmission in error please notify the sender immediately. All email traffic sent to or from us, including without limitation all GCSX traffic, may be subject to recording and/or monitoring in accordance with relevant legislation.
------------------------------------------------------------------------------
Shape the Mobile Experience: Free Subscription Software experts and developers: Be at the forefront of tech innovation.
Intel(R) Software Adrenaline delivers strategic insight and game-changing conversations that shape the rapidly evolving mobile landscape. Sign up now.
http://pubads.g.doubleclick.net/gampad/clk?id=63431311&iu=/4140/ostg.clktrk
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users