[Geoserver-users] Image Mosaic & Gaps

Hi I am currently running GeoServer 2.1.1 and have run in to a problem with creating an image mosaic and the appearance of gaps between the images. My creation process is thus; Source raster is 1km square imagery tiles in jpg with accompnaying .jgw files. I convert these to GeoTIFF using gdal_translate, and create index file using gdaltindex. When connected to Geoserver and drawn up you can see white gaps between the images which reduce in size as I zoom in, until they disappear at the highest zoom level. I have never experienced this problem with the numerous other raster datasets I have been using. Does anyone have any advice? Cheers


View this message in context: Image Mosaic & Gaps
Sent from the GeoServer - User mailing list archive at Nabble.com.

On Fri, Aug 5, 2011 at 7:27 PM, AndyC <andyclark666@anonymised.com> wrote:

Hi I am currently running GeoServer 2.1.1 and have run in to a problem with creating an image mosaic and the appearance of gaps between the images. My creation process is thus; Source raster is 1km square imagery tiles in jpg with accompnaying .jgw files. I convert these to GeoTIFF using gdal_translate, and create index file using gdaltindex. When connected to Geoserver and drawn up you can see white gaps between the images which reduce in size as I zoom in, until they disappear at the highest zoom level. I have never experienced this problem with the numerous other raster datasets I have been using. Does anyone have any advice? Cheers

Without knowing anything my guess is that there is something going on in the raster to world
trasformation, maybe a numerical error that is only visible at lower scales?
Or else some issue in the overviews, if there are any.

Hard to say without looking at the data, can you share a group of 4 tiles for further investigation?

Cheers
Andrea

Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf


Hi Andrea

Attached are four 1km jpegs and accompanying .jgw files.

https://rcpt.yousendit.com/1189606781/e17c9b5691e425ad7b00d3d51c223591
https://rcpt.yousendit.com/1189606781/e17c9b5691e425ad7b00d3d51c223591

My process was to geotiff them thus;

gdal_translate -of GTiff -q -co "TILED=YES" -co COMPRESS=JPEG
OSIM_SV8306.JPG OSIM_SV8306.tif

and then build index .shp file using gdaltindex. All files are in EPSG 27700
(British National Grid). I have also ntoiced that these "gaps" do not occur
in the north of the projection, just in the south if that helps.

--
View this message in context: http://osgeo-org.1803224.n2.nabble.com/Image-Mosaic-Gaps-tp6657350p6663564.html
Sent from the GeoServer - User mailing list archive at Nabble.com.

For further infomation I have also tried the following; Converting jpg to tiff files, creating new .tfw files and gdal_translate to geotiff with the same gaps appearing. Adding full projection information to the geotiffs eg; gdal_translate -of GTiff -a_srs “+proj=tmerc +lat_0=49 +lon_0=-2 +k=0.9996012717 +x_0=400000 +y_0=-100000 +ellps=airy +datum=OSGB36 +units=m +no_defs” -co TILED=YES -co COMPRESS=JPEG OSIM_TV4299.jpg OSIM_TV4299.tif but with the same gaps appearing as well. This one has me stumped but I have the feeling that it is projection related.


View this message in context: Re: Image Mosaic & Gaps
Sent from the GeoServer - User mailing list archive at Nabble.com.

and further to my example, the gaps and black areas started appearing after GeoServer upgrade to 2.1.1 which I did just after a Java upgrade to 1.6.0_26 (on Ubuntu 64-bit server 10.04 LTS).

If I try to revert to 2.1.0, Geoserver starts then crashes (I thought it should revert easily back to previous minor version?) so I’ve had to go back to 2.1.1.

This is how I generated the images that go into my ImageMosaic, which was working in 2.1.0 on previous Java version, so I doubt it’s a projection problem:

gdalwarp -s_srs “+proj=tmerc +lat_0=0 +lon_0=19 +k=1 +x_0=0 +y_0=0 +ellps=WGS84 +datum=WGS84 +units=m +no_defs” -t_srs “EPSG:3857” -r cubic -multi -tr 0.5 0.5 -of GTiff file_a /tmp/file_b

then

gdal_translate -co “TILED=YES” -co “COMPRESS=JPEG” -co “PHOTOMETRIC=YCBCR” -co “JPEG_QUALITY=75” -projwin ulx uly lrx lry /tmp/file_b file_c

then

gdaladdo -r average --config COMPRESS_OVERVIEW JPEG --config PHOTOMETRIC_OVERVIEW YCBCR --config INTERLEAVE_OVERVIEW PIXEL file_c 2 4 8 16 32 64

On Tue, 2011-08-09 at 05:44 -0700, AndyC wrote:

For further infomation I have also tried the following; Converting jpg to tiff files, creating new .tfw files and gdal_translate to geotiff with the same gaps appearing. Adding full projection information to the geotiffs eg; gdal_translate -of GTiff -a_srs “+proj=tmerc +lat_0=49 +lon_0=-2 +k=0.9996012717 +x_0=400000 +y_0=-100000 +ellps=airy +datum=OSGB36 +units=m +no_defs” -co TILED=YES -co COMPRESS=JPEG OSIM_TV4299.jpg OSIM_TV4299.tif but with the same gaps appearing as well. This one has me stumped but I have the feeling that it is projection related.


View this message in context: Re: Image Mosaic & Gaps
Sent from the GeoServer - User mailing list archive at Nabble.com.

------------------------------------------------------------------------------
uberSVN's rich system and user administration capabilities and model 
configuration take the hassle out of deploying and managing Subversion and 
the tools developers use with it. Learn more about uberSVN and get a free 
download at:  [http://p.sf.net/sfu/wandisco-dev2dev](http://p.sf.net/sfu/wandisco-dev2dev)
_______________________________________________
Geoserver-users mailing list
[Geoserver-users@lists.sourceforge.net](mailto:Geoserver-users@lists.sourceforge.net)
[https://lists.sourceforge.net/lists/listinfo/geoserver-users](https://lists.sourceforge.net/lists/listinfo/geoserver-users)


regards

Gavin

Gavin Fleming
http://afrispatial.co.za
t: 0218620670
c: 0845965680
f: 0866164820

<br><br><br>

Gavin Do you think it is a Java issue then? I am currently running Jave SE 1.6.0_23-b03, I will try the latest JDK version on another machine and see if it makes any difference? Andy


View this message in context: Re: Image Mosaic & Gaps
Sent from the GeoServer - User mailing list archive at Nabble.com.

Okay after much pain I have worked out what the problem is. Using the same data set I tried using GeoServer 2.1.0 instead of Geoserver 2.1.1 and my imagemosaic worked without any errors. Using the same Tomcat 7 deployment I first created the imagemosaic in 2.1.1 (and had the same problem as the screen shot in the first post), then undeployed Geoserver and deployed the previous version of 2.1.0. Connected to the same data via imagemosaic and it worked perfectly. It would appear that there is a bug in imagemosaic of 2.1.1. How do I raise a ticket to have this looked at?


View this message in context: Re: Image Mosaic & Gaps (Solved issue but possible 2.1.1 Bug)
Sent from the GeoServer - User mailing list archive at Nabble.com.

On Wed, Aug 10, 2011 at 12:40 PM, AndyC <andyclark666@anonymised.com> wrote:

Okay after much pain I have worked out what the problem is. Using the same
data set I tried using GeoServer 2.1.0 instead of Geoserver 2.1.1 and my
imagemosaic worked without any errors. Using the same Tomcat 7 deployment I
first created the imagemosaic in 2.1.1 (and had the same problem as the
screen shot in the first post), then undeployed Geoserver and deployed the
previous version of 2.1.0. Connected to the same data via imagemosaic and it
worked perfectly. It would appear that there is a bug in imagemosaic of
2.1.1. How do I raise a ticket to have this looked at?

jira.codehaus.org, and attach a data sample and a wms request to reproduce
the issue.
And then wait, developers have a day job and community fixes are often
done in their personal spare time.

Cheers
Andrea

--
-------------------------------------------------------
Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

-------------------------------------------------------

On Tue, Aug 9, 2011 at 3:55 PM, Gavin <gavin@anonymised.com> wrote:

and further to my example, the gaps and black areas started appearing after GeoServer upgrade to 2.1.1 which I did just after a Java upgrade to 1.6.0_26 (on Ubuntu 64-bit server 10.04 LTS).

If I try to revert to 2.1.0, Geoserver starts then crashes (I thought it should revert easily back to previous minor version?) so I’ve had to go back to 2.1.1.

Yep, supposedly one should be able to go back and forth with the same series.
To know why it happened we need a stack trace though

Cheers
Anrdea

This is how I generated the images that go into my ImageMosaic, which was working in 2.1.0 on previous Java version, so I doubt it’s a projection problem:

gdalwarp -s_srs “+proj=tmerc +lat_0=0 +lon_0=19 +k=1 +x_0=0 +y_0=0 +ellps=WGS84 +datum=WGS84 +units=m +no_defs” -t_srs “EPSG:3857” -r cubic -multi -tr 0.5 0.5 -of GTiff file_a /tmp/file_b

then

gdal_translate -co “TILED=YES” -co “COMPRESS=JPEG” -co “PHOTOMETRIC=YCBCR” -co “JPEG_QUALITY=75” -projwin ulx uly lrx lry /tmp/file_b file_c

then

gdaladdo -r average --config COMPRESS_OVERVIEW JPEG --config PHOTOMETRIC_OVERVIEW YCBCR --config INTERLEAVE_OVERVIEW PIXEL file_c 2 4 8 16 32 64

Hmm… this looks nothing like the data Andy sent me. Are you working on the same project?

Cheers
Andrea

Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf


On Mon, Aug 8, 2011 at 11:06 AM, AndyC <andyclark666@anonymised.com> wrote:

Hi Andrea

Attached are four 1km jpegs and accompanying .jgw files.

https://rcpt.yousendit.com/1189606781/e17c9b5691e425ad7b00d3d51c223591
https://rcpt.yousendit.com/1189606781/e17c9b5691e425ad7b00d3d51c223591

My process was to geotiff them thus;

gdal_translate -of GTiff -q -co "TILED=YES" -co COMPRESS=JPEG
OSIM_SV8306.JPG OSIM_SV8306.tif

and then build index .shp file using gdaltindex. All files are in EPSG 27700
(British National Grid). I have also ntoiced that these "gaps" do not occur
in the north of the projection, just in the south if that helps.

Hi Andy,
I just played with your files, but I'm getting no gaps, here are two images
I got from the files used as is, and reprojected into 4326:

http://demo.geo-solutions.it/share/nogaps_27700.jpg
http://demo.geo-solutions.it/share/nogaps_4326.jpg

I made one significant change to your conversion process though, forced
the right projection into the geotiff files:

find . -name "*.JPG" -exec gdal_translate -a_srs EPSG:27700 -of GTiff
-q -co "TILED=YES" -co COMPRESS=JPEG {} {}.tif \;

And then moved all the tiff files in their own folder.

I did not notice any gaps in the few projections I've tried, but sure the
display was slow. This is due partly to the jpeg compression (files
are smaller, but it takes more work to read them) and partly because
overviews were missing
Added the overviews to the files with the following:

find . -name "*.tif" -exec gdaladdo -r gauss {} 2 4 8 26 32 64 \;

and now it displays both properly and fast

Cheers
Andrea

--
-------------------------------------------------------
Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

-------------------------------------------------------

Morning Andrea

Would you be kind enough to share the image mosaic shape file that you
created for the tiles I posted?

When I repeat the process as you mentioned thus;

gdal_translate -a_srs EPSG:27700 -of GTiff -q -co "TILED=YES" -co
COMPRESS=JPEG in.jpg out.tif

and then created the shape file using the attached .prj and .properties
http://osgeo-org.1803224.n2.nabble.com/file/n6686753/sv_monday.prj
sv_monday.prj and
http://osgeo-org.1803224.n2.nabble.com/file/n6686753/sv_monday.properties
sv_monday.properties i still get gaps.

So it must be something I am doing or misunderstanding in 2.1.1.

Thanks

--
View this message in context: http://osgeo-org.1803224.n2.nabble.com/Image-Mosaic-Gaps-possible-bug-in-2-1-1-tp6657350p6686753.html
Sent from the GeoServer - User mailing list archive at Nabble.com.

On Mon, Aug 15, 2011 at 10:49 AM, AndyC <andyclark666@anonymised.com> wrote:

Morning Andrea

Would you be kind enough to share the image mosaic shape file that you
created for the tiles I posted?

I think I can do better, here is the mosaic:
http://demo.geo-solutions.it/share/gaps.zip

So it must be something I am doing or misunderstanding in 2.1.1.

Or maybe it's a bug solved since the last release, I never use the released
versions, always today's source build (releases are carved in stone, only
the current source can be fixed).

Can you try a nightly build you too?
http://gridlock.opengeo.org/geoserver/2.1.x/

Cheers
Andrea

--
-------------------------------------------------------
Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

-------------------------------------------------------

AndyC wrote:

Hi

I am currently running GeoServer 2.1.1 and have run in to a problem with
creating an image mosaic and the appearance of gaps between the images.

My creation process is thus;

Source raster is 1km square imagery tiles in jpg with accompnaying .jgw
files.
I convert these to GeoTIFF using gdal_translate, and create index file
using gdaltindex.
When connected to Geoserver and drawn up you can see white gaps between
the images which reduce in size as I zoom in, until they disappear at the
highest zoom level.

I have never experienced this problem with the numerous other raster
datasets I have been using.
Does anyone have any advice?

Cheers

http://osgeo-org.1803224.n2.nabble.com/file/n6657350/image1.jpg
http://osgeo-org.1803224.n2.nabble.com/file/n6657350/image2.jpg

I have a similar issue.
I set up a mosaic with the mosaic plug in of the geoserver.
The set of rasters are continous and non-overlapping.
Anyway there are white gaps between the single raster data sets when i
display the mosaic in the layer preview.
I already tried different configurations but the result in the GeoServer
Layer Preview as well as in the GeoExplorer looks like the picture added.

Can you help me?

Here are the main configuration details and the content of the cofiguration
file:

GeoServer Version 2.1-SNAPSHOT
Raster Format: tif

• MaxAllowedTiles: 2147483647
• BackgroundValues: default value.
• OutputTransparentColor: ‘no color’.
• InputTransparentColor: #FFFFFF.
• AllowMultiThreading: true
• USE_JAI_IMAGEREAD: false.
• SUGGESTED_TILE_SIZE: 512,512.

#-Automagically created from GeoTools-
#Tue Jul 05 16:03:40 CEST 2011
Name=TOPOMAP
LocationAttribute=location
AbsolutePath=false
ExpandToRGB=false
Levels=5.700000000000209E-5,5.699999999999986E-5
Caching=true
LevelsNum=1
Envelope2D=113.750,3.250 114.50000199999999,4.0000019999999994
NumFiles=9
Heterogeneous=false

http://osgeo-org.1803224.n2.nabble.com/file/n6746894/mosaic1.jpg

--
View this message in context: http://osgeo-org.1803224.n2.nabble.com/Image-Mosaic-Gaps-possible-bug-in-2-1-1-tp6657350p6746894.html
Sent from the GeoServer - User mailing list archive at Nabble.com.

Dominik Affolter wrote:

AndyC wrote:

Hi

I am currently running GeoServer 2.1.1 and have run in to a problem with
creating an image mosaic and the appearance of gaps between the images.

My creation process is thus;

Source raster is 1km square imagery tiles in jpg with accompnaying .jgw
files.
I convert these to GeoTIFF using gdal_translate, and create index file
using gdaltindex.
When connected to Geoserver and drawn up you can see white gaps between
the images which reduce in size as I zoom in, until they disappear at the
highest zoom level.

I have never experienced this problem with the numerous other raster
datasets I have been using.
Does anyone have any advice?

Cheers

http://osgeo-org.1803224.n2.nabble.com/file/n6657350/image1.jpg
http://osgeo-org.1803224.n2.nabble.com/file/n6657350/image2.jpg

I have a similar issue.
I set up a mosaic with the mosaic plug in of the geoserver.
The set of rasters are continous and non-overlapping.
Anyway there are white gaps between the single raster data sets when i
display the mosaic in the layer preview.
I already tried different configurations but the result in the GeoServer
Layer Preview as well as in the GeoExplorer looks like the picture added.

Can you help me?

Here are the main configuration details and the content of the
cofiguration file:

GeoServer Version 2.1-SNAPSHOT
Raster Format: tif

• MaxAllowedTiles: 2147483647
• BackgroundValues: default value.
• OutputTransparentColor: ‘no color’.
• InputTransparentColor: #FFFFFF.
• AllowMultiThreading: true
• USE_JAI_IMAGEREAD: false.
• SUGGESTED_TILE_SIZE: 512,512.

#-Automagically created from GeoTools-
#Tue Jul 05 16:03:40 CEST 2011
Name=TOPOMAP
LocationAttribute=location
AbsolutePath=false
ExpandToRGB=false
Levels=5.700000000000209E-5,5.699999999999986E-5
Caching=true
LevelsNum=1
Envelope2D=113.750,3.250 114.50000199999999,4.0000019999999994
NumFiles=9
Heterogeneous=false

http://osgeo-org.1803224.n2.nabble.com/file/n6746894/mosaic1.jpg

I still have this issue with my version of GeoServer.
I work with the latest version of the OpenGeo Suite Community Edition.
Perhaps this problem only occurs in this version of GeoServer.
Here are the gdalinfo outputs of my GeoTiff, I tried it with different
configurations:

http://osgeo-org.1803224.n2.nabble.com/file/n6831159/rasterInfo1.txt
rasterInfo1.txt
http://osgeo-org.1803224.n2.nabble.com/file/n6831159/rasterInfo2.txt
rasterInfo2.txt

I spent a lot of time on this issue, and did not make progress. These
mosaics are a central point in our project.
Please, can anybody help me or give me a hint to solve this issue.

I think to clarify this topic would not only bring forward me:
http://osgeo-org.1803224.n2.nabble.com/is-it-possible-to-use-imagemosaic-plugin-for-georasters-td6779952.html
http://osgeo-org.1803224.n2.nabble.com/Image-Mosaic-amp-Overviews-td6687068.html
http://osgeo-org.1803224.n2.nabble.com/Problems-of-white-color-tile-borders-amp-gaps-in-raster-imagery-display-td6741946.html

--
View this message in context: http://osgeo-org.1803224.n2.nabble.com/Image-Mosaic-Gaps-possible-bug-in-2-1-1-tp6657350p6831159.html
Sent from the GeoServer - User mailing list archive at Nabble.com.

On Mon, Sep 26, 2011 at 10:43 AM, Dominik Affolter <d1affolter@anonymised.com> wrote:

I still have this issue with my version of GeoServer.

I work with the latest version of the OpenGeo Suite Community Edition.
Perhaps this problem only occurs in this version of GeoServer.

You should probably ask for help on the OpenGeo support forums, indeed we
cannot help with that one.

Here are the gdalinfo outputs of my GeoTiff, I tried it with different
configurations:

http://osgeo-org.1803224.n2.nabble.com/file/n6831159/rasterInfo1.txt
rasterInfo1.txt
http://osgeo-org.1803224.n2.nabble.com/file/n6831159/rasterInfo2.txt
rasterInfo2.txt

I spent a lot of time on this issue, and did not make progress. These
mosaics are a central point in our project.
Please, can anybody help me or give me a hint to solve this issue.

I’d suggest to try out a vanilla GeoServer build, a nightly one of the
stable series, here:
http://gridlock.opengeo.org/geoserver/2.1.x/

If the problem persist try to generate a smaller mosaic, maybe just 2-3
tiles, that shows the problem, and attach them to a bug report
on jira.codehaus.org

Cheers
Andrea

Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf


geowolf wrote:

You should probably ask for help on the OpenGeo support forums, indeed we
cannot help with that one.

I already posted this problem on the OpenGeo forum but did not suceed there:

http://getsatisfaction.com/opengeo/topics/mosaic_gaps_between_granules

they refered me to this mailing list.

geowolf wrote:

I'd suggest to try out a vanilla GeoServer build, a nightly one of the
stable series, here:
http://gridlock.opengeo.org/geoserver/2.1.x/

Thanks, I'll try this and inform here on this list about the progress.

--
View this message in context: http://osgeo-org.1803224.n2.nabble.com/Image-Mosaic-Gaps-possible-bug-in-2-1-1-tp6657350p6831550.html
Sent from the GeoServer - User mailing list archive at Nabble.com.

geowolf wrote:

You should probably ask for help on the OpenGeo support forums, indeed we
cannot help with that one.

I already posted this problem on the OpenGeo forum but did not suceed there:

http://getsatisfaction.com/opengeo/topics/mosaic_gaps_between_granules

they refered me to this mailing list.

geowolf wrote:

I'd suggest to try out a vanilla GeoServer build, a nightly one of the
stable series, here:
http://gridlock.opengeo.org/geoserver/2.1.x/

Thanks, I'll try this and inform here on this list about the progress.

--
View this message in context: http://osgeo-org.1803224.n2.nabble.com/Image-Mosaic-Gaps-possible-bug-in-2-1-1-tp6657350p6831558.html
Sent from the GeoServer - User mailing list archive at Nabble.com.

Dominik Affolter wrote:

geowolf wrote:

I'd suggest to try out a vanilla GeoServer build, a nightly one of the
stable series, here:
http://gridlock.opengeo.org/geoserver/2.1.x/

Thanks, I'll try this and inform here on this list about the progress.

It works! With the latest stand-alone version of geoserver.
But I have still the issue with the GeoServer Version of OpenGeo Suite,
respectively the technique it is deployed on the server.
I work with the default component tomcat 6 and the folowing JVM version:
Sun Microsystems Inc.: 1.6.0_26 (Java HotSpot(TM) 64-Bit Server VM).
Could this be the problem?

--
View this message in context: http://osgeo-org.1803224.n2.nabble.com/Image-Mosaic-Gaps-possible-bug-in-2-1-1-tp6657350p6854274.html
Sent from the GeoServer - User mailing list archive at Nabble.com.

Dear Geoserver Users I am getting same error (Image Gabs) in my application . Currently i am using Geoserver 2.1.0 version and image mosaic jdbc – My data s taking from oracle database . Please provide any advice to me .
T.Manikandan GIS R&D


View this message in context: Image Mosaic JDBC & Gaps
Sent from the GeoServer - User mailing list archive at Nabble.com.