[Geoserver-devel] ImageMosaic error when no Data in a tile

Hi all,

I have a problem with an ImageMosaic Layer that might be linked to this issue : http://jira.codehaus.org/browse/GEOS-3628

I am using Geoserver 2.1.3.

My problem is that I am combining two layers in one wms request :

· Background layer has data everywhere

· The overlaying layer is a Mosaic with no data in some places(where I would like to see the background layer)

This issue GEOS-3628 has been closed because not reproduced but here is maybe a way to reproduce just considering the overlaying layer because some tiles throws an exception when there is no data in the whole tile area=>so combination with transparency doen’t work for those tiles.

To reproduce:

· Use the nurc:mosaic layer in sample data and remove some data in the shapefile to have something like this in the preview

· Use this URL http://localhost:8080/geoserver/nurc/wms?service=WMS&version=1.1.0&request=GetMap&layers=nurc:mosaic&styles=&bbox=6.346,36.492,20.83,46.591&width=1000&height=800&srs=EPSG:4326&format=application/openlayers&Exceptions=application/vnd.ogc.se_inimage

· Select Tiling=Tiled in options and zoom into one blank area, result should be like this

As is, this is not really a problem because Openlayers can ignore those tiles in error but when combining with another layer I can’t see the background layer in theses areas.

*When I set OutputTransparentColor to black it works fine for tiles where there is a part with “data” and a part with « no data » but tiles with « no data » only return same exception

*When I remove OutputTransparentColor it works for tiles with « no data » (I see the background layer) but mixed tiles display the “no data” part in black…

Thanks in advance for your help, please don’t hesitate if this is not clear enough,

Damien DESPRES - Ingénieur Développement
+33 (0)5 62 24 70 64


24,rue Hermès - BP 12113
31521 Ramonville Saint-Agne cedex
Tel: +33 (0)5 62 24 70 00
Fax: +33 (0)5 62 24 70 01
www.magellium.fr

Description : untitled

Ciao Damien,
I just tried to do the same thing with 2.1.x without any problems.
I would recommend testing 2.1.x (stable branch, no api changes, you should be safe) and then report back to us.

Regards,
Simone Giannecchini

Ing. Simone Giannecchini
GeoSolutions S.A.S.
Founder

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 333 8128928

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/simonegiannecchini
http://twitter.com/simogeo


On Thu, Feb 16, 2012 at 1:53 PM, Despres <damien.despres@anonymised.com> wrote:

Hi all,

I have a problem with an ImageMosaic Layer that might be linked to this issue : http://jira.codehaus.org/browse/GEOS-3628

I am using Geoserver 2.1.3.

My problem is that I am combining two layers in one wms request :

· Background layer has data everywhere

· The overlaying layer is a Mosaic with no data in some places(where I would like to see the background layer)

This issue GEOS-3628 has been closed because not reproduced but here is maybe a way to reproduce just considering the overlaying layer because some tiles throws an exception when there is no data in the whole tile area=>so combination with transparency doen’t work for those tiles.

To reproduce:

· Use the nurc:mosaic layer in sample data and remove some data in the shapefile to have something like this in the preview

· Use this URL http://localhost:8080/geoserver/nurc/wms?service=WMS&version=1.1.0&request=GetMap&layers=nurc:mosaic&styles=&bbox=6.346,36.492,20.83,46.591&width=1000&height=800&srs=EPSG:4326&format=application/openlayers&Exceptions=application/vnd.ogc.se_inimage

· Select Tiling=Tiled in options and zoom into one blank area, result should be like this

As is, this is not really a problem because Openlayers can ignore those tiles in error but when combining with another layer I can’t see the background layer in theses areas.

*When I set OutputTransparentColor to black it works fine for tiles where there is a part with “data” and a part with « no data » but tiles with « no data » only return same exception

*When I remove OutputTransparentColor it works for tiles with « no data » (I see the background layer) but mixed tiles display the “no data” part in black…

Thanks in advance for your help, please don’t hesitate if this is not clear enough,

Damien DESPRES - Ingénieur Développement
+33 (0)5 62 24 70 64


24,rue Hermès - BP 12113
31521 Ramonville Saint-Agne cedex
Tel: +33 (0)5 62 24 70 00
Fax: +33 (0)5 62 24 70 01
www.magellium.fr

Description : untitled


Virtualization & Cloud Management Using Capacity Planning
Cloud computing makes use of virtualization - but cloud computing
also focuses on allowing computing to be delivered as a service.
http://www.accelacomm.com/jaw/sfnl/114/51521223/


Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

Hi Simone,

Thanks for your help, I’ve tried with latest 2.1.x nightly build :

· Build Information

  • Version 2.1-SNAPSHOT
  • Subversion Revision 16772
  • Build Date 20-Feb-2012 05:02
  • GeoTools Version 2.7-SNAPSHOT (rev 38575)

#### But I still face the same problem.#### An extract of my logs is attached with the mosaic_sample shapefiles I am using to reproduce the issue. I have delete two features in sample data.#### Don’t’ hesitate If you need more information to reproduce. Here is the URL of tile in error:#### http://localhost:8080/geoserver/nurc/wms?EXCEPTIONS=application%2Fvnd.ogc.se_inimage&LAYERS=nurc%3Amosaic&STYLES=&FORMAT=image%2Fjpeg&TILED=true&TILESORIGIN=6.346%2C36.492&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&SRS=EPSG%3A4326&BBOX=13.135375,41.470875,13.588,41.9235&WIDTH=256&HEIGHT=256#### #### Thanks#### #### Damien

De : simboss1@anonymised.com [mailto:simboss1@anonymised.com] De la part de Simone Giannecchini
Envoyé : dimanche 19 février 2012 23:20
À : Despres
Cc : geoserver-devel@anonymised.comet
Objet : Re: [Geoserver-devel] ImageMosaic error when no Data in a tile

Ciao Damien,

I just tried to do the same thing with 2.1.x without any problems.

I would recommend testing 2.1.x (stable branch, no api changes, you should be safe) and then report back to us.

Regards,
Simone Giannecchini

Ing. Simone Giannecchini
GeoSolutions S.A.S.
Founder

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 333 8128928

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/simonegiannecchini
http://twitter.com/simogeo


On Thu, Feb 16, 2012 at 1:53 PM, Despres <damien.despres@anonymised.com…2904…> wrote:

Hi all,

I have a problem with an ImageMosaic Layer that might be linked to this issue : http://jira.codehaus.org/browse/GEOS-3628

I am using Geoserver 2.1.3.

My problem is that I am combining two layers in one wms request :

· Background layer has data everywhere

· The overlaying layer is a Mosaic with no data in some places(where I would like to see the background layer)

This issue GEOS-3628 has been closed because not reproduced but here is maybe a way to reproduce just considering the overlaying layer because some tiles throws an exception when there is no data in the whole tile area=>so combination with transparency doen’t work for those tiles.

To reproduce:

· Use the nurc:mosaic layer in sample data and remove some data in the shapefile to have something like this in the preview

· Use this URL http://localhost:8080/geoserver/nurc/wms?service=WMS&version=1.1.0&request=GetMap&layers=nurc:mosaic&styles=&bbox=6.346,36.492,20.83,46.591&width=1000&height=800&srs=EPSG:4326&format=application/openlayers&Exceptions=application/vnd.ogc.se_inimage

· Select Tiling=Tiled in options and zoom into one blank area, result should be like this

As is, this is not really a problem because Openlayers can ignore those tiles in error but when combining with another layer I can’t see the background layer in theses areas.

*When I set OutputTransparentColor to black it works fine for tiles where there is a part with “data” and a part with « no data » but tiles with « no data » only return same exception

*When I remove OutputTransparentColor it works for tiles with « no data » (I see the background layer) but mixed tiles display the “no data” part in black…

Thanks in advance for your help, please don’t hesitate if this is not clear enough,

Damien DESPRES - Ingénieur Développement
+33 (0)5 62 24 70 64


24,rue Hermès - BP 12113
31521 Ramonville Saint-Agne cedex
Tel: +33 (0)5 62 24 70 00
Fax: +33 (0)5 62 24 70 01
www.magellium.fr

Description : untitled


Virtualization & Cloud Management Using Capacity Planning
Cloud computing makes use of virtualization - but cloud computing
also focuses on allowing computing to be delivered as a service.
http://www.accelacomm.com/jaw/sfnl/114/51521223/


Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

(attachments)

extractlog.log (36 KB)
mosaic_sample.zip (2.23 KB)

On Mon, Feb 20, 2012 at 10:01 AM, Despres <damien.despres@anonymised.com> wrote:

Hi Simone,

Thanks for your help, I’ve tried with latest 2.1.x nightly build :

· Build Information

  • Version 2.1-SNAPSHOT
  • Subversion Revision 16772
  • Build Date 20-Feb-2012 05:02
  • GeoTools Version 2.7-SNAPSHOT (rev 38575)

I think there might be some misunderstanding here, as far as I can see Damien has
removed records from the shapefile, not removed images from disk, right?

Cheers
Andrea

Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf


Hi,

I have first set the location field to « empty string » in the shapefile but i have also tried with removing the feature and the results are the same.

But you’re right I have not tried to remove images without any changes to the shapefiles. I will try this.

Thanks,

Damien

De : andrea.aime@anonymised.com [mailto:andrea.aime@anonymised.com] De la part de Andrea Aime
Envoyé : lundi 20 février 2012 10:44
À : Despres
Cc : Simone Giannecchini; geoserver-devel@lists.sourceforge.net
Objet : Re: [Geoserver-devel] ImageMosaic error when no Data in a tile

On Mon, Feb 20, 2012 at 10:01 AM, Despres <damien.despres@anonymised.com> wrote:

Hi Simone,

Thanks for your help, I’ve tried with latest 2.1.x nightly build :

· Build Information

  • Version 2.1-SNAPSHOT
  • Subversion Revision 16772
  • Build Date 20-Feb-2012 05:02
  • GeoTools Version 2.7-SNAPSHOT (rev 38575)

I think there might be some misunderstanding here, as far as I can see Damien has

removed records from the shapefile, not removed images from disk, right?

Cheers

Andrea

Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf


Hi,

I’ve the same issue if i remove an image without any change in the shape ;-(

Damien

De : Despres [mailto:damien.despres@anonymised.com]
Envoyé : lundi 20 février 2012 10:58
À : ‘Andrea Aime’
Cc : geoserver-devel@lists.sourceforge.net
Objet : Re: [Geoserver-devel] ImageMosaic error when no Data in a tile

Hi,

I have first set the location field to « empty string » in the shapefile but i have also tried with removing the feature and the results are the same.

But you’re right I have not tried to remove images without any changes to the shapefiles. I will try this.

Thanks,

Damien

De : andrea.aime@anonymised.com [mailto:andrea.aime@anonymised.com] De la part de Andrea Aime
Envoyé : lundi 20 février 2012 10:44
À : Despres
Cc : Simone Giannecchini; geoserver-devel@anonymised.comforge.net
Objet : Re: [Geoserver-devel] ImageMosaic error when no Data in a tile

On Mon, Feb 20, 2012 at 10:01 AM, Despres <damien.despres@anonymised.com04…> wrote:

Hi Simone,

Thanks for your help, I’ve tried with latest 2.1.x nightly build :

· Build Information

  • Version 2.1-SNAPSHOT
  • Subversion Revision 16772
  • Build Date 20-Feb-2012 05:02
  • GeoTools Version 2.7-SNAPSHOT (rev 38575)

I think there might be some misunderstanding here, as far as I can see Damien has

removed records from the shapefile, not removed images from disk, right?

Cheers

Andrea

Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf


Hi all,

I have pushed further the investigation into the ImageMosaic plugin.

Here is a test case I have added to ImageMosaicReaderTest to reproduce the exception with the data sample provided in attachement.

@Test

public void testMosaic_sample() throws Exception {

URL dds = TestData.url(this, “mosaic/mosaic.shp”);

final AbstractGridFormat format = getFormat(dds);

final ImageMosaicReader reader = getReader(dds, format);

final ParameterValue gg = AbstractGridFormat.READ_GRIDGEOMETRY2D.createValue();

// BBOX corresponding to a tile in error => Zone with no data only

// 13.135375,41.470875,13.588,41.9235&WIDTH=256&HEIGHT=256

double x1 = 13.135375;

double y1 = 41.470875;

double x2 = 13.588;

double y2 = 41.9235;

final GeneralEnvelope cropEnvelope = new GeneralEnvelope(new double { x1, y1 }, new double { x2, y2 });

cropEnvelope.setCoordinateReferenceSystem(reader.getCrs());

gg.setValue(new GridGeometry2D(PixelInCell.CELL_CENTER, reader.getOriginalGridToWorld(PixelInCell.CELL_CENTER), cropEnvelope, null));

final ParameterValue outTransp = ImageMosaicFormat.OUTPUT_TRANSPARENT_COLOR.createValue();

outTransp.setValue(Color.black);

final GridCoverage2D read = reader.read(new GeneralParameterValue { gg, outTransp });

}

//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

What I have found is that in my case, in this function of ImageWorker, the colorModel is a FloatDoubleColorModel and this case is not handled.

public final ImageWorker makeColorTransparent(final Color transparentColor)

throws IllegalStateException

{

if (transparentColor == null) {

throw new IllegalArgumentException(

Errors.format(ErrorKeys.NULL_ARGUMENT_$1, “transparentColor”));

}

final ColorModel cm = image.getColorModel();

if (cm instanceof IndexColorModel) {

return maskIndexColorModel(transparentColor);

} else if (cm instanceof ComponentColorModel) {

switch (image.getSampleModel().getDataType()) {

case DataBuffer.TYPE_BYTE: {

return maskComponentColorModelByte(transparentColor);

}

// Add other types here if we support them…

}

}

throw new IllegalStateException(Errors.format(ErrorKeys.UNSUPPORTED_DATA_TYPE));

}

So I’ve tested a quick and dirty fix in RasterLayerResponse and that solved my problem (but it’s dirty)

private RenderedImage postProcessRaster(RenderedImage image) {

// alpha on the final mosaic

if (finalTransparentColor != null) {

if (LOGGER.isLoggable(Level.FINE))

LOGGER.fine(“Support for alpha on final mosaic”);

final ColorModel cm = image.getColorModel();

if (cm instanceof FloatDoubleColorModel) {

return image;

}

return ImageUtilities.maskColor(finalTransparentColor, image);

}

Don’t hesitate if you need more informations,

Regards,

Damien

De : Despres [mailto:damien.despres@anonymised.com]
Envoyé : lundi 20 février 2012 11:09
À : ‘Andrea Aime’
Cc : geoserver-devel@lists.sourceforge.net
Objet : Re: [Geoserver-devel] ImageMosaic error when no Data in a tile

Hi,

I’ve the same issue if i remove an image without any change in the shape ;-(

Damien

De : Despres [mailto:damien.despres@anonymised.com]
Envoyé : lundi 20 février 2012 10:58
À : ‘Andrea Aime’
Cc : geoserver-devel@lists.sourceforge.net
Objet : Re: [Geoserver-devel] ImageMosaic error when no Data in a tile

Hi,

I have first set the location field to « empty string » in the shapefile but i have also tried with removing the feature and the results are the same.

But you’re right I have not tried to remove images without any changes to the shapefiles. I will try this.

Thanks,

Damien

De : andrea.aime@anonymised.com [mailto:andrea.aime@anonymised.com] De la part de Andrea Aime
Envoyé : lundi 20 février 2012 10:44
À : Despres
Cc : Simone Giannecchini; geoserver-devel@lists.sourceforge.net
Objet : Re: [Geoserver-devel] ImageMosaic error when no Data in a tile

On Mon, Feb 20, 2012 at 10:01 AM, Despres <damien.despres@anonymised.com> wrote:

Hi Simone,

Thanks for your help, I’ve tried with latest 2.1.x nightly build :

· Build Information

  • Version 2.1-SNAPSHOT
  • Subversion Revision 16772
  • Build Date 20-Feb-2012 05:02
  • GeoTools Version 2.7-SNAPSHOT (rev 38575)

I think there might be some misunderstanding here, as far as I can see Damien has

removed records from the shapefile, not removed images from disk, right?

Cheers

Andrea

Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf


(attachments)

mosaic.zip (156 KB)