[Geoserver-users] How to set InputTransparentColor for MrSid layers?

Many thanks for your helpful response Simone, much appreciated. I am now using an ImageMosaic source which was very easy to do, and as you say it supports setting of the transparent colour for my MrSid images which works great.

One thing I did notice is that it can be slower to generate tile images when using ImageMosaic when compared to using a Layer Group for the same set of images. It only seems to be slower though when the requested tiles cover an area which requires imagery from more than one of the source images. By way of example, requesting 35 tiles at TMS zoom level 19, all coming from a single source image, took 5 seconds to serve on my platform with both the ImageMosaic and Layer Group approaches, but requesting 35 tiles at zoom level 15, which in this case requires parts from 3 of the source images, takes 65 seconds using ImageMosaic but only 16 seconds using a LayerGroup.

I realise the ImageMosaic would have to do some extra work to take the transparent colour into account at the image borders, so to check this I tried deleting the InputTansparentColor for the ImageMosaic. This speeded things up from 65 seconds to 37 seconds, but this is still less than half the speed of the layer group. I guess the ImageMosaic must also be doing something extra that the layer group is not.

As you suggested, I filed a JIRA request for an enhancement for layer groups to support setting of a transparent colour at https://jira.codehaus.org/browse/GEOS-5501.

Regards

Tim

-----Original Message-----
From: Simone Giannecchini [mailto:simone.giannecchini@anonymised.com]
Sent: 01 December 2012 09:08
To: Tim Hayton
Cc: geoserver-users@lists.sourceforge.net
Subject: Re: [Geoserver-users] How to set InputTransparentColor for MrSid layers?

Ciao Tim,
please, find my aswer inline below....

Regards,
Simone Giannecchini

Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Simone Giannecchini
@simogeo
Founder/Director

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 333 8128928

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

On Wed, Nov 28, 2012 at 4:08 PM, Tim Hayton <tim.hayton@anonymised.com> wrote:

I've just started using GeoServer/GeoWebCache to serve map imagery to
web clients which use the TMS service. My source images are in either
GeoTIFF or MrSid format, and are satellite imagery for adjacent areas
which slightly overlap. I have successfully set these up in GeoServer
as data sources and layers, and then made a Layer Group which combines
each individual image layer into a single group so I can easily get
tiles for anywhere in the coverage area via a single TMS request.

I would suggest to use the ImageMosaic plugin.
I am assuming the images have:
- same CRS
- same data type
- same number of bands
- similar resolution

With the ImageMosaic plugin you do not need to use the layer group and your GetCapabilities will be much smaller.
ImageMosaic handle the transparent color both on put images as well as on the output image. Check the layer configuration.
Last but not least it does handle MrSID as well as GeoTiff files.

My problem is related to nodata areas, which are black in the
individual images, at the borders of each image file where there is an
overlap with the next image. For the GeoTIFF layers, I can easily set
the InputTransparentColor in the GeoServer Layers -> Coverage
Parameters screen, and this works great with the combined layer group
correctly serving the actual data rather than the black pixels for tiles where two images overlap.
However, for my MrSid images, there is no InputTransparentColor option
in the Layers screen, and when I get border tiles with TMS requests I
am often getting the black pixels in preference to the actual data
where tiles overlap.

See suggestion above about using ImageMosaic. That said you are right there is no support for input transparent color on GDAL Formats. Could file a jira enhancement report for this?
It would be dead easy to add it and I believe we should do it right away.

So my question is basically is there any way to set the
InputTransparentColor for layers based on MrSid data sources, or is
there something peculiar to this format that prevent this option from being set?

I also see the same problem if I get a tile layer preview from the web
interface, but interestingly if I ask for a layer preview (rather than
a tile layer preview) the I do not see the problem, and the image
layers blend correctly at the borders. I wonder then if the problem
might be related to the way GeoWebCache requests the images from
GeoServer in a way that causes the nodata to take preference somehow?

System details:

GeoServer 2.2.2 from windows installer

GDAL extensions

Windows XP

Oracle Java 1.7.0_09

Thanks in advance for any help!

Tim

--
MIRA Ltd

Watling Street, Nuneaton, Warwickshire, CV10 0TU, England
Registered in England and Wales No. 402570
VAT Registration GB 100 1464 84

The contents of this e-mail are confidential and are solely for the use of the intended recipient. If you receive this e-mail in error, please delete it and notify us either by e-mail, telephone or fax. You should not copy, forward or otherwise disclose the content of the e-mail as this is prohibited.

Ciao Tim,
can you paste here the gdalinfo output of some of you tiles as well as
an indication
of the configuration of the parameters in the mosaic layer page in GeoServer?

Regards,
Simone Giannecchini

Our support, Your Success! Visit http://opensdi.geo-solutions.it for
more information.

Ing. Simone Giannecchini
@simogeo
Founder/Director

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 333 8128928

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

On Thu, Dec 13, 2012 at 4:21 PM, Tim Hayton <tim.hayton@anonymised.com> wrote:

Many thanks for your helpful response Simone, much appreciated. I am now using an ImageMosaic source which was very easy to do, and as you say it supports setting of the transparent colour for my MrSid images which works great.

One thing I did notice is that it can be slower to generate tile images when using ImageMosaic when compared to using a Layer Group for the same set of images. It only seems to be slower though when the requested tiles cover an area which requires imagery from more than one of the source images. By way of example, requesting 35 tiles at TMS zoom level 19, all coming from a single source image, took 5 seconds to serve on my platform with both the ImageMosaic and Layer Group approaches, but requesting 35 tiles at zoom level 15, which in this case requires parts from 3 of the source images, takes 65 seconds using ImageMosaic but only 16 seconds using a LayerGroup.

I realise the ImageMosaic would have to do some extra work to take the transparent colour into account at the image borders, so to check this I tried deleting the InputTansparentColor for the ImageMosaic. This speeded things up from 65 seconds to 37 seconds, but this is still less than half the speed of the layer group. I guess the ImageMosaic must also be doing something extra that the layer group is not.

As you suggested, I filed a JIRA request for an enhancement for layer groups to support setting of a transparent colour at https://jira.codehaus.org/browse/GEOS-5501.

Regards

Tim

-----Original Message-----
From: Simone Giannecchini [mailto:simone.giannecchini@anonymised.com]
Sent: 01 December 2012 09:08
To: Tim Hayton
Cc: geoserver-users@lists.sourceforge.net
Subject: Re: [Geoserver-users] How to set InputTransparentColor for MrSid layers?

Ciao Tim,
please, find my aswer inline below....

Regards,
Simone Giannecchini

Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Simone Giannecchini
@simogeo
Founder/Director

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 333 8128928

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

On Wed, Nov 28, 2012 at 4:08 PM, Tim Hayton <tim.hayton@anonymised.com> wrote:

I've just started using GeoServer/GeoWebCache to serve map imagery to
web clients which use the TMS service. My source images are in either
GeoTIFF or MrSid format, and are satellite imagery for adjacent areas
which slightly overlap. I have successfully set these up in GeoServer
as data sources and layers, and then made a Layer Group which combines
each individual image layer into a single group so I can easily get
tiles for anywhere in the coverage area via a single TMS request.

I would suggest to use the ImageMosaic plugin.
I am assuming the images have:
- same CRS
- same data type
- same number of bands
- similar resolution

With the ImageMosaic plugin you do not need to use the layer group and your GetCapabilities will be much smaller.
ImageMosaic handle the transparent color both on put images as well as on the output image. Check the layer configuration.
Last but not least it does handle MrSID as well as GeoTiff files.

My problem is related to nodata areas, which are black in the
individual images, at the borders of each image file where there is an
overlap with the next image. For the GeoTIFF layers, I can easily set
the InputTransparentColor in the GeoServer Layers -> Coverage
Parameters screen, and this works great with the combined layer group
correctly serving the actual data rather than the black pixels for tiles where two images overlap.
However, for my MrSid images, there is no InputTransparentColor option
in the Layers screen, and when I get border tiles with TMS requests I
am often getting the black pixels in preference to the actual data
where tiles overlap.

See suggestion above about using ImageMosaic. That said you are right there is no support for input transparent color on GDAL Formats. Could file a jira enhancement report for this?
It would be dead easy to add it and I believe we should do it right away.

So my question is basically is there any way to set the
InputTransparentColor for layers based on MrSid data sources, or is
there something peculiar to this format that prevent this option from being set?

I also see the same problem if I get a tile layer preview from the web
interface, but interestingly if I ask for a layer preview (rather than
a tile layer preview) the I do not see the problem, and the image
layers blend correctly at the borders. I wonder then if the problem
might be related to the way GeoWebCache requests the images from
GeoServer in a way that causes the nodata to take preference somehow?

System details:

GeoServer 2.2.2 from windows installer

GDAL extensions

Windows XP

Oracle Java 1.7.0_09

Thanks in advance for any help!

Tim

--
MIRA Ltd

Watling Street, Nuneaton, Warwickshire, CV10 0TU, England
Registered in England and Wales No. 402570
VAT Registration GB 100 1464 84

The contents of this e-mail are confidential and are solely for the use of the intended recipient. If you receive this e-mail in error, please delete it and notify us either by e-mail, telephone or fax. You should not copy, forward or otherwise disclose the content of the e-mail as this is prohibited.

I found what was causing the difference in timings - it was the Windows disk read cache. If I make sure this cache is clear first (and of course that the GeoWebCache tile cache is also empty), then using either ImageMosaic of a Layer Group takes about the same execution time to generate the tiles. However, if I repeat the test, but this time only clear the GWC tile cache and leave the Windows disk cache untouched, then it seems the Layer Group makes much better use of the cached source images and is much faster than ImageMosaic at generating the tiles again.

Understanding this, it seems ImageMosaic is a good solution for me in practice, as once the tiles have been generated once for an area they will be cached by GeoWebCache anyway so I'll never need to generate them again unless that cache fills up.

Many thanks again for your previous help Simone.

Tim

-----Original Message-----
From: simboss1@anonymised.com [mailto:simboss1@anonymised.com] On Behalf Of Simone Giannecchini
Sent: 14 December 2012 08:40
To: Tim Hayton
Cc: geoserver-users@lists.sourceforge.net
Subject: Re: [Geoserver-users] How to set InputTransparentColor for MrSid layers?

Ciao Tim,
can you paste here the gdalinfo output of some of you tiles as well as an indication of the configuration of the parameters in the mosaic layer page in GeoServer?

Regards,
Simone Giannecchini

Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Simone Giannecchini
@simogeo
Founder/Director

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 333 8128928

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

On Thu, Dec 13, 2012 at 4:21 PM, Tim Hayton <tim.hayton@anonymised.com> wrote:

Many thanks for your helpful response Simone, much appreciated. I am now using an ImageMosaic source which was very easy to do, and as you say it supports setting of the transparent colour for my MrSid images which works great.

One thing I did notice is that it can be slower to generate tile images when using ImageMosaic when compared to using a Layer Group for the same set of images. It only seems to be slower though when the requested tiles cover an area which requires imagery from more than one of the source images. By way of example, requesting 35 tiles at TMS zoom level 19, all coming from a single source image, took 5 seconds to serve on my platform with both the ImageMosaic and Layer Group approaches, but requesting 35 tiles at zoom level 15, which in this case requires parts from 3 of the source images, takes 65 seconds using ImageMosaic but only 16 seconds using a LayerGroup.

I realise the ImageMosaic would have to do some extra work to take the transparent colour into account at the image borders, so to check this I tried deleting the InputTansparentColor for the ImageMosaic. This speeded things up from 65 seconds to 37 seconds, but this is still less than half the speed of the layer group. I guess the ImageMosaic must also be doing something extra that the layer group is not.

As you suggested, I filed a JIRA request for an enhancement for layer groups to support setting of a transparent colour at https://jira.codehaus.org/browse/GEOS-5501.

Regards

Tim

--
MIRA Ltd

Watling Street, Nuneaton, Warwickshire, CV10 0TU, England
Registered in England and Wales No. 402570
VAT Registration GB 100 1464 84

The contents of this e-mail are confidential and are solely for the use of the intended recipient. If you receive this e-mail in error, please delete it and notify us either by e-mail, telephone or fax. You should not copy, forward or otherwise disclose the content of the e-mail as this is prohibited.