[Geoserver-devel] Geotiff: pyramids non transparant while original is transparent

GeoServer GeoServer Developer
,
1

Hi All,

We have some Geotiff rasters with small lines (cables) on it.
Serving these transparent with Geoserver is fine, using
InputTransparentColor set to #ffffff.

But adding pyramids/overviews to it, the lines look better in QGIS, so
we added the pyramids using gdaladdo.

While that works for QGIS, for Geoserver (2.12.0) it looks as if the
InputTransparentColor in the overviews is NOT used?

The map is not transparent anymore, untill you zoom in at the base
level, THEN the map will be transparent again.

Using default raster sld.

Anybody a clue?

Regards,

Richard Duivenvoorde

PS:

Gdalinfo for the original tiff (without pyramids):

Driver: GTiff/GeoTIFF
Files: WATER.tif
Size is 13422, 20898
Coordinate System is:
PROJCS["Amersfoort / RD New",
    GEOGCS["Amersfoort",
        DATUM["Amersfoort",
            SPHEROID["Bessel 1841",6377397.155,299.1528128,
                AUTHORITY["EPSG","7004"]],

TOWGS84[565.2369,50.0087,465.658,-0.406857,0.350733,-1.87035,4.0812],
            AUTHORITY["EPSG","6289"]],
        PRIMEM["Greenwich",0,
            AUTHORITY["EPSG","8901"]],
        UNIT["degree",0.0174532925199433,
            AUTHORITY["EPSG","9122"]],
        AUTHORITY["EPSG","4289"]],
    PROJECTION["Oblique_Stereographic"],
    PARAMETER["latitude_of_origin",52.15616055555555],
    PARAMETER["central_meridian",5.38763888888889],
    PARAMETER["scale_factor",0.9999079],
    PARAMETER["false_easting",155000],
    PARAMETER["false_northing",463000],
    UNIT["metre",1,
        AUTHORITY["EPSG","9001"]],
    AXIS["X",EAST],
    AXIS["Y",NORTH],
    AUTHORITY["EPSG","28992"]]
Origin = (132801.000000000000000,452878.000000000000000)
Pixel Size = (0.333333300000000,-0.333333300000000)
Metadata:
  AREA_OR_POINT=Area
Image Structure Metadata:
  COMPRESSION=LZW
  INTERLEAVE=BAND
Corner Coordinates:
Upper Left ( 132801.000, 452878.000) ( 5d 3'49.93"E, 52d 3'53.02"N)
Lower Left ( 132801.000, 445912.001) ( 5d 3'51.56"E, 52d 0' 7.60"N)
Upper Right ( 137275.000, 452878.000) ( 5d 7'44.84"E, 52d 3'53.60"N)
Lower Right ( 137275.000, 445912.001) ( 5d 7'46.14"E, 52d 0' 8.18"N)
Center ( 135038.000, 449395.000) ( 5d 5'48.11"E, 52d 2' 0.62"N)
Band 1 Block=13422x1 Type=Byte, ColorInterp=Palette
  NoData Value=0
  Color Table (RGB with 256 entries)
    0: 255,255,255,0
    1: 0,0,255,255
    2: 0,0,0,255
    3: 0,0,0,255
    4: 0,0,0,255
    5: 0,0,0,255
    6: 0,0,0,255
    7: 0,0,0,255
    8: 0,0,0,255
    9: 0,0,0,255
   10: 0,0,0,255
etc....

Gdalinfo for the tiff with pyramids:

Driver: GTiff/GeoTIFF
Files: WATER.tif
Size is 13422, 20898
Coordinate System is:
PROJCS["Amersfoort / RD New",
    GEOGCS["Amersfoort",
        DATUM["Amersfoort",
            SPHEROID["Bessel 1841",6377397.155,299.1528128,
                AUTHORITY["EPSG","7004"]],

TOWGS84[565.2369,50.0087,465.658,-0.406857,0.350733,-1.87035,4.0812],
            AUTHORITY["EPSG","6289"]],
        PRIMEM["Greenwich",0,
            AUTHORITY["EPSG","8901"]],
        UNIT["degree",0.0174532925199433,
            AUTHORITY["EPSG","9122"]],
        AUTHORITY["EPSG","4289"]],
    PROJECTION["Oblique_Stereographic"],
    PARAMETER["latitude_of_origin",52.15616055555555],
    PARAMETER["central_meridian",5.38763888888889],
    PARAMETER["scale_factor",0.9999079],
    PARAMETER["false_easting",155000],
    PARAMETER["false_northing",463000],
    UNIT["metre",1,
        AUTHORITY["EPSG","9001"]],
    AXIS["X",EAST],
    AXIS["Y",NORTH],
    AUTHORITY["EPSG","28992"]]
Origin = (132801.000000000000000,452878.000000000000000)
Pixel Size = (0.333333300000000,-0.333333300000000)
Metadata:
  AREA_OR_POINT=Area
Image Structure Metadata:
  COMPRESSION=LZW
  INTERLEAVE=BAND
Corner Coordinates:
Upper Left ( 132801.000, 452878.000) ( 5d 3'49.93"E, 52d 3'53.02"N)
Lower Left ( 132801.000, 445912.001) ( 5d 3'51.56"E, 52d 0' 7.60"N)
Upper Right ( 137275.000, 452878.000) ( 5d 7'44.84"E, 52d 3'53.60"N)
Lower Right ( 137275.000, 445912.001) ( 5d 7'46.14"E, 52d 0' 8.18"N)
Center ( 135038.000, 449395.000) ( 5d 5'48.11"E, 52d 2' 0.62"N)
Band 1 Block=13422x1 Type=Byte, ColorInterp=Palette
  NoData Value=0
  Overviews: 6711x10449, 3356x5225, 1678x2613, 839x1307, 420x654,
210x327, 105x164, 53x82
  Color Table (RGB with 256 entries)
    0: 255,255,255,0
    1: 0,0,255,255
    2: 0,0,0,255
    3: 0,0,0,255
    4: 0,0,0,255
    5: 0,0,0,255
    6: 0,0,0,255
    7: 0,0,0,255
    8: 0,0,0,255
    9: 0,0,0,255
   10: 0,0,0,255
etc...

2

Hi Richard,
that’s just strange, overviews have nothing special, there is just a switch
read wise to choose which image to use and the rest is the same and unaware of the image origin.
Unless… are the overviews external?

Cheers
Andrea

···

On Thu, Mar 8, 2018 at 2:37 PM, Richard Duivenvoorde <rdmailings@anonymised.com> wrote:

Hi All,

We have some Geotiff rasters with small lines (cables) on it.
Serving these transparent with Geoserver is fine, using
InputTransparentColor set to #ffffff.

But adding pyramids/overviews to it, the lines look better in QGIS, so
we added the pyramids using gdaladdo.

While that works for QGIS, for Geoserver (2.12.0) it looks as if the
InputTransparentColor in the overviews is NOT used?

The map is not transparent anymore, untill you zoom in at the base
level, THEN the map will be transparent again.

Using default raster sld.

Anybody a clue?

Regards,

Richard Duivenvoorde

PS:

Gdalinfo for the original tiff (without pyramids):

Driver: GTiff/GeoTIFF
Files: WATER.tif
Size is 13422, 20898
Coordinate System is:
PROJCS[“Amersfoort / RD New”,
GEOGCS[“Amersfoort”,
DATUM[“Amersfoort”,
SPHEROID[“Bessel 1841”,6377397.155,299.1528128,
AUTHORITY[“EPSG”,“7004”]],

TOWGS84[565.2369,50.0087,465.658,-0.406857,0.350733,-1.87035,4.0812],
AUTHORITY[“EPSG”,“6289”]],
PRIMEM[“Greenwich”,0,
AUTHORITY[“EPSG”,“8901”]],
UNIT[“degree”,0.0174532925199433,
AUTHORITY[“EPSG”,“9122”]],
AUTHORITY[“EPSG”,“4289”]],
PROJECTION[“Oblique_Stereographic”],
PARAMETER[“latitude_of_origin”,52.15616055555555],
PARAMETER[“central_meridian”,5.38763888888889],
PARAMETER[“scale_factor”,0.9999079],
PARAMETER[“false_easting”,155000],
PARAMETER[“false_northing”,463000],
UNIT[“metre”,1,
AUTHORITY[“EPSG”,“9001”]],
AXIS[“X”,EAST],
AXIS[“Y”,NORTH],
AUTHORITY[“EPSG”,“28992”]]
Origin = (132801.000000000000000,452878.000000000000000)
Pixel Size = (0.333333300000000,-0.333333300000000)
Metadata:
AREA_OR_POINT=Area
Image Structure Metadata:
COMPRESSION=LZW
INTERLEAVE=BAND
Corner Coordinates:
Upper Left ( 132801.000, 452878.000) ( 5d 3’49.93"E, 52d 3’53.02"N)
Lower Left ( 132801.000, 445912.001) ( 5d 3’51.56"E, 52d 0’ 7.60"N)
Upper Right ( 137275.000, 452878.000) ( 5d 7’44.84"E, 52d 3’53.60"N)
Lower Right ( 137275.000, 445912.001) ( 5d 7’46.14"E, 52d 0’ 8.18"N)
Center ( 135038.000, 449395.000) ( 5d 5’48.11"E, 52d 2’ 0.62"N)
Band 1 Block=13422x1 Type=Byte, ColorInterp=Palette
NoData Value=0
Color Table (RGB with 256 entries)
0: 255,255,255,0
1: 0,0,255,255
2: 0,0,0,255
3: 0,0,0,255
4: 0,0,0,255
5: 0,0,0,255
6: 0,0,0,255
7: 0,0,0,255
8: 0,0,0,255
9: 0,0,0,255
10: 0,0,0,255
etc…

Gdalinfo for the tiff with pyramids:

Driver: GTiff/GeoTIFF
Files: WATER.tif
Size is 13422, 20898
Coordinate System is:
PROJCS[“Amersfoort / RD New”,
GEOGCS[“Amersfoort”,
DATUM[“Amersfoort”,
SPHEROID[“Bessel 1841”,6377397.155,299.1528128,
AUTHORITY[“EPSG”,“7004”]],

TOWGS84[565.2369,50.0087,465.658,-0.406857,0.350733,-1.87035,4.0812],
AUTHORITY[“EPSG”,“6289”]],
PRIMEM[“Greenwich”,0,
AUTHORITY[“EPSG”,“8901”]],
UNIT[“degree”,0.0174532925199433,
AUTHORITY[“EPSG”,“9122”]],
AUTHORITY[“EPSG”,“4289”]],
PROJECTION[“Oblique_Stereographic”],
PARAMETER[“latitude_of_origin”,52.15616055555555],
PARAMETER[“central_meridian”,5.38763888888889],
PARAMETER[“scale_factor”,0.9999079],
PARAMETER[“false_easting”,155000],
PARAMETER[“false_northing”,463000],
UNIT[“metre”,1,
AUTHORITY[“EPSG”,“9001”]],
AXIS[“X”,EAST],
AXIS[“Y”,NORTH],
AUTHORITY[“EPSG”,“28992”]]
Origin = (132801.000000000000000,452878.000000000000000)
Pixel Size = (0.333333300000000,-0.333333300000000)
Metadata:
AREA_OR_POINT=Area
Image Structure Metadata:
COMPRESSION=LZW
INTERLEAVE=BAND
Corner Coordinates:
Upper Left ( 132801.000, 452878.000) ( 5d 3’49.93"E, 52d 3’53.02"N)
Lower Left ( 132801.000, 445912.001) ( 5d 3’51.56"E, 52d 0’ 7.60"N)
Upper Right ( 137275.000, 452878.000) ( 5d 7’44.84"E, 52d 3’53.60"N)
Lower Right ( 137275.000, 445912.001) ( 5d 7’46.14"E, 52d 0’ 8.18"N)
Center ( 135038.000, 449395.000) ( 5d 5’48.11"E, 52d 2’ 0.62"N)
Band 1 Block=13422x1 Type=Byte, ColorInterp=Palette
NoData Value=0
Overviews: 6711x10449, 3356x5225, 1678x2613, 839x1307, 420x654,
210x327, 105x164, 53x82
Color Table (RGB with 256 entries)
0: 255,255,255,0
1: 0,0,255,255
2: 0,0,0,255
3: 0,0,0,255
4: 0,0,0,255
5: 0,0,0,255
6: 0,0,0,255
7: 0,0,0,255
8: 0,0,0,255
9: 0,0,0,255
10: 0,0,0,255
etc…

Check out the vibrant tech community on one of the world’s most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

Geoserver-devel mailing list
Geoserver-devel@anonymised.com.366…sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

–

Regards,

Andrea Aime

==
GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

3

On 08-03-18 14:47, Andrea Aime wrote:

Hi Richard,
that's just strange, overviews have nothing special, there is just a switch
read wise to choose which image to use and the rest is the same and
unaware of the image origin.
Unless... are the overviews external?

Nope, added with:
gdaladdo -r average file.tif 2 4 8 16 32 64 128 256

I also tried to create it with QGIS (hoping that this would change
something...), but both have the same issue.

If anybody interested: http://duif.net/water.zip contains 3 zips
(epsg:28992)) to test.

Happy to create an issue for it.

Regards,

Richard Duivenvoorde

4

Hi Richard,
I think that the problem is in how GDAL produces overviews with palettes:
https://lists.osgeo.org/pipermail/gdal-dev/2016-November/045589.html

The different behavior between QGIS and GeoServer is due to this:
QGIS (based on GDAL) uses the palette of the main image whilst some other readers, as an instance, the JAVA TiffReader used by GeoServer, uses overview’s specific palette.

I’m also going to ask if there is any news on that on GDAL.

Hope this helps.
Regards,
Daniele

As further references:

···

On Thu, Mar 8, 2018 at 2:37 PM, Richard Duivenvoorde <rdmailings@anonymised.com> wrote:

Hi All,

We have some Geotiff rasters with small lines (cables) on it.
Serving these transparent with Geoserver is fine, using
InputTransparentColor set to #ffffff.

But adding pyramids/overviews to it, the lines look better in QGIS, so
we added the pyramids using gdaladdo.

While that works for QGIS, for Geoserver (2.12.0) it looks as if the
InputTransparentColor in the overviews is NOT used?

The map is not transparent anymore, untill you zoom in at the base
level, THEN the map will be transparent again.

Using default raster sld.

Anybody a clue?

Regards,

Richard Duivenvoorde

PS:

Gdalinfo for the original tiff (without pyramids):

Driver: GTiff/GeoTIFF
Files: WATER.tif
Size is 13422, 20898
Coordinate System is:
PROJCS[“Amersfoort / RD New”,
GEOGCS[“Amersfoort”,
DATUM[“Amersfoort”,
SPHEROID[“Bessel 1841”,6377397.155,299.1528128,
AUTHORITY[“EPSG”,“7004”]],

TOWGS84[565.2369,50.0087,465.658,-0.406857,0.350733,-1.87035,4.0812],
AUTHORITY[“EPSG”,“6289”]],
PRIMEM[“Greenwich”,0,
AUTHORITY[“EPSG”,“8901”]],
UNIT[“degree”,0.0174532925199433,
AUTHORITY[“EPSG”,“9122”]],
AUTHORITY[“EPSG”,“4289”]],
PROJECTION[“Oblique_Stereographic”],
PARAMETER[“latitude_of_origin”,52.15616055555555],
PARAMETER[“central_meridian”,5.38763888888889],
PARAMETER[“scale_factor”,0.9999079],
PARAMETER[“false_easting”,155000],
PARAMETER[“false_northing”,463000],
UNIT[“metre”,1,
AUTHORITY[“EPSG”,“9001”]],
AXIS[“X”,EAST],
AXIS[“Y”,NORTH],
AUTHORITY[“EPSG”,“28992”]]
Origin = (132801.000000000000000,452878.000000000000000)
Pixel Size = (0.333333300000000,-0.333333300000000)
Metadata:
AREA_OR_POINT=Area
Image Structure Metadata:
COMPRESSION=LZW
INTERLEAVE=BAND
Corner Coordinates:
Upper Left ( 132801.000, 452878.000) ( 5d 3’49.93"E, 52d 3’53.02"N)
Lower Left ( 132801.000, 445912.001) ( 5d 3’51.56"E, 52d 0’ 7.60"N)
Upper Right ( 137275.000, 452878.000) ( 5d 7’44.84"E, 52d 3’53.60"N)
Lower Right ( 137275.000, 445912.001) ( 5d 7’46.14"E, 52d 0’ 8.18"N)
Center ( 135038.000, 449395.000) ( 5d 5’48.11"E, 52d 2’ 0.62"N)
Band 1 Block=13422x1 Type=Byte, ColorInterp=Palette
NoData Value=0
Color Table (RGB with 256 entries)
0: 255,255,255,0
1: 0,0,255,255
2: 0,0,0,255
3: 0,0,0,255
4: 0,0,0,255
5: 0,0,0,255
6: 0,0,0,255
7: 0,0,0,255
8: 0,0,0,255
9: 0,0,0,255
10: 0,0,0,255
etc…

Gdalinfo for the tiff with pyramids:

Driver: GTiff/GeoTIFF
Files: WATER.tif
Size is 13422, 20898
Coordinate System is:
PROJCS[“Amersfoort / RD New”,
GEOGCS[“Amersfoort”,
DATUM[“Amersfoort”,
SPHEROID[“Bessel 1841”,6377397.155,299.1528128,
AUTHORITY[“EPSG”,“7004”]],

TOWGS84[565.2369,50.0087,465.658,-0.406857,0.350733,-1.87035,4.0812],
AUTHORITY[“EPSG”,“6289”]],
PRIMEM[“Greenwich”,0,
AUTHORITY[“EPSG”,“8901”]],
UNIT[“degree”,0.0174532925199433,
AUTHORITY[“EPSG”,“9122”]],
AUTHORITY[“EPSG”,“4289”]],
PROJECTION[“Oblique_Stereographic”],
PARAMETER[“latitude_of_origin”,52.15616055555555],
PARAMETER[“central_meridian”,5.38763888888889],
PARAMETER[“scale_factor”,0.9999079],
PARAMETER[“false_easting”,155000],
PARAMETER[“false_northing”,463000],
UNIT[“metre”,1,
AUTHORITY[“EPSG”,“9001”]],
AXIS[“X”,EAST],
AXIS[“Y”,NORTH],
AUTHORITY[“EPSG”,“28992”]]
Origin = (132801.000000000000000,452878.000000000000000)
Pixel Size = (0.333333300000000,-0.333333300000000)
Metadata:
AREA_OR_POINT=Area
Image Structure Metadata:
COMPRESSION=LZW
INTERLEAVE=BAND
Corner Coordinates:
Upper Left ( 132801.000, 452878.000) ( 5d 3’49.93"E, 52d 3’53.02"N)
Lower Left ( 132801.000, 445912.001) ( 5d 3’51.56"E, 52d 0’ 7.60"N)
Upper Right ( 137275.000, 452878.000) ( 5d 7’44.84"E, 52d 3’53.60"N)
Lower Right ( 137275.000, 445912.001) ( 5d 7’46.14"E, 52d 0’ 8.18"N)
Center ( 135038.000, 449395.000) ( 5d 5’48.11"E, 52d 2’ 0.62"N)
Band 1 Block=13422x1 Type=Byte, ColorInterp=Palette
NoData Value=0
Overviews: 6711x10449, 3356x5225, 1678x2613, 839x1307, 420x654,
210x327, 105x164, 53x82
Color Table (RGB with 256 entries)
0: 255,255,255,0
1: 0,0,255,255
2: 0,0,0,255
3: 0,0,0,255
4: 0,0,0,255
5: 0,0,0,255
6: 0,0,0,255
7: 0,0,0,255
8: 0,0,0,255
9: 0,0,0,255
10: 0,0,0,255
etc…

Check out the vibrant tech community on one of the world’s most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

Geoserver-devel mailing list
Geoserver-devel@anonymised.com.366…sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

–

Regards,
Daniele Romagnoli

GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.

Ing. Daniele Romagnoli
Senior Software Engineer

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

5

Hi,

I would like to add also that with paletted images no other resampling method than ”nearest” makes sense. Try if removing “–r average” helps.

The safe way is to expand the paletted images into RGB(A) with the pct2pgb utility but the price is increased need for disk space.

-Jukka Rahkonen-

Lähettäjä: Daniele Romagnoli [mailto:daniele.romagnoli@…1268…]
Lähetetty: 8. maaliskuuta 2018 16:17
Vastaanottaja: richard@…3975…
Kopio: geoserver-devel geoserver-devel@lists.sourceforge.net
Aihe: Re: [Geoserver-devel] Geotiff: pyramids non transparant while original is transparent

···

Hi Richard,

I think that the problem is in how GDAL produces overviews with palettes:

https://lists.osgeo.org/pipermail/gdal-dev/2016-November/045589.html

The different behavior between QGIS and GeoServer is due to this:

QGIS (based on GDAL) uses the palette of the main image whilst some other readers, as an instance, the JAVA TiffReader used by GeoServer, uses overview’s specific palette.

I’m also going to ask if there is any news on that on GDAL.

Hope this helps.

Regards,

Daniele

As further references:

https://trac.osgeo.org/gdal/ticket/2213

https://lists.osgeo.org/pipermail/gdal-dev/2016-November/045590.html

https://github.com/geosolutions-it/imageio-ext/issues/122

On Thu, Mar 8, 2018 at 2:37 PM, Richard Duivenvoorde <rdmailings@…3975…> wrote:

Hi All,

We have some Geotiff rasters with small lines (cables) on it.
Serving these transparent with Geoserver is fine, using
InputTransparentColor set to #ffffff.

But adding pyramids/overviews to it, the lines look better in QGIS, so
we added the pyramids using gdaladdo.

While that works for QGIS, for Geoserver (2.12.0) it looks as if the
InputTransparentColor in the overviews is NOT used?

The map is not transparent anymore, untill you zoom in at the base
level, THEN the map will be transparent again.

Using default raster sld.

Anybody a clue?

Regards,

Richard Duivenvoorde

PS:

Gdalinfo for the original tiff (without pyramids):

Driver: GTiff/GeoTIFF
Files: WATER.tif
Size is 13422, 20898
Coordinate System is:
PROJCS[“Amersfoort / RD New”,
GEOGCS[“Amersfoort”,
DATUM[“Amersfoort”,
SPHEROID[“Bessel 1841”,6377397.155,299.1528128,
AUTHORITY[“EPSG”,“7004”]],

TOWGS84[565.2369,50.0087,465.658,-0.406857,0.350733,-1.87035,4.0812],
AUTHORITY[“EPSG”,“6289”]],
PRIMEM[“Greenwich”,0,
AUTHORITY[“EPSG”,“8901”]],
UNIT[“degree”,0.0174532925199433,
AUTHORITY[“EPSG”,“9122”]],
AUTHORITY[“EPSG”,“4289”]],
PROJECTION[“Oblique_Stereographic”],
PARAMETER[“latitude_of_origin”,52.15616055555555],
PARAMETER[“central_meridian”,5.38763888888889],
PARAMETER[“scale_factor”,0.9999079],
PARAMETER[“false_easting”,155000],
PARAMETER[“false_northing”,463000],
UNIT[“metre”,1,
AUTHORITY[“EPSG”,“9001”]],
AXIS[“X”,EAST],
AXIS[“Y”,NORTH],
AUTHORITY[“EPSG”,“28992”]]
Origin = (132801.000000000000000,452878.000000000000000)
Pixel Size = (0.333333300000000,-0.333333300000000)
Metadata:
AREA_OR_POINT=Area
Image Structure Metadata:
COMPRESSION=LZW
INTERLEAVE=BAND
Corner Coordinates:
Upper Left ( 132801.000, 452878.000) ( 5d 3’49.93"E, 52d 3’53.02"N)
Lower Left ( 132801.000, 445912.001) ( 5d 3’51.56"E, 52d 0’ 7.60"N)
Upper Right ( 137275.000, 452878.000) ( 5d 7’44.84"E, 52d 3’53.60"N)
Lower Right ( 137275.000, 445912.001) ( 5d 7’46.14"E, 52d 0’ 8.18"N)
Center ( 135038.000, 449395.000) ( 5d 5’48.11"E, 52d 2’ 0.62"N)
Band 1 Block=13422x1 Type=Byte, ColorInterp=Palette
NoData Value=0
Color Table (RGB with 256 entries)
0: 255,255,255,0
1: 0,0,255,255
2: 0,0,0,255
3: 0,0,0,255
4: 0,0,0,255
5: 0,0,0,255
6: 0,0,0,255
7: 0,0,0,255
8: 0,0,0,255
9: 0,0,0,255
10: 0,0,0,255
etc…

Gdalinfo for the tiff with pyramids:

Driver: GTiff/GeoTIFF
Files: WATER.tif
Size is 13422, 20898
Coordinate System is:
PROJCS[“Amersfoort / RD New”,
GEOGCS[“Amersfoort”,
DATUM[“Amersfoort”,
SPHEROID[“Bessel 1841”,6377397.155,299.1528128,
AUTHORITY[“EPSG”,“7004”]],

TOWGS84[565.2369,50.0087,465.658,-0.406857,0.350733,-1.87035,4.0812],
AUTHORITY[“EPSG”,“6289”]],
PRIMEM[“Greenwich”,0,
AUTHORITY[“EPSG”,“8901”]],
UNIT[“degree”,0.0174532925199433,
AUTHORITY[“EPSG”,“9122”]],
AUTHORITY[“EPSG”,“4289”]],
PROJECTION[“Oblique_Stereographic”],
PARAMETER[“latitude_of_origin”,52.15616055555555],
PARAMETER[“central_meridian”,5.38763888888889],
PARAMETER[“scale_factor”,0.9999079],
PARAMETER[“false_easting”,155000],
PARAMETER[“false_northing”,463000],
UNIT[“metre”,1,
AUTHORITY[“EPSG”,“9001”]],
AXIS[“X”,EAST],
AXIS[“Y”,NORTH],
AUTHORITY[“EPSG”,“28992”]]
Origin = (132801.000000000000000,452878.000000000000000)
Pixel Size = (0.333333300000000,-0.333333300000000)
Metadata:
AREA_OR_POINT=Area
Image Structure Metadata:
COMPRESSION=LZW
INTERLEAVE=BAND
Corner Coordinates:
Upper Left ( 132801.000, 452878.000) ( 5d 3’49.93"E, 52d 3’53.02"N)
Lower Left ( 132801.000, 445912.001) ( 5d 3’51.56"E, 52d 0’ 7.60"N)
Upper Right ( 137275.000, 452878.000) ( 5d 7’44.84"E, 52d 3’53.60"N)
Lower Right ( 137275.000, 445912.001) ( 5d 7’46.14"E, 52d 0’ 8.18"N)
Center ( 135038.000, 449395.000) ( 5d 5’48.11"E, 52d 2’ 0.62"N)
Band 1 Block=13422x1 Type=Byte, ColorInterp=Palette
NoData Value=0
Overviews: 6711x10449, 3356x5225, 1678x2613, 839x1307, 420x654,
210x327, 105x164, 53x82
Color Table (RGB with 256 entries)
0: 255,255,255,0
1: 0,0,255,255
2: 0,0,0,255
3: 0,0,0,255
4: 0,0,0,255
5: 0,0,0,255
6: 0,0,0,255
7: 0,0,0,255
8: 0,0,0,255
9: 0,0,0,255
10: 0,0,0,255
etc…

Check out the vibrant tech community on one of the world’s most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

–

Regards,

Daniele Romagnoli

==

GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.

==

Ing. Daniele Romagnoli

Senior Software Engineer

GeoSolutions S.A.S.

Via di Montramito 3/A

55054 Massarosa (LU)

Italy

phone: +39 0584 962313

fax: +39 0584 1660272

http://www.geo-solutions.it

http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

6

On Thu, Mar 8, 2018 at 3:16 PM, Richard Duivenvoorde <rdmailings@anonymised.com>
wrote:

On 08-03-18 14:47, Andrea Aime wrote:
> Hi Richard,
> that's just strange, overviews have nothing special, there is just a
switch
> read wise to choose which image to use and the rest is the same and
> unaware of the image origin.
> Unless... are the overviews external?

Nope, added with:
gdaladdo -r average file.tif 2 4 8 16 32 64 128 256

Note that for paletted images, not nearest interpolations may produce weird
results. (Since it interpolates indexes).

Daniele

I also tried to create it with QGIS (hoping that this would change
something...), but both have the same issue.

If anybody interested: http://duif.net/water.zip contains 3 zips
(epsg:28992)) to test.

Happy to create an issue for it.

Regards,

Richard Duivenvoorde

------------------------------------------------------------
------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

--
Regards,
Daniele Romagnoli

GeoServer Professional Services from the experts! Visit http://goo.gl/it488V
for more information.

Ing. Daniele Romagnoli
Senior Software Engineer

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------
AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

7

Hi Richard,
did you have any chance to reprocess the image by using one of the suggestions emerged in this thread, such as RGB expansion as suggested by Jukka or recreating the overviews with a nightly GDAL version after Even’s fix 1?
Please, let us know if you managed to have it working.

Regards,
Daniele

···

On Thu, Mar 8, 2018 at 3:50 PM, Daniele Romagnoli <daniele.romagnoli@anonymised.com> wrote:

–

On Thu, Mar 8, 2018 at 3:16 PM, Richard Duivenvoorde <rdmailings@anonymised.com> wrote:

On 08-03-18 14:47, Andrea Aime wrote:

Hi Richard,
that’s just strange, overviews have nothing special, there is just a switch
read wise to choose which image to use and the rest is the same and
unaware of the image origin.
Unless… are the overviews external?

Nope, added with:
gdaladdo -r average file.tif 2 4 8 16 32 64 128 256

Note that for paletted images, not nearest interpolations may produce weird results. (Since it interpolates indexes).

Daniele

I also tried to create it with QGIS (hoping that this would change
something…), but both have the same issue.

If anybody interested: http://duif.net/water.zip contains 3 zips
(epsg:28992)) to test.

Happy to create an issue for it.

Regards,

Richard Duivenvoorde

Check out the vibrant tech community on one of the world’s most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

Geoserver-devel mailing list
Geoserver-devel@anonymised.comrge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

–

Regards,
Daniele Romagnoli

==
GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.

Ing. Daniele Romagnoli
Senior Software Engineer

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)

Italy

phone: +39 0584 962313
fax: +39 0584 1660272

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

Regards,
Daniele Romagnoli

GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.

Ing. Daniele Romagnoli
Senior Software Engineer

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

8

Hi Daniele,

YEP:

- Tried thursday, on the original Windows Osgeo4w server
gdaladdo file.tif 2 4 8 16 32 64 128 256 (that is without '-r average')
did NOT fix the problem

- then thought to double check and run
from my own laptop (with gdal 2.2.3 from distro/Debian sid)
(Geoserver 2.13-RC1)

gdaladdo -r average file.tif 2 4 8 16 32 64 128 256 (2.2.3 on Debian)
NOT working

gdaladdo file.tif 2 4 8 16 32 64 128 256
NOT working

- compiled gdal master here on Debian sid, use that:
gdaladdo file.tif 2 4 8 16 32 64 128 256
WORKS \o/

gdaladdo -r average file.tif 2 4 8 16 32 64 128 256 (gdal master)
WORKS \o/

So my conclusion ( but please try yourself :slight_smile: ):

- Even's patch works BOTH for -r average as for the 'default' (nearest)
version of gdaladdo.

Thanks all!!

Regards,

Richard Duivenvoorde

On 09-03-18 17:17, Daniele Romagnoli wrote:

Hi Richard,
did you have any chance to reprocess the image by using one of the
suggestions emerged in this thread, such as RGB expansion as suggested
by Jukka or recreating the overviews with a nightly GDAL version after
Even's fix [1]?
Please, let us know if you managed to have it working.

Regards,
Daniele

[1]: https://lists.osgeo.org/pipermail/gdal-dev/2018-March/048204.html

On Thu, Mar 8, 2018 at 3:50 PM, Daniele Romagnoli
<daniele.romagnoli@anonymised.com
<mailto:daniele.romagnoli@anonymised.com>> wrote:

    On Thu, Mar 8, 2018 at 3:16 PM, Richard Duivenvoorde
    <rdmailings@anonymised.com <mailto:rdmailings@anonymised.com>> wrote:

        On 08-03-18 14:47, Andrea Aime wrote:
        > Hi Richard,
        > that's just strange, overviews have nothing special, there is just a switch
        > read wise to choose which image to use and the rest is the same and
        > unaware of the image origin.
        > Unless... are the overviews external?

        Nope, added with:
        gdaladdo -r average file.tif 2 4 8 16 32 64 128 256

    Note that for paletted images, not nearest interpolations may
    produce weird results. (Since it interpolates indexes).

    Daniele

    

        I also tried to create it with QGIS (hoping that this would change
        something...), but both have the same issue.

        If anybody interested: http://duif.net/water.zip contains 3 zips
        (epsg:28992)) to test.

        Happy to create an issue for it.

        Regards,

        Richard Duivenvoorde

        ------------------------------------------------------------------------------
        Check out the vibrant tech community on one of the world's most
        engaging tech sites, Slashdot.org! http://sdm.link/slashdot
        _______________________________________________
        Geoserver-devel mailing list
        Geoserver-devel@lists.sourceforge.net
        <mailto:Geoserver-devel@lists.sourceforge.net>
        https://lists.sourceforge.net/lists/listinfo/geoserver-devel
        <https://lists.sourceforge.net/lists/listinfo/geoserver-devel&gt;

    --
    Regards,
    Daniele Romagnoli
    ==
    GeoServer Professional Services from the experts! Visit
    http://goo.gl/it488V for more information.
    ==

    Ing. Daniele Romagnoli
    Senior Software Engineer

    GeoSolutions S.A.S.
    Via di Montramito 3/A
    55054 Massarosa (LU)
    Italy
    phone: +39 0584 962313 <tel:+39%200584%20962313>
    fax: +39 0584 1660272 <tel:+39%200584%20166%200272>

    http://www.geo-solutions.it
    http://twitter.com/geosolutions_it

    -------------------------------------------------------
    AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
    Le informazioni contenute in questo messaggio di posta elettronica
    e/o nel/i file/s allegato/i sono da considerarsi strettamente
    riservate. Il loro utilizzo è consentito esclusivamente al
    destinatario del messaggio, per le finalità indicate nel messaggio
    stesso. Qualora riceviate questo messaggio senza esserne il
    destinatario, Vi preghiamo cortesemente di darcene notizia via
    e-mail e di procedere alla distruzione del messaggio stesso,
    cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
    divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo,
    od utilizzarlo per finalità diverse, costituisce comportamento
    contrario ai principi dettati dal D.Lgs. 196/2003.
    
    The information in this message and/or attachments, is intended
    solely for the attention and use of the named addressee(s) and may
    be confidential or proprietary in nature or covered by the
    provisions of privacy act (Legislative Decree June, 30 2003, no.196
    - Italy's New Data Protection Code).Any use not in accord with its
    purpose, any disclosure, reproduction, copying, distribution, or
    either dissemination, either whole or partial, is strictly forbidden
    except previous formal approval of the named addressee(s). If you
    are not the intended recipient, please contact immediately the
    sender by telephone, fax or e-mail and delete the information in
    this message that has been received in error. The sender does not
    give any warranty or accept liability as the content, accuracy or
    completeness of sent messages and accepts no responsibility for
    changes made after they were sent or for other risks which arise as
    a result of e-mail transmission, viruses, etc.

--
Regards,
Daniele Romagnoli

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Daniele Romagnoli
Senior Software Engineer

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------
AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio
stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti,
copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento
contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely
for the attention and use of the named addressee(s) and may be
confidential or proprietary in nature or covered by the provisions of
privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data
Protection Code).Any use not in accord with its purpose, any disclosure,
reproduction, copying, distribution, or either dissemination, either
whole or partial, is strictly forbidden except previous formal approval
of the named addressee(s). If you are not the intended recipient, please
contact immediately the sender by telephone, fax or e-mail and delete
the information in this message that has been received in error. The
sender does not give any warranty or accept liability as the content,
accuracy or completeness of sent messages and accepts no responsibility
for changes made after they were sent or for other risks which arise as
a result of e-mail transmission, viruses, etc.

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel