[Geoserver-users] WMS requests fail in some instances

James_Rapaport · August 8, 2019, 2:44pm

I have found a curious problem concerning WMS, group layers and images with transparency. I start by creating a group layer containing two layers. Each layer sources its data from a TIFF image and the images have RGBA bands. I load the group layer in a client via WMS. I find that there is a range of requests which fail to draw. I have included some example requests below. The logged error is included below too.

If I replace the source TIFFs with RGB (no transparency) the problem does not occur.

If I add the individual layers (ie the layers that are included in the group layer) separately to the client, the problem does not occur. But if I add the individual layers to the client in a single request (ie layers=layer-1,layer-2) the problem does occur.

I found this problem when using GeoServer 2.15.2 (Oracle Corporation: 1.8.0_162 (Java HotSpot™ 64-Bit Server VM) and Windows 10). I found it did not occur when using GeoServer 2.14.0 but it did occur in 2.14.1 and higher.

I wonder if it is either a bug (that I’ll happily report with some sample data) or whether perhaps I have missed a configuration step or something. I’d be grateful for any advice.

Kind regards,

James

jive · August 8, 2019, 10:02pm

Interesting, can you experiment with:

a) supplying a background color to see if it has an effect
b) drawing a subset of the raster both in the centre and on the edge
c) turning JAI-EXT operations on or off …

There is a “fast path” when rendering a single raster layer where it is not drawn but read directly from disk - so the case that works may be this case.

The message sounds like rendering is trying to lookup a pixel using a negative row/col pixel. Perhaps have a look at gdalinfo? You can also experiment with using an alpha mask (1 bit) vs band (8 bit)…

I think we need to dig a bit more before even known what project to report this issue to…

···

–
Jody Garnett

James_Rapaport · August 10, 2019, 2:37pm

Thanks Jody,

I’ve carried out the experiments – I’ll explain what I did in case I’ve misunderstood:

···

I added the bgcolor parameter to the GetMap request
1. I tried bgcolor=0xFFFFFF (which I think is the WMS spec default anyway)
2. I tried bgcolor=0x000000
3. The problem still occurred
I think I know where you’re going with this one, because I thought I had seen something in my earlier testing but I was unable to confirm it. I’ve now tested with different datasets that are geographically further apart. In short, when I make a request to draw in the centre, approximately, of the raster (specifically the group layer extent) the problem occurs. When I make a request to draw towards the edge of the raster the request is successful and the problem does not occur.
I added -Dorg.geotools.coverage.jaiext.enabled=false to the JVM options
1. The problem still occurred
2. I enabled it again and then changed the configuration in the UI (on the Image Processing page, I moved all the operations from JAIEXT to JAI) and the problem still occurred.
I looked at gdalinfo for each source TIFF. Nothing stands out to me but I’ve copied the output below in case it’s of any interest.
I tried looking into setting up an alpha mask (1 bit) vs band (8 bit). I couldn’t work out how to do it. If you think it’s a useful test to do still, can you please provide some guidance and I’ll happily do it? You’ll see in the gdalinfo below that the alpha band is 8 bit now.

All the tests were carried out in GeoServer 2.15.2 (Oracle Corporation: 1.8.0_162 (Java HotSpot™ 64-Bit Server VM) and Windows 10).

Kind regards,

James

=======================================================

Driver: GTiff/GeoTIFF

Files: 4000-1-0.tif

Size is 3464, 1344

Coordinate System is:

PROJCS[“WGS 84 / Pseudo-Mercator”,

GEOGCS[“WGS 84”,

DATUM[“WGS_1984”,

SPHEROID[“WGS 84”,6378137,298.257223563,

AUTHORITY[“EPSG”,“7030”]],

AUTHORITY[“EPSG”,“6326”]],

PRIMEM[“Greenwich”,0,

AUTHORITY[“EPSG”,“8901”]],

UNIT[“degree”,0.0174532925199433,

AUTHORITY[“EPSG”,“9122”]],

AUTHORITY[“EPSG”,“4326”]],

PROJECTION[“Mercator_1SP”],

PARAMETER[“central_meridian”,0],

PARAMETER[“scale_factor”,1],

PARAMETER[“false_easting”,0],

PARAMETER[“false_northing”,0],

UNIT[“metre”,1,

AUTHORITY[“EPSG”,“9001”]],

AXIS[“X”,EAST],

AXIS[“Y”,NORTH],

EXTENSION[“PROJ4”,“+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +wktext +no_defs”],

AUTHORITY[“EPSG”,“3857”]]

Origin = (-11140949.022222205996513,18543042.967076625674963)

Pixel Size = (9000.705069007237398,-9000.705069007237398)

Metadata:

AREA_OR_POINT=Area

Image Structure Metadata:

COMPRESSION=LZW

INTERLEAVE=PIXEL

Corner Coordinates:

Upper Left (-11140949.022,18543042.967) (100d 4’51.05"W, 83d44’48.36"N)

Lower Left (-11140949.022, 6446095.354) (100d 4’51.05"W, 49d59’56.25"N)

Upper Right (20037493.337,18543042.967) (179d59’59.51"E, 83d44’48.36"N)

Lower Right (20037493.337, 6446095.354) (179d59’59.51"E, 49d59’56.25"N)

Center ( 4448272.157,12494569.161) ( 39d57’34.23"E, 73d56’52.73"N)

Band 1 Block=3464x1 Type=Byte, ColorInterp=Red

NoData Value=254

Band 2 Block=3464x1 Type=Byte, ColorInterp=Green

NoData Value=254

Band 3 Block=3464x1 Type=Byte, ColorInterp=Blue

NoData Value=254

Band 4 Block=3464x1 Type=Byte, ColorInterp=Alpha

NoData Value=254

Driver: GTiff/GeoTIFF

Files: 4000-1-1.tif

Size is 3461, 1431

Coordinate System is:

PROJCS[“WGS 84 / Pseudo-Mercator”,

GEOGCS[“WGS 84”,

DATUM[“WGS_1984”,

SPHEROID[“WGS 84”,6378137,298.257223563,

AUTHORITY[“EPSG”,“7030”]],

AUTHORITY[“EPSG”,“6326”]],

PRIMEM[“Greenwich”,0,

AUTHORITY[“EPSG”,“8901”]],

UNIT[“degree”,0.0174532925199433,

AUTHORITY[“EPSG”,“9122”]],

AUTHORITY[“EPSG”,“4326”]],

PROJECTION[“Mercator_1SP”],

PARAMETER[“central_meridian”,0],

PARAMETER[“scale_factor”,1],

PARAMETER[“false_easting”,0],

PARAMETER[“false_northing”,0],

UNIT[“metre”,1,

AUTHORITY[“EPSG”,“9001”]],

AXIS[“X”,EAST],

AXIS[“Y”,NORTH],

EXTENSION[“PROJ4”,“+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +wktext +no_defs”],

AUTHORITY[“EPSG”,“3857”]]

Origin = (-11140949.022210156545043,6453257.157739197835326)

Pixel Size = (9006.638862780311683,-9006.638862780311683)

Metadata:

AREA_OR_POINT=Area

Image Structure Metadata:

COMPRESSION=LZW

INTERLEAVE=PIXEL

Corner Coordinates:

Upper Left (-11140949.022, 6453257.158) (100d 4’51.05"W, 50d 2’25.06"N)

Lower Left (-11140949.022,-6435243.055) (100d 4’51.05"W, 49d56’10.51"S)

Upper Right (20031028.082, 6453257.158) (179d56’30.43"E, 50d 2’25.06"N)

Lower Right (20031028.082,-6435243.055) (179d56’30.43"E, 49d56’10.51"S)

Center ( 4445039.530, 9007.051) ( 39d55’49.69"E, 0d 4’51.28"N)

Band 1 Block=3461x1 Type=Byte, ColorInterp=Red

NoData Value=254

Band 2 Block=3461x1 Type=Byte, ColorInterp=Green

NoData Value=254

Band 3 Block=3461x1 Type=Byte, ColorInterp=Blue

NoData Value=254

Band 4 Block=3461x1 Type=Byte, ColorInterp=Alpha

NoData Value=254

From: Jody Garnett <jody.garnett@…84…>
Sent: August 8, 2019 19:02
To: James Rapaport <james.rapaport@…10043…>
Cc: geoserver-users@lists.sourceforge.net
Subject: Re: [Geoserver-users] WMS requests fail in some instances

Interesting, can you experiment with:

a) supplying a background color to see if it has an effect

b) drawing a subset of the raster both in the centre and on the edge

c) turning JAI-EXT operations on or off …

There is a “fast path” when rendering a single raster layer where it is not drawn but read directly from disk - so the case that works may be this case.

The message sounds like rendering is trying to lookup a pixel using a negative row/col pixel. Perhaps have a look at gdalinfo? You can also experiment with using an alpha mask (1 bit) vs band (8 bit)…

I think we need to dig a bit more before even known what project to report this issue to…

–

Jody Garnett

On Thu, 8 Aug 2019 at 08:01, James Rapaport <james.rapaport@…10043…> wrote:

I have found a curious problem concerning WMS, group layers and images with transparency. I start by creating a group layer containing two layers. Each layer sources its data from a TIFF image and the images have RGBA bands. I load the group layer in a client via WMS. I find that there is a range of requests which fail to draw. I have included some example requests below. The logged error is included below too.

If I replace the source TIFFs with RGB (no transparency) the problem does not occur.

If I add the individual layers (ie the layers that are included in the group layer) separately to the client, the problem does not occur. But if I add the individual layers to the client in a single request (ie layers=layer-1,layer-2) the problem does occur.

I found this problem when using GeoServer 2.15.2 (Oracle Corporation: 1.8.0_162 (Java HotSpot™ 64-Bit Server VM) and Windows 10). I found it did not occur when using GeoServer 2.14.0 but it did occur in 2.14.1 and higher.

I wonder if it is either a bug (that I’ll happily report with some sample data) or whether perhaps I have missed a configuration step or something. I’d be grateful for any advice.

Kind regards,

James

===================================

Here’s some more detail on the problem:

When an error occurs the following is logged:

07 Aug 10:51:23 WARN [renderer.lite] - The specified dimensional parameter is non-positive.

java.lang.IllegalArgumentException: The specified dimensional parameter is non-positive.

at javax.media.jai.ImageLayout.setTileHeight(ImageLayout.java:585)

07 Aug 10:51:23 ERROR [renderer.lite] - The specified dimensional parameter is non-positive.

java.lang.IllegalArgumentException: The specified dimensional parameter is non-positive.

at javax.media.jai.ImageLayout.setTileHeight(ImageLayout.java:585)

07 Aug 10:51:23 ERROR [geoserver.ows] -

org.geoserver.platform.ServiceException: Rendering process failed

at org.geoserver.wms.map.RenderedImageMapOutputFormat.produceMap(RenderedImageMapOutputFormat.java:628)

The following request results in an error:-

http://localhost:8081/geoserver/ows?WIDTH=636&HEIGHT=636&SERVICE=WMS&VERSION=1.3.0&REQUEST=GetMap&BBOX=-20037508.34278924391,-20048966.10401460156,20037508.34278924391,20048966.10401459411&CRS=EPSG:3857&LAYERS=oceanwise:pm-ps&STYLES=&FORMAT=image/png&TRANSPARENT=TRUE

The following request results in an image:-

http://localhost:8081/geoserver/ows?WIDTH=637&HEIGHT=637&SERVICE=WMS&VERSION=1.3.0&REQUEST=GetMap&BBOX=-20037508.34278924391,-20048966.10401460156,20037508.34278924391,20048966.10401459411&CRS=EPSG:3857&LAYERS=oceanwise:pm-ps&STYLES=&FORMAT=image/png&TRANSPARENT=TRUE

The requests are for a box 636 x 636 and 637 x 637 respectively.

The following request results in an error:-

http://localhost:8081/geoserver/ows?WIDTH=318&HEIGHT=318&SERVICE=WMS&VERSION=1.3.0&REQUEST=GetMap&BBOX=-20037508.34278924391,-20048966.10401460156,20037508.34278924391,20048966.10401459411&CRS=EPSG:3857&LAYERS=oceanwise:pm-ps&STYLES=&FORMAT=image/png&TRANSPARENT=TRUE

The following request results in an image:-

http://localhost:8081/geoserver/ows?WIDTH=317&HEIGHT=317&SERVICE=WMS&VERSION=1.3.0&REQUEST=GetMap&BBOX=-20037508.34278924391,-20048966.10401460156,20037508.34278924391,20048966.10401459411&CRS=EPSG:3857&LAYERS=oceanwise:pm-ps&STYLES=&FORMAT=image/png&TRANSPARENT=TRUE

The requests are for a box 318 x 318 and 317 x 317 respectively. Boxes between 318 x 318 and 636 x 636 also fail. But requests for boxes smaller than 318 x 318 and larger than 636 x 636 succeed. I don’t want to attach too much significance to the formulation of the request. Save to say that there’s a set of requests that fail and a set of requests that do not fail.

Affected version: 2.15.2 - checking previous versions shows that the problem does not occur in 2.14.0 but it does occur in 2.14.1 onward.

Environment:

Client: QGIS 8.3.1-Zanzibar

Container: Tomcat 8.5.43

JVM: Oracle Corporation: 1.8.0_162 (Java HotSpot™ 64-Bit Server VM)

Native JAI: false

Native JAI ImageIO: false

OS: Windows 10

This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com

Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:

Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/

The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer

Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com

Andrea_Aime4 · August 10, 2019, 4:30pm

Hi James,
having a bit of both alpha channel and nodata is unusual, might be the cause of the issue.
Anyhow, good analysis, I suggest you open a bug report with sample data and requests that will make the issue appear.
Thanks!

Cheers
Andrea

···

Regards, Andrea Aime == GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.

jive · August 13, 2019, 9:55pm

I think I know where you’re going with this one, because I thought I had seen something in my earlier testing but I was unable to confirm it. I’ve now tested with different datasets that are geographically further apart. In short, when I make a request to draw in the centre, approximately, of the raster (specifically the group layer extent) the problem occurs. When I make a request to draw towards the edge of the raster the request is successful and the problem does not occur.

This indicates to me that the “fast path” is indeed having a failure, which we can report. I agree with Andrea - thanks for the detailed testing.

For a workaround to disable fast-path:

Looking in RenderedImageMapOutputFormat.java

// fast path for pure coverage rendering
if (DefaultWebMapService.isDirectRasterPathEnabled()
&& mapContent.layers().size() == 1
&& mapContent.getAngle() == 0.0
&& (layout == null || layout.isEmpty())) {

DefaultWebMapService

public static boolean isDirectRasterPathEnabled() {
return !BYPASS_DIRECT;
}

defined by:

/** This variable is used to bypass direct raster rendering. */
private static boolean BYPASS_DIRECT =
Boolean.getBoolean(“org.geoserver.render.raster.direct.disable”);

So if you can try starting up with -Dorg.geoserver.render.raster.direct.disable=true it should act as a workaround.

I tried looking into setting up an alpha mask (1 bit) vs band (8 bit). I couldn’t work out how to do it. If you think it’s a useful test to do still, can you please provide some guidance and I’ll happily do it? You’ll see in the gdalinfo below that the alpha band is 8 bit now.

Yeah I am not the best at that part either There are several ways to define transparency - I would report the issue as is, and then try each method to seek a work around or to learn more.

Kind regards,

James

Cheers,
Jody

Andrea_Aime4 · August 14, 2019, 5:26am

Hmm… might be, but does not fit with how I know the fast path works: it kicks in when there is a single raster source layer,
and not smart enough to figure out a layer is out of bounds (generally speaking the rendering does not trust the source
bounds, they are quite often out of whack)

Cheers
Andrea

···

GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.

James_Rapaport · August 14, 2019, 11:11am

I tested starting with -Dorg.geoserver.render.raster.direct.disable=true. Unfortunately, I don’t see a change and the problem persists.

Cheers,

James

···

On Wed, Aug 14, 2019 at 12:04 AM Jody Garnett <jody.garnett@…84…> wrote:

I think I know where you’re going with this one, because I thought I had seen something in my earlier testing but I was unable to confirm it. I’ve now tested with different datasets that are geographically further apart. In short, when I make a request to draw in the centre, approximately, of the raster (specifically the group layer extent) the problem occurs. When I make a request to draw towards the edge of the raster the request is successful and the problem does not occur.

This indicates to me that the “fast path” is indeed having a failure, which we can report. I agree with Andrea - thanks for the detailed testing.

Hmm… might be, but does not fit with how I know the fast path works: it kicks in when there is a single raster source layer,

and not smart enough to figure out a layer is out of bounds (generally speaking the rendering does not trust the source

bounds, they are quite often out of whack)

Cheers

Andrea

==

GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.

This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com

jive · August 14, 2019, 4:59pm

Thanks, that is my last idea for a workaround, time to debug

···

–
Jody Garnett