[Geoserver-users] Geoserver WFS - app-schema plugin - java.lang.StackOverflowErrors

GeoServer GeoServer User
1

I’m using Geoserver 2.11.dev and app-schema plugin. WFS max request limit is set to 1mil features. When serving large WFS 2.0 requests (over 100000 fetures) from app-schema store (PostGIS backend) requests fail with error:

19 pro 09:49:48 ERROR [geoserver.ows] -
java.lang.StackOverflowError
at com.sun.org.apache.xalan.internal.xsltc.util.IntegerArray.quicksort(IntegerArray.java:162)
at com.sun.org.apache.xalan.internal.xsltc.util.IntegerArray.quicksort(IntegerArray.java:163)
at com.sun.org.apache.xalan.internal.xsltc.util.IntegerArray.quicksort(IntegerArray.java:163)
at com.sun.org.apache.xalan.internal.xsltc.util.IntegerArray.quicksort(IntegerArray.java:163)
…

Serving smaller set of features (<100000) doesn’t cause any error. When I set maximum thread stack size for JAVA in JAVA_OPTS to -Xss4096K, errors disappear, but I’m still wondering why this setting is necessary. Any idea? Can something like reprojection cause this?

Best regards
Davor

2

Hi,

This happens because quicksort (sorting algorithm) is being called recursively so
many times that it exhausts the available stack. Sun Java Virtual Machine -Xss option
allow us to control the size of the stack (which stores method calls, arguments,
recursive calls, …) per thread. So either increasing the stack size per thread or
reducing the number of the involved features makes the error go away.

That say, since your data is stored in a PostGIS database GeoServer should delegate
the sorting on the database, or at least try really hard to do it. 100000 looks like a lot
of features … pagination is not suitable for your use case ? Note that the way GeoServer
pagination is implemented inserts or deletes between calls may be lost, usually this is not
an issue and an acceptable limitation.

Hope this helps,

Nuno Oliveira

···

On 12/19/2017 02:41 PM, Davor Racic wrote:

I’m using Geoserver 2.11.dev and app-schema plugin. WFS max request limit is set to 1mil features. When serving large WFS 2.0 requests (over 100000 fetures) from app-schema store (PostGIS backend) requests fail with error:

19 pro 09:49:48 ERROR [geoserver.ows] -
java.lang.StackOverflowError
at com.sun.org.apache.xalan.internal.xsltc.util.IntegerArray.quicksort(IntegerArray.java:162)
at com.sun.org.apache.xalan.internal.xsltc.util.IntegerArray.quicksort(IntegerArray.java:163)
at com.sun.org.apache.xalan.internal.xsltc.util.IntegerArray.quicksort(IntegerArray.java:163)
at com.sun.org.apache.xalan.internal.xsltc.util.IntegerArray.quicksort(IntegerArray.java:163)
…

Serving smaller set of features (<100000) doesn’t cause any error. When I set maximum thread stack size for JAVA in JAVA_OPTS to -Xss4096K, errors disappear, but I’m still wondering why this setting is necessary. Any idea? Can something like reprojection cause this?

Best regards
Davor

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! [http://sdm.link/slashdot](http://sdm.link/slashdot)

_______________________________________________
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:
- Earning your support instead of buying it, but Ian Turton: [http://www.ianturton.com/talks/foss4g.html#/](http://www.ianturton.com/talks/foss4g.html#/)
- The GeoServer user list posting guidelines: [http://geoserver.org/comm/userlist-guidelines.html](http://geoserver.org/comm/userlist-guidelines.html)

If you want to request a feature or an improvement, also see this: [https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer](https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer)

[Geoserver-users@lists.sourceforge.net](mailto:Geoserver-users@lists.sourceforge.net)
[https://lists.sourceforge.net/lists/listinfo/geoserver-users](https://lists.sourceforge.net/lists/listinfo/geoserver-users)


-- 
Regards,
Nuno Oliveira
==
GeoServer Professional Services from the experts! Visit [http://goo.gl/it488V](http://goo.gl/it488V) for more information.
==

Nuno Miguel Carvalho Oliveira
@nmcoliveira
Software Engineer

GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
Italy
phone: +39 0584 962313
fax:      +39 0584 1660272

[http://www.geo-solutions.it](http://www.geo-solutions.it)
[http://twitter.com/geosolutions_it](http://twitter.com/geosolutions_it)

-------------------------------------------------------
AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.
 
The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility  for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.
3

Davor,

GeoServer uses XSLT to post-process WFS responses (and halve the number of database queries). Do you have the other end of the stack trace so we can see the origin and confirm this as the cause? Try *reducing* your stack size to make the trace shorter.

It looks like this failure might be caused by the Xalan XSLT embedded in the JVM standard library using Quicksort to sort an array of integers. Quicksort is a terrible algorithm because its worst-case performance is O(n**2):
https://en.wikipedia.org/wiki/Quicksort

Some Quicksort implementations can exhibit this behaviour when the input is already sorted, already reverse-sorted, or all inputs are the same (first two criteria combined). Recursive implementations will then use O(n**2) time and stack.

Heapsort is often preferred because its worst-case performance is O(n log n):
https://en.wikipedia.org/wiki/Heapsort

Kind regards,
Ben.

On 20/12/17 03:41, Davor Racic wrote:

I'm using Geoserver 2.11.dev and app-schema plugin. WFS max request limit
is set to 1mil features. When serving large WFS 2.0 requests (over 100000
fetures) from app-schema store (PostGIS backend) requests fail with error:

19 pro 09:49:48 ERROR [geoserver.ows] -
java.lang.StackOverflowError
        at
com.sun.org.apache.xalan.internal.xsltc.util.IntegerArray.quicksort(IntegerArray.java:162)

        at
com.sun.org.apache.xalan.internal.xsltc.util.IntegerArray.quicksort(IntegerArray.java:163)

        at
com.sun.org.apache.xalan.internal.xsltc.util.IntegerArray.quicksort(IntegerArray.java:163)

        at
com.sun.org.apache.xalan.internal.xsltc.util.IntegerArray.quicksort(IntegerArray.java:163)
        ...

Serving smaller set of features (<100000) doesn't cause any error. When I
set maximum thread stack size for JAVA in JAVA_OPTS to -Xss4096K, errors
disappear, but I'm still wondering why this setting is necessary. Any idea?
Can something like reprojection cause this?

Best regards
Davor

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

_______________________________________________
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:
- Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer

Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
Ben Caradoc-Davies <ben@anonymised.com>
Director
Transient Software Limited <https://transient.nz/&gt;
New Zealand

4

Hi Ben & Nuno, thank you for your suggestions.

things seem a little bit clearer. We are testing INSPIRE download service, not an application, so we assume that someone can and should be able to download/transfer whole set of data (cca 130000 features). I suppose some sort of pagination should be done by Geoserver itself in background when data PG store is configured with some value set on “fetch size”, and suppose that this XLST post-processing is done page by page after each fetch? In our case, that property was missing in app-schema store definition. We are testing this at the moment.

Best regards
Davor

sri, 20. pro 2017. u 00:12 Ben Caradoc-Davies <ben@anonymised.com> napisao je:

Davor,

GeoServer uses XSLT to post-process WFS responses (and halve the number
of database queries). Do you have the other end of the stack trace so we
can see the origin and confirm this as the cause? Try reducing your
stack size to make the trace shorter.

It looks like this failure might be caused by the Xalan XSLT embedded in
the JVM standard library using Quicksort to sort an array of integers.
Quicksort is a terrible algorithm because its worst-case performance is
O(n**2):
https://en.wikipedia.org/wiki/Quicksort

Some Quicksort implementations can exhibit this behaviour when the input
is already sorted, already reverse-sorted, or all inputs are the same
(first two criteria combined). Recursive implementations will then use
O(n**2) time and stack.

Heapsort is often preferred because its worst-case performance is O(n
log n):
https://en.wikipedia.org/wiki/Heapsort

Kind regards,
Ben.

On 20/12/17 03:41, Davor Racic wrote:

I’m using Geoserver 2.11.dev and app-schema plugin. WFS max request limit
is set to 1mil features. When serving large WFS 2.0 requests (over 100000
fetures) from app-schema store (PostGIS backend) requests fail with error:

19 pro 09:49:48 ERROR [geoserver.ows] -
java.lang.StackOverflowError
at
com.sun.org.apache.xalan.internal.xsltc.util.IntegerArray.quicksort(IntegerArray.java:162)

at
com.sun.org.apache.xalan.internal.xsltc.util.IntegerArray.quicksort(IntegerArray.java:163)

at
com.sun.org.apache.xalan.internal.xsltc.util.IntegerArray.quicksort(IntegerArray.java:163)

at
com.sun.org.apache.xalan.internal.xsltc.util.IntegerArray.quicksort(IntegerArray.java:163)
…

Serving smaller set of features (<100000) doesn’t cause any error. When I
set maximum thread stack size for JAVA in JAVA_OPTS to -Xss4096K, errors
disappear, but I’m still wondering why this setting is necessary. Any idea?
Can something like reprojection cause this?

Best regards
Davor

Check out the vibrant tech community on one of the world’s most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:

If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer

Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

–
Ben Caradoc-Davies <ben@anonymised.com>
Director
Transient Software Limited <https://transient.nz/>
New Zealand

5

For the record, I made one small mistake here: for stack-based recursive implementations, worst case Quicksort is O(n**2) time and O(n) stack. By comparison, worst case Heapsort is O(n log n) time and O(log n) stack.

Kind regards,
Ben.

On 20/12/17 12:12, Ben Caradoc-Davies wrote:

It looks like this failure might be caused by the Xalan XSLT embedded in the JVM standard library using Quicksort to sort an array of integers. Quicksort is a terrible algorithm because its worst-case performance is O(n**2):
https://en.wikipedia.org/wiki/Quicksort

Some Quicksort implementations can exhibit this behaviour when the input is already sorted, already reverse-sorted, or all inputs are the same (first two criteria combined). Recursive implementations will then use O(n**2) time and stack.

Heapsort is often preferred because its worst-case performance is O(n log n):
https://en.wikipedia.org/wiki/Heapsort

--
Ben Caradoc-Davies <ben@anonymised.com>
Director
Transient Software Limited <https://transient.nz/&gt;
New Zealand

6

Davor,

the XSLT is applied after the response has been encoded to XML. The size of the response will be affected by maxFeatures/count but not I think by the database fetch size. Pagination is applied at the WFS level with startIndex (WFS 2.0, supported in GeoServer WFS 1.1.0 as an extension).

The stack trace of the failing request will help us investigate the problem.

Kind regards,
Ben.

On 20/12/17 22:59, Davor Racic wrote:

Hi Ben & Nuno, thank you for your suggestions.

things seem a little bit clearer. We are testing INSPIRE download service,
not an application, so we assume that someone can and should be able to
download/transfer whole set of data (cca 130000 features). I suppose some
sort of pagination should be done by Geoserver itself in background when
data PG store is configured with some value set on "fetch size", and
suppose that this XLST post-processing is done page by page after each
fetch? In our case, that property was missing in app-schema store
definition. We are testing this at the moment.

Best regards
Davor

sri, 20. pro 2017. u 00:12 Ben Caradoc-Davies <ben@anonymised.com> napisao je:

Davor,

GeoServer uses XSLT to post-process WFS responses (and halve the number
of database queries). Do you have the other end of the stack trace so we
can see the origin and confirm this as the cause? Try *reducing* your
stack size to make the trace shorter.

It looks like this failure might be caused by the Xalan XSLT embedded in
the JVM standard library using Quicksort to sort an array of integers.
Quicksort is a terrible algorithm because its worst-case performance is
O(n**2):
https://en.wikipedia.org/wiki/Quicksort

Some Quicksort implementations can exhibit this behaviour when the input
is already sorted, already reverse-sorted, or all inputs are the same
(first two criteria combined). Recursive implementations will then use
O(n**2) time and stack.

Heapsort is often preferred because its worst-case performance is O(n
log n):
https://en.wikipedia.org/wiki/Heapsort

Kind regards,
Ben.

On 20/12/17 03:41, Davor Racic wrote:

I'm using Geoserver 2.11.dev and app-schema plugin. WFS max request limit
is set to 1mil features. When serving large WFS 2.0 requests (over 100000
fetures) from app-schema store (PostGIS backend) requests fail with

error:

19 pro 09:49:48 ERROR [geoserver.ows] -
java.lang.StackOverflowError
         at

com.sun.org.apache.xalan.internal.xsltc.util.IntegerArray.quicksort(IntegerArray.java:162)

         at

com.sun.org.apache.xalan.internal.xsltc.util.IntegerArray.quicksort(IntegerArray.java:163)

         at

com.sun.org.apache.xalan.internal.xsltc.util.IntegerArray.quicksort(IntegerArray.java:163)

         at

com.sun.org.apache.xalan.internal.xsltc.util.IntegerArray.quicksort(IntegerArray.java:163)

         ...

Serving smaller set of features (<100000) doesn't cause any error. When I
set maximum thread stack size for JAVA in JAVA_OPTS to -Xss4096K, errors
disappear, but I'm still wondering why this setting is necessary. Any

idea?

Can something like reprojection cause this?

Best regards
Davor

------------------------------------------------------------------------------

Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

_______________________________________________
Geoserver-users mailing list

Please make sure you read the following two resources before posting to

this list:

- Earning your support instead of buying it, but Ian Turton:

http://www.ianturton.com/talks/foss4g.html#/

- The GeoServer user list posting guidelines:

http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this:

https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer

Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
Ben Caradoc-Davies <ben@anonymised.com>
Director
Transient Software Limited <https://transient.nz/&gt;
New Zealand

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

_______________________________________________
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:
- Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer

Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
Ben Caradoc-Davies <ben@anonymised.com>
Director
Transient Software Limited <https://transient.nz/&gt;
New Zealand

7

We have encountered the same issue with our app schema (used for INSPIRE). We
are using GeoServer 2.13 and our backend database is Oracle 10g.

Davor, were you able to resolve your issue and how did you do it?

Ben, here is the full call stack when the error happens:

Regards,
Jure

--
Sent from: http://osgeo-org.1560.x6.nabble.com/GeoServer-User-f3786390.html

8

Yes, I set maximum thread stack size for JAVA in JAVA_OPTS to -Xss4096K.

--
Sent from: http://osgeo-org.1560.x6.nabble.com/GeoServer-User-f3786390.html