[Geoserver-users] WFS service request, produces a partial response

Hello.

I have installed GeoServer as Tomcat webapp on a DigitalOcean droplet.

It is working fine, for what i see.

The problem i have is with this relatively larger shape file, “/home/pftd_ag/CarteBase/sottocategorie_ag” that has 64 multipart polygons.

I see them all if i dowload the files to the PC while, if I access through the WFS service of my GeoServer, I can only see 17 multipart polygons.

I can see that the answer to the WFS request is not complete: “This page contains the following errors: error on line 1 at column 4733180: expected ‘>’”, nonetheless qGIS throws no error and shows the map, though not complete.

Who is truncating the output?

Is there a limitation with DO connection or is the effect due to GeoServer?

Moreover: can the problem be solved?

Many thanks, Roberto Scotti

On Mon, Aug 10, 2015 at 1:43 PM, Scotti Roberto <roberto.scotti@anonymised.com>
wrote:

Who is truncating the output?

Most likely a reading error, which is often caused by invalid geometries in
the data source. Check the GeoServer logs to get more information.

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------

Hello
Many thanks for the suggestion.I downloaded the log and tried to examine it.
I do read the word “ERROR” in connection with the conclusion of the WFS request for the shapefile “sottocategorie_ag”.
I do not see the beginning of the request.
I suppose the last line documents the initial request.
The ERROR says:
“2015-08-11 05:30:32,567 ERROR [geoserver.ows] -
java.lang.OutOfMemoryError: Java heap space”

Can this be due to “invalid geometries”?

The Geoserver is running on a PVS (Digital Ocean Droplet) with 512 MB

excerpt_form_geoserver.log.txt (13.2 KB)

···

memory. Is this cause of the error? Is the software not able to forecast that heap will need more space than what is available in order to manage the situation? Such a problem can occur with any memory dimension or is there a minimum memory requirement in order to avoid it?

I am attaching an excerpt of the log with what I understand as useful lines.

Hoping you can answer again, many thanks in advance,
Cheers
Roberto

2015-08-10 13:57 GMT+02:00 Andrea Aime <andrea.aime@anonymised.com>:

On Mon, Aug 10, 2015 at 1:43 PM, Scotti Roberto <roberto.scotti@anonymised.com> wrote:

Who is truncating the output?

Most likely a reading error, which is often caused by invalid geometries in
the data source. Check the GeoServer logs to get more information.

Cheers
Andrea

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


On Tue, Aug 11, 2015 at 12:57 PM, Scotti Roberto <roberto.scotti@anonymised.com>
wrote:

Hello
Many thanks for the suggestion.
I downloaded the log and tried to examine it.
I do read the word "ERROR" in connection with the conclusion of the WFS
request for the shapefile "sottocategorie_ag".
I do not see the beginning of the request.
I suppose the last line documents the initial request.
The ERROR says:
"2015-08-11 05:30:32,567 ERROR [geoserver.ows] -
java.lang.OutOfMemoryError: Java heap space"

Can this be due to "invalid geometries"?

The Geoserver is running on a PVS (Digital Ocean Droplet) with 512 MB

Ah, this might be the cause. How did you install/deploy GeoServer?
It might be running with just 64MB of memory on such a small machine
(Java server side ergononomics not kicking in, and as result, the JVM
running as if it was working on an applet).

We should ensure that it runs with at least 256MB in that environment, but
to
do so, we need to know how you deployed it.

memory. Is this cause of the error? Is the software not able to forecast
that heap will need more space than what is available in order to manage
the situation? Such a problem can occur with any memory dimension or is
there a minimum memory requirement in order to avoid it?

We normally recommend to run GeoServer with 2GB of heap, on a machine that
has at least 4GB
of memory, it's possible to run it with less of course, but it will
severely limit the amount of
concurrent requests one can handle.
Your WFS download can be going OOM because of many reasons, large geometries
being one, concurrent requests being another (and if you're running
GeoServer with
just 64m of memory, well, we should be thankful it started at all)

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------

On Tue, Aug 11, 2015 at 5:46 PM, Scotti Roberto <roberto.scotti@anonymised.com>
wrote:

Hello.
My GeoServer is deployed as a Tomcat webapp on a (DigitalOcean droplet)
Personal Virtual Server offering a 32bit Ubuntu 10.04 system with 512 MB
memory and 20 GB HD.
Could I deploy GeoServer better?

Maybe... did you install Tomcat yourself?

Can I verify/set the amount of memory i can use?

Unfortunately we don't have a way to get the max memory allowed from the
UI, but you
try to make it go OOM, and then go in the "status" page and report back JVM
version
and memory usage.

Another interesting information can be provided by running "ps aux | grep
java"
in your PVS, and again, report back the output

We do not have a problem of concurrent access, the shape file I am
requesting is complex but not that big (see attachment)

Agreed, does not seem big enough to cause troubles

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------

1 - Yes I installed Tomcat following simple directions: https://www.digitalocean.com/community/tutorials/how-to-install-apache-tomcat-on-ubuntu-12-04

tomcat7 18800 0.6 46.0 342252 233496 ? Sl 13:58 1:18 /usr/lib/jvm/default-java/bin/java -Djava.util.logging.config.file=/var/lib/tomcat7/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.awt.headless=true -Xmx128m -XX:+UseConcMarkSweepGC -Djava.endorsed.dirs=/usr/share/tomcat7/endorsed -classpath /usr/share/tomcat7/bin/bootstrap.jar:/usr/share/tomcat7/bin/tomcat-juli.jar -Dcatalina.base=/var/lib/tomcat7 -Dcatalina.home=/usr/share/tomcat7 -Djava.io.tmpdir=/tmp/tomcat7-tomcat7-tmp org.apache.catalina.startup.Bootstrap start
scotti 20126 0.0 0.1 4672 824 pts/3 S+ 17:17 0:00 grep --color=auto java
scotti@anonymised.com87…:~$

Thanks again
Roberto

···

Actually step 3 seemed unnecessary.
Are the following more complex instructions necessary? https://www.digitalocean.com/community/tutorials/how-to-install-apache-tomcat-7-on-ubuntu-14-04-via-apt-get
2 - “report back JVM version and memory usage.” this is to be done with this “tomcat7-admin” ?
3 - scotti@anonymised.com:~$ ps aux | grep java

2015-08-11 18:13 GMT+02:00 Andrea Aime <andrea.aime@anonymised.com>:

On Tue, Aug 11, 2015 at 5:46 PM, Scotti Roberto <roberto.scotti@anonymised.com> wrote:

Hello.
My GeoServer is deployed as a Tomcat webapp on a (DigitalOcean droplet) Personal Virtual Server offering a 32bit Ubuntu 10.04 system with 512 MB memory and 20 GB HD.
Could I deploy GeoServer better?

Maybe… did you install Tomcat yourself?

Can I verify/set the amount of memory i can use?

Unfortunately we don’t have a way to get the max memory allowed from the UI, but you
try to make it go OOM, and then go in the “status” page and report back JVM version
and memory usage.

Another interesting information can be provided by running “ps aux | grep java”
in your PVS, and again, report back the output

We do not have a problem of concurrent access, the shape file I am requesting is complex but not that big (see attachment)

Agreed, does not seem big enough to cause troubles

Cheers

Andrea

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.