[Geoserver-users] Problems with REST on some coveragestores

GeoServer GeoServer User
1

Hello,

The issue comes probably from the open connections to your database.
If you have a standard postgres/postgis installation the number of simultanous connection has limits.
You can augment this limit but, if the flow introduces always new coverages then soon or later this will happen again.

The best solution is to work with a jndi datasource defined in your tomcat/jetty container and use this connection in
the geoserver definition.

Best regards,

Eric Smets

-- 
Eric Smets                                    [eric.smets@anonymised.com](mailto:eric.smets@anonymised.com)
FKS bvba - Formal and Knowledge Systems       [http://www.fks.be](http://www.fks.be/)/
Schampbergstraat 32                           Tel:  ++32-(0)11-21 49 11
B-3511 Hasselt                                Fax:  ++32-(0)11-22 04 19

-----Oorspronkelijk bericht-----
Afzender: Sabine Ohlendorf sabine.ohlendorf@anonymised.com
Verstuurd: Maandag 1 Juni 2015 12:26
Aan: Geoserver Mailinglist geoserver-users@lists.sourceforge.net
Onderwerp: [Geoserver-users] Problems with REST on some coveragestores

Hi,

I have a geoserver version 2.7.0 running on Ubuntu 14.4. and I am using the REST interface to create new coveragestores and layers or to add further data to an existing coveragestore. In general this is working quite good.
I have some python scripts sending the REST requests to the server, via python package ‘requests’. But since a few days I have problems ingesting new data to some of my coveragestores. For other coveragestores everything is still working without problems. I am always using the same requests.
POST-Reqeust to the URL:
http://ipaddress/geoserver/rest/workspaces/my-workspace/coveragestores/my-coveragestore/file.imagemosaic
A zip file with one tif image is sent.
I also tried the corresponding request via curl:
curl -v -u username:password -XPOST -H “Content-type:application/zip” --data-binary @path-to-zip http://ipaddress/geoserver/rest/workspaces/my-workspace/coveragestores/my-coveragestore/file.imagemosaic

The result is the same. The data is not included into the coveragestore. But I don’t get any error message. The HTTP return code is ‘202 - Accepted’ and the tif file is copied to the geoserver to the correct data directory of the coveragestore but it is not included into the coveragestore. We are using a database with the following database.properties file for the coveragestores:

JNDI specific

SPI=org.geotools.data.postgis.PostgisNGJNDIDataStoreFactory
jndiReferenceName=jdbc/postgres
Loose\ bbox=true
preparedStatements=false

Looking through the log files of geoserver and postgresql I could not find any error message.
/var/lib/tomcat/geoserver_master/logs/catalina.out
/var/lib/tomcat/geoserver_master/logs/localhost_access_log.2015-06-01.txt
/var/log/postgresql/postgresql-9.3-main.log
Everything looks normal. But the image is not ingested to the coveragestore. The strange thing is that for some coveragestores it is still working and for others I have these problem which just occured from one day to the other.
Does anyone has an idea? Are the maybe some more logfiles I could check?

Thanks.
Sabine

 

------------------------------------------------------------------------------


_______________________________________________

Geoserver-users mailing list

Geoserver-users@lists.sourceforge.net

https://lists.sourceforge.net/lists/listinfo/geoserver-users
2

On Mon, Jun 1, 2015 at 12:31 PM, Eric Smets <eric.smets@anonymised.com> wrote:

Hello,

The issue comes probably from the open connections to your database.
If you have a standard postgres/postgis installation the number of
simultanous connection has limits.
You can augment this limit but, if the flow introduces always new
coverages then soon or later this will happen again.

The best solution is to work with a jndi datasource defined in your
tomcat/jetty container and use this connection in
the geoserver definition.

By the way, the issue of leaking connections in mosaics backed by jdbc
stores should be solved in 2.7.1,
and in 2.6.x nightly builds too

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------

3

Hi,

@Eric:thanks for the hint. We are already working with a jndi datasource.

@Andrea: good to know :slight_smile:

Cheers,
Sabine

···

On Mon, Jun 1, 2015 at 12:31 PM, Eric Smets <eric.smets@anonymised.com51…> wrote:

Hello,

The issue comes probably from the open connections to your database.
If you have a standard postgres/postgis installation the number of simultanous connection has limits.
You can augment this limit but, if the flow introduces always new coverages then soon or later this will happen again.

The best solution is to work with a jndi datasource defined in your tomcat/jetty container and use this connection in
the geoserver definition.

By the way, the issue of leaking connections in mosaics backed by jdbc stores should be solved in 2.7.1,
and in 2.6.x nightly builds too

Cheers
Andrea

–

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

4

Hi,

by chance I found the solution of the problem myself:
I found out that I was trying to ingest 32bit tif data to a layer with datatype 64bits. That was the reason why it didn't accept the data.

Still it is strange that this didn't throw an error or appeared in any logfile. It would have been good to receive an Error as answer on the REST request.

Regards,
Sabine

5

On Tue, Jun 2, 2015 at 1:31 PM, Sabine Ohlendorf <sabine.ohlendorf@anonymised.com.>
wrote:

Hi,

by chance I found the solution of the problem myself:
I found out that I was trying to ingest 32bit tif data to a layer with
datatype 64bits. That was the reason why it didn't accept the data.

It is documented, see point 2 in the documentation
http://docs.geoserver.org/latest/en/user/tutorials/image_mosaic_plugin/imagemosaic.html

Still it is strange that this didn't throw an error or appeared in any
logfile. It would have been good to receive an Error as answer on the REST
request.

Indeed it would be nice. You can file a request for improvement at
https://osgeo-org.atlassian.net/projects/GEOS
Pull request are welcomed too :slight_smile:

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------