Hi list,
I have an imagepyramid made of of 16 imagemosaic layers.
The imagemosaic indexes are stored in a postgis database.
I've had connection exhaustion problems (i.e. "sorry, too many clients" from the PostgreSQL backend).
My guess is that even though the imagepyramid has its own datastore.properties file, each imagemosaic layer uses the "max\ connections" parameter from its own datastore.properties file. So for 16 imagemosaic layers with the default max connections setting (10), we could go as high as 160 idle open connections.
Is this assumption correct ? If I am right, then the only way to keep these connections at a reasonable level would be to set the max connections per imagemosaic to something like 4 or 5.
Thanks for your advices on this !
Cheers !
--
Arnaud
Hi Arnaud,
for that case you need to “step up” your installation and have everything use the
same JNDI connection pool (same connection pool for all stores), see also:
https://docs.geoserver.org/stable/en/user/tutorials/tomcat-jndi/tomcat-jndi.html
Cheers
Andrea
···
Regards, Andrea Aime == GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.
Hi Andrea
Le 05/02/2019 à 12:14, Andrea Aime a écrit :
for that case you need to "step up" your installation and have everything use the
same JNDI connection pool (same connection pool for all stores), see also:
https://docs.geoserver.org/stable/en/user/tutorials/tomcat-jndi/tomcat-jndi.html
That looks like a good option.
For the time being I was thinking about configuring a pgBouncer service (i.e. connection pooling middleware), which I guess would achieve the same result.
But I'll look into JNDI first if that can spare me an additional software to maintain. Thanks for the suggestion !
Cheers
--
Arnaud
Le 05/02/2019 à 12:14, Andrea Aime a écrit :
for that case you need to "step up" your installation and have everything use the
same JNDI connection pool (same connection pool for all stores), see also:
https://docs.geoserver.org/stable/en/user/tutorials/tomcat-jndi/tomcat-jndi.html
JNDI works like a charm ! Thanks !
Just a note, I don't know whether this is a bug or something missing in the documentation, or maybe even something I have overlooked :
the "schema" parameter in the Datastore Configuration page does not seem to have any effect.
I've set it to the name of my postgresql's database schema, but the log was flooded with geotools.jdbc trying to read the primary keys of all the tables in my database (and not in my schema alone). I had not checked "expose primary keys".
More problematic, geoserver that the relation containing my mosaic index did not exist.
In the datastore.properties of each mosaic (and of my imagepyramid) I added "schema=myschema", and everything worked perfectly afterwards.
Is the the expected behaviour ? Should I propose a quick patch in the jndi configuration doc ?
Also for my datastore.properties file, I had followed the guide from the "geoserver training" section of your website : https://geoserver.geo-solutions.it/edu/en/multidim/imagemosaic/mosaic_datastore.html#using-a-jndi-connection-pool
which did not mention this "schema" parameter, so mabe I could add this somewhere ?
Cheers
--
Arnaud
Just a note, I don’t know whether this is a bug or something missing in
the documentation, or maybe even something I have overlooked :
the “schema” parameter in the Datastore Configuration page does not seem
to have any effect.
Hum… the schema in the datastore configuration should affect only the feature
types served by the datastore itself.
Are you expecting it to affect the mosaics somehow? Did you setup any connection
between the datastore config and the mosaic?
In the datastore.properties of each mosaic (and of my imagepyramid) I
added “schema=myschema”, and everything worked perfectly afterwards.
Is the the expected behaviour ? Should I propose a quick patch in the
jndi configuration doc ?
I think it’s the expected behavior, but without seeing if there is any connection
between the mosaic and the store page in the UI I cannot comment further
Also for my datastore.properties file, I had followed the guide from the
“geoserver training” section of your website :
https://geoserver.geo-solutions.it/edu/en/multidim/imagemosaic/mosaic_datastore.html#using-a-jndi-connection-pool
which did not mention this “schema” parameter, so mabe I could add this
somewhere ?
I’m afraid not, while the generated output of the training material is under a creative commons license,
the source code of it is in a private repo. But it’s a good suggestion, I’ve opened a ticket to remind
us a fix the next time we work on the material.
Cheers
Andrea
···
GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.
Hi Andrea
Le 07/02/2019 à 10:09, Andrea Aime a écrit :
Hum... the schema in the datastore configuration should affect only the feature types served by the datastore itself.
Are you expecting it to affect the mosaics somehow?
No, sorry, I got lost in the process. Now I believe I understand.
For some reason I thought that the JNDI datastore would be used by the imagemosaic.
But the imagemosaic directely references the JNDI connection, not the JNDI datastore. Makes sense.
Did you setup any connection between the datastore config and the mosaic?
No, that's what I thought was happening when I set up the jndiReferenceName in the datastore.properties file to be the same as the jndiReferenceName in the JNDI datastore configuration. Now I understand that both refernce the jndiReferenceName defined in the context.xml of my tomcat server. Totally logical, but as you can see I'm not very knowledgeable in that matter.
I'm afraid not, while the generated output of the training material is under a creative commons license, the source code of it is in a private repo. But it's a good suggestion, I've opened a ticket to remind us a fix the next time we work on the material.
OK, thanks for your help Andrea.
Cheers !
--
Arnaud