[Geoserver-users] GeoServer slowdowns during PostGres performance testing

Hi List,
Doing some performance testing (WMS GetMap) on a GeoServer (2.7.2) with PostGreSQL (9.5.2) using JMeter, we’re getting some odd slowdowns. It seems that for a short while (up to a few minutes) GeoServer massively slows down in responding to requests; the “max” times shoot up from a few seconds to about 60-70 seconds! We’ve had this across several environments with PostGres, but the same tests on Oracle have never exhibited this.
The issue isn’t on the PostGres end - the slowest query there during the tests takes about 4 seconds, there’s no queuing and the resource usage on its (separate to GeoServer) box is negligible.
If I make the same request to GeoServer directly when not part of this testing, it only takes a second or two, so there’s nothing obviously wrong with the requests either. The problem is also proving difficult to replicate reliably.
The connections to PG are via Tomcat using a JNDI pool.
Has anyone encountered this sort of behaviour before or have any inkling as to the cause?
Thanks,
Jonathan

most likely cause is that you have saturated the postgis connection pool and are waiting for earlier requests to complete.

I can’t think of anything else obvious that might cause that problem.

Ian

···

On 12 May 2016 at 13:08, Jonathan Moules <jonathan-lists@anonymised.com> wrote:

Hi List,
Doing some performance testing (WMS GetMap) on a GeoServer (2.7.2) with PostGreSQL (9.5.2) using JMeter, we’re getting some odd slowdowns. It seems that for a short while (up to a few minutes) GeoServer massively slows down in responding to requests; the “max” times shoot up from a few seconds to about 60-70 seconds! We’ve had this across several environments with PostGres, but the same tests on Oracle have never exhibited this.
The issue isn’t on the PostGres end - the slowest query there during the tests takes about 4 seconds, there’s no queuing and the resource usage on its (separate to GeoServer) box is negligible.
If I make the same request to GeoServer directly when not part of this testing, it only takes a second or two, so there’s nothing obviously wrong with the requests either. The problem is also proving difficult to replicate reliably.
The connections to PG are via Tomcat using a JNDI pool.
Has anyone encountered this sort of behaviour before or have any inkling as to the cause?
Thanks,
Jonathan


Mobile security can be enabling, not merely restricting. Employees who
bring their own devices (BYOD) to work are irked by the imposition of MDM
restrictions. Mobile Device Manager Plus allows you to control only the
apps on BYO-devices by containerizing them, leaving personal data untouched!
https://ad.doubleclick.net/ddm/clk/304595813;131938128;j


Geoserver-users mailing list
Geoserver-users@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Ian Turton

Hi Ian,
Thanks for the suggestion but we don’t think it’s that. On the PG box there were no logs indicating that the pool had run out, and on the GeoServer side I can’t seem to hit the pool limit when it’s as low as 20 but using 98 threads to test a single instance.

We’re thinking the behaviour we’re seeing seems more like garbage collection locking up an set of instances for a while (once its freed up, those instances were going better than the not-locked-up ones). Is GC a plausible mechanism?

Cheers,
Jonathan

---- On Thu, 12 May 2016 13:23:02 +0100 Ian Turton<ijturton@anonymised.com…> wrote ----

most likely cause is that you have saturated the postgis connection pool and are waiting for earlier requests to complete.

I can’t think of anything else obvious that might cause that problem.

Ian

On 12 May 2016 at 13:08, Jonathan Moules <jonathan-lists@anonymised.com> wrote:

Hi List,
Doing some performance testing (WMS GetMap) on a GeoServer (2.7.2) with PostGreSQL (9.5.2) using JMeter, we’re getting some odd slowdowns. It seems that for a short while (up to a few minutes) GeoServer massively slows down in responding to requests; the “max” times shoot up from a few seconds to about 60-70 seconds! We’ve had this across several environments with PostGres, but the same tests on Oracle have never exhibited this.
The issue isn’t on the PostGres end - the slowest query there during the tests takes about 4 seconds, there’s no queuing and the resource usage on its (separate to GeoServer) box is negligible.
If I make the same request to GeoServer directly when not part of this testing, it only takes a second or two, so there’s nothing obviously wrong with the requests either. The problem is also proving difficult to replicate reliably.
The connections to PG are via Tomcat using a JNDI pool.
Has anyone encountered this sort of behaviour before or have any inkling as to the cause?
Thanks,
Jonathan


Mobile security can be enabling, not merely restricting. Employees who
bring their own devices (BYOD) to work are irked by the imposition of MDM
restrictions. Mobile Device Manager Plus allows you to control only the
apps on BYO-devices by containerizing them, leaving personal data untouched!
https://ad.doubleclick.net/ddm/clk/304595813;131938128;j


Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Ian Turton

On Thu, May 12, 2016 at 3:44 PM, Jonathan Moules <
jonathan-lists@anonymised.com> wrote:

Hi Ian,
Thanks for the suggestion but we don't think it's that. On the PG box
there were no logs indicating that the pool had run out, and on the
GeoServer side I can't seem to hit the pool limit when it's as low as 20
but using 98 threads to test a single instance.

We're thinking the behaviour we're seeing seems more like garbage
collection locking up an set of instances for a while (once its freed up,
those instances were going better than the not-locked-up ones). Is GC a
plausible mechanism?

You can use VisualVM connected to the JVM to see if and when it's running
GC.
Do you have high CPU usage during the slowdonw?

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------