[Geoserver-users] CPU drain

GeoServer GeoServer User
1

Hi,

Before you start optimizing anything you should evaluate if the performance that you have is normal. For example in this 11 years old benchmark https://www.idee.es/resources/presentaciones/JIDEE07/POWERPOINT_JIDEE2007/PowerPoint.7-Mapserver_vs._Geoserver.pdf which was made with a computer that is today classified as a toy

(Dual core (1.8Ghz per core). 2GB RAM. 7200RPM disk. Linux. PostgreSQL 8.2.4. PostGIS 1.2) shows much better performance.

Shapefile of size 6 MB is really very small dataset. If it takes a second to render it with a powerful computer there must be something special in the data or in your installation. I am sure that there are people on this list who get better results with Raspberry Pi.

Can you share your 6 MB dataset? Or could you repeat your tests with some public data for example from http://www.naturalearthdata.com/downloads/?

-Jukka Rahkonen-

···

Lähettäjä: Tripple Tee [mailto:trippletee96@…84…]
Lähetetty: 9. elokuuta 2018 7:51
Vastaanottaja: geoserver-users@lists.sourceforge.net
Aihe: Re: [Geoserver-users] CPU drain

Shape file is not doing well even on a powerful CPU. I have the same set of data on a proper server with Xeon chip (Virtual Windows 2016 with 10 vCPU) the performance is a mark improvement compare with the poor Celeron. The CPU usage jump to 15% every time I zoom in or pan, but it still take a second to render.

I did some googling on Java and multi core/thread CPU. Seem like Java and most software can not take full advantage of the multi core/thread provided by hardware.

Thanks Brad for the info about PostGIS, I might have to stuck with shape file and it lean, my design goal is portable:

1 geoserver on main server, when a team going away, all user need to do is to robocopy the whole folder from the main server to the portable server. Lucky GeoServer was written in Java so I can achieve this while using different OS (main server is Windows 2016, portable server is Linux)

Might be able to do this with PostGIS but I will have to copy the database from Windows to Linux, but it will not be a simple process for user.

Sent from Mail for Windows 10

From: bradh@…7296…
Sent: Thursday, 9 August 2018 11:43 AM
To: ‘Tripple Tee’; geoserver-users@lists.sourceforge.net
Subject: RE: [Geoserver-users] CPU drain

There are a bunch of tips at http://docs.geoserver.org/latest/en/user/production/index.html and you need to benchmark (e.g. with jmeter) against your expected workload when performance tuning.

In general, shapefile is not going to be very high performance, as noted at http://docs.geoserver.org/latest/en/user/production/data.html#use-a-spatial-database

Brad

From: Tripple Tee <trippletee96@…84…>
Sent: Thursday, 9 August 2018 11:16 AM
To: geoserver-users@lists.sourceforge.net
Subject: [Geoserver-users] CPU drain

Hi,

Please give some pointer on optimising for better resource consumption

I am running GeoServer on Linux with Intel Celeron 4-core processor, which I know is slow

When I open an individual layer (or layer group) of a region 200km each way (with details like roads, buildings …) the shape files size is under 0.5MB it loads quite comfortably, effortless to zoom and pan.

When I open a layer (or layer group) of the whole world with less detail (only land and water lines) it seem to struggle. The size of the shape file is around 5-7MB, I can see the CPU spike and it takes seconds to render as I zoom in or pan the map. Note: I am preview this layer on the default 750x400 resolution box. Should GeoServer only query data of the sub region for that preview box ?

1 - Is this the expected performance ?

2 – How can I optimize it to stop the CPU spike and lag,

Thanks

Sent from Mail for Windows 10

2

Thanks Jukka,

Thanks for the link, I do think I have it wrong somewhere.

The shape files I am using are indeed from NaturalEarth.

https://www.naturalearthdata.com/http//www.naturalearthdata.com/download/10m/physical/ne_10m_land.zip

https://www.naturalearthdata.com/http//www.naturalearthdata.com/download/10m/physical/ne_10m_minor_islands.zip

or

https://www.naturalearthdata.com/http//www.naturalearthdata.com/download/10m/physical/ne_10m_ocean.zip

Not sure if my Bounding box is correctly published. I press “Compute from data”

Native and Lat/Long bounding box are same

MinX: -179.9999 MinY: -89.9999 MaxX: 180 MaxY: 83.6341

This is what layer preview show

Look like the earth was render twice

Other setting while creating new layer is by default, the Coordinate Refernece System: EPSG:4326

Thanks

Sent from Mail for Windows 10

From: Rahkonen Jukka (MML)
Sent: Thursday, 9 August 2018 5:00 PM
To: Tripple Tee; geoserver-users@lists.sourceforge.net
Subject: Re: [Geoserver-users] CPU drain

Hi,

Before you start optimizing anything you should evaluate if the performance that you have is normal. For example in this 11 years old benchmark https://www.idee.es/resources/presentaciones/JIDEE07/POWERPOINT_JIDEE2007/PowerPoint.7-Mapserver_vs._Geoserver.pdf which was made with a computer that is today classified as a toy

(Dual core (1.8Ghz per core). 2GB RAM. 7200RPM disk. Linux. PostgreSQL 8.2.4. PostGIS 1.2) shows much better performance.

Shapefile of size 6 MB is really very small dataset. If it takes a second to render it with a powerful computer there must be something special in the data or in your installation. I am sure that there are people on this list who get better results with Raspberry Pi.

Can you share your 6 MB dataset? Or could you repeat your tests with some public data for example from http://www.naturalearthdata.com/downloads/?

-Jukka Rahkonen-

Lähettäjä: Tripple Tee [mailto:trippletee96@anonymised.com]
Lähetetty: 9. elokuuta 2018 7:51
Vastaanottaja: geoserver-users@lists.sourceforge.net
Aihe: Re: [Geoserver-users] CPU drain

Shape file is not doing well even on a powerful CPU. I have the same set of data on a proper server with Xeon chip (Virtual Windows 2016 with 10 vCPU) the performance is a mark improvement compare with the poor Celeron. The CPU usage jump to 15% every time I zoom in or pan, but it still take a second to render.

I did some googling on Java and multi core/thread CPU. Seem like Java and most software can not take full advantage of the multi core/thread provided by hardware.

Thanks Brad for the info about PostGIS, I might have to stuck with shape file and it lean, my design goal is portable:

1 geoserver on main server, when a team going away, all user need to do is to robocopy the whole folder from the main server to the portable server. Lucky GeoServer was written in Java so I can achieve this while using different OS (main server is Windows 2016, portable server is Linux)

Might be able to do this with PostGIS but I will have to copy the database from Windows to Linux, but it will not be a simple process for user.

Sent from Mail for Windows 10

From: bradh@anonymised.comuth.net
Sent: Thursday, 9 August 2018 11:43 AM
To: ‘Tripple Tee’; geoserver-users@lists.sourceforge.net
Subject: RE: [Geoserver-users] CPU drain

There are a bunch of tips at http://docs.geoserver.org/latest/en/user/production/index.html and you need to benchmark (e.g. with jmeter) against your expected workload when performance tuning.

In general, shapefile is not going to be very high performance, as noted at http://docs.geoserver.org/latest/en/user/production/data.html#use-a-spatial-database

Brad

From: Tripple Tee <trippletee96@anonymised.com>
Sent: Thursday, 9 August 2018 11:16 AM
To: geoserver-users@lists.sourceforge.net
Subject: [Geoserver-users] CPU drain

Hi,

Please give some pointer on optimising for better resource consumption

I am running GeoServer on Linux with Intel Celeron 4-core processor, which I know is slow

When I open an individual layer (or layer group) of a region 200km each way (with details like roads, buildings …) the shape files size is under 0.5MB it loads quite comfortably, effortless to zoom and pan.

When I open a layer (or layer group) of the whole world with less detail (only land and water lines) it seem to struggle. The size of the shape file is around 5-7MB, I can see the CPU spike and it takes seconds to render as I zoom in or pan the map. Note: I am preview this layer on the default 750x400 resolution box. Should GeoServer only query data of the sub region for that preview box ?

1 - Is this the expected performance ?

2 – How can I optimize it to stop the CPU spike and lag,

Thanks

Sent from Mail for Windows 10

3

Hi guys,

I haven’t found the problem with my setup, but I found the shape file that cause CPU drain, the natural-earth-ocean-shape file, without it I can see the respond time is nearly instance

On a similar topic, I have a layer made up by 1 sat image GeoTiff file 5G in size, zoom and pan took 2s, by changing the Tile Caching setting, untick the jpeg and png, and tick the png8 I can cut the zoom/pan time down to 1s . Data transfer rate drop from 12MB to 2MB. And I cannot see any degrade in image quality (not with my eyes anyway)

The response time of the main server (proper Dell server) and the portable server (Celeron SMB NAS) is now roughly on par

Client to Main server via VPN route over public network

Client to portable server via 1G ethernet LAN

Cheers

Sent from Mail for Windows 10

From: Tripple Tee
Sent: Friday, 10 August 2018 8:10 AM
To: Rahkonen Jukka (MML); geoserver-users@lists.sourceforge.net
Subject: RE: [Geoserver-users] CPU drain

Thanks Jukka,

Thanks for the link, I do think I have it wrong somewhere.

The shape files I am using are indeed from NaturalEarth.

https://www.naturalearthdata.com/http//www.naturalearthdata.com/download/10m/physical/ne_10m_land.zip

https://www.naturalearthdata.com/http//www.naturalearthdata.com/download/10m/physical/ne_10m_minor_islands.zip

or

https://www.naturalearthdata.com/http//www.naturalearthdata.com/download/10m/physical/ne_10m_ocean.zip

Not sure if my Bounding box is correctly published. I press “Compute from data”

Native and Lat/Long bounding box are same

MinX: -179.9999 MinY: -89.9999 MaxX: 180 MaxY: 83.6341

This is what layer preview show

cid:image002.png@anonymised.com

Look like the earth was render twice

Other setting while creating new layer is by default, the Coordinate Refernece System: EPSG:4326

Thanks

Sent from Mail for Windows 10

From: Rahkonen Jukka (MML)
Sent: Thursday, 9 August 2018 5:00 PM
To: Tripple Tee; geoserver-users@lists.sourceforge.net
Subject: Re: [Geoserver-users] CPU drain

Hi,

Before you start optimizing anything you should evaluate if the performance that you have is normal. For example in this 11 years old benchmark https://www.idee.es/resources/presentaciones/JIDEE07/POWERPOINT_JIDEE2007/PowerPoint.7-Mapserver_vs._Geoserver.pdf which was made with a computer that is today classified as a toy

(Dual core (1.8Ghz per core). 2GB RAM. 7200RPM disk. Linux. PostgreSQL 8.2.4. PostGIS 1.2) shows much better performance.

Shapefile of size 6 MB is really very small dataset. If it takes a second to render it with a powerful computer there must be something special in the data or in your installation. I am sure that there are people on this list who get better results with Raspberry Pi.

Can you share your 6 MB dataset? Or could you repeat your tests with some public data for example from http://www.naturalearthdata.com/downloads/?

-Jukka Rahkonen-

Lähettäjä: Tripple Tee [mailto:trippletee96@anonymised.com]
Lähetetty: 9. elokuuta 2018 7:51
Vastaanottaja: geoserver-users@anonymised.comt
Aihe: Re: [Geoserver-users] CPU drain

Shape file is not doing well even on a powerful CPU. I have the same set of data on a proper server with Xeon chip (Virtual Windows 2016 with 10 vCPU) the performance is a mark improvement compare with the poor Celeron. The CPU usage jump to 15% every time I zoom in or pan, but it still take a second to render.

I did some googling on Java and multi core/thread CPU. Seem like Java and most software can not take full advantage of the multi core/thread provided by hardware.

Thanks Brad for the info about PostGIS, I might have to stuck with shape file and it lean, my design goal is portable:

1 geoserver on main server, when a team going away, all user need to do is to robocopy the whole folder from the main server to the portable server. Lucky GeoServer was written in Java so I can achieve this while using different OS (main server is Windows 2016, portable server is Linux)

Might be able to do this with PostGIS but I will have to copy the database from Windows to Linux, but it will not be a simple process for user.

Sent from Mail for Windows 10

From: bradh@anonymised.com
Sent: Thursday, 9 August 2018 11:43 AM
To: ‘Tripple Tee’; geoserver-users@anonymised.comourceforge.net
Subject: RE: [Geoserver-users] CPU drain

There are a bunch of tips at http://docs.geoserver.org/latest/en/user/production/index.html and you need to benchmark (e.g. with jmeter) against your expected workload when performance tuning.

In general, shapefile is not going to be very high performance, as noted at http://docs.geoserver.org/latest/en/user/production/data.html#use-a-spatial-database

Brad

From: Tripple Tee <trippletee96@anonymised.com>
Sent: Thursday, 9 August 2018 11:16 AM
To: geoserver-users@lists.sourceforge.net
Subject: [Geoserver-users] CPU drain

Hi,

Please give some pointer on optimising for better resource consumption

I am running GeoServer on Linux with Intel Celeron 4-core processor, which I know is slow

When I open an individual layer (or layer group) of a region 200km each way (with details like roads, buildings …) the shape files size is under 0.5MB it loads quite comfortably, effortless to zoom and pan.

When I open a layer (or layer group) of the whole world with less detail (only land and water lines) it seem to struggle. The size of the shape file is around 5-7MB, I can see the CPU spike and it takes seconds to render as I zoom in or pan the map. Note: I am preview this layer on the default 750x400 resolution box. Should GeoServer only query data of the sub region for that preview box ?

1 - Is this the expected performance ?

2 – How can I optimize it to stop the CPU spike and lag,

Thanks

Sent from Mail for Windows 10

EC39884172924B7EBEB04896EB8B052D.png

4

Yes, the ocean shape file is one I use when I teach performance optimization.

option a) easy draw your map with background color the same as you would your ocean

option b) make a whole world rectangle and draw it behind your other content

option c) process the ocean - simplify for when you are zoomed out - and then process into smaller polygons for when you are zoomed in

In this case process the data into smaller multi polygons

···

–
Jody Garnett

5

OSM provides a split polygon file here, in case you’re interested, to be used when zoomed in:
http://openstreetmapdata.com/data/water-polygons

Cheers
Andrea

···

Regards, Andrea Aime == GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.