[Geoserver-devel] [veille] geoserver vs. mapserver : round 3

GeoServer GeoServer Developer
1

Hi,

Thomas Bonfort, an employee here at camptocamp just ran the Geoserver/Mapserver tests presented at Foss4G 2008 against the mapserver that has been realized after the code sprint in Toronto. I thought you might be interested in the results:

http://docs.google.com/Doc?id=dgm385hv_125dbj8hc2s&hl=en

The biggest changes were:
* Added a cache for proj4
* Some shapefile access changes (doesn't specify what other than they resulted in better performance)

I don't know if the mapserver is public yet I will ask.

Jesse

2

Jesse Eichar ha scritto:

Hi,

Thomas Bonfort, an employee here at camptocamp just ran the Geoserver/ Mapserver tests presented at Foss4G 2008 against the mapserver that has been realized after the code sprint in Toronto. I thought you might be interested in the results:

http://docs.google.com/Doc?id=dgm385hv_125dbj8hc2s&hl=en

The biggest changes were:
* Added a cache for proj4
* Some shapefile access changes (doesn't specify what other than they resulted in better performance)

I don't know if the mapserver is public yet I will ask.

I had a mail exchange with Thomas and I tried out GeoServer in
a similarly spec-ed machine getting wildly different results,
thought I actually found a slowdown which I since then fixed.

Sent him a mail detailing my results, but never got a response.
Here is the relevant part of the mail I sent him (nothing personal,
and since we're removing curtains from the results, better do
everything in public):

---------------------------------------------------------------

...

On my desktop I run some profile sessions and found a single
but fatal performance screwup (http://jira.codehaus.org/browse/GEOT-2401).

Then I installed GS 1.7.3 on my Ubuntu 64bit notebook
(the one I used for the FOSS4G 2008 benchmarks):
T8100 2.1Ghz core 2 duo, 4GB memory,
using JDK 1.6.0_12, JAI and JAI Image I/O, both installed
using the installers (you have to export a variable in
order to have the jai-imageio installer work, see here:
http://markmail.org/message/xbeo6nfqlrg4dmu4).

The "out of the box" behaviour of GeoServer places it
at 61 req/s, without me doing anything (JMeter running
from my desktop, connected with a small 100Mbit switch,
an ADSL modem with an integrated 4 port switch actually).
Even taking into consideration the 8100 is newer than the
6400, this is a long shot from your results.
Wondering if your jai imageio is properly installed?

I then run the tests with the fixed shapefile reader
and got up to 90req/s.

...

--------------------------------------------------------------

Soo... there is something amiss in those graphs...
unless someone can tell me going from his e6400 to my t8100
CPU actually doubles the CPU power at same operating frequency?
I am dubious the t8100 is really that much faster... but
maybe it is?

Cheers
Andrea

--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.

3

I have recently spoken with Thomas and he told me that he found that he had installed JAI incorrectly so he wants to redo the tests. I look forward to seeing them.

Most of the results I have seen have been from you guys so I want the mapserver guys to get into the party too. I expect that process could help us find some geoserver bugs that we can fix.

I will ask him to post to this list as well when he gets his new results. He's busy though so it could take him a little while.

Jesse

Andrea Aime wrote:

Jesse Eichar ha scritto:

Hi,

Thomas Bonfort, an employee here at camptocamp just ran the Geoserver/ Mapserver tests presented at Foss4G 2008 against the mapserver that has been realized after the code sprint in Toronto. I thought you might be interested in the results:

http://docs.google.com/Doc?id=dgm385hv_125dbj8hc2s&hl=en

The biggest changes were:
* Added a cache for proj4
* Some shapefile access changes (doesn't specify what other than they resulted in better performance)

I don't know if the mapserver is public yet I will ask.

I had a mail exchange with Thomas and I tried out GeoServer in
a similarly spec-ed machine getting wildly different results,
thought I actually found a slowdown which I since then fixed.

Sent him a mail detailing my results, but never got a response.
Here is the relevant part of the mail I sent him (nothing personal,
and since we're removing curtains from the results, better do
everything in public):

---------------------------------------------------------------

...

On my desktop I run some profile sessions and found a single
but fatal performance screwup (http://jira.codehaus.org/browse/GEOT-2401).

Then I installed GS 1.7.3 on my Ubuntu 64bit notebook
(the one I used for the FOSS4G 2008 benchmarks):
T8100 2.1Ghz core 2 duo, 4GB memory,
using JDK 1.6.0_12, JAI and JAI Image I/O, both installed
using the installers (you have to export a variable in
order to have the jai-imageio installer work, see here:
http://markmail.org/message/xbeo6nfqlrg4dmu4).

The "out of the box" behaviour of GeoServer places it
at 61 req/s, without me doing anything (JMeter running
from my desktop, connected with a small 100Mbit switch,
an ADSL modem with an integrated 4 port switch actually).
Even taking into consideration the 8100 is newer than the
6400, this is a long shot from your results.
Wondering if your jai imageio is properly installed?

I then run the tests with the fixed shapefile reader
and got up to 90req/s.

...

--------------------------------------------------------------

Soo... there is something amiss in those graphs...
unless someone can tell me going from his e6400 to my t8100
CPU actually doubles the CPU power at same operating frequency?
I am dubious the t8100 is really that much faster... but
maybe it is?

Cheers
Andrea

4

Jesse Eichar ha scritto:

I have recently spoken with Thomas and he told me that he found that he had installed JAI incorrectly so he wants to redo the tests. I look forward to seeing them.

Most of the results I have seen have been from you guys so I want the mapserver guys to get into the party too. I expect that process could help us find some geoserver bugs that we can fix.

Indeed, this year I would like to redo the performance comparison
presentation again involving the MapServer people (and maybe MapGuide,
DeeGree, Mapnik too?), and having tests running in some neutral place
(maybe the conference can give us a server to test onto?).

I know performance wise MapServer will eventually have the upper
hand, it's just a matter of giving the MS devs enough incentive,
but the charts published before seem flawed to me. Plus,
it's comparing the latest and the greatest of MS with a released
version of GeoServer. Let's use GS nightly builds to compare apples to
apples :wink:
http://gridlock.openplans.org/geoserver/1.7.x/

I will ask him to post to this list as well when he gets his new results. He's busy though so it could take him a little while.

Yeah, I also want to run the current MapServer on the same box
I cited before, but I have to download and recompile and install
everything from sources, and... oh well, I've been lazy :slight_smile:
I'll try to amend that next weekend.

Cheers
Andrea

--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.

5

hi,

those results weren't supposed to go public for the very reason that I was
suspecting something fishy was going on with my jai installation.

I'll run them again some time with the latest geoserver build so we can
compare apples to apples. From some quick preliminary tests, you guys still
have a clear lead with the postgis data sources.

I'd be interested to participate if Andrea reruns the benchmarks for a foss
paper. Until then, here are a few suggestions:

* have geoserver print a warning message if jai-imageio isn't installed
correctly (there's one for plain jai)
* change the number of total requests, especially for the single-threaded
tests. 10 requests as done actually has a *very* low SNR.
* Mapserver does round caps by default, whereas I think GeoServer does butt
ones. Don't know if this is configurable in geoserver, it is for mapserver.

best regards,

thomas

Jesse Eichar-4 wrote:

Hi,

Thomas Bonfort, an employee here at camptocamp just ran the Geoserver/
Mapserver tests presented at Foss4G 2008 against the mapserver that
has been realized after the code sprint in Toronto. I thought you
might be interested in the results:

http://docs.google.com/Doc?id=dgm385hv_125dbj8hc2s&hl=en

The biggest changes were:
* Added a cache for proj4
* Some shapefile access changes (doesn't specify what other than they
resulted in better performance)

I don't know if the mapserver is public yet I will ask.

Jesse

------------------------------------------------------------------------------
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

--
View this message in context: http://www.nabble.com/Fwd%3A--veille--geoserver-vs.-mapserver-%3A-round-3-tp22783965p23033763.html
Sent from the GeoServer - Dev mailing list archive at Nabble.com.

6

tbonfort ha scritto:

hi,

those results weren't supposed to go public for the very reason that I was
suspecting something fishy was going on with my jai installation.

I'll run them again some time with the latest geoserver build so we can
compare apples to apples. From some quick preliminary tests, you guys still
have a clear lead with the postgis data sources.

I managed to get a nice speedup on shapefiles as well since the 1.7.3 release.
Did you check with a recent nightly build? Here:
http://gridlock.openplans.org/geoserver/1.7.x/

I'd be interested to participate if Andrea reruns the benchmarks for a foss
paper.

Yep, looking forward to this. What about we mention this on
osgeo-discuss and see if anyone from MapGuide, DeeGree or Mapnik wants
to participate as well?

Until then, here are a few suggestions:

* have geoserver print a warning message if jai-imageio isn't installed
correctly (there's one for plain jai)

That seems like a good idea.

* change the number of total requests, especially for the single-threaded
tests. 10 requests as done actually has a *very* low SNR.

Agreed, thought I would pump up only the numbers in the low threads count, maybe in a way that each test does at least 100 requests (total).

* Mapserver does round caps by default, whereas I think GeoServer does butt
ones. Don't know if this is configurable in geoserver, it is for mapserver.

We don't have a global defaults toggle, but it's an option in SLD that
can can tune so that the two servers are doing the same job.

Cheers
Andrea