[Geoserver-users] Poor Throughput

GeoServer GeoServer User
1

I've been experimenting with GeoServer 1.7.4 as a possible alternative to an expensive, proprietary product.

I've set up a WMS service with 4 polygon layers inside it, nothing particularly complicated and simple rendering. Data is served from (DataStore) ArcSDE, with the feature classes in the web mercator projection. I making requests for EPSG:900913. I'm trying to get the source projection and requested projection to be the same to avoid reprojecting on-the-fly, but I'm not sure that I'm succeeding at that.

I'm using Tomcat 6.0 and JDK 1.6 on a four-processor, 2.5 GHz Windows Server 2003 box with 4 GB RAM.

Single, isolated WMS requests to this service have a response time of about 2.5 seconds. That's a little slow, but our real concern is throughput - being able to handle a high load of simultaneous requests.

To measure that, I have a home-grown multi-threaded application that can throw multiple requests at a map service, at whatever rate I want, and report the response times.

When I throw a new request at my WMS service on GeoServer every 3000 milliseconds for 30 seconds, the average response time is 2.5 seconds.

When I throw a new request every 2400 ms for 30 seconds, the average response time is 4.6 seconds.

When I throw a new request every 1000 ms for 30 seconds, the average response time is 31.9 seconds. The response rate does not "ramp up" slowly... it pretty quickly gets to responses in the 30 second range, and stays up there.

Clearly it's not scaling well. Even when hitting it "hard" like in the above test, the CPUs on the server are showing very low utilization (can hardly tell there's anything special going on).

I've tried many of the suggestions in http://geoserver.org/display/GEOSDOC/2.6+GeoServer+in+Production+Environment but haven't seen any significant benefit.

For comparison, an ArcGIS Server map service on the same machine with the same layers using the same ArcSDE data sources (plus some labels and a hill-shade raster layer that the similar GeoServer WMS service doesn't have) gives an average response time of 1.1 seconds when I throw a new request at it every 1000 ms for 30 seconds.

When I throw a new request at ArcGIS Server every 500 ms for 30 seconds, the average response time is 1.2 seconds.

And I know there's some throughput tuning on ArcGIS Server that will improve throughput that I have not yet done.

So, any suggestions for getting better throughput and scaling out of GeoServer?

David

2

Hi David,
I had Geoserver pointing at ArcIMS for WMS requests and the response
time was a little slow, throughput was also lacking. But I traced most
of it down to ArcIMS and the ims plugin for geoserver. We switched to
shapefiles and performance greatly increased. 50-60 users requesting
over 15 layers, thousands of features per request, with complex
styling, panning/zooming constantly and the response time was a
consistent 1.5-2.5 seconds with no sign of slowing down. And no
complaints from the users at all. (this was geoserver 1.6)
I don't have any hard stats for you, but I do know from a fair amount
of experience with geoserver that shapefile and postgis are a good way
to go, and ArcIMS has caused me nothing but grief.
As for scalability, depending on how intense the usage is going to be,
you can load balance geoserver, or better yet, throw tiling/caching in
front of it.

cheers,
Brent

On Fri, May 29, 2009 at 7:39 AM, David Rush <david@anonymised.com> wrote:

I've been experimenting with GeoServer 1.7.4 as a possible alternative to an expensive, proprietary product.

I've set up a WMS service with 4 polygon layers inside it, nothing particularly complicated and simple rendering. Data is served from (DataStore) ArcSDE, with the feature classes in the web mercator projection. I making requests for EPSG:900913. I'm trying to get the source projection and requested projection to be the same to avoid reprojecting on-the-fly, but I'm not sure that I'm succeeding at that.

I'm using Tomcat 6.0 and JDK 1.6 on a four-processor, 2.5 GHz Windows Server 2003 box with 4 GB RAM.

Single, isolated WMS requests to this service have a response time of about 2.5 seconds. That's a little slow, but our real concern is throughput - being able to handle a high load of simultaneous requests.

To measure that, I have a home-grown multi-threaded application that can throw multiple requests at a map service, at whatever rate I want, and report the response times.

When I throw a new request at my WMS service on GeoServer every 3000 milliseconds for 30 seconds, the average response time is 2.5 seconds.

When I throw a new request every 2400 ms for 30 seconds, the average response time is 4.6 seconds.

When I throw a new request every 1000 ms for 30 seconds, the average response time is 31.9 seconds. The response rate does not "ramp up" slowly... it pretty quickly gets to responses in the 30 second range, and stays up there.

Clearly it's not scaling well. Even when hitting it "hard" like in the above test, the CPUs on the server are showing very low utilization (can hardly tell there's anything special going on).

I've tried many of the suggestions in http://geoserver.org/display/GEOSDOC/2.6+GeoServer+in+Production+Environment but haven't seen any significant benefit.

For comparison, an ArcGIS Server map service on the same machine with the same layers using the same ArcSDE data sources (plus some labels and a hill-shade raster layer that the similar GeoServer WMS service doesn't have) gives an average response time of 1.1 seconds when I throw a new request at it every 1000 ms for 30 seconds.

When I throw a new request at ArcGIS Server every 500 ms for 30 seconds, the average response time is 1.2 seconds.

And I know there's some throughput tuning on ArcGIS Server that will improve throughput that I have not yet done.

So, any suggestions for getting better throughput and scaling out of GeoServer?

David

------------------------------------------------------------------------------
Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
is a gathering of tech-side developers & brand creativity professionals. Meet
the minds behind Google Creative Lab, Visual Complexity, Processing, &
iPhoneDevCamp as they present alongside digital heavyweights like Barbarian
Group, R/GA, & Big Spaceship. http://p.sf.net/sfu/creativitycat-com
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
Brent Owens

3

Brent Owens ha scritto:

Hi David,
I had Geoserver pointing at ArcIMS for WMS requests and the response
time was a little slow, throughput was also lacking. But I traced most
of it down to ArcIMS and the ims plugin for geoserver. We switched to
shapefiles and performance greatly increased. 50-60 users requesting
over 15 layers, thousands of features per request, with complex
styling, panning/zooming constantly and the response time was a
consistent 1.5-2.5 seconds with no sign of slowing down. And no
complaints from the users at all. (this was geoserver 1.6)
I don't have any hard stats for you, but I do know from a fair amount
of experience with geoserver that shapefile and postgis are a good way
to go, and ArcIMS has caused me nothing but grief.
As for scalability, depending on how intense the usage is going to be,
you can load balance geoserver, or better yet, throw tiling/caching in
front of it.

Brent, when you say IMS, do you really mean SDE?

Afaik Gabriel has improved quite a bit the SDE connector in the GeoServer 1.7.x series and there is people using it in production,
but I have no idea of how they set it up and how that can
influence performance.

Cheers
Andrea

--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.

4

Yea sorry, I mean ArcSDE.
I will have to try 1.7 with SDE and see how it works.

cheers,
-Brent

On Fri, May 29, 2009 at 8:28 AM, Andrea Aime <aaime@anonymised.com> wrote:

Brent Owens ha scritto:

Hi David,
I had Geoserver pointing at ArcIMS for WMS requests and the response
time was a little slow, throughput was also lacking. But I traced most
of it down to ArcIMS and the ims plugin for geoserver. We switched to
shapefiles and performance greatly increased. 50-60 users requesting
over 15 layers, thousands of features per request, with complex
styling, panning/zooming constantly and the response time was a
consistent 1.5-2.5 seconds with no sign of slowing down. And no
complaints from the users at all. (this was geoserver 1.6)
I don't have any hard stats for you, but I do know from a fair amount
of experience with geoserver that shapefile and postgis are a good way
to go, and ArcIMS has caused me nothing but grief.
As for scalability, depending on how intense the usage is going to be,
you can load balance geoserver, or better yet, throw tiling/caching in
front of it.

Brent, when you say IMS, do you really mean SDE?

Afaik Gabriel has improved quite a bit the SDE connector in the GeoServer
1.7.x series and there is people using it in production,
but I have no idea of how they set it up and how that can
influence performance.

Cheers
Andrea

--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.

--
Brent Owens

5

Hi David,

sad to hear that. Yet, as you know perf tunning is a complicated matter. The following are the factors I can think off the top of my head that you may need to double check:
- Did you assess the impact of the number of available ArcSDE connections to GeoServer? (you can change the pool.maxConnections parameter on the DataStore's edit page)
- Do the layers have proper spatial indexes? do they have indexes on the non spatial attributes used by your SLD, if any?
- You said ArcGIS Server is performing well so the question of if you compared with other products does not apply. But my main suspect here is the number of connections. Did you check how many connections does ArcGIS Server use to serve your requests, and how they relate to the ones GeoServer is allowed to use?

If you wish I would be glad of setting up an environment similar to yours (can you share an sdeexport of your 4 polygon layers with me?) and do some profiling. I'm very interested on helping you with this cause my latest perf chek on the plugin was getting better results than the postgis one, but that may have changed over my feet.

Ah... btw, I just fixed a scalability issue that's surely related to this. Check this out: http://jira.codehaus.org/browse/GEOT-1956
The fix is for geotools 2.5.6 and GeoServer 1.7.5, so you may want to try out the latest nightly with the fix applied:

<http://gridlock.openplans.org/geoserver/1.7.x/geoserver-1.7.x-latest-war.zip&gt;
and
<http://gridlock.openplans.org/geoserver/1.7.x/ext-latest/geoserver-1.7.5-SNAPSHOT-arcsde-plugin.zip&gt;

Let me know how it goes.

Cheers,
Gabriel

David Rush wrote:

I've been experimenting with GeoServer 1.7.4 as a possible alternative to an expensive, proprietary product.

I've set up a WMS service with 4 polygon layers inside it, nothing particularly complicated and simple rendering. Data is served from (DataStore) ArcSDE, with the feature classes in the web mercator projection. I making requests for EPSG:900913. I'm trying to get the source projection and requested projection to be the same to avoid reprojecting on-the-fly, but I'm not sure that I'm succeeding at that.

I'm using Tomcat 6.0 and JDK 1.6 on a four-processor, 2.5 GHz Windows Server 2003 box with 4 GB RAM.

Single, isolated WMS requests to this service have a response time of about 2.5 seconds. That's a little slow, but our real concern is throughput - being able to handle a high load of simultaneous requests.

To measure that, I have a home-grown multi-threaded application that can throw multiple requests at a map service, at whatever rate I want, and report the response times.

When I throw a new request at my WMS service on GeoServer every 3000 milliseconds for 30 seconds, the average response time is 2.5 seconds.

When I throw a new request every 2400 ms for 30 seconds, the average response time is 4.6 seconds.

When I throw a new request every 1000 ms for 30 seconds, the average response time is 31.9 seconds. The response rate does not "ramp up" slowly... it pretty quickly gets to responses in the 30 second range, and stays up there.

Clearly it's not scaling well. Even when hitting it "hard" like in the above test, the CPUs on the server are showing very low utilization (can hardly tell there's anything special going on).

I've tried many of the suggestions in http://geoserver.org/display/GEOSDOC/2.6+GeoServer+in+Production+Environment but haven't seen any significant benefit.

For comparison, an ArcGIS Server map service on the same machine with the same layers using the same ArcSDE data sources (plus some labels and a hill-shade raster layer that the similar GeoServer WMS service doesn't have) gives an average response time of 1.1 seconds when I throw a new request at it every 1000 ms for 30 seconds.

When I throw a new request at ArcGIS Server every 500 ms for 30 seconds, the average response time is 1.2 seconds.

And I know there's some throughput tuning on ArcGIS Server that will improve throughput that I have not yet done.

So, any suggestions for getting better throughput and scaling out of GeoServer?

David

------------------------------------------------------------------------------
Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT is a gathering of tech-side developers & brand creativity professionals. Meet
the minds behind Google Creative Lab, Visual Complexity, Processing, & iPhoneDevCamp as they present alongside digital heavyweights like Barbarian Group, R/GA, & Big Spaceship. http://p.sf.net/sfu/creativitycat-com _______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
Gabriel Roldan
OpenGeo - http://opengeo.org
Expert service straight from the developers.

6

Thanks for your comment. I was hoping that might be the solution to my problem but unfortunately it wasn’t.
The"type" column of the “geometry_columns” table was set to “GEOMETRY”.

So, everything appears to be correct, but yet I only get polygons. An easy test is to set up a table of type “GEOMETRY” and inert a point, line, and polygon, and then have geoserver hit it.

Do you have a running example of this that works? Maybe it’s something to do with my installation? Geoserver version? We’re using 1.7.4

Thanks for your help…

Alexei

From: A. Soroka [mailto:ajs6f@anonymised.com]
Sent: Fri 5/26/2009 7:39 AM
To: geoserver-users@lists.sourceforge.net
Subject: [Geoserver-users] Using geometry (not point, line,poly) data type in PostGIS causes geoserver to output all features aspolygon

On Tue, 26 May 2009 14:35:36 -0400, ajs6f wrote:

Have you put the “GEOMETRY” keyword in your ‘geometry_columns’ table
in the appropriate row/column?

A. Soroka
Digital Research and Scholarship R & D
the University of Virginia Library

On May 26, 2009, at 2:25 PM, Alexei Peters wrote:

Hello everyone,
I have a quick question about geoserver and the geometry data type
in PostGIS. We’re using the geometry data type in PostGIS, in which
you can store points, lines and polygons in one table. Geoserver
seems to have no issues connecting to this table but seems to
convert all geometries to type polygon for WFS output.
Has anyone else experienced this and is there a way to get the
geometry to output as its actual geometry type?

Thanks for any help with this…

Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
is a gathering of tech-side developers & brand creativity
professionals. Meet
the minds behind Google Creative Lab, Visual Complexity, Processing, &
iPhoneDevCamp as they present alongside digital heavyweights like
Barbarian
Group, R/GA, & Big Spaceship. http://p.sf.net/sfu/creativitycat-com

Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

7

Gabriel:

I tried replacing the gt-arcsde-2.5.5.jar with gt-arcsde-2.5-SNAPSHOT.jar from your link.

Wow.

Isolated requests are now responding in about 0.26 seconds, a nearly ten-fold improvement.

Sending a new request every 1000 ms for 30 seconds got an average response time of 0.245 sec.
500 ms for 30 sec average 0.233 sec.
200 ms for 30 sec avg 0.322 sec.
100 ms for 30 sec avg 1.987 sec.
50 ms for 30 sec avg 2.170 sec.

I tried a 20 ms rate, but my tool cratered at that rate.

The faster rates also started showing significant CPU load (40%-ish) on the geoserver machine, which is a good sign.

At present I have the min/max poolConnnections at 10/10. Earlier I had it at the default of 2/10, and didn't notice a big diff when I switched it to 10/10.

Thanks! This is looking a lot more promising.

David

Gabriel Roldan wrote:

Hi David,

sad to hear that. Yet, as you know perf tunning is a complicated matter.
The following are the factors I can think off the top of my head that
you may need to double check:
- Did you assess the impact of the number of available ArcSDE
connections to GeoServer? (you can change the pool.maxConnections
parameter on the DataStore's edit page)
- Do the layers have proper spatial indexes? do they have indexes on the
non spatial attributes used by your SLD, if any?
- You said ArcGIS Server is performing well so the question of if you
compared with other products does not apply. But my main suspect here is
the number of connections. Did you check how many connections does
ArcGIS Server use to serve your requests, and how they relate to the
ones GeoServer is allowed to use?

If you wish I would be glad of setting up an environment similar to
yours (can you share an sdeexport of your 4 polygon layers with me?) and
do some profiling. I'm very interested on helping you with this cause my
latest perf chek on the plugin was getting better results than the
postgis one, but that may have changed over my feet.

Ah... btw, I just fixed a scalability issue that's surely related to
this. Check this out: http://jira.codehaus.org/browse/GEOT-1956
The fix is for geotools 2.5.6 and GeoServer 1.7.5, so you may want to
try out the latest nightly with the fix applied:

<http://gridlock.openplans.org/geoserver/1.7.x/geoserver-1.7.x-latest-war.zip&gt;

and
<http://gridlock.openplans.org/geoserver/1.7.x/ext-latest/geoserver-1.7.5-SNAPSHOT-arcsde-plugin.zip&gt;

Let me know how it goes.

Cheers,
Gabriel

David Rush wrote:

I've been experimenting with GeoServer 1.7.4 as a possible alternative
to an expensive, proprietary product.

I've set up a WMS service with 4 polygon layers inside it, nothing
particularly complicated and simple rendering. Data is served from
(DataStore) ArcSDE, with the feature classes in the web mercator
projection. I making requests for EPSG:900913. I'm trying to get the
source projection and requested projection to be the same to avoid
reprojecting on-the-fly, but I'm not sure that I'm succeeding at that.

I'm using Tomcat 6.0 and JDK 1.6 on a four-processor, 2.5 GHz Windows
Server 2003 box with 4 GB RAM.

Single, isolated WMS requests to this service have a response time of
about 2.5 seconds. That's a little slow, but our real concern is
throughput - being able to handle a high load of simultaneous requests.

To measure that, I have a home-grown multi-threaded application that
can throw multiple requests at a map service, at whatever rate I want,
and report the response times.

When I throw a new request at my WMS service on GeoServer every 3000
milliseconds for 30 seconds, the average response time is 2.5 seconds.

When I throw a new request every 2400 ms for 30 seconds, the average
response time is 4.6 seconds.

When I throw a new request every 1000 ms for 30 seconds, the average
response time is 31.9 seconds. The response rate does not "ramp up"
slowly... it pretty quickly gets to responses in the 30 second range,
and stays up there.

Clearly it's not scaling well. Even when hitting it "hard" like in
the above test, the CPUs on the server are showing very low
utilization (can hardly tell there's anything special going on).

I've tried many of the suggestions in
http://geoserver.org/display/GEOSDOC/2.6+GeoServer+in+Production+Environment
but haven't seen any significant benefit.

For comparison, an ArcGIS Server map service on the same machine with
the same layers using the same ArcSDE data sources (plus some labels
and a hill-shade raster layer that the similar GeoServer WMS service
doesn't have) gives an average response time of 1.1 seconds when I
throw a new request at it every 1000 ms for 30 seconds.

When I throw a new request at ArcGIS Server every 500 ms for 30
seconds, the average response time is 1.2 seconds.

And I know there's some throughput tuning on ArcGIS Server that will
improve throughput that I have not yet done.

So, any suggestions for getting better throughput and scaling out of
GeoServer?

David

------------------------------------------------------------------------------

Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
is a gathering of tech-side developers & brand creativity
professionals. Meet
the minds behind Google Creative Lab, Visual Complexity, Processing, &
iPhoneDevCamp as they present alongside digital heavyweights like
Barbarian Group, R/GA, & Big Spaceship.
http://p.sf.net/sfu/creativitycat-com
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

8

Phew! :slight_smile:
Glad to hear that. Just for reference, the offending code was 1) the connection validation code when one is requested out of the pool. It was making a full roundtrip to the server to get the user name. Replaced by SeSonnection.testServer(). And 2) the list of SeLayer objects being eagerly cached on a per connection basis. Now they are cached on an as-needed basis and the list of available layers is updated by a background thread every 30 seconds to catch up with layer deletions/additions. Which makes me love what a good profiler may give you.

Cheers,

Gabriel

David Rush wrote:

Gabriel:

I tried replacing the gt-arcsde-2.5.5.jar with gt-arcsde-2.5-SNAPSHOT.jar from your link.

Wow.

Isolated requests are now responding in about 0.26 seconds, a nearly ten-fold improvement.

Sending a new request every 1000 ms for 30 seconds got an average response time of 0.245 sec.
500 ms for 30 sec average 0.233 sec.
200 ms for 30 sec avg 0.322 sec.
100 ms for 30 sec avg 1.987 sec.
50 ms for 30 sec avg 2.170 sec.

I tried a 20 ms rate, but my tool cratered at that rate.

The faster rates also started showing significant CPU load (40%-ish) on the geoserver machine, which is a good sign.

At present I have the min/max poolConnnections at 10/10. Earlier I had it at the default of 2/10, and didn't notice a big diff when I switched it to 10/10.

Thanks! This is looking a lot more promising.

David

Gabriel Roldan wrote:

Hi David,

sad to hear that. Yet, as you know perf tunning is a complicated matter.
The following are the factors I can think off the top of my head that
you may need to double check:
- Did you assess the impact of the number of available ArcSDE
connections to GeoServer? (you can change the pool.maxConnections
parameter on the DataStore's edit page)
- Do the layers have proper spatial indexes? do they have indexes on the
non spatial attributes used by your SLD, if any?
- You said ArcGIS Server is performing well so the question of if you
compared with other products does not apply. But my main suspect here is
the number of connections. Did you check how many connections does
ArcGIS Server use to serve your requests, and how they relate to the
ones GeoServer is allowed to use?

If you wish I would be glad of setting up an environment similar to
yours (can you share an sdeexport of your 4 polygon layers with me?) and
do some profiling. I'm very interested on helping you with this cause my
latest perf chek on the plugin was getting better results than the
postgis one, but that may have changed over my feet.

Ah... btw, I just fixed a scalability issue that's surely related to
this. Check this out: http://jira.codehaus.org/browse/GEOT-1956
The fix is for geotools 2.5.6 and GeoServer 1.7.5, so you may want to
try out the latest nightly with the fix applied:

<http://gridlock.openplans.org/geoserver/1.7.x/geoserver-1.7.x-latest-war.zip&gt;

and
<http://gridlock.openplans.org/geoserver/1.7.x/ext-latest/geoserver-1.7.5-SNAPSHOT-arcsde-plugin.zip&gt;

Let me know how it goes.

Cheers,
Gabriel

David Rush wrote:

I've been experimenting with GeoServer 1.7.4 as a possible alternative
to an expensive, proprietary product.

I've set up a WMS service with 4 polygon layers inside it, nothing
particularly complicated and simple rendering. Data is served from
(DataStore) ArcSDE, with the feature classes in the web mercator
projection. I making requests for EPSG:900913. I'm trying to get the
source projection and requested projection to be the same to avoid
reprojecting on-the-fly, but I'm not sure that I'm succeeding at that.

I'm using Tomcat 6.0 and JDK 1.6 on a four-processor, 2.5 GHz Windows
Server 2003 box with 4 GB RAM.

Single, isolated WMS requests to this service have a response time of
about 2.5 seconds. That's a little slow, but our real concern is
throughput - being able to handle a high load of simultaneous requests.

To measure that, I have a home-grown multi-threaded application that
can throw multiple requests at a map service, at whatever rate I want,
and report the response times.

When I throw a new request at my WMS service on GeoServer every 3000
milliseconds for 30 seconds, the average response time is 2.5 seconds.

When I throw a new request every 2400 ms for 30 seconds, the average
response time is 4.6 seconds.

When I throw a new request every 1000 ms for 30 seconds, the average
response time is 31.9 seconds. The response rate does not "ramp up"
slowly... it pretty quickly gets to responses in the 30 second range,
and stays up there.

Clearly it's not scaling well. Even when hitting it "hard" like in
the above test, the CPUs on the server are showing very low
utilization (can hardly tell there's anything special going on).

I've tried many of the suggestions in
http://geoserver.org/display/GEOSDOC/2.6+GeoServer+in+Production+Environment
but haven't seen any significant benefit.

For comparison, an ArcGIS Server map service on the same machine with
the same layers using the same ArcSDE data sources (plus some labels
and a hill-shade raster layer that the similar GeoServer WMS service
doesn't have) gives an average response time of 1.1 seconds when I
throw a new request at it every 1000 ms for 30 seconds.

When I throw a new request at ArcGIS Server every 500 ms for 30
seconds, the average response time is 1.2 seconds.

And I know there's some throughput tuning on ArcGIS Server that will
improve throughput that I have not yet done.

So, any suggestions for getting better throughput and scaling out of
GeoServer?

David

------------------------------------------------------------------------------

Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
is a gathering of tech-side developers & brand creativity
professionals. Meet
the minds behind Google Creative Lab, Visual Complexity, Processing, &
iPhoneDevCamp as they present alongside digital heavyweights like
Barbarian Group, R/GA, & Big Spaceship.
http://p.sf.net/sfu/creativitycat-com
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

------------------------------------------------------------------------------
Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT is a gathering of tech-side developers & brand creativity professionals. Meet
the minds behind Google Creative Lab, Visual Complexity, Processing, & iPhoneDevCamp as they present alongside digital heavyweights like Barbarian Group, R/GA, & Big Spaceship. http://p.sf.net/sfu/creativitycat-com _______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
Gabriel Roldan
OpenGeo - http://opengeo.org
Expert service straight from the developers.

9

Btw, David, what's an "hill-shade raster layer"?

Is it a raster layer you have on your ArcSDE instance? if so, did you try configuring it on geoserver? With the latest development on support for ArcSDE rasters it should sort of rock too, but I'd certainly love some feedback on how it behaves under load.

Cheers,
Gabriel

David Rush wrote:

Gabriel:

I tried replacing the gt-arcsde-2.5.5.jar with gt-arcsde-2.5-SNAPSHOT.jar from your link.

Wow.

Isolated requests are now responding in about 0.26 seconds, a nearly ten-fold improvement.

Sending a new request every 1000 ms for 30 seconds got an average response time of 0.245 sec.
500 ms for 30 sec average 0.233 sec.
200 ms for 30 sec avg 0.322 sec.
100 ms for 30 sec avg 1.987 sec.
50 ms for 30 sec avg 2.170 sec.

I tried a 20 ms rate, but my tool cratered at that rate.

The faster rates also started showing significant CPU load (40%-ish) on the geoserver machine, which is a good sign.

At present I have the min/max poolConnnections at 10/10. Earlier I had it at the default of 2/10, and didn't notice a big diff when I switched it to 10/10.

Thanks! This is looking a lot more promising.

David

Gabriel Roldan wrote:

Hi David,

sad to hear that. Yet, as you know perf tunning is a complicated matter.
The following are the factors I can think off the top of my head that
you may need to double check:
- Did you assess the impact of the number of available ArcSDE
connections to GeoServer? (you can change the pool.maxConnections
parameter on the DataStore's edit page)
- Do the layers have proper spatial indexes? do they have indexes on the
non spatial attributes used by your SLD, if any?
- You said ArcGIS Server is performing well so the question of if you
compared with other products does not apply. But my main suspect here is
the number of connections. Did you check how many connections does
ArcGIS Server use to serve your requests, and how they relate to the
ones GeoServer is allowed to use?

If you wish I would be glad of setting up an environment similar to
yours (can you share an sdeexport of your 4 polygon layers with me?) and
do some profiling. I'm very interested on helping you with this cause my
latest perf chek on the plugin was getting better results than the
postgis one, but that may have changed over my feet.

Ah... btw, I just fixed a scalability issue that's surely related to
this. Check this out: http://jira.codehaus.org/browse/GEOT-1956
The fix is for geotools 2.5.6 and GeoServer 1.7.5, so you may want to
try out the latest nightly with the fix applied:

<http://gridlock.openplans.org/geoserver/1.7.x/geoserver-1.7.x-latest-war.zip&gt;

and
<http://gridlock.openplans.org/geoserver/1.7.x/ext-latest/geoserver-1.7.5-SNAPSHOT-arcsde-plugin.zip&gt;

Let me know how it goes.

Cheers,
Gabriel

David Rush wrote:

I've been experimenting with GeoServer 1.7.4 as a possible alternative
to an expensive, proprietary product.

I've set up a WMS service with 4 polygon layers inside it, nothing
particularly complicated and simple rendering. Data is served from
(DataStore) ArcSDE, with the feature classes in the web mercator
projection. I making requests for EPSG:900913. I'm trying to get the
source projection and requested projection to be the same to avoid
reprojecting on-the-fly, but I'm not sure that I'm succeeding at that.

I'm using Tomcat 6.0 and JDK 1.6 on a four-processor, 2.5 GHz Windows
Server 2003 box with 4 GB RAM.

Single, isolated WMS requests to this service have a response time of
about 2.5 seconds. That's a little slow, but our real concern is
throughput - being able to handle a high load of simultaneous requests.

To measure that, I have a home-grown multi-threaded application that
can throw multiple requests at a map service, at whatever rate I want,
and report the response times.

When I throw a new request at my WMS service on GeoServer every 3000
milliseconds for 30 seconds, the average response time is 2.5 seconds.

When I throw a new request every 2400 ms for 30 seconds, the average
response time is 4.6 seconds.

When I throw a new request every 1000 ms for 30 seconds, the average
response time is 31.9 seconds. The response rate does not "ramp up"
slowly... it pretty quickly gets to responses in the 30 second range,
and stays up there.

Clearly it's not scaling well. Even when hitting it "hard" like in
the above test, the CPUs on the server are showing very low
utilization (can hardly tell there's anything special going on).

I've tried many of the suggestions in
http://geoserver.org/display/GEOSDOC/2.6+GeoServer+in+Production+Environment
but haven't seen any significant benefit.

For comparison, an ArcGIS Server map service on the same machine with
the same layers using the same ArcSDE data sources (plus some labels
and a hill-shade raster layer that the similar GeoServer WMS service
doesn't have) gives an average response time of 1.1 seconds when I
throw a new request at it every 1000 ms for 30 seconds.

When I throw a new request at ArcGIS Server every 500 ms for 30
seconds, the average response time is 1.2 seconds.

And I know there's some throughput tuning on ArcGIS Server that will
improve throughput that I have not yet done.

So, any suggestions for getting better throughput and scaling out of
GeoServer?

David

------------------------------------------------------------------------------

Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
is a gathering of tech-side developers & brand creativity
professionals. Meet
the minds behind Google Creative Lab, Visual Complexity, Processing, &
iPhoneDevCamp as they present alongside digital heavyweights like
Barbarian Group, R/GA, & Big Spaceship.
http://p.sf.net/sfu/creativitycat-com
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

------------------------------------------------------------------------------
Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT is a gathering of tech-side developers & brand creativity professionals. Meet
the minds behind Google Creative Lab, Visual Complexity, Processing, & iPhoneDevCamp as they present alongside digital heavyweights like Barbarian Group, R/GA, & Big Spaceship. http://p.sf.net/sfu/creativitycat-com _______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
Gabriel Roldan
OpenGeo - http://opengeo.org
Expert service straight from the developers.

10

Gabriel:

A hill-shade raster layer is a map layer showing the topography of the land by shading the sides of all the hills.

I think it usually (if not always) uses a DEM (digital elevation model) to artificially generate the appearance of light coming in from one direction, so slopes facing towards the light are lighter, and slopes facing away are darker. There must be another name for it.... it's a very common technique.

Here's a link to an example: http://maps.unomaha.edu/atlas2002/pages/06-Hillshade.pdf

Hillshade is the brown background.

If anyone could give me some information on creating such a layer for use in geoserver, I'd appreciate it.

David

Gabriel Roldan wrote:

Btw, David, what's an "hill-shade raster layer"?

Is it a raster layer you have on your ArcSDE instance? if so, did you
try configuring it on geoserver? With the latest development on support
for ArcSDE rasters it should sort of rock too, but I'd certainly love
some feedback on how it behaves under load.

Cheers,
Gabriel

David Rush wrote:

Gabriel:

I tried replacing the gt-arcsde-2.5.5.jar with
gt-arcsde-2.5-SNAPSHOT.jar from your link.

Wow.

Isolated requests are now responding in about 0.26 seconds, a nearly
ten-fold improvement.

Sending a new request every 1000 ms for 30 seconds got an average
response time of 0.245 sec.
500 ms for 30 sec average 0.233 sec.
200 ms for 30 sec avg 0.322 sec.
100 ms for 30 sec avg 1.987 sec.
50 ms for 30 sec avg 2.170 sec.

I tried a 20 ms rate, but my tool cratered at that rate.

The faster rates also started showing significant CPU load (40%-ish)
on the geoserver machine, which is a good sign.

At present I have the min/max poolConnnections at 10/10. Earlier I
had it at the default of 2/10, and didn't notice a big diff when I
switched it to 10/10.

Thanks! This is looking a lot more promising.

David

Gabriel Roldan wrote:

Hi David,

sad to hear that. Yet, as you know perf tunning is a complicated matter.
The following are the factors I can think off the top of my head that
you may need to double check:
- Did you assess the impact of the number of available ArcSDE
connections to GeoServer? (you can change the pool.maxConnections
parameter on the DataStore's edit page)
- Do the layers have proper spatial indexes? do they have indexes on the
non spatial attributes used by your SLD, if any?
- You said ArcGIS Server is performing well so the question of if you
compared with other products does not apply. But my main suspect here is
the number of connections. Did you check how many connections does
ArcGIS Server use to serve your requests, and how they relate to the
ones GeoServer is allowed to use?

If you wish I would be glad of setting up an environment similar to
yours (can you share an sdeexport of your 4 polygon layers with me?) and
do some profiling. I'm very interested on helping you with this cause my
latest perf chek on the plugin was getting better results than the
postgis one, but that may have changed over my feet.

Ah... btw, I just fixed a scalability issue that's surely related to
this. Check this out: http://jira.codehaus.org/browse/GEOT-1956
The fix is for geotools 2.5.6 and GeoServer 1.7.5, so you may want to
try out the latest nightly with the fix applied:

<http://gridlock.openplans.org/geoserver/1.7.x/geoserver-1.7.x-latest-war.zip&gt;

and
<http://gridlock.openplans.org/geoserver/1.7.x/ext-latest/geoserver-1.7.5-SNAPSHOT-arcsde-plugin.zip&gt;

Let me know how it goes.

Cheers,
Gabriel

David Rush wrote:

I've been experimenting with GeoServer 1.7.4 as a possible alternative
to an expensive, proprietary product.

I've set up a WMS service with 4 polygon layers inside it, nothing
particularly complicated and simple rendering. Data is served from
(DataStore) ArcSDE, with the feature classes in the web mercator
projection. I making requests for EPSG:900913. I'm trying to get the
source projection and requested projection to be the same to avoid
reprojecting on-the-fly, but I'm not sure that I'm succeeding at that.

I'm using Tomcat 6.0 and JDK 1.6 on a four-processor, 2.5 GHz Windows
Server 2003 box with 4 GB RAM.

Single, isolated WMS requests to this service have a response time of
about 2.5 seconds. That's a little slow, but our real concern is
throughput - being able to handle a high load of simultaneous requests.

To measure that, I have a home-grown multi-threaded application that
can throw multiple requests at a map service, at whatever rate I want,
and report the response times.

When I throw a new request at my WMS service on GeoServer every 3000
milliseconds for 30 seconds, the average response time is 2.5 seconds.

When I throw a new request every 2400 ms for 30 seconds, the average
response time is 4.6 seconds.

When I throw a new request every 1000 ms for 30 seconds, the average
response time is 31.9 seconds. The response rate does not "ramp up"
slowly... it pretty quickly gets to responses in the 30 second range,
and stays up there.

Clearly it's not scaling well. Even when hitting it "hard" like in
the above test, the CPUs on the server are showing very low
utilization (can hardly tell there's anything special going on).

I've tried many of the suggestions in
http://geoserver.org/display/GEOSDOC/2.6+GeoServer+in+Production+Environment

but haven't seen any significant benefit.

For comparison, an ArcGIS Server map service on the same machine with
the same layers using the same ArcSDE data sources (plus some labels
and a hill-shade raster layer that the similar GeoServer WMS service
doesn't have) gives an average response time of 1.1 seconds when I
throw a new request at it every 1000 ms for 30 seconds.

When I throw a new request at ArcGIS Server every 500 ms for 30
seconds, the average response time is 1.2 seconds.

And I know there's some throughput tuning on ArcGIS Server that will
improve throughput that I have not yet done.

So, any suggestions for getting better throughput and scaling out of
GeoServer?

David

------------------------------------------------------------------------------

Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
is a gathering of tech-side developers & brand creativity
professionals. Meet
the minds behind Google Creative Lab, Visual Complexity, Processing, &
iPhoneDevCamp as they present alongside digital heavyweights like
Barbarian Group, R/GA, & Big Spaceship.
http://p.sf.net/sfu/creativitycat-com
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

------------------------------------------------------------------------------

Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
is a gathering of tech-side developers & brand creativity
professionals. Meet
the minds behind Google Creative Lab, Visual Complexity, Processing, &
iPhoneDevCamp as they present alongside digital heavyweights like
Barbarian Group, R/GA, & Big Spaceship.
http://p.sf.net/sfu/creativitycat-com
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

11

If anyone could give me some information on creating such a layer for use in geoserver, I'd appreciate it.

I think you may be wanting to use the SLD element <ShadedRelief>, but as far as I know, support for this has not been implemented in GeoServer yet. Simone, am I correct?

Thanks,
Mike Pumphrey
OpenGeo - http://opengeo.org

David Rush wrote:

Gabriel:

A hill-shade raster layer is a map layer showing the topography of the land by shading the sides of all the hills.

I think it usually (if not always) uses a DEM (digital elevation model) to artificially generate the appearance of light coming in from one direction, so slopes facing towards the light are lighter, and slopes facing away are darker. There must be another name for it.... it's a very common technique.

Here's a link to an example: http://maps.unomaha.edu/atlas2002/pages/06-Hillshade.pdf

Hillshade is the brown background.

If anyone could give me some information on creating such a layer for use in geoserver, I'd appreciate it.

David

Gabriel Roldan wrote:

Btw, David, what's an "hill-shade raster layer"?

Is it a raster layer you have on your ArcSDE instance? if so, did you
try configuring it on geoserver? With the latest development on support
for ArcSDE rasters it should sort of rock too, but I'd certainly love
some feedback on how it behaves under load.

Cheers,
Gabriel

David Rush wrote:

Gabriel:

I tried replacing the gt-arcsde-2.5.5.jar with
gt-arcsde-2.5-SNAPSHOT.jar from your link.

Wow.

Isolated requests are now responding in about 0.26 seconds, a nearly
ten-fold improvement.

Sending a new request every 1000 ms for 30 seconds got an average
response time of 0.245 sec.
500 ms for 30 sec average 0.233 sec.
200 ms for 30 sec avg 0.322 sec.
100 ms for 30 sec avg 1.987 sec.
50 ms for 30 sec avg 2.170 sec.

I tried a 20 ms rate, but my tool cratered at that rate.

The faster rates also started showing significant CPU load (40%-ish)
on the geoserver machine, which is a good sign.

At present I have the min/max poolConnnections at 10/10. Earlier I
had it at the default of 2/10, and didn't notice a big diff when I
switched it to 10/10.

Thanks! This is looking a lot more promising.

David

Gabriel Roldan wrote:

Hi David,

sad to hear that. Yet, as you know perf tunning is a complicated matter.
The following are the factors I can think off the top of my head that
you may need to double check:
- Did you assess the impact of the number of available ArcSDE
connections to GeoServer? (you can change the pool.maxConnections
parameter on the DataStore's edit page)
- Do the layers have proper spatial indexes? do they have indexes on the
non spatial attributes used by your SLD, if any?
- You said ArcGIS Server is performing well so the question of if you
compared with other products does not apply. But my main suspect here is
the number of connections. Did you check how many connections does
ArcGIS Server use to serve your requests, and how they relate to the
ones GeoServer is allowed to use?

If you wish I would be glad of setting up an environment similar to
yours (can you share an sdeexport of your 4 polygon layers with me?) and
do some profiling. I'm very interested on helping you with this cause my
latest perf chek on the plugin was getting better results than the
postgis one, but that may have changed over my feet.

Ah... btw, I just fixed a scalability issue that's surely related to
this. Check this out: http://jira.codehaus.org/browse/GEOT-1956
The fix is for geotools 2.5.6 and GeoServer 1.7.5, so you may want to
try out the latest nightly with the fix applied:

<http://gridlock.openplans.org/geoserver/1.7.x/geoserver-1.7.x-latest-war.zip&gt;

and
<http://gridlock.openplans.org/geoserver/1.7.x/ext-latest/geoserver-1.7.5-SNAPSHOT-arcsde-plugin.zip&gt;

Let me know how it goes.

Cheers,
Gabriel

David Rush wrote:

I've been experimenting with GeoServer 1.7.4 as a possible alternative
to an expensive, proprietary product.

I've set up a WMS service with 4 polygon layers inside it, nothing
particularly complicated and simple rendering. Data is served from
(DataStore) ArcSDE, with the feature classes in the web mercator
projection. I making requests for EPSG:900913. I'm trying to get the
source projection and requested projection to be the same to avoid
reprojecting on-the-fly, but I'm not sure that I'm succeeding at that.

I'm using Tomcat 6.0 and JDK 1.6 on a four-processor, 2.5 GHz Windows
Server 2003 box with 4 GB RAM.

Single, isolated WMS requests to this service have a response time of
about 2.5 seconds. That's a little slow, but our real concern is
throughput - being able to handle a high load of simultaneous requests.

To measure that, I have a home-grown multi-threaded application that
can throw multiple requests at a map service, at whatever rate I want,
and report the response times.

When I throw a new request at my WMS service on GeoServer every 3000
milliseconds for 30 seconds, the average response time is 2.5 seconds.

When I throw a new request every 2400 ms for 30 seconds, the average
response time is 4.6 seconds.

When I throw a new request every 1000 ms for 30 seconds, the average
response time is 31.9 seconds. The response rate does not "ramp up"
slowly... it pretty quickly gets to responses in the 30 second range,
and stays up there.

Clearly it's not scaling well. Even when hitting it "hard" like in
the above test, the CPUs on the server are showing very low
utilization (can hardly tell there's anything special going on).

I've tried many of the suggestions in
http://geoserver.org/display/GEOSDOC/2.6+GeoServer+in+Production+Environment

but haven't seen any significant benefit.

For comparison, an ArcGIS Server map service on the same machine with
the same layers using the same ArcSDE data sources (plus some labels
and a hill-shade raster layer that the similar GeoServer WMS service
doesn't have) gives an average response time of 1.1 seconds when I
throw a new request at it every 1000 ms for 30 seconds.

When I throw a new request at ArcGIS Server every 500 ms for 30
seconds, the average response time is 1.2 seconds.

And I know there's some throughput tuning on ArcGIS Server that will
improve throughput that I have not yet done.

So, any suggestions for getting better throughput and scaling out of
GeoServer?

David

------------------------------------------------------------------------------

Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
is a gathering of tech-side developers & brand creativity
professionals. Meet
the minds behind Google Creative Lab, Visual Complexity, Processing, &
iPhoneDevCamp as they present alongside digital heavyweights like
Barbarian Group, R/GA, & Big Spaceship.
http://p.sf.net/sfu/creativitycat-com
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

------------------------------------------------------------------------------

Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
is a gathering of tech-side developers & brand creativity
professionals. Meet
the minds behind Google Creative Lab, Visual Complexity, Processing, &
iPhoneDevCamp as they present alongside digital heavyweights like
Barbarian Group, R/GA, & Big Spaceship.
http://p.sf.net/sfu/creativitycat-com
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

------------------------------------------------------------------------------
OpenSolaris 2009.06 is a cutting edge operating system for enterprises looking to deploy the next generation of Solaris that includes the latest innovations from Sun and the OpenSource community. Download a copy and enjoy capabilities such as Networking, Storage and Virtualization. Go to: http://p.sf.net/sfu/opensolaris-get
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

12

Ciao Mike,
you are right, hill shading has not been implemented in current
version of the raster symbolizer support.
It would not be extremely hard to implement but so far I have not
been able to find someone who was willing to fund such a development.

Simone.
-------------------------------------------------------
Ing. Simone Giannecchini
GeoSolutions S.A.S.
Owner - Software Engineer
Via Carignoni 51
55041 Camaiore (LU)
Italy

phone: +39 0584983027
fax: +39 0584983027
mob: +39 333 8128928

http://www.geo-solutions.it
http://simboss.blogspot.com/
http://www.linkedin.com/in/simonegiannecchini

-------------------------------------------------------

On Thu, Jun 4, 2009 at 11:24 PM, Mike Pumphrey <mike@anonymised.com> wrote:

If anyone could give me some information on creating such a layer for use
in geoserver, I'd appreciate it.

I think you may be wanting to use the SLD element <ShadedRelief>, but as far
as I know, support for this has not been implemented in GeoServer yet.
Simone, am I correct?

Thanks,
Mike Pumphrey
OpenGeo - http://opengeo.org

David Rush wrote:

Gabriel:

A hill-shade raster layer is a map layer showing the topography of the
land by shading the sides of all the hills.

I think it usually (if not always) uses a DEM (digital elevation model) to
artificially generate the appearance of light coming in from one direction,
so slopes facing towards the light are lighter, and slopes facing away are
darker. There must be another name for it.... it's a very common technique.

Here's a link to an example:
http://maps.unomaha.edu/atlas2002/pages/06-Hillshade.pdf

Hillshade is the brown background.

If anyone could give me some information on creating such a layer for use
in geoserver, I'd appreciate it.

David

Gabriel Roldan wrote:

Btw, David, what's an "hill-shade raster layer"?

Is it a raster layer you have on your ArcSDE instance? if so, did you
try configuring it on geoserver? With the latest development on support
for ArcSDE rasters it should sort of rock too, but I'd certainly love
some feedback on how it behaves under load.

Cheers,
Gabriel

David Rush wrote:

Gabriel:

I tried replacing the gt-arcsde-2.5.5.jar with
gt-arcsde-2.5-SNAPSHOT.jar from your link.

Wow.

Isolated requests are now responding in about 0.26 seconds, a nearly
ten-fold improvement.

Sending a new request every 1000 ms for 30 seconds got an average
response time of 0.245 sec.
500 ms for 30 sec average 0.233 sec.
200 ms for 30 sec avg 0.322 sec.
100 ms for 30 sec avg 1.987 sec.
50 ms for 30 sec avg 2.170 sec.

I tried a 20 ms rate, but my tool cratered at that rate.

The faster rates also started showing significant CPU load (40%-ish)
on the geoserver machine, which is a good sign.

At present I have the min/max poolConnnections at 10/10. Earlier I
had it at the default of 2/10, and didn't notice a big diff when I
switched it to 10/10.

Thanks! This is looking a lot more promising.

David

Gabriel Roldan wrote:

Hi David,

sad to hear that. Yet, as you know perf tunning is a complicated
matter.
The following are the factors I can think off the top of my head that
you may need to double check:
- Did you assess the impact of the number of available ArcSDE
connections to GeoServer? (you can change the pool.maxConnections
parameter on the DataStore's edit page)
- Do the layers have proper spatial indexes? do they have indexes on
the
non spatial attributes used by your SLD, if any?
- You said ArcGIS Server is performing well so the question of if you
compared with other products does not apply. But my main suspect here
is
the number of connections. Did you check how many connections does
ArcGIS Server use to serve your requests, and how they relate to the
ones GeoServer is allowed to use?

If you wish I would be glad of setting up an environment similar to
yours (can you share an sdeexport of your 4 polygon layers with me?)
and
do some profiling. I'm very interested on helping you with this cause
my
latest perf chek on the plugin was getting better results than the
postgis one, but that may have changed over my feet.

Ah... btw, I just fixed a scalability issue that's surely related to
this. Check this out: http://jira.codehaus.org/browse/GEOT-1956
The fix is for geotools 2.5.6 and GeoServer 1.7.5, so you may want to
try out the latest nightly with the fix applied:

<http://gridlock.openplans.org/geoserver/1.7.x/geoserver-1.7.x-latest-war.zip&gt;

and

<http://gridlock.openplans.org/geoserver/1.7.x/ext-latest/geoserver-1.7.5-SNAPSHOT-arcsde-plugin.zip&gt;

Let me know how it goes.

Cheers,
Gabriel

David Rush wrote:

I've been experimenting with GeoServer 1.7.4 as a possible alternative
to an expensive, proprietary product.

I've set up a WMS service with 4 polygon layers inside it, nothing
particularly complicated and simple rendering. Data is served from
(DataStore) ArcSDE, with the feature classes in the web mercator
projection. I making requests for EPSG:900913. I'm trying to get the
source projection and requested projection to be the same to avoid
reprojecting on-the-fly, but I'm not sure that I'm succeeding at that.

I'm using Tomcat 6.0 and JDK 1.6 on a four-processor, 2.5 GHz Windows
Server 2003 box with 4 GB RAM.

Single, isolated WMS requests to this service have a response time of
about 2.5 seconds. That's a little slow, but our real concern is
throughput - being able to handle a high load of simultaneous
requests.

To measure that, I have a home-grown multi-threaded application that
can throw multiple requests at a map service, at whatever rate I want,
and report the response times.

When I throw a new request at my WMS service on GeoServer every 3000
milliseconds for 30 seconds, the average response time is 2.5 seconds.

When I throw a new request every 2400 ms for 30 seconds, the average
response time is 4.6 seconds.

When I throw a new request every 1000 ms for 30 seconds, the average
response time is 31.9 seconds. The response rate does not "ramp up"
slowly... it pretty quickly gets to responses in the 30 second range,
and stays up there.

Clearly it's not scaling well. Even when hitting it "hard" like in
the above test, the CPUs on the server are showing very low
utilization (can hardly tell there's anything special going on).

I've tried many of the suggestions in

http://geoserver.org/display/GEOSDOC/2.6+GeoServer+in+Production+Environment

but haven't seen any significant benefit.

For comparison, an ArcGIS Server map service on the same machine with
the same layers using the same ArcSDE data sources (plus some labels
and a hill-shade raster layer that the similar GeoServer WMS service
doesn't have) gives an average response time of 1.1 seconds when I
throw a new request at it every 1000 ms for 30 seconds.

When I throw a new request at ArcGIS Server every 500 ms for 30
seconds, the average response time is 1.2 seconds.

And I know there's some throughput tuning on ArcGIS Server that will
improve throughput that I have not yet done.

So, any suggestions for getting better throughput and scaling out of
GeoServer?

David

------------------------------------------------------------------------------

Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
is a gathering of tech-side developers & brand creativity
professionals. Meet
the minds behind Google Creative Lab, Visual Complexity, Processing, &
iPhoneDevCamp as they present alongside digital heavyweights like
Barbarian Group, R/GA, & Big Spaceship.
http://p.sf.net/sfu/creativitycat-com
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

------------------------------------------------------------------------------

Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
is a gathering of tech-side developers & brand creativity
professionals. Meet
the minds behind Google Creative Lab, Visual Complexity, Processing, &
iPhoneDevCamp as they present alongside digital heavyweights like
Barbarian Group, R/GA, & Big Spaceship.
http://p.sf.net/sfu/creativitycat-com
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

------------------------------------------------------------------------------
OpenSolaris 2009.06 is a cutting edge operating system for enterprises
looking to deploy the next generation of Solaris that includes the latest
innovations from Sun and the OpenSource community. Download a copy and enjoy
capabilities such as Networking, Storage and Virtualization. Go to:
http://p.sf.net/sfu/opensolaris-get
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

13

Hi David,

you may want to update your jar again to the latest nightly.
No perf. boost this time, but a couple important bugfixes.
For instance:
- set the connection pool (the commons-pool one) "min iddle" count to zero. It was set to pool.minConnections and lead to a commons-pool background thread trying to create connections to keep up with the minIddle, which caused resource exhaustion if the server was unreachable.
- if a connection failed to be created (due to unreachable server or wrong parameters) the exception was being propagated without first cleaning it´s execution thread. Now it´s caught and executor.shutDownNow() is called)

Cheers,

Gabriel

David Rush wrote:

Gabriel:

I tried replacing the gt-arcsde-2.5.5.jar with gt-arcsde-2.5-SNAPSHOT.jar from your link.

Wow.

Isolated requests are now responding in about 0.26 seconds, a nearly ten-fold improvement.

Sending a new request every 1000 ms for 30 seconds got an average response time of 0.245 sec.
500 ms for 30 sec average 0.233 sec.
200 ms for 30 sec avg 0.322 sec.
100 ms for 30 sec avg 1.987 sec.
50 ms for 30 sec avg 2.170 sec.

I tried a 20 ms rate, but my tool cratered at that rate.

The faster rates also started showing significant CPU load (40%-ish) on the geoserver machine, which is a good sign.

At present I have the min/max poolConnnections at 10/10. Earlier I had it at the default of 2/10, and didn't notice a big diff when I switched it to 10/10.

Thanks! This is looking a lot more promising.

David

Gabriel Roldan wrote:

Hi David,

sad to hear that. Yet, as you know perf tunning is a complicated matter.
The following are the factors I can think off the top of my head that
you may need to double check:
- Did you assess the impact of the number of available ArcSDE
connections to GeoServer? (you can change the pool.maxConnections
parameter on the DataStore's edit page)
- Do the layers have proper spatial indexes? do they have indexes on the
non spatial attributes used by your SLD, if any?
- You said ArcGIS Server is performing well so the question of if you
compared with other products does not apply. But my main suspect here is
the number of connections. Did you check how many connections does
ArcGIS Server use to serve your requests, and how they relate to the
ones GeoServer is allowed to use?

If you wish I would be glad of setting up an environment similar to
yours (can you share an sdeexport of your 4 polygon layers with me?) and
do some profiling. I'm very interested on helping you with this cause my
latest perf chek on the plugin was getting better results than the
postgis one, but that may have changed over my feet.

Ah... btw, I just fixed a scalability issue that's surely related to
this. Check this out: http://jira.codehaus.org/browse/GEOT-1956
The fix is for geotools 2.5.6 and GeoServer 1.7.5, so you may want to
try out the latest nightly with the fix applied:

<http://gridlock.openplans.org/geoserver/1.7.x/geoserver-1.7.x-latest-war.zip&gt;

and
<http://gridlock.openplans.org/geoserver/1.7.x/ext-latest/geoserver-1.7.5-SNAPSHOT-arcsde-plugin.zip&gt;

Let me know how it goes.

Cheers,
Gabriel

David Rush wrote:

I've been experimenting with GeoServer 1.7.4 as a possible alternative
to an expensive, proprietary product.

I've set up a WMS service with 4 polygon layers inside it, nothing
particularly complicated and simple rendering. Data is served from
(DataStore) ArcSDE, with the feature classes in the web mercator
projection. I making requests for EPSG:900913. I'm trying to get the
source projection and requested projection to be the same to avoid
reprojecting on-the-fly, but I'm not sure that I'm succeeding at that.

I'm using Tomcat 6.0 and JDK 1.6 on a four-processor, 2.5 GHz Windows
Server 2003 box with 4 GB RAM.

Single, isolated WMS requests to this service have a response time of
about 2.5 seconds. That's a little slow, but our real concern is
throughput - being able to handle a high load of simultaneous requests.

To measure that, I have a home-grown multi-threaded application that
can throw multiple requests at a map service, at whatever rate I want,
and report the response times.

When I throw a new request at my WMS service on GeoServer every 3000
milliseconds for 30 seconds, the average response time is 2.5 seconds.

When I throw a new request every 2400 ms for 30 seconds, the average
response time is 4.6 seconds.

When I throw a new request every 1000 ms for 30 seconds, the average
response time is 31.9 seconds. The response rate does not "ramp up"
slowly... it pretty quickly gets to responses in the 30 second range,
and stays up there.

Clearly it's not scaling well. Even when hitting it "hard" like in
the above test, the CPUs on the server are showing very low
utilization (can hardly tell there's anything special going on).

I've tried many of the suggestions in
http://geoserver.org/display/GEOSDOC/2.6+GeoServer+in+Production+Environment
but haven't seen any significant benefit.

For comparison, an ArcGIS Server map service on the same machine with
the same layers using the same ArcSDE data sources (plus some labels
and a hill-shade raster layer that the similar GeoServer WMS service
doesn't have) gives an average response time of 1.1 seconds when I
throw a new request at it every 1000 ms for 30 seconds.

When I throw a new request at ArcGIS Server every 500 ms for 30
seconds, the average response time is 1.2 seconds.

And I know there's some throughput tuning on ArcGIS Server that will
improve throughput that I have not yet done.

So, any suggestions for getting better throughput and scaling out of
GeoServer?

David

------------------------------------------------------------------------------

Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
is a gathering of tech-side developers & brand creativity
professionals. Meet
the minds behind Google Creative Lab, Visual Complexity, Processing, &
iPhoneDevCamp as they present alongside digital heavyweights like
Barbarian Group, R/GA, & Big Spaceship.
http://p.sf.net/sfu/creativitycat-com
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

------------------------------------------------------------------------------
Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT is a gathering of tech-side developers & brand creativity professionals. Meet
the minds behind Google Creative Lab, Visual Complexity, Processing, & iPhoneDevCamp as they present alongside digital heavyweights like Barbarian Group, R/GA, & Big Spaceship. http://p.sf.net/sfu/creativitycat-com _______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
Gabriel Roldan
OpenGeo - http://opengeo.org
Expert service straight from the developers.