[Geoserver-users] max features

All,

I currently have a postgis Feature Type with approx 13 million point
features. My geoserver instance is set for 1 million max features. When I
send a WMS request with a filter that should return about 40,000 features,
geosever just hangs....the same query is returned quickly when I hit postgis
directly. My question is: does the Maximum Features number refer to the
total number of features in the Feature Type (13 million) or the number
returned for the GetMap request (40,000)?

Thanks,
Steve

Stephen Crawford
Center for Environmental Informatics
The Pennsylvania State University

I believe the maxfeatures setting in the featuretype configuration only applies to WFS getFeature requests. However, you can specify maxfeatures as a WMS parameter (just add it to the query string in the URL).

Hope this helps,
David Winslow

Stephen Crawford wrote:

All,

I currently have a postgis Feature Type with approx 13 million point
features. My geoserver instance is set for 1 million max features. When I
send a WMS request with a filter that should return about 40,000 features,
geosever just hangs....the same query is returned quickly when I hit postgis
directly. My question is: does the Maximum Features number refer to the
total number of features in the Feature Type (13 million) or the number
returned for the GetMap request (40,000)?

Thanks,
Steve

Stephen Crawford
Center for Environmental Informatics
The Pennsylvania State University

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

!DSPAM:4040,4885efcc177393327367457!

Stephen,

We are having a similar issue that may be related. We have a postgis Feature Type with about 800k features. We notice a sharp decrease in performance when we send a WMS request with a filter that returns ~50k features. On our server, we can handle 50-80k feature requests (with 4-5 seconds delay) but as the number of features increase the delays get significantly longer (45 seconds). Similarly, the postgis table doesn’t seem to be the problem, as I can render all 800k points in MapServer with only a few seconds of processing max.

Let me know if you resolve your issue, and if you are able to get good performance rendering your points. Thanks!

–j

On Tue, Jul 22, 2008 at 10:33 AM, Stephen Crawford <src176@anonymised.com> wrote:

All,

I currently have a postgis Feature Type with approx 13 million point
features. My geoserver instance is set for 1 million max features. When I
send a WMS request with a filter that should return about 40,000 features,
geosever just hangs…the same query is returned quickly when I hit postgis
directly. My question is: does the Maximum Features number refer to the
total number of features in the Feature Type (13 million) or the number
returned for the GetMap request (40,000)?

Thanks,
Steve

Stephen Crawford
Center for Environmental Informatics
The Pennsylvania State University


This SF.Net email is sponsored by the Moblin Your Move Developer’s challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/


Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

I just tried a WFS request that works well for a filter that returns 4500 features (from the 13 million Feature Type), but the same filter does not work with the WMS…it just hangs, with no error in the logs or anything.


From: geoserver-users-bounces@lists.sourceforge.net [mailto:geoserver-users-bounces@lists.sourceforge.net] On Behalf Of Josh Marcus
Sent: Tuesday, July 22, 2008 12:16 PM
To: Stephen Crawford
Cc: geoserver-users@lists.sourceforge.net
Subject: Re: [Geoserver-users] max features

Stephen,

We are having a similar issue that may be related. We have a postgis Feature Type with about 800k features. We notice a sharp decrease in performance when we send a WMS request with a filter that returns ~50k features. On our server, we can handle 50-80k feature requests (with 4-5 seconds delay) but as the number of features increase the delays get significantly longer (45 seconds). Similarly, the postgis table doesn’t seem to be the problem, as I can render all 800k points in MapServer with only a few seconds of processing max.

Let me know if you resolve your issue, and if you are able to get good performance rendering your points. Thanks!

–j

On Tue, Jul 22, 2008 at 10:33 AM, Stephen Crawford <src176@anonymised.com> wrote:

All,

I currently have a postgis Feature Type with approx 13 million point
features. My geoserver instance is set for 1 million max features. When I
send a WMS request with a filter that should return about 40,000 features,
geosever just hangs…the same query is returned quickly when I hit postgis
directly. My question is: does the Maximum Features number refer to the
total number of features in the Feature Type (13 million) or the number
returned for the GetMap request (40,000)?

Thanks,
Steve

Stephen Crawford
Center for Environmental Informatics
The Pennsylvania State University


This SF.Net email is sponsored by the Moblin Your Move Developer’s challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/


Geoserver-users mailing list
Geoserver-users@anonymised.comts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

All,

In regards to my last note, I made a new table in postgis that is a 300k
feature subset of the original 13 million feature table. I made this into a
new feature type in geoserver, and the WMS request is returned easily. So
this leads me to believe the problem is in the filter. In short, this
works:

http://***/geoserver/wms?bbox=-6100000,-4500000,4800000,4500000&request=GetM
ap&layers=cei:world,cei:cardinal&width=723&height=600&styles=other_polygons,
default_point&bgcolor=0x99ccff&format=image/png&srs=EPSG:102008

..and this doesn't:

http://***/geoserver/wms?bbox=-6100000,-4500000,4800000,4500000&request=GetM
ap&layers=cei:obs_subset&width=723&height=600&styles=default_point&bgcolor=0
x99ccff&format=image/png&srs=EPSG:102008&filter=<Filter><PropertyIsEqualTo><
PropertyName>sci_name</PropertyName><Literal>Cardinalis
cardinalis</Literal></PropertyIsEqualTo></Filter>

...where 'cardinal' is the table derived from 'obs_subset'

I tried to increase the loggin level to see if I could learn anything, but
then it would log out of control with the following exception; any help is
greatly appreciated.

Thanks,
Steve

2008-07-22 15:43:54,900 ERROR [geotools.rendering] - Pool not open
java.lang.IllegalStateException: Pool not open
        at
org.apache.commons.pool.BaseObjectPool.assertOpen(BaseObjectPool.java:78)
        at
org.apache.commons.pool.impl.GenericObjectPool.returnObject(GenericObjectPoo
l.java:916)
        at
org.apache.commons.dbcp.PoolableConnection.close(PoolableConnection.java:87)
        at
org.apache.commons.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.close(P
oolingDataSource.java:181)
        at org.geotools.data.jdbc.JDBCUtils.close(JDBCUtils.java:149)
        at org.geotools.data.jdbc.QueryData.close(QueryData.java:199)
        at
org.geotools.data.jdbc.JDBCFeatureReader.close(JDBCFeatureReader.java:66)
        at
org.geotools.data.jdbc.JDBCFeatureReader.close(JDBCFeatureReader.java:62)
        at
org.geotools.data.store.FeatureReaderIterator.close(FeatureReaderIterator.ja
va:87)
        at
org.geotools.data.store.FeatureReaderIterator.hasNext(FeatureReaderIterator.
java:50)
        at
org.geotools.renderer.lite.StreamingRenderer.processStylers(StreamingRendere
r.java:1596)
        at
org.geotools.renderer.lite.StreamingRenderer.paint(StreamingRenderer.java:64
1)
        at
org.geotools.renderer.shape.ShapefileRenderer.renderWithStreamingRenderer(Sh
apefileRenderer.java:1461)
        at
org.geotools.renderer.shape.ShapefileRenderer.paint(ShapefileRenderer.java:1
331)
        at
org.geotools.renderer.shape.ShapefileRenderer.paint(ShapefileRenderer.java:2
62)
        at
org.vfny.geoserver.wms.responses.DefaultRasterMapProducer.produceMap(Default
RasterMapProducer.java:310)
        at
org.vfny.geoserver.wms.responses.GetMapResponse.execute(GetMapResponse.java:
424)
        at
org.geoserver.ows.adapters.ResponseAdapter.getMimeType(ResponseAdapter.java:
48)
        at org.geoserver.ows.Dispatcher.response(Dispatcher.java:616)
        at
org.geoserver.ows.Dispatcher.handleRequestInternal(Dispatcher.java:192)
        at
org.springframework.web.servlet.mvc.AbstractController.handleRequest(Abstrac
tController.java:139)
        at
org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(Si
mpleControllerHandlerAdapter.java:44)
        at
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServl
et.java:684)
        at
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServle
t.java:625)
        at
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkSer
vlet.java:392)
        at
org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java
:347)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
        at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
FilterChain.java:290)
        at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
ain.java:206)
        at
org.vfny.geoserver.filters.SetCharacterEncodingFilter.doFilter(SetCharacterE
ncodingFilter.java:108)
        at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
FilterChain.java:235)
        at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
ain.java:206)
        at
org.geoserver.filters.ReverseProxyFilter.doFilter(ReverseProxyFilter.java:17
0)
        at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
FilterChain.java:235)
        at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
ain.java:206)
        at
org.geoserver.filters.LoggingFilter.doFilter(LoggingFilter.java:73)
        at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
FilterChain.java:235)
        at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
ain.java:206)
        at org.geoserver.filters.GZIPFilter.doFilter(GZIPFilter.java:41)
        at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
FilterChain.java:235)
        at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
ain.java:206)
        at
org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterCh
ainProxy.java:264)
        at
org.acegisecurity.intercept.web.FilterSecurityInterceptor.invoke(FilterSecur
ityInterceptor.java:107)
        at
org.acegisecurity.intercept.web.FilterSecurityInterceptor.doFilter(FilterSec
urityInterceptor.java:72)
        at
org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterCh
ainProxy.java:274)
        at
org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslatio
nFilter.java:110)
        at
org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterCh
ainProxy.java:274)
        at
org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(Ano
nymousProcessingFilter.java:125)
        at
org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterCh
ainProxy.java:274)
        at
org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessin
gFilter.java:178)
        at
org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterCh
ainProxy.java:274)
        at
org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpS
essionContextIntegrationFilter.java:229)
        at
org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterCh
ainProxy.java:274)
        at
org.acegisecurity.util.FilterChainProxy.doFilter(FilterChainProxy.java:148)
        at
org.acegisecurity.util.FilterToBeanProxy.doFilter(FilterToBeanProxy.java:98)
        at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
FilterChain.java:235)
        at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
ain.java:206)
        at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.ja
va:233)
        at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.ja
va:175)
        at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128
)
        at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102
)
        at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java
:109)
        at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:263)
        at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)
        at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http
11Protocol.java:584)
        at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
        at java.lang.Thread.run(Unknown Source)

________________________________

From: geoserver-users-bounces@lists.sourceforge.net
[mailto:geoserver-users-bounces@lists.sourceforge.net] On Behalf Of Josh
Marcus
Sent: Tuesday, July 22, 2008 12:16 PM
To: Stephen Crawford
Cc: geoserver-users@lists.sourceforge.net
Subject: Re: [Geoserver-users] max features

Stephen,

We are having a similar issue that may be related. We have a postgis
Feature Type with about 800k features. We notice a sharp decrease in
performance when we send a WMS request with a filter that returns ~50k
features. On our server, we can handle 50-80k feature requests (with 4-5
seconds delay) but as the number of features increase the delays get
significantly longer (45 seconds). Similarly, the postgis table doesn't seem
to be the problem, as I can render all 800k points in MapServer with only a
few seconds of processing max.

Let me know if you resolve your issue, and if you are able to get good
performance rendering your points. Thanks!

--j

On Tue, Jul 22, 2008 at 10:33 AM, Stephen Crawford <src176@anonymised.com> wrote:

  All,
  
  I currently have a postgis Feature Type with approx 13 million point
  features. My geoserver instance is set for 1 million max features.
When I
  send a WMS request with a filter that should return about 40,000
features,
  geosever just hangs....the same query is returned quickly when I hit
postgis
  directly. My question is: does the Maximum Features number refer
to the
  total number of features in the Feature Type (13 million) or the
number
  returned for the GetMap request (40,000)?
  
  Thanks,
  Steve
  
  Stephen Crawford
  Center for Environmental Informatics
  The Pennsylvania State University
  
-------------------------------------------------------------------------
  This SF.Net email is sponsored by the Moblin Your Move Developer's
challenge
  Build the coolest Linux based applications with Moblin SDK & win
great prizes
  Grand prize is a trip for two to an Open Source event anywhere in
the world
  http://moblin-contest.org/redirect.php?banner_id=100&url=/
  _______________________________________________
  Geoserver-users mailing list
  Geoserver-users@lists.sourceforge.net
  https://lists.sourceforge.net/lists/listinfo/geoserver-users
  

Can you try a cql_filter? Much more compact...

&CQL_FILTER=sci_name%20=%20'Cardinalis%20cardinalis'&

For what is worth--'filter' parameter should work too... I haven't
looked closely at your filter, but I am using it successfully with WMS
requests.

Alex

On 7/22/08, Stephen Crawford <src176@anonymised.com> wrote:

All,

In regards to my last note, I made a new table in postgis that is a 300k
feature subset of the original 13 million feature table. I made this into a
new feature type in geoserver, and the WMS request is returned easily. So
this leads me to believe the problem is in the filter. In short, this
works:

http://***/geoserver/wms?bbox=-6100000,-4500000,4800000,4500000&request=GetM
ap&layers=cei:world,cei:cardinal&width=723&height=600&styles=other_polygons,
default_point&bgcolor=0x99ccff&format=image/png&srs=EPSG:102008

..and this doesn't:

http://***/geoserver/wms?bbox=-6100000,-4500000,4800000,4500000&request=GetM
ap&layers=cei:obs_subset&width=723&height=600&styles=default_point&bgcolor=0
x99ccff&format=image/png&srs=EPSG:102008&filter=<Filter><PropertyIsEqualTo><
PropertyName>sci_name</PropertyName><Literal>Cardinalis
cardinalis</Literal></PropertyIsEqualTo></Filter>

...where 'cardinal' is the table derived from 'obs_subset'

I tried to increase the loggin level to see if I could learn anything, but
then it would log out of control with the following exception; any help is
greatly appreciated.

Thanks,
Steve

2008-07-22 15:43:54,900 ERROR [geotools.rendering] - Pool not open
java.lang.IllegalStateException: Pool not open
        at
org.apache.commons.pool.BaseObjectPool.assertOpen(BaseObjectPool.java:78)
        at
org.apache.commons.pool.impl.GenericObjectPool.returnObject(GenericObjectPoo
l.java:916)
        at
org.apache.commons.dbcp.PoolableConnection.close(PoolableConnection.java:87)
        at
org.apache.commons.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.close(P
oolingDataSource.java:181)
        at org.geotools.data.jdbc.JDBCUtils.close(JDBCUtils.java:149)
        at org.geotools.data.jdbc.QueryData.close(QueryData.java:199)
        at
org.geotools.data.jdbc.JDBCFeatureReader.close(JDBCFeatureReader.java:66)
        at
org.geotools.data.jdbc.JDBCFeatureReader.close(JDBCFeatureReader.java:62)
        at
org.geotools.data.store.FeatureReaderIterator.close(FeatureReaderIterator.ja
va:87)
        at
org.geotools.data.store.FeatureReaderIterator.hasNext(FeatureReaderIterator.
java:50)
        at
org.geotools.renderer.lite.StreamingRenderer.processStylers(StreamingRendere
r.java:1596)
        at
org.geotools.renderer.lite.StreamingRenderer.paint(StreamingRenderer.java:64
1)
        at
org.geotools.renderer.shape.ShapefileRenderer.renderWithStreamingRenderer(Sh
apefileRenderer.java:1461)
        at
org.geotools.renderer.shape.ShapefileRenderer.paint(ShapefileRenderer.java:1
331)
        at
org.geotools.renderer.shape.ShapefileRenderer.paint(ShapefileRenderer.java:2
62)
        at
org.vfny.geoserver.wms.responses.DefaultRasterMapProducer.produceMap(Default
RasterMapProducer.java:310)
        at
org.vfny.geoserver.wms.responses.GetMapResponse.execute(GetMapResponse.java:
424)
        at
org.geoserver.ows.adapters.ResponseAdapter.getMimeType(ResponseAdapter.java:
48)
        at org.geoserver.ows.Dispatcher.response(Dispatcher.java:616)
        at
org.geoserver.ows.Dispatcher.handleRequestInternal(Dispatcher.java:192)
        at
org.springframework.web.servlet.mvc.AbstractController.handleRequest(Abstrac
tController.java:139)
        at
org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(Si
mpleControllerHandlerAdapter.java:44)
        at
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServl
et.java:684)
        at
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServle
t.java:625)
        at
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkSer
vlet.java:392)
        at
org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java
:347)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
        at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
FilterChain.java:290)
        at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
ain.java:206)
        at
org.vfny.geoserver.filters.SetCharacterEncodingFilter.doFilter(SetCharacterE
ncodingFilter.java:108)
        at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
FilterChain.java:235)
        at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
ain.java:206)
        at
org.geoserver.filters.ReverseProxyFilter.doFilter(ReverseProxyFilter.java:17
0)
        at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
FilterChain.java:235)
        at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
ain.java:206)
        at
org.geoserver.filters.LoggingFilter.doFilter(LoggingFilter.java:73)
        at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
FilterChain.java:235)
        at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
ain.java:206)
        at org.geoserver.filters.GZIPFilter.doFilter(GZIPFilter.java:41)
        at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
FilterChain.java:235)
        at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
ain.java:206)
        at
org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterCh
ainProxy.java:264)
        at
org.acegisecurity.intercept.web.FilterSecurityInterceptor.invoke(FilterSecur
ityInterceptor.java:107)
        at
org.acegisecurity.intercept.web.FilterSecurityInterceptor.doFilter(FilterSec
urityInterceptor.java:72)
        at
org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterCh
ainProxy.java:274)
        at
org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslatio
nFilter.java:110)
        at
org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterCh
ainProxy.java:274)
        at
org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(Ano
nymousProcessingFilter.java:125)
        at
org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterCh
ainProxy.java:274)
        at
org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessin
gFilter.java:178)
        at
org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterCh
ainProxy.java:274)
        at
org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpS
essionContextIntegrationFilter.java:229)
        at
org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterCh
ainProxy.java:274)
        at
org.acegisecurity.util.FilterChainProxy.doFilter(FilterChainProxy.java:148)
        at
org.acegisecurity.util.FilterToBeanProxy.doFilter(FilterToBeanProxy.java:98)
        at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
FilterChain.java:235)
        at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
ain.java:206)
        at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.ja
va:233)
        at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.ja
va:175)
        at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128
)
        at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102
)
        at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java
:109)
        at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:263)
        at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)
        at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http
11Protocol.java:584)
        at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
        at java.lang.Thread.run(Unknown Source)

________________________________

From: geoserver-users-bounces@lists.sourceforge.net
[mailto:geoserver-users-bounces@lists.sourceforge.net] On Behalf Of Josh
Marcus
Sent: Tuesday, July 22, 2008 12:16 PM
To: Stephen Crawford
Cc: geoserver-users@lists.sourceforge.net
Subject: Re: [Geoserver-users] max features

Stephen,

We are having a similar issue that may be related. We have a postgis
Feature Type with about 800k features. We notice a sharp decrease in
performance when we send a WMS request with a filter that returns ~50k
features. On our server, we can handle 50-80k feature requests (with 4-5
seconds delay) but as the number of features increase the delays get
significantly longer (45 seconds). Similarly, the postgis table doesn't seem
to be the problem, as I can render all 800k points in MapServer with only a
few seconds of processing max.

Let me know if you resolve your issue, and if you are able to get good
performance rendering your points. Thanks!

--j

On Tue, Jul 22, 2008 at 10:33 AM, Stephen Crawford <src176@anonymised.com> wrote:

        All,

        I currently have a postgis Feature Type with approx 13 million point
        features. My geoserver instance is set for 1 million max features.
When I
        send a WMS request with a filter that should return about 40,000
features,
        geosever just hangs....the same query is returned quickly when I hit
postgis
        directly. My question is: does the Maximum Features number refer
to the
        total number of features in the Feature Type (13 million) or the
number
        returned for the GetMap request (40,000)?

        Thanks,
        Steve

        Stephen Crawford
        Center for Environmental Informatics
        The Pennsylvania State University

-------------------------------------------------------------------------
        This SF.Net email is sponsored by the Moblin Your Move Developer's
challenge
        Build the coolest Linux based applications with Moblin SDK & win
great prizes
        Grand prize is a trip for two to an Open Source event anywhere in
the world
        http://moblin-contest.org/redirect.php?banner_id=100&url=/
        _______________________________________________
        Geoserver-users mailing list
        Geoserver-users@lists.sourceforge.net
        https://lists.sourceforge.net/lists/listinfo/geoserver-users

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Any chance one of you can include the log of the query being made against postgis during the WMS request. Definitely seems something is a miss. And as I mentioned before a dump of some sample data would definitely help us debug this one.

-Justin

Josh Marcus wrote:

Stephen,

We are having a similar issue that may be related. We have a postgis Feature Type with about 800k features. We notice a sharp decrease in performance when we send a WMS request with a filter that returns ~50k features. On our server, we can handle 50-80k feature requests (with 4-5 seconds delay) but as the number of features increase the delays get significantly longer (45 seconds). Similarly, the postgis table doesn't seem to be the problem, as I can render all 800k points in MapServer with only a few seconds of processing max.

Let me know if you resolve your issue, and if you are able to get good performance rendering your points. Thanks!

--j

On Tue, Jul 22, 2008 at 10:33 AM, Stephen Crawford <src176@anonymised.com <mailto:src176@anonymised.com>> wrote:

    All,

    I currently have a postgis Feature Type with approx 13 million point
    features. My geoserver instance is set for 1 million max features.
     When I
    send a WMS request with a filter that should return about 40,000
    features,
    geosever just hangs....the same query is returned quickly when I hit
    postgis
    directly. My question is: does the Maximum Features number refer
    to the
    total number of features in the Feature Type (13 million) or the number
    returned for the GetMap request (40,000)?

    Thanks,
    Steve

    Stephen Crawford
    Center for Environmental Informatics
    The Pennsylvania State University

    -------------------------------------------------------------------------
    This SF.Net email is sponsored by the Moblin Your Move Developer's
    challenge
    Build the coolest Linux based applications with Moblin SDK & win
    great prizes
    Grand prize is a trip for two to an Open Source event anywhere in
    the world
    http://moblin-contest.org/redirect.php?banner_id=100&url=/
    <http://moblin-contest.org/redirect.php?banner_id=100&url=/&gt;
    _______________________________________________
    Geoserver-users mailing list
    Geoserver-users@lists.sourceforge.net
    <mailto:Geoserver-users@lists.sourceforge.net>
    https://lists.sourceforge.net/lists/listinfo/geoserver-users

!DSPAM:4007,488607db260511637810514!

------------------------------------------------------------------------

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/

!DSPAM:4007,488607db260511637810514!

------------------------------------------------------------------------

_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

!DSPAM:4007,488607db260511637810514!

--
Justin Deoliveira
The Open Planning Project
jdeolive@anonymised.com

Hi Justin,

I found that my problem is on the postgis side of things. By looking at the
sql sent from geoserver, I can see that my wfs request works fine because it
is not looking for the bounding box. That is, the query from the wfs works
well:

SELECT "oid", encode(asBinary(force_2d("the_geom"),'XDR'),'base64') FROM
"public"."obs_subset" WHERE "sci_name" = 'Scolopax minor';

...but the wms query doesn't (actually, it does come up after about 10
minutes):

SELECT "oid", encode(asBinary(force_2d("the_geom"),'XDR'),'base64') FROM
"public"."obs_subset" WHERE ((TRUE AND "sci_name" = 'Scolopax minor') AND
"the_geom" && GeometryFromText('POLYGON ((-6190456.43153527 -4590000,
-6190456.43153527 4590000, 4890456.43153527 4590000, 4890456.43153527
-4590000, -6190456.43153527 -4590000))', 102008));

I have a spatial index, and the query planner says it is using the index.
I've run vacuum analyze and update_geometry_stats (we're using pg 7.4 and
postgis 1.2) The table is all point data, again about 13 million records.
This query returns about 4500 rows. Should it be this slow? I did note
that each of the points does not have a bbox (hasbbox=f)....but I wouldn't
think that would matter for points.

Thanks,
Steve

-----Original Message-----
From: Justin Deoliveira [mailto:jdeolive@anonymised.com]
Sent: Thursday, July 24, 2008 10:14 PM
To: Josh Marcus
Cc: Stephen Crawford; geoserver-users@lists.sourceforge.net
Subject: Re: [Geoserver-users] max features

Any chance one of you can include the log of the query being made against
postgis during the WMS request. Definitely seems something is a miss. And as
I mentioned before a dump of some sample data would definitely help us debug
this one.

-Justin

Josh Marcus wrote:

Stephen,

We are having a similar issue that may be related. We have a postgis
Feature Type with about 800k features. We notice a sharp decrease in
performance when we send a WMS request with a filter that returns ~50k
features. On our server, we can handle 50-80k feature requests (with
4-5 seconds delay) but as the number of features increase the delays
get significantly longer (45 seconds). Similarly, the postgis table
doesn't seem to be the problem, as I can render all 800k points in
MapServer with only a few seconds of processing max.

Let me know if you resolve your issue, and if you are able to get good
performance rendering your points. Thanks!

--j

On Tue, Jul 22, 2008 at 10:33 AM, Stephen Crawford <src176@anonymised.com
<mailto:src176@anonymised.com>> wrote:

    All,

    I currently have a postgis Feature Type with approx 13 million point
    features. My geoserver instance is set for 1 million max features.
     When I
    send a WMS request with a filter that should return about 40,000
    features,
    geosever just hangs....the same query is returned quickly when I hit
    postgis
    directly. My question is: does the Maximum Features number refer
    to the
    total number of features in the Feature Type (13 million) or the

number

    returned for the GetMap request (40,000)?

    Thanks,
    Steve

    Stephen Crawford
    Center for Environmental Informatics
    The Pennsylvania State University

-------------------------------------------------------------------------

    This SF.Net email is sponsored by the Moblin Your Move Developer's
    challenge
    Build the coolest Linux based applications with Moblin SDK & win
    great prizes
    Grand prize is a trip for two to an Open Source event anywhere in
    the world
    http://moblin-contest.org/redirect.php?banner_id=100&url=/
    <http://moblin-contest.org/redirect.php?banner_id=100&url=/&gt;
    _______________________________________________
    Geoserver-users mailing list
    Geoserver-users@lists.sourceforge.net
    <mailto:Geoserver-users@lists.sourceforge.net>
    https://lists.sourceforge.net/lists/listinfo/geoserver-users

!DSPAM:4007,488607db260511637810514!

----------------------------------------------------------------------
--

----------------------------------------------------------------------
--- This SF.Net email is sponsored by the Moblin Your Move Developer's
challenge Build the coolest Linux based applications with Moblin SDK &
win great prizes Grand prize is a trip for two to an Open Source event
anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/

!DSPAM:4007,488607db260511637810514!

----------------------------------------------------------------------
--

_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

!DSPAM:4007,488607db260511637810514!

--
Justin Deoliveira
The Open Planning Project
jdeolive@anonymised.com

Stephen Crawford ha scritto:

Hi Justin,

I found that my problem is on the postgis side of things. By looking at the
sql sent from geoserver, I can see that my wfs request works fine because it
is not looking for the bounding box. That is, the query from the wfs works
well:

SELECT "oid", encode(asBinary(force_2d("the_geom"),'XDR'),'base64') FROM
"public"."obs_subset" WHERE "sci_name" = 'Scolopax minor';

...but the wms query doesn't (actually, it does come up after about 10
minutes):

SELECT "oid", encode(asBinary(force_2d("the_geom"),'XDR'),'base64') FROM
"public"."obs_subset" WHERE ((TRUE AND "sci_name" = 'Scolopax minor') AND
"the_geom" && GeometryFromText('POLYGON ((-6190456.43153527 -4590000,
-6190456.43153527 4590000, 4890456.43153527 4590000, 4890456.43153527
-4590000, -6190456.43153527 -4590000))', 102008));

I have a spatial index, and the query planner says it is using the index.
I've run vacuum analyze and update_geometry_stats (we're using pg 7.4 and
postgis 1.2) The table is all point data, again about 13 million records.
This query returns about 4500 rows. Should it be this slow? I did note
that each of the points does not have a bbox (hasbbox=f)....but I wouldn't
think that would matter for points.

Hum, this reminds me of a post from Paul Ramsey:
http://blog.cleverelephant.ca/2008/05/elephant-vs-dolphin.html

That PostGIS is old, might it be that it's planning the query
badly? Any chance I can put my hands on a dump of that database
and have a look?

Cheers
Andrea

Stephen Crawford ha scritto:

All,

I currently have a postgis Feature Type with approx 13 million point
features. My geoserver instance is set for 1 million max features. When I
send a WMS request with a filter that should return about 40,000 features,
geosever just hangs....the same query is returned quickly when I hit postgis
directly. My question is: does the Maximum Features number refer to the
total number of features in the Feature Type (13 million) or the number
returned for the GetMap request (40,000)?

The "Maximum features" setting you find in the feature type and in the server configuration page are a WFS only settings, it won't affect WMS in any way.

We could make it work on WMS as well, thought I believe WMS should have
a different number than WFS. The rationale is that WFS encoding is much
more expensive, and much bigger in size than a raster output.

There is actually a JIRA already related to this topic:
http://jira.codehaus.org/browse/GEOS-1127
We didn't work on it so far since it would require working on the
config and UI, something we'll do only on 2.0.x onwards since the
old UI and config are too painful to code against and are going
to be completely replaced in the 2.0.x series.

If you need/want that support right now, you still have options,
such as DIY and attach the patch to that jira issue, or sponsor
some developer to code it for you (commercial support page here:
http://geoserver.org/display/GEOS/Commercial+Support)

Cheers
Andrea