[Geoserver-users] Geoserver memory & connections management

Harikumar_Reddy · May 11, 2011, 6:04am

Hi All,

We are using Geoserver 2.0.2 (recently migrated from 2.0.1) and using Geoserver since version 1.7. we are happy by its map rendering results but it is going down very often. we are restarting geoserver at least once in two days, after that it works for 24 hours very well. We are using oracle spatial 11g database as datastore for gis layers (vector & raster). It is actually for one state we are using and facing these problems. Any memory leakage or any other configurations to be done. The total data is around 80 GB ( 78 gb of satellite images & 2 gb gis layers)

I am sorry for asking very basic questions of memory management & geoserver connections related to systems side as i am pure GIS professional.

Now we are going to implement project for entire country (India) with 25 gis layers and satellite images for 100 cities. If we need to restart Geoserver once in two days may not be good idea. Any special configarations/settings is required (hardware side, software side). The production Geoserver will be having the given below linux environment.

Linux CentOS version 5
Geoserver version 2.0.2
Open layers 2.9
Oracle spatial 11 g R2

we are also thinking of alterante map servers because this kind of issues. If anyone knows, please suggest. For country level applications, which will be having an approximate user base of 50,000 day can be handled well by geoserver for the vector layers & satellite images ?

I have seen many posts by andrea & others, guiding users with good insights.

Many Many Thanks in advance for any guidance, suggestions & help.

Regards,
Hari.

Ben_Caradoc-Davies · May 11, 2011, 6:29am

We will need a bit more information before we can help you:

- What Java implementation (and version) are you using? What JVM options?

- What servlet container? Tomcat?

- Are you using Apache HTTPD in front of Tomcat?

- What are the symptoms of "going down"? Do you get any response? From Apache? Tomcat?

- Do you get anything in the logs (Apache, Tomcat or GeoServer)?

- Do you see heap errors in the logs?

- Do you get any mysterious database errors (ORA-something) or hangs?

- Is there a firewall between GeoServer and Oracle? from the app-schema manual but relevant to all users:
http://docs.geoserver.org/latest/en/user/data/app-schema/data-stores.html#jndi
"Firewall timeouts can silently sever idle connections to the database and cause GeoServer to hang. If there is a firewall between GeoServer and the database, a connection pool configured to shut down idle connections before the firewall can drop them will prevent GeoServer from hanging."

- Have you tried turning on VERBOSE logging in GeoServer to diagnose the problem?

Kind regards,
Ben.

On 11/05/11 14:04, Harikumar Reddy wrote:

Hi All,

We are using Geoserver 2.0.2 (recently migrated from 2.0.1) and using Geoserver since version 1.7. we are happy by its map rendering results but it is going down very often. we are restarting geoserver at least once in two days, after that it works for 24 hours very well. We are using oracle spatial 11g database as datastore for gis layers (vector& raster). It is actually for one state we are using and facing these problems. Any memory leakage or any other configurations to be done. The total data is around 80 GB ( 78 gb of satellite images& 2 gb gis layers)

I am sorry for asking very basic questions of memory management& geoserver connections related to systems side as i am pure GIS professional.

Now we are going to implement project for entire country (India) with 25 gis layers and satellite images for 100 cities. If we need to restart Geoserver once in two days may not be good idea. Any special configarations/settings is required (hardware side, software side). The production Geoserver will be having the given below linux environment.

Linux CentOS version 5
Geoserver version 2.0.2
Open layers 2.9
Oracle spatial 11 g R2

we are also thinking of alterante map servers because this kind of issues. If anyone knows, please suggest. For country level applications, which will be having an approximate user base of 50,000 day can be handled well by geoserver for the vector layers& satellite images ?

I have seen many posts by andrea& others, guiding users with good insights.

Many Many Thanks in advance for any guidance, suggestions& help.

Regards,
Hari.

--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineering Team Leader
CSIRO Earth Science and Resource Engineering
Australian Resources Research Centre

Christian_Mueller · May 11, 2011, 6:34am

Can you check if you have http sessions. You should have not.

Look here
http://jira.codehaus.org/browse/GEOS-4477

This was fixed with geoserver 2.1 RC5

Quoting Harikumar Reddy <harik.gis@anonymised.com>:

Hi All,

We are using Geoserver 2.0.2 (recently migrated from 2.0.1) and using
Geoserver since version 1.7. we are happy by its map rendering results but
it is going down very often. we are restarting geoserver at least once in
two days, after that it works for 24 hours very well. We are using oracle
spatial 11g database as datastore for gis layers (vector & raster). It is
actually for one state we are using and facing these problems. Any memory
leakage or any other configurations to be done. The total data is around 80
GB ( 78 gb of satellite images & 2 gb gis layers)

I am sorry for asking very basic questions of memory management & geoserver
connections related to systems side as i am pure GIS professional.

Now we are going to implement project for entire country (India) with 25 gis
layers and satellite images for 100 cities. If we need to restart Geoserver
once in two days may not be good idea. Any special configarations/settings
is required (hardware side, software side). The production Geoserver will be
having the given below linux environment.

Linux CentOS version 5
Geoserver version 2.0.2
Open layers 2.9
Oracle spatial 11 g R2

we are also thinking of alterante map servers because this kind of issues.
If anyone knows, please suggest. For country level applications, which will
be having an approximate user base of 50,000 day can be handled well by
geoserver for the vector layers & satellite images ?

I have seen many posts by andrea & others, guiding users with good insights.

Many Many Thanks in advance for any guidance, suggestions & help.

Regards,
Hari.

----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.

Harikumar_Reddy · May 11, 2011, 7:44am

Hi Ben Caradoc-Davies, christian.mueller

Thanks for the very quick responses.

As per my observations, Most of the times, in Geoserver errors of heap memory present. I didnot check logs of the tomcat (we are using
tomcat 6 & tcserver).

I will provide required details & logs by contacting system admin team.

Thankyou verymuch.

Hari.

On Wed, May 11, 2011 at 12:04 PM, <christian.mueller@anonymised.com> wrote:

Can you check if you have http sessions. You should have not.

Look here
http://jira.codehaus.org/browse/GEOS-4477

This was fixed with geoserver 2.1 RC5

Quoting Harikumar Reddy <harik.gis@anonymised.com>:

Hi All,

We are using Geoserver 2.0.2 (recently migrated from 2.0.1) and using
Geoserver since version 1.7. we are happy by its map rendering results but
it is going down very often. we are restarting geoserver at least once in
two days, after that it works for 24 hours very well. We are using oracle
spatial 11g database as datastore for gis layers (vector & raster). It is
actually for one state we are using and facing these problems. Any memory
leakage or any other configurations to be done. The total data is around 80
GB ( 78 gb of satellite images & 2 gb gis layers)

I am sorry for asking very basic questions of memory management & geoserver
connections related to systems side as i am pure GIS professional.

Now we are going to implement project for entire country (India) with 25 gis
layers and satellite images for 100 cities. If we need to restart Geoserver
once in two days may not be good idea. Any special configarations/settings
is required (hardware side, software side). The production Geoserver will be
having the given below linux environment.

Linux CentOS version 5
Geoserver version 2.0.2
Open layers 2.9
Oracle spatial 11 g R2

we are also thinking of alterante map servers because this kind of issues.
If anyone knows, please suggest. For country level applications, which will
be having an approximate user base of 50,000 day can be handled well by
geoserver for the vector layers & satellite images ?

I have seen many posts by andrea & others, guiding users with good insights.

Many Many Thanks in advance for any guidance, suggestions & help.

Regards,
Hari.

This message was sent using IMP, the Internet Messaging Program.

Ben_Caradoc-Davies · May 11, 2011, 8:04am

You can increase the maximum size of your heap by setting the JVM option -Xmx512m (or higher). If you are using a 64-bit JVM and have less than 32 GB RAM, -XX:+UseCompressedOops is highly recommended. These are known to reduce your risk of heap exhaustion.

On 11/05/11 15:44, Harikumar Reddy wrote:

Hi Ben Caradoc-Davies, christian.mueller

Thanks for the very quick responses.

As per my observations, Most of the times, in Geoserver errors of heap memory present. I didnot check logs of the tomcat (we are using
tomcat 6& tcserver).

I will provide required details& logs by contacting system admin team.

Thankyou verymuch.

Hari.

On Wed, May 11, 2011 at 12:04 PM,<christian.mueller@anonymised.com<mailto:christian.mueller@anonymised.com>> wrote:
Can you check if you have http sessions. You should have not.

Look here
http://jira.codehaus.org/browse/GEOS-4477

This was fixed with geoserver 2.1 RC5

Quoting Harikumar Reddy<harik.gis@anonymised.com<mailto:harik.gis@anonymised.com>>:

Hi All,

We are using Geoserver 2.0.2 (recently migrated from 2.0.1) and using
Geoserver since version 1.7. we are happy by its map rendering results but
it is going down very often. we are restarting geoserver at least once in
two days, after that it works for 24 hours very well. We are using oracle
spatial 11g database as datastore for gis layers (vector& raster). It is
actually for one state we are using and facing these problems. Any memory
leakage or any other configurations to be done. The total data is around 80
GB ( 78 gb of satellite images& 2 gb gis layers)

I am sorry for asking very basic questions of memory management& geoserver
connections related to systems side as i am pure GIS professional.

Now we are going to implement project for entire country (India) with 25 gis
layers and satellite images for 100 cities. If we need to restart Geoserver
once in two days may not be good idea. Any special configarations/settings
is required (hardware side, software side). The production Geoserver will be
having the given below linux environment.

Linux CentOS version 5
Geoserver version 2.0.2
Open layers 2.9
Oracle spatial 11 g R2

we are also thinking of alterante map servers because this kind of issues.
If anyone knows, please suggest. For country level applications, which will
be having an approximate user base of 50,000 day can be handled well by
geoserver for the vector layers& satellite images ?

I have seen many posts by andrea& others, guiding users with good insights.

Many Many Thanks in advance for any guidance, suggestions& help.

Regards,
Hari.

----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.

--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineering Team Leader
CSIRO Earth Science and Resource Engineering
Australian Resources Research Centre

Harikumar_Reddy · May 11, 2011, 11:52am

Hi Ben,

Below i am providing answers for your questions (apart from that i have some doubts also written).

Did you observed any limitations (known or unknown limitations like howmuch maximum data it can support or maximum number of workspaces/datastore we can able to create (raster& vectors)/how many gis layers can be configured in one geoserver) for geoserver version 2.0.2 & other newer versions ?

We have deployed a webgis application a month before (vote.mapview.in), which was working very fine and didnt have any issues till now with Geoserver (but it has very less gis data layers).

Please find below required information to analyse problem.

What Java implementation (and version) are you using? What JVM options?

"java version “1.6.0_23”

Java™ SE Runtime Environment (build 1.6.0_23-b05)

Java HotSpot™ 64-Bit Server VM (build 19.0-b09, mixed mode)"

What servlet container? Tomcat?

We are using Tomcat 6 servlet container in spring source tc server cloud environment

Are you using Apache HTTPD in front of Tomcat?

Not using Apache HTTPD infront of Tomcat.

What are the symptoms of “going down”? Do you get any response? From Apache? Tomcat?

Geoserver log file piled up and occupied 250 MB disk space. So i have carefully extracted log of may 11th for geoserver (ver 2.0.1) and sending as an attachment.

Do you get anything in the logs (Apache, Tomcat or GeoServer)?

Yes, attached logs of tomcat & geoserver.

Do you see heap errors in the logs?
Heap we have increased, so now a days we are not getting.
Do you get any mysterious database errors (ORA-something) or hangs?

Not exactly ORA-xxxxx errors, but i suspect db related issues might also happen.

Is there a firewall between GeoServer and Oracle? from the app-schema manual but relevant to all users:

There is no firewall between Geoserver & Oracle, because of testing & connection problems, removed firewall as of now.
Have you tried turning on VERBOSE logging in GeoServer to diagnose the problem?

Yes, VERBOSE logging has been already set. It is become easier to analyse logs after this setting change.

Thanks,
Hari.

On Wed, May 11, 2011 at 1:14 PM, Harikumar Reddy <harik.gis@anonymised.com> wrote:

Hi Ben Caradoc-Davies, christian.mueller

Thanks for the very quick responses.

As per my observations, Most of the times, in Geoserver errors of heap memory present. I didnot check logs of the tomcat (we are using
tomcat 6 & tcserver).

I will provide required details & logs by contacting system admin team.

Thankyou verymuch.

Hari.

On Wed, May 11, 2011 at 12:04 PM, <christian.mueller@anonymised.com> wrote:

Can you check if you have http sessions. You should have not.

Look here
http://jira.codehaus.org/browse/GEOS-4477

This was fixed with geoserver 2.1 RC5

Quoting Harikumar Reddy <harik.gis@anonymised.com>:

Hi All,

We are using Geoserver 2.0.2 (recently migrated from 2.0.1) and using
Geoserver since version 1.7. we are happy by its map rendering results but
it is going down very often. we are restarting geoserver at least once in
two days, after that it works for 24 hours very well. We are using oracle
spatial 11g database as datastore for gis layers (vector & raster). It is
actually for one state we are using and facing these problems. Any memory
leakage or any other configurations to be done. The total data is around 80
GB ( 78 gb of satellite images & 2 gb gis layers)

I am sorry for asking very basic questions of memory management & geoserver
connections related to systems side as i am pure GIS professional.

Now we are going to implement project for entire country (India) with 25 gis
layers and satellite images for 100 cities. If we need to restart Geoserver
once in two days may not be good idea. Any special configarations/settings
is required (hardware side, software side). The production Geoserver will be
having the given below linux environment.

Linux CentOS version 5
Geoserver version 2.0.2
Open layers 2.9
Oracle spatial 11 g R2

we are also thinking of alterante map servers because this kind of issues.
If anyone knows, please suggest. For country level applications, which will
be having an approximate user base of 50,000 day can be handled well by
geoserver for the vector layers & satellite images ?

I have seen many posts by andrea & others, guiding users with good insights.

Many Many Thanks in advance for any guidance, suggestions & help.

Regards,
Hari.

This message was sent using IMP, the Internet Messaging Program.

Ben_Caradoc-Davies · May 12, 2011, 2:23am

Hari's full logs are too large for the list, so I have included some excepts of the geoserver, catalina, and localhost logs. Ideas, anyone?

The geoserver log contains hundreds of these:

11 May 04:02:17 INFO [geoserver.ows] - Problem writing exception information back to calling client:
ClientAbortException: java.net.SocketException: Broken pipe
  at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:358)
  at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java:432)
  at org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:309)
  at org.apache.catalina.connector.OutputBuffer.flush(OutputBuffer.java:288)
  at org.apache.catalina.connector.CoyoteOutputStream.flush(CoyoteOutputStream.java:98)
  at javax.imageio.stream.MemoryCacheImageOutputStream.flushBefore(MemoryCacheImageOutputStream.java:177)
  at javax.imageio.stream.MemoryCacheImageOutputStream.close(MemoryCacheImageOutputStream.java:163)
  at javax.imageio.ImageIO.write(ImageIO.java:1567)
  at org.geoserver.wms.WmsExceptionHandler.handleServiceException(WmsExceptionHandler.java:96)
  at org.geoserver.ows.Dispatcher.handleServiceException(Dispatcher.java:1346)
  at org.geoserver.ows.Dispatcher.exception(Dispatcher.java:1301)
  at org.geoserver.ows.Dispatcher.handleRequestInternal(Dispatcher.java:241)
  at org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:153)
  at org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:48)
  at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:875)
  at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:809)
  at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:571)
  at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:501)
  at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
  at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
  at org.vfny.geoserver.filters.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:108)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
  at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:265)
  at org.acegisecurity.intercept.web.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:107)
  at org.acegisecurity.intercept.web.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:72)
  at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
  at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:124)
  at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
  at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
  at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
  at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:174)
  at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
  at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
  at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
  at org.acegisecurity.util.FilterChainProxy.doFilter(FilterChainProxy.java:149)
  at org.acegisecurity.util.FilterToBeanProxy.doFilter(FilterToBeanProxy.java:98)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
  at org.geoserver.filters.LoggingFilter.doFilter(LoggingFilter.java:73)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
  at org.geoserver.filters.ReverseProxyFilter.doFilter(ReverseProxyFilter.java:183)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
  at org.geoserver.filters.GZIPFilter.doFilter(GZIPFilter.java:47)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
  at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
  at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
  at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
  at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
  at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
  at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
  at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
  at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
  at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:409)
  at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
  at java.lang.Thread.run(Thread.java:662)
Caused by: java.net.SocketException: Broken pipe
  at java.net.SocketOutputStream.socketWrite0(Native Method)
  at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
  at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
  at org.apache.coyote.http11.InternalOutputBuffer.realWriteBytes(InternalOutputBuffer.java:741)
  at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java:432)
  at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java:347)
  at org.apache.coyote.http11.InternalOutputBuffer$OutputStreamOutputBuffer.doWrite(InternalOutputBuffer.java:765)
  at org.apache.coyote.http11.filters.ChunkedOutputFilter.doWrite(ChunkedOutputFilter.java:124)
  at org.apache.coyote.http11.InternalOutputBuffer.doWrite(InternalOutputBuffer.java:574)
  at org.apache.coyote.Response.doWrite(Response.java:560)
  at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:353)
  ... 61 more
11 May 04:02:17 ERROR [geoserver.ows] -
ClientAbortException: java.net.SocketException: Connection reset
  at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:358)
  at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java:352)
  at org.apache.catalina.connector.OutputBuffer.writeBytes(OutputBuffer.java:381)
  at org.apache.catalina.connector.OutputBuffer.write(OutputBuffer.java:370)
  at org.apache.catalina.connector.CoyoteOutputStream.write(CoyoteOutputStream.java:89)
  at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
  at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
  at org.vfny.geoserver.servlets.FileStrategy.flush(FileStrategy.java:126)
  at org.geoserver.ows.Dispatcher.response(Dispatcher.java:727)
  at org.geoserver.ows.Dispatcher.handleRequestInternal(Dispatcher.java:233)
  at org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:153)
  at org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:48)
  at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:875)
  at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:809)
  at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:571)
  at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:501)
  at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
  at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
  at org.vfny.geoserver.filters.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:108)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
  at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:265)
  at org.acegisecurity.intercept.web.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:107)
  at org.acegisecurity.intercept.web.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:72)
  at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
  at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:124)
  at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
  at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
  at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
  at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:174)
  at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
  at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
  at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
  at org.acegisecurity.util.FilterChainProxy.doFilter(FilterChainProxy.java:149)
  at org.acegisecurity.util.FilterToBeanProxy.doFilter(FilterToBeanProxy.java:98)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
  at org.geoserver.filters.LoggingFilter.doFilter(LoggingFilter.java:73)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
  at org.geoserver.filters.ReverseProxyFilter.doFilter(ReverseProxyFilter.java:183)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
  at org.geoserver.filters.GZIPFilter.doFilter(GZIPFilter.java:47)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
  at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
  at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
  at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
  at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
  at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
  at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
  at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
  at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
  at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:409)
  at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
  at java.lang.Thread.run(Thread.java:662)
Caused by: java.net.SocketException: Connection reset
  at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:96)
  at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
  at org.apache.coyote.http11.InternalOutputBuffer.realWriteBytes(InternalOutputBuffer.java:741)
  at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java:432)
  at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java:347)
  at org.apache.coyote.http11.InternalOutputBuffer$OutputStreamOutputBuffer.doWrite(InternalOutputBuffer.java:765)
  at org.apache.coyote.http11.filters.ChunkedOutputFilter.doWrite(ChunkedOutputFilter.java:126)
  at org.apache.coyote.http11.InternalOutputBuffer.doWrite(InternalOutputBuffer.java:574)
  at org.apache.coyote.Response.doWrite(Response.java:560)
  at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:353)
  ... 59 more

catalina log contains:

10-May-2011 09:27:25.987 SEVERE org.apache.catalina.connector.CoyoteAdapter.service An exception or error occurred in the container during the request processing
  java.lang.IllegalStateException: Cannot call sendRedirect() after the response has been committed
  at org.apache.catalina.connector.Response.sendRedirect(Response.java:1331)
  at org.apache.catalina.connector.CoyoteAdapter.postParseRequest(CoyoteAdapter.java:509)
  at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:296)
  at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
  at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
  at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:409)
  at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
  at java.lang.Thread.run(Thread.java:662)

localhost log contains:

10-May-2011 09:27:31.272 SEVERE org.apache.catalina.core.StandardWrapperValve.invoke Servlet.service() for servlet dispatcher threw exception
  java.lang.IllegalStateException
  at org.apache.catalina.connector.ResponseFacade.sendRedirect(ResponseFacade.java:435)
  at javax.servlet.http.HttpServletResponseWrapper.sendRedirect(HttpServletResponseWrapper.java:126)
  at org.acegisecurity.context.HttpSessionContextIntegrationFilter$OnRedirectUpdateSessionResponseWrapper.sendRedirect(HttpSessionContextIntegrationFilter.java:525)
  at org.apache.wicket.protocol.http.WicketFilter.doGet(WicketFilter.java:281)
  at org.apache.wicket.protocol.http.WicketServlet.doGet(WicketServlet.java:124)
  at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
  at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
  at org.springframework.web.servlet.mvc.ServletWrappingController.handleRequestInternal(ServletWrappingController.java:158)
  at org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:153)
  at org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:48)
  at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:875)
  at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:809)
  at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:571)
  at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:501)
  at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
  at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
  at org.vfny.geoserver.filters.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:108)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
  at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:265)
  at org.acegisecurity.intercept.web.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:107)
  at org.acegisecurity.intercept.web.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:72)
  at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
  at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:124)
  at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
  at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
  at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
  at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:135)
  at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
  at org.acegisecurity.wrapper.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:81)
  at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
  at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
  at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
  at org.acegisecurity.ui.logout.LogoutFilter.doFilter(LogoutFilter.java:110)
  at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
  at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
  at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
  at org.acegisecurity.util.FilterChainProxy.doFilter(FilterChainProxy.java:149)
  at org.acegisecurity.util.FilterToBeanProxy.doFilter(FilterToBeanProxy.java:98)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
  at org.geoserver.filters.LoggingFilter.doFilter(LoggingFilter.java:73)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
  at org.geoserver.filters.ReverseProxyFilter.doFilter(ReverseProxyFilter.java:183)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
  at org.geoserver.filters.GZIPFilter.doFilter(GZIPFilter.java:47)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
  at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
  at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
  at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
  at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
  at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
  at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
  at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
  at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
  at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:409)
  at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
  at java.lang.Thread.run(Thread.java:662)

On 11/05/11 19:53, Harikumar Reddy wrote:

Hi Ben,

Below i am providing answers for your questions (apart from that i have some doubts also written).

Did you observed any limitations (known or unknown limitations like howmuch maximum data it can support or maximum number of workspaces/datastore we can able to create (raster& vectors)/how many gis layers can be configured in one geoserver) for geoserver version 2.0.2& other newer versions ?

We have deployed a webgis application a month before (vote.mapview.in<http://vote.mapview.in>), which was working very fine and didnt have any issues till now with Geoserver (but it has very less gis data layers).

Please find below required information to analyse problem.

- What Java implementation (and version) are you using? What JVM options?

"java version "1.6.0_23"
Java(TM) SE Runtime Environment (build 1.6.0_23-b05)
Java HotSpot(TM) 64-Bit Server VM (build 19.0-b09, mixed mode)"

- What servlet container? Tomcat?

We are using Tomcat 6 servlet container in spring source tc server cloud environment

- Are you using Apache HTTPD in front of Tomcat?

Not using Apache HTTPD infront of Tomcat.

- What are the symptoms of "going down"? Do you get any response? From Apache? Tomcat?

Geoserver log file piled up and occupied 250 MB disk space. So i have carefully extracted log of may 11th for geoserver (ver 2.0.1) and sending as an attachment.

- Do you get anything in the logs (Apache, Tomcat or GeoServer)?

Yes, attached logs of tomcat& geoserver.

- Do you see heap errors in the logs?
Heap we have increased, so now a days we are not getting.

- Do you get any mysterious database errors (ORA-something) or hangs?

Not exactly ORA-xxxxx errors, but i suspect db related issues might also happen.

- Is there a firewall between GeoServer and Oracle? from the app-schema manual but relevant to all users:
<http://docs.geoserver.org/latest/en/user/data/app-schema/data-stores.html#jndi>
There is no firewall between Geoserver& Oracle, because of testing& connection problems, removed firewall as of now.

- Have you tried turning on VERBOSE logging in GeoServer to diagnose the problem?

Yes, VERBOSE logging has been already set. It is become easier to analyse logs after this setting change.

Hi Christian

Thanks,
Hari.

On Wed, May 11, 2011 at 1:14 PM, Harikumar Reddy<harik.gis@anonymised.com<mailto:harik.gis@anonymised.com>> wrote:
Hi Ben Caradoc-Davies, christian.mueller

Thanks for the very quick responses.

As per my observations, Most of the times, in Geoserver errors of heap memory present. I didnot check logs of the tomcat (we are using
tomcat 6& tcserver).

I will provide required details& logs by contacting system admin team.

Thankyou verymuch.

Hari.

On Wed, May 11, 2011 at 12:04 PM,<christian.mueller@anonymised.com<mailto:christian.mueller@anonymised.com>> wrote:
Can you check if you have http sessions. You should have not.

Look here
http://jira.codehaus.org/browse/GEOS-4477

This was fixed with geoserver 2.1 RC5

Quoting Harikumar Reddy<harik.gis@anonymised.com<mailto:harik.gis@anonymised.com>>:

Hi All,

We are using Geoserver 2.0.2 (recently migrated from 2.0.1) and using
Geoserver since version 1.7. we are happy by its map rendering results but
it is going down very often. we are restarting geoserver at least once in
two days, after that it works for 24 hours very well. We are using oracle
spatial 11g database as datastore for gis layers (vector& raster). It is
actually for one state we are using and facing these problems. Any memory
leakage or any other configurations to be done. The total data is around 80
GB ( 78 gb of satellite images& 2 gb gis layers)

I am sorry for asking very basic questions of memory management& geoserver
connections related to systems side as i am pure GIS professional.

Now we are going to implement project for entire country (India) with 25 gis
layers and satellite images for 100 cities. If we need to restart Geoserver
once in two days may not be good idea. Any special configarations/settings
is required (hardware side, software side). The production Geoserver will be
having the given below linux environment.

Linux CentOS version 5
Geoserver version 2.0.2
Open layers 2.9
Oracle spatial 11 g R2

we are also thinking of alterante map servers because this kind of issues.
If anyone knows, please suggest. For country level applications, which will
be having an approximate user base of 50,000 day can be handled well by
geoserver for the vector layers& satellite images ?

I have seen many posts by andrea& others, guiding users with good insights.

Many Many Thanks in advance for any guidance, suggestions& help.

Regards,
Hari.

----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.

--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineering Team Leader
CSIRO Earth Science and Resource Engineering
Australian Resources Research Centre

Andrea_Aime4 · May 12, 2011, 8:37am

On Thu, May 12, 2011 at 4:23 AM, Ben Caradoc-Davies
<Ben.Caradoc-Davies@anonymised.com> wrote:

Hari's full logs are too large for the list, so I have included some
excepts of the geoserver, catalina, and localhost logs. Ideas, anyone?

The geoserver log contains hundreds of these:

11 May 04:02:17 INFO [geoserver.ows] - Problem writing exception
information back to calling client:
ClientAbortException: java.net.SocketException: Broken pipe
at

This is a client shutting down the connection in GS face. Nothing to worry about
(indeed it is reported just at INFO level, not warn or error)

10-May-2011 09:27:25.987 SEVERE
org.apache.catalina.connector.CoyoteAdapter.service An exception or
error occurred in the container during the request processing
java.lang.IllegalStateException: Cannot call sendRedirect() after the
response has been committed
at org.apache.catalina.connector.Response.sendRedirect(Response.java:1331)
at
org.apache.catalina.connector.CoyoteAdapter.postParseRequest(CoyoteAdapter.java:509)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:296)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
at
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:409)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)

This one might have some merit, thought only the GUI uses redirects
afaik, so not
a concern for services.

localhost log contains:

10-May-2011 09:27:31.272 SEVERE
org.apache.catalina.core.StandardWrapperValve.invoke Servlet.service()
for servlet dispatcher threw exception
java.lang.IllegalStateException
at
org.apache.catalina.connector.ResponseFacade.sendRedirect(ResponseFacade.java:435)
at
javax.servlet.http.HttpServletResponseWrapper.sendRedirect(HttpServletResponseWrapper.java:126)
at

Same as before, a failing redirect, but not sure why.

None of these error are patologic.

If the GS was not responding at some point and then started responding
afterwards it might just be sign of an overload, too much requests
in parallel for the connection pool to handle them in parallel, maybe
some of them very long lived and were keeping the connections busy
for a long time.

If this is the case it's more a case of dimensioning the connection pool
according to the load one wants to handle, setting the service limits
accordingly, and using control-flow to make sure GeoServer does not
try to bite more than it can chew.

Cheers
Andrea

--
-------------------------------------------------------
Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

-------------------------------------------------------

Harikumar_Reddy · May 12, 2011, 8:55am

Hi Andrea,

Thnakyou verymuch for control flow configartion advise. I will definitely implement.

Regards,
Hari.

On Thu, May 12, 2011 at 2:07 PM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Thu, May 12, 2011 at 4:23 AM, Ben Caradoc-Davies
Ben.Caradoc-Davies@anonymised.com wrote:

Hari’s full logs are too large for the list, so I have included some
excepts of the geoserver, catalina, and localhost logs. Ideas, anyone?

The geoserver log contains hundreds of these:

11 May 04:02:17 INFO [geoserver.ows] - Problem writing exception
information back to calling client:
ClientAbortException: java.net.SocketException: Broken pipe
at

This is a client shutting down the connection in GS face. Nothing to worry about
(indeed it is reported just at INFO level, not warn or error)

10-May-2011 09:27:25.987 SEVERE
org.apache.catalina.connector.CoyoteAdapter.service An exception or
error occurred in the container during the request processing
java.lang.IllegalStateException: Cannot call sendRedirect() after the
response has been committed
at org.apache.catalina.connector.Response.sendRedirect(Response.java:1331)
at
org.apache.catalina.connector.CoyoteAdapter.postParseRequest(CoyoteAdapter.java:509)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:296)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
at
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:409)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)

This one might have some merit, thought only the GUI uses redirects
afaik, so not
a concern for services.

localhost log contains:

10-May-2011 09:27:31.272 SEVERE
org.apache.catalina.core.StandardWrapperValve.invoke Servlet.service()
for servlet dispatcher threw exception
java.lang.IllegalStateException
at
org.apache.catalina.connector.ResponseFacade.sendRedirect(ResponseFacade.java:435)
at
javax.servlet.http.HttpServletResponseWrapper.sendRedirect(HttpServletResponseWrapper.java:126)
at

Same as before, a failing redirect, but not sure why.

None of these error are patologic.

If the GS was not responding at some point and then started responding
afterwards it might just be sign of an overload, too much requests
in parallel for the connection pool to handle them in parallel, maybe
some of them very long lived and were keeping the connections busy
for a long time.

If this is the case it’s more a case of dimensioning the connection pool
according to the load one wants to handle, setting the service limits
accordingly, and using control-flow to make sure GeoServer does not
try to bite more than it can chew.

Cheers
Andrea

–

Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

Chris_Holmes1 · May 12, 2011, 6:38pm

Hey Hari, vote.mapview.in looks cool. Have you considered using GeoWebCache for it? Looks like you’re doing large, single tile requests. If you stick GWC in front of it then you could put a lot less of a load on the server, since most places that people look at would get cached.

C

On Wed, May 11, 2011 at 7:52 AM, Harikumar Reddy <harik.gis@anonymised.com> wrote:

Hi Ben,

Below i am providing answers for your questions (apart from that i have some doubts also written).

Did you observed any limitations (known or unknown limitations like howmuch maximum data it can support or maximum number of workspaces/datastore we can able to create (raster& vectors)/how many gis layers can be configured in one geoserver) for geoserver version 2.0.2 & other newer versions ?

We have deployed a webgis application a month before (vote.mapview.in), which was working very fine and didnt have any issues till now with Geoserver (but it has very less gis data layers).

Please find below required information to analyse problem.

What Java implementation (and version) are you using? What JVM options?

"java version “1.6.0_23”

Java™ SE Runtime Environment (build 1.6.0_23-b05)

Java HotSpot™ 64-Bit Server VM (build 19.0-b09, mixed mode)"

What servlet container? Tomcat?

We are using Tomcat 6 servlet container in spring source tc server cloud environment

Are you using Apache HTTPD in front of Tomcat?

Not using Apache HTTPD infront of Tomcat.

What are the symptoms of “going down”? Do you get any response? From Apache? Tomcat?

Geoserver log file piled up and occupied 250 MB disk space. So i have carefully extracted log of may 11th for geoserver (ver 2.0.1) and sending as an attachment.

Do you get anything in the logs (Apache, Tomcat or GeoServer)?

Yes, attached logs of tomcat & geoserver.

Do you see heap errors in the logs?

Heap we have increased, so now a days we are not getting.

Do you get any mysterious database errors (ORA-something) or hangs?

Not exactly ORA-xxxxx errors, but i suspect db related issues might also happen.

Is there a firewall between GeoServer and Oracle? from the app-schema manual but relevant to all users:

There is no firewall between Geoserver & Oracle, because of testing & connection problems, removed firewall as of now.

Have you tried turning on VERBOSE logging in GeoServer to diagnose the problem?

Yes, VERBOSE logging has been already set. It is become easier to analyse logs after this setting change.

Thanks,
Hari.

On Wed, May 11, 2011 at 1:14 PM, Harikumar Reddy <harik.gis@anonymised.com> wrote:

Hi Ben Caradoc-Davies, christian.mueller

Thanks for the very quick responses.

As per my observations, Most of the times, in Geoserver errors of heap memory present. I didnot check logs of the tomcat (we are using
tomcat 6 & tcserver).

I will provide required details & logs by contacting system admin team.

Thankyou verymuch.

Hari.

On Wed, May 11, 2011 at 12:04 PM, <christian.mueller@anonymised.com> wrote:

Can you check if you have http sessions. You should have not.

Look here
http://jira.codehaus.org/browse/GEOS-4477

This was fixed with geoserver 2.1 RC5

Quoting Harikumar Reddy <harik.gis@anonymised.com>:

Hi All,

We are using Geoserver 2.0.2 (recently migrated from 2.0.1) and using
Geoserver since version 1.7. we are happy by its map rendering results but
it is going down very often. we are restarting geoserver at least once in
two days, after that it works for 24 hours very well. We are using oracle
spatial 11g database as datastore for gis layers (vector & raster). It is
actually for one state we are using and facing these problems. Any memory
leakage or any other configurations to be done. The total data is around 80
GB ( 78 gb of satellite images & 2 gb gis layers)

I am sorry for asking very basic questions of memory management & geoserver
connections related to systems side as i am pure GIS professional.

Now we are going to implement project for entire country (India) with 25 gis
layers and satellite images for 100 cities. If we need to restart Geoserver
once in two days may not be good idea. Any special configarations/settings
is required (hardware side, software side). The production Geoserver will be
having the given below linux environment.

Linux CentOS version 5
Geoserver version 2.0.2
Open layers 2.9
Oracle spatial 11 g R2

we are also thinking of alterante map servers because this kind of issues.
If anyone knows, please suggest. For country level applications, which will
be having an approximate user base of 50,000 day can be handled well by
geoserver for the vector layers & satellite images ?

I have seen many posts by andrea & others, guiding users with good insights.

Many Many Thanks in advance for any guidance, suggestions & help.

Regards,
Hari.

This message was sent using IMP, the Internet Messaging Program.

Achieve unprecedented app performance and reliability
What every C/C++ and Fortran developer should know.
Learn how Intel has extended the reach of its next-generation tools
to help boost performance applications - inlcuding clusters.
http://p.sf.net/sfu/intel-dev2devmay

Geoserver-users mailing list
Geoserver-users@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Andrea_Aime4 · May 12, 2011, 7:19pm

On Thu, May 12, 2011 at 8:38 PM, Chris Holmes <cholmes@anonymised.com> wrote:

Hey Hari, vote.mapview.in looks cool. Have you considered using GeoWebCache
for it? Looks like you're doing large, single tile requests. If you stick
GWC in front of it then you could put a lot less of a load on the server,
since most places that people look at would get cached.

Just followed the link and got an exception stating that rendering the map
would have used 80MB and the limit was 25MB.
Wow, you must have used a ton of FeatureTypeStyle declarations in your
style, I was just using a 15" notebook, the request couldn't have been
larger than 900*900.
Doing a style with a ton of FeatureTypeStyle elements will require either
a lot of passes over the data or (what GS does) allocating many rendering
surfaces in parallel. Better rework the SLD to use a small number of
FeatureTypeStyle in it: consider this, how many Z levels do you actually
need for a specific layer? That's the number of FeatureTypeStyle
elements you actually need.

Cheers
Andrea

--
-------------------------------------------------------
Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

-------------------------------------------------------

Harikumar_Reddy · May 13, 2011, 3:47am

Hi Chris,

Thanks for suggestions and webgis application feedback on vote.mapview.in. On SLD of this webapplication, we did lot of work to provide beautiful maps view to indian citizens/ general public. After this application launch, observed map serving performance of geoserver and have decided to continue with Geoserver forever for all applications which are enterprise level/country based data serving applications. but still lot of study and observation is on the way.

currently we have data for one state of India (Kerala), we are planning to for entire India soon. That is the reason we are taking experts (in my view, experts will be actual users who are encountering problems everyday and resolving by their own knowledge and with help of similar kind of users knowledge base, one of the supprt people/forums is geoserver-users@anonymised.comrceforge.net).

As you suggested, we are already using GWC, internally for many applications which are in development and accessing layers with IP address:Port number/geowebcache/service/wms. When we are seeding layers, if it take much time, if we cancel operation, observed that geoserver referring to geowebcache is also going down. need to study a lot on this process. once it is done, we can implement GWC for all applications launched for public.

Hi Andrea,

As usual, Thankyou for useful insights to increase performance& reliability, applications availability. Could you please guide/advise me on below points.

Just followed the link and got an exception stating that rendering the map
would have used 80MB and the limit was 25MB.

Yes, it was happening sometime (How can we manage this kind of memory issues). From default 16 MB, i have increased to 25 MB. Is it really more than 25 MB of wms image that has been rendering in web page ? how can be the maximum size…

Geoserver is serving layers for other applications also, but there are no satellite images/rasters. The total GIS layers on disk space may be around 35 MB maximum, stored in oracle tables( i have conveyed only map data and there are other tables for election results and candidates/constituency profiles which are not GIS layers).

Wow, you must have used a ton of FeatureTypeStyle declarations in your
style, I was just using a 15" notebook, the request couldn’t have been
larger than 900*900.

Could you please explain, how to implement this?

Doing a style with a ton of FeatureTypeStyle elements will require either
a lot of passes over the data or (what GS does) allocating many rendering
surfaces in parallel. Better rework the SLD to use a small number of
FeatureTypeStyle in it: consider this, how many Z levels do you actually
need for a specific layer? That’s the number of FeatureTypeStyle
elements you actually need.

Its true that Road SLD xml file is having more than 3000 lines description(i need to redefine it, will do asap), we have a large road dataset, which i will devide into national highway, state highway, DMR (district major roads), streets (5 types). I will add it as group layer, and access in web applications.

Thanks,
Hari.

On Fri, May 13, 2011 at 12:08 AM, Chris Holmes <cholmes@anonymised.com> wrote:

Hey Hari, vote.mapview.in looks cool. Have you considered using GeoWebCache for it? Looks like you’re doing large, single tile requests. If you stick GWC in front of it then you could put a lot less of a load on the server, since most places that people look at would get cached.

C

On Wed, May 11, 2011 at 7:52 AM, Harikumar Reddy <harik.gis@anonymised.com> wrote:

Hi Ben,

Below i am providing answers for your questions (apart from that i have some doubts also written).

Did you observed any limitations (known or unknown limitations like howmuch maximum data it can support or maximum number of workspaces/datastore we can able to create (raster& vectors)/how many gis layers can be configured in one geoserver) for geoserver version 2.0.2 & other newer versions ?

We have deployed a webgis application a month before (vote.mapview.in), which was working very fine and didnt have any issues till now with Geoserver (but it has very less gis data layers).

Please find below required information to analyse problem.

What Java implementation (and version) are you using? What JVM options?

"java version “1.6.0_23”

Java™ SE Runtime Environment (build 1.6.0_23-b05)

Java HotSpot™ 64-Bit Server VM (build 19.0-b09, mixed mode)"

What servlet container? Tomcat?

We are using Tomcat 6 servlet container in spring source tc server cloud environment

Are you using Apache HTTPD in front of Tomcat?

Not using Apache HTTPD infront of Tomcat.

What are the symptoms of “going down”? Do you get any response? From Apache? Tomcat?

Geoserver log file piled up and occupied 250 MB disk space. So i have carefully extracted log of may 11th for geoserver (ver 2.0.1) and sending as an attachment.

Do you get anything in the logs (Apache, Tomcat or GeoServer)?

Yes, attached logs of tomcat & geoserver.

Do you see heap errors in the logs?

Heap we have increased, so now a days we are not getting.

Do you get any mysterious database errors (ORA-something) or hangs?

Not exactly ORA-xxxxx errors, but i suspect db related issues might also happen.

Is there a firewall between GeoServer and Oracle? from the app-schema manual but relevant to all users:

There is no firewall between Geoserver & Oracle, because of testing & connection problems, removed firewall as of now.

Have you tried turning on VERBOSE logging in GeoServer to diagnose the problem?

Yes, VERBOSE logging has been already set. It is become easier to analyse logs after this setting change.

Thanks,
Hari.

On Wed, May 11, 2011 at 1:14 PM, Harikumar Reddy <harik.gis@anonymised.com> wrote:

Hi Ben Caradoc-Davies, christian.mueller

Thanks for the very quick responses.

As per my observations, Most of the times, in Geoserver errors of heap memory present. I didnot check logs of the tomcat (we are using
tomcat 6 & tcserver).

I will provide required details & logs by contacting system admin team.

Thankyou verymuch.

Hari.

On Wed, May 11, 2011 at 12:04 PM, <christian.mueller@anonymised.com> wrote:

Can you check if you have http sessions. You should have not.

Look here
http://jira.codehaus.org/browse/GEOS-4477

This was fixed with geoserver 2.1 RC5

Quoting Harikumar Reddy <harik.gis@anonymised.com>:

Hi All,

We are using Geoserver 2.0.2 (recently migrated from 2.0.1) and using
Geoserver since version 1.7. we are happy by its map rendering results but
it is going down very often. we are restarting geoserver at least once in
two days, after that it works for 24 hours very well. We are using oracle
spatial 11g database as datastore for gis layers (vector & raster). It is
actually for one state we are using and facing these problems. Any memory
leakage or any other configurations to be done. The total data is around 80
GB ( 78 gb of satellite images & 2 gb gis layers)

I am sorry for asking very basic questions of memory management & geoserver
connections related to systems side as i am pure GIS professional.

Now we are going to implement project for entire country (India) with 25 gis
layers and satellite images for 100 cities. If we need to restart Geoserver
once in two days may not be good idea. Any special configarations/settings
is required (hardware side, software side). The production Geoserver will be
having the given below linux environment.

Linux CentOS version 5
Geoserver version 2.0.2
Open layers 2.9
Oracle spatial 11 g R2

we are also thinking of alterante map servers because this kind of issues.
If anyone knows, please suggest. For country level applications, which will
be having an approximate user base of 50,000 day can be handled well by
geoserver for the vector layers & satellite images ?

I have seen many posts by andrea & others, guiding users with good insights.

Many Many Thanks in advance for any guidance, suggestions & help.

Regards,
Hari.

This message was sent using IMP, the Internet Messaging Program.

Achieve unprecedented app performance and reliability
What every C/C++ and Fortran developer should know.
Learn how Intel has extended the reach of its next-generation tools
to help boost performance applications - inlcuding clusters.
http://p.sf.net/sfu/intel-dev2devmay

Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Ben_Caradoc-Davies · May 13, 2011, 4:09am

Hari,

you may also benefit from Andrea Aime's FOSS4G 2009 presentation "GeoServer in Production":

Here is the FOSS4G information for this presentation:
http://2009.foss4g.org/presentations/#presentation_163

The slides are an OpenOffice Impress document:
http://download.osgeo.org/osgeo/foss4g/2009/SPREP/3Fri/Parkside%20Auditorium/1300/fri%20paud%201400%20aime.odp

If you don't have OpenOffice, get it here:
http://www.openoffice.org/

Or you can just watch the video of the slides with presenter audio:
http://blip.tv/file/2796322/

Kind regards,

--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineering Team Leader
CSIRO Earth Science and Resource Engineering
Australian Resources Research Centre

Harikumar_Reddy · May 13, 2011, 7:43am

Hi Ben,

Its very useful information( especially video file). Thanks.

Regards,
Hari.

On Fri, May 13, 2011 at 9:39 AM, Ben Caradoc-Davies Ben.Caradoc-Davies@anonymised.com wrote:

Hari,

you may also benefit from Andrea Aime’s FOSS4G 2009 presentation “GeoServer in Production”:

Here is the FOSS4G information for this presentation:
http://2009.foss4g.org/presentations/#presentation_163

The slides are an OpenOffice Impress document:
http://download.osgeo.org/osgeo/foss4g/2009/SPREP/3Fri/Parkside%20Auditorium/1300/fri%20paud%201400%20aime.odp

If you don’t have OpenOffice, get it here:
http://www.openoffice.org/

Or you can just watch the video of the slides with presenter audio:
http://blip.tv/file/2796322/

Kind regards,

–

Ben Caradoc-Davies Ben.Caradoc-Davies@anonymised.com

Software Engineering Team Leader
CSIRO Earth Science and Resource Engineering
Australian Resources Research Centre