I am unable to upload large shapefiles onto geoserver, it takes extremely long and I eventually have to cancel the process, more than an hour. Initially I was using geonode interface to upload and when this did not work, I saved the file on disk and and tried to use the geoserver import layer but this did not work either. Any help will be appreciated.
Kind Regards,
Viola
On Tue, Nov 17, 2015 at 9:22 AM, Viola Anne <violaanne@anonymised.com> wrote:
I am unable to upload large shapefiles onto geoserver, it takes extremely
long and I eventually have to cancel the process, more than an hour.
Initially I was using geonode interface to upload and when this did not
work, I saved the file on disk and and tried to use the geoserver import
layer but this did not work either. Any help will be appreciated.
Hi Viola,
are you trying to import into another store, like postgis?
Also, what does "geoserver import did not work" mean, do you have stack
traces or error logs?
Cheers
Andrea
Ing. Andrea Aime
@geowolf
Technical Lead
GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549
http://www.geo-solutions.it
http://twitter.com/geosolutions_it
*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*
Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.
The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.
-------------------------------------------------------
Hi Viola,
Could you provide more information (for example, the specific Error you receive, what you initially used to upload the file, Linux or Windows, Database Shapefile or Directly in a folder)?
Also, are you sure the settings allow you to upload files of such a size? If you use a Linux Server for example, you will use Tomcat which has a default maximum allowed file size of (I think) 50MB.
Cheers,
Dré Vromans
On Tue, Nov 17, 2015 at 9:22 AM, Viola Anne <violaanne@anonymised.com> wrote:
I am unable to upload large shapefiles onto geoserver, it takes extremely long and I eventually have to cancel the process, more than an hour. Initially I was using geonode interface to upload and when this did not work, I saved the file on disk and and tried to use the geoserver import layer but this did not work either. Any help will be appreciated.
Hi Viola,
are you trying to import into another store, like postgis?
Also, what does “geoserver import did not work” mean, do you have stack traces or error logs?
Cheers
Andrea
–
Ing. Andrea Aime
@geowolf
Technical Lead
GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549
http://www.geo-solutions.it
http://twitter.com/geosolutions_it
AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.
The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.
Hi Andrea and Dré Vromans,
Yes, I am trying to import a shapefile into an already existing postgis store. I am using Ubuntu 14.04, I have been able to successfully upload files greater than 50MB previously.
A little background on the issue: What happened is I have a raster file showing land degradation, I converted this to polygons and ended up with a shapefile of 428MB which though slow I was able to successfully upload using Geonode interface (sometime last month). However, I ran into an issue, when downloading this file through Geonode (Geonode allows users to download layers) only part of the shapefile was being downloaded. The original shapefile has over 2M features the resulting downloaded shapefile always has only 1M features. I then decided to ‘dissolve’ the features using ArcMap so that I ended up with only 5 features instead. So now when trying to upload this new dissolved shapefile which is still 428MB I run into this problem. From yesterdays attempt to upload this shapefile using geoservers’ import data spatial files from directory or archive function this is the stack trace:
2015-11-16 17:06:29,609 INFO [geoserver.importer] - begining import
2015-11-16 19:13:44,312 WARN [geotools.jdbc] - There is code leaving feature readers/iterators open, this is leaking statements and connections!
2015-11-16 23:57:50,798 INFO [geoserver.wms] -
Request: getServiceInfo
And from trying to replicate this process this morning I end up with the following stack trace
2015-11-17 12:40:09,548 INFO [geoserver.importer] - begining import
2015-11-17 13:01:39,969 ERROR [wicket.RequestCycle] - Error attaching this container for rendering: [MarkupContainer [Component id = listContainer]]
org.apache.wicket.WicketRuntimeException: Error attaching this container for rendering: [MarkupContainer [Component id = listContainer]]
at org.apache.wicket.MarkupContainer.onBeforeRenderChildren(MarkupContainer.java:1765)
at org.apache.wicket.Component.onBeforeRender(Component.java:3946)
at org.apache.wicket.Component.internalBeforeRender(Component.java:1071)
at org.apache.wicket.Component.beforeRender(Component.java:1105)
at org.apache.wicket.MarkupContainer.onBeforeRenderChildren(MarkupContainer.java:1753)
at org.apache.wicket.Component.onBeforeRender(Component.java:3946)
at org.apache.wicket.Component.internalBeforeRender(Component.java:1071)
at org.apache.wicket.Component.beforeRender(Component.java:1105)
at org.apache.wicket.Component.prepareForRender(Component.java:2292)
at org.apache.wicket.Component.prepareForRender(Component.java:2329)
at org.apache.wicket.ajax.AjaxRequestTarget.respondComponent(AjaxRequestTarget.java:830)
at org.apache.wicket.ajax.AjaxRequestTarget.respondComponents(AjaxRequestTarget.java:682)
at org.apache.wicket.ajax.AjaxRequestTarget.respond(AjaxRequestTarget.java:592)
at org.apache.wicket.request.AbstractRequestCycleProcessor.respond(AbstractRequestCycleProcessor.java:105)
at org.apache.wicket.RequestCycle.processEventsAndRespond(RequestCycle.java:1258)
at org.apache.wicket.RequestCycle.step(RequestCycle.java:1329)
at org.apache.wicket.RequestCycle.steps(RequestCycle.java:1436)
at org.apache.wicket.RequestCycle.request(RequestCycle.java:545)
at org.apache.wicket.protocol.http.WicketFilter.doGet(WicketFilter.java:484)
at org.apache.wicket.protocol.http.WicketServlet.doGet(WicketServlet.java:138)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:620)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
at org.springframework.web.servlet.mvc.ServletWrappingController.handleRequestInternal(ServletWrappingController.java:159)
at org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:153)
at org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:48)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:923)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:852)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:882)
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:778)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:620)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.geoserver.filters.ThreadLocalsCleanupFilter.doFilter(ThreadLocalsCleanupFilter.java:27)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.geoserver.filters.SpringDelegatingFilter$Chain.doFilter(SpringDelegatingFilter.java:74)
at org.geoserver.wms.animate.AnimatorFilter.doFilter(AnimatorFilter.java:70)
at org.geoserver.filters.SpringDelegatingFilter$Chain.doFilter(SpringDelegatingFilter.java:70)
at org.geoserver.filters.SpringDelegatingFilter.doFilter(SpringDelegatingFilter.java:45)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.geoserver.platform.AdvancedDispatchFilter.doFilter(AdvancedDispatchFilter.java:49)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.vfny.geoserver.filters.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:109)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:311)
at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:68)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:116)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:83)
at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:72)
at org.geoserver.security.filter.GeoServerCompositeFilter.doFilter(GeoServerCompositeFilter.java:91)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:68)
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)
at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:72)
at org.geoserver.security.filter.GeoServerCompositeFilter.doFilter(GeoServerCompositeFilter.java:91)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at org.geoserver.security.filter.GeoServerAnonymousAuthenticationFilter.doFilter(GeoServerAnonymousAuthenticationFilter.java:53)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:68)
at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:182)
at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:72)
at org.geoserver.security.filter.GeoServerCompositeFilter.doFilter(GeoServerCompositeFilter.java:91)
at org.geoserver.security.filter.GeoServerUserNamePasswordAuthenticationFilter.doFilter(GeoServerUserNamePasswordAuthenticationFilter.java:115)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:68)
at org.springframework.security.web.authentication.rememberme.RememberMeAuthenticationFilter.doFilter(RememberMeAuthenticationFilter.java:146)
at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:72)
at org.geoserver.security.filter.GeoServerCompositeFilter.doFilter(GeoServerCompositeFilter.java:91)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at org.geonode.security.GeoNodeCookieProcessingFilter.doFilter(GeoNodeCookieProcessingFilter.java:107)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:68)
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)
at org.geoserver.security.filter.GeoServerSecurityContextPersistenceFilter$1.doFilter(GeoServerSecurityContextPersistenceFilter.java:52)
at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:72)
at org.geoserver.security.filter.GeoServerCompositeFilter.doFilter(GeoServerCompositeFilter.java:91)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:173)
at org.geoserver.security.GeoServerSecurityFilterChainProxy.doFilter(GeoServerSecurityFilterChainProxy.java:134)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.geoserver.filters.LoggingFilter.doFilter(LoggingFilter.java:75)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.geoserver.filters.GZIPFilter.doFilter(GZIPFilter.java:42)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.geoserver.filters.SessionDebugFilter.doFilter(SessionDebugFilter.java:47)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:501)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1041)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:315)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
at org.geoserver.importer.web.ImportTaskProvider.getItems(ImportTaskProvider.java:40)
at org.geoserver.web.wicket.GeoServerDataProvider.getFilteredItems(GeoServerDataProvider.java:209)
at org.geoserver.web.wicket.GeoServerDataProvider.size(GeoServerDataProvider.java:227)
at org.apache.wicket.markup.repeater.data.DataViewBase.internalGetItemCount(DataViewBase.java:141)
at org.apache.wicket.markup.repeater.AbstractPageableView.getItemCount(AbstractPageableView.java:258)
at org.apache.wicket.markup.repeater.AbstractPageableView.getRowCount(AbstractPageableView.java:240)
at org.apache.wicket.markup.repeater.AbstractPageableView.getViewSize(AbstractPageableView.java:352)
at org.apache.wicket.markup.repeater.AbstractPageableView.getItemModels(AbstractPageableView.java:105)
at org.apache.wicket.markup.repeater.RefreshingView.onPopulate(RefreshingView.java:94)
at org.apache.wicket.markup.repeater.AbstractRepeater.onBeforeRender(AbstractRepeater.java:131)
at org.apache.wicket.markup.repeater.AbstractPageableView.onBeforeRender(AbstractPageableView.java:121)
at org.apache.wicket.Component.internalBeforeRender(Component.java:1071)
at org.apache.wicket.Component.beforeRender(Component.java:1105)
at org.apache.wicket.MarkupContainer.onBeforeRenderChildren(MarkupContainer.java:1753)
… 109 more
On Tue, Nov 17, 2015 at 2:09 PM, Dré Vromans <bahumut5@anonymised.com> wrote:
Hi Viola,
Could you provide more information (for example, the specific Error you receive, what you initially used to upload the file, Linux or Windows, Database Shapefile or Directly in a folder)?
Also, are you sure the settings allow you to upload files of such a size? If you use a Linux Server for example, you will use Tomcat which has a default maximum allowed file size of (I think) 50MB.Cheers,
Dré VromansOp di 17 nov. 2015 om 10:04 schreef Andrea Aime <andrea.aime@anonymised.com>:
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users
–
On Tue, Nov 17, 2015 at 9:22 AM, Viola Anne <violaanne@anonymised.com> wrote:
I am unable to upload large shapefiles onto geoserver, it takes extremely long and I eventually have to cancel the process, more than an hour. Initially I was using geonode interface to upload and when this did not work, I saved the file on disk and and tried to use the geoserver import layer but this did not work either. Any help will be appreciated.
Hi Viola,
are you trying to import into another store, like postgis?
Also, what does “geoserver import did not work” mean, do you have stack traces or error logs?
Cheers
Andrea
–
Ing. Andrea Aime
@geowolf
Technical Lead
GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549
http://www.geo-solutions.it
http://twitter.com/geosolutions_it
AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.
The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.
Kind Regards,
Viola
Hi Viola,
So am I right in guess that you have one shapefile which has five features, but those features have probably thousands maybe millions of vertices? That’s quite a lot, though I don’t know if it’s enough to cause GeoServer to balk (PostGIS can theoretically handle > 2 billion vertices, so no problem there - http://gis.stackexchange.com/questions/59249/what-is-the-maximum-number-of-vertices-for-polygons-polylines-in-postgis ).
I’d guess it’s something relating to that if the only difference is that they’ve been dissolved.
One possible solution - you could try uploading the shapefile into postgis using a different tool. QGIS has one built in I believe, or ogr2ogr if you lean towards the command line.
Cheers,
Jonathan
On 17/11/2015 12:45, Viola Anne wrote:
Hi Andrea and Dré Vromans,
Yes, I am trying to import a shapefile into an already existing postgis store. I am using Ubuntu 14.04, I have been able to successfully upload files greater than 50MB previously.A little background on the issue: What happened is I have a raster file showing land degradation, I converted this to polygons and ended up with a shapefile of 428MB which though slow I was able to successfully upload using Geonode interface (sometime last month). However, I ran into an issue, when downloading this file through Geonode (Geonode allows users to download layers) only part of the shapefile was being downloaded. The original shapefile has over 2M features the resulting downloaded shapefile always has only 1M features. I then decided to ‘dissolve’ the features using ArcMap so that I ended up with only 5 features instead. So now when trying to upload this new dissolved shapefile which is still 428MB I run into this problem. From yesterdays attempt to upload this shapefile using geoservers’ import data spatial files from directory or archive function this is the stack trace:
2015-11-16 17:06:29,609 INFO [geoserver.importer] - begining import
2015-11-16 19:13:44,312 WARN [geotools.jdbc] - There is code leaving feature readers/iterators open, this is leaking statements and connections!
2015-11-16 23:57:50,798 INFO [geoserver.wms] -
Request: getServiceInfoAnd from trying to replicate this process this morning I end up with the following stack trace
2015-11-17 12:40:09,548 INFO [geoserver.importer] - begining import
2015-11-17 13:01:39,969 ERROR [wicket.RequestCycle] - Error attaching this container for rendering: [MarkupContainer [Component id = listContainer]]
org.apache.wicket.WicketRuntimeException: Error attaching this container for rendering: [MarkupContainer [Component id = listContainer]]
at org.apache.wicket.MarkupContainer.onBeforeRenderChildren(MarkupContainer.java:1765)
at org.apache.wicket.Component.onBeforeRender(Component.java:3946)
at org.apache.wicket.Component.internalBeforeRender(Component.java:1071)
at org.apache.wicket.Component.beforeRender(Component.java:1105)
at org.apache.wicket.MarkupContainer.onBeforeRenderChildren(MarkupContainer.java:1753)
at org.apache.wicket.Component.onBeforeRender(Component.java:3946)
at org.apache.wicket.Component.internalBeforeRender(Component.java:1071)
at org.apache.wicket.Component.beforeRender(Component.java:1105)
at org.apache.wicket.Component.prepareForRender(Component.java:2292)
at org.apache.wicket.Component.prepareForRender(Component.java:2329)
at org.apache.wicket.ajax.AjaxRequestTarget.respondComponent(AjaxRequestTarget.java:830)
at org.apache.wicket.ajax.AjaxRequestTarget.respondComponents(AjaxRequestTarget.java:682)
at org.apache.wicket.ajax.AjaxRequestTarget.respond(AjaxRequestTarget.java:592)
at org.apache.wicket.request.AbstractRequestCycleProcessor.respond(AbstractRequestCycleProcessor.java:105)
at org.apache.wicket.RequestCycle.processEventsAndRespond(RequestCycle.java:1258)
at org.apache.wicket.RequestCycle.step(RequestCycle.java:1329)
at org.apache.wicket.RequestCycle.steps(RequestCycle.java:1436)
at org.apache.wicket.RequestCycle.request(RequestCycle.java:545)
at org.apache.wicket.protocol.http.WicketFilter.doGet(WicketFilter.java:484)
at org.apache.wicket.protocol.http.WicketServlet.doGet(WicketServlet.java:138)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:620)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
at org.springframework.web.servlet.mvc.ServletWrappingController.handleRequestInternal(ServletWrappingController.java:159)
at org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:153)
at org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:48)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:923)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:852)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:882)
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:778)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:620)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.geoserver.filters.ThreadLocalsCleanupFilter.doFilter(ThreadLocalsCleanupFilter.java:27)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.geoserver.filters.SpringDelegatingFilter$Chain.doFilter(SpringDelegatingFilter.java:74)
at org.geoserver.wms.animate.AnimatorFilter.doFilter(AnimatorFilter.java:70)
at org.geoserver.filters.SpringDelegatingFilter$Chain.doFilter(SpringDelegatingFilter.java:70)
at org.geoserver.filters.SpringDelegatingFilter.doFilter(SpringDelegatingFilter.java:45)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.geoserver.platform.AdvancedDispatchFilter.doFilter(AdvancedDispatchFilter.java:49)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.vfny.geoserver.filters.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:109)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:311)
at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:68)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:116)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:83)
at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:72)
at org.geoserver.security.filter.GeoServerCompositeFilter.doFilter(GeoServerCompositeFilter.java:91)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:68)
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)
at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:72)
at org.geoserver.security.filter.GeoServerCompositeFilter.doFilter(GeoServerCompositeFilter.java:91)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at org.geoserver.security.filter.GeoServerAnonymousAuthenticationFilter.doFilter(GeoServerAnonymousAuthenticationFilter.java:53)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:68)
at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:182)
at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:72)
at org.geoserver.security.filter.GeoServerCompositeFilter.doFilter(GeoServerCompositeFilter.java:91)
at org.geoserver.security.filter.GeoServerUserNamePasswordAuthenticationFilter.doFilter(GeoServerUserNamePasswordAuthenticationFilter.java:115)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:68)
at org.springframework.security.web.authentication.rememberme.RememberMeAuthenticationFilter.doFilter(RememberMeAuthenticationFilter.java:146)
at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:72)
at org.geoserver.security.filter.GeoServerCompositeFilter.doFilter(GeoServerCompositeFilter.java:91)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at org.geonode.security.GeoNodeCookieProcessingFilter.doFilter(GeoNodeCookieProcessingFilter.java:107)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:68)
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)
at org.geoserver.security.filter.GeoServerSecurityContextPersistenceFilter$1.doFilter(GeoServerSecurityContextPersistenceFilter.java:52)
at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:72)
at org.geoserver.security.filter.GeoServerCompositeFilter.doFilter(GeoServerCompositeFilter.java:91)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:173)
at org.geoserver.security.GeoServerSecurityFilterChainProxy.doFilter(GeoServerSecurityFilterChainProxy.java:134)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.geoserver.filters.LoggingFilter.doFilter(LoggingFilter.java:75)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.geoserver.filters.GZIPFilter.doFilter(GZIPFilter.java:42)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.geoserver.filters.SessionDebugFilter.doFilter(SessionDebugFilter.java:47)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:501)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1041)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:315)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
at org.geoserver.importer.web.ImportTaskProvider.getItems(ImportTaskProvider.java:40)
at org.geoserver.web.wicket.GeoServerDataProvider.getFilteredItems(GeoServerDataProvider.java:209)
at org.geoserver.web.wicket.GeoServerDataProvider.size(GeoServerDataProvider.java:227)
at org.apache.wicket.markup.repeater.data.DataViewBase.internalGetItemCount(DataViewBase.java:141)
at org.apache.wicket.markup.repeater.AbstractPageableView.getItemCount(AbstractPageableView.java:258)
at org.apache.wicket.markup.repeater.AbstractPageableView.getRowCount(AbstractPageableView.java:240)
at org.apache.wicket.markup.repeater.AbstractPageableView.getViewSize(AbstractPageableView.java:352)
at org.apache.wicket.markup.repeater.AbstractPageableView.getItemModels(AbstractPageableView.java:105)
at org.apache.wicket.markup.repeater.RefreshingView.onPopulate(RefreshingView.java:94)
at org.apache.wicket.markup.repeater.AbstractRepeater.onBeforeRender(AbstractRepeater.java:131)
at org.apache.wicket.markup.repeater.AbstractPageableView.onBeforeRender(AbstractPageableView.java:121)
at org.apache.wicket.Component.internalBeforeRender(Component.java:1071)
at org.apache.wicket.Component.beforeRender(Component.java:1105)
at org.apache.wicket.MarkupContainer.onBeforeRenderChildren(MarkupContainer.java:1753)
… 109 moreOn Tue, Nov 17, 2015 at 2:09 PM, Dré Vromans <bahumut5@anonymised.com> wrote:
Hi Viola,
Could you provide more information (for example, the specific Error you receive, what you initially used to upload the file, Linux or Windows, Database Shapefile or Directly in a folder)?
Also, are you sure the settings allow you to upload files of such a size? If you use a Linux Server for example, you will use Tomcat which has a default maximum allowed file size of (I think) 50MB.Cheers,
Dré VromansOp di 17 nov. 2015 om 10:04 schreef Andrea Aime <andrea.aime@anonymised.com>:
On Tue, Nov 17, 2015 at 9:22 AM, Viola Anne <violaanne@anonymised.com> wrote:
I am unable to upload large shapefiles onto geoserver, it takes extremely long and I eventually have to cancel the process, more than an hour. Initially I was using geonode interface to upload and when this did not work, I saved the file on disk and and tried to use the geoserver import layer but this did not work either. Any help will be appreciated.
Hi Viola,
are you trying to import into another store, like postgis?
Also, what does “geoserver import did not work” mean, do you have stack traces or error logs?Cheers
Andrea–
==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.Ing. Andrea Aime
@geowolf
Technical LeadGeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549http://www.geo-solutions.it
http://twitter.com/geosolutions_itAVVERTENZE AI SENSI DEL D.Lgs. 196/2003
Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.
The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users–
Kind Regards,
Viola------------------------------------------------------------------------------_______________________________________________ Geoserver-users mailing list [Geoserver-users@lists.sourceforge.net](mailto:Geoserver-users@lists.sourceforge.net) [https://lists.sourceforge.net/lists/listinfo/geoserver-users](https://lists.sourceforge.net/lists/listinfo/geoserver-users)