[Geoserver-users] transform columns of excel output plugin

I originally talked with Jody about my question at FOSS4G and I followed up by email. I am documenting this email here… My original problem/question is that I have looked at the excel plugin for geoserver. This plugin copies the names of the columns in my database. The database column names are not user friendly names and the excel plugin could allow an easy way to grab the data filtered out by CQL if I was able to change the column names through some means in geoserver.

Andrea said he would follow up with this request this week.

Jody mentioned there may be a way using the Transform Operation. I followed up by email today, and he suggested a couple of things. I am hoping to spark discussion to figure out if anyone has done what I am hoping to do or if anyone has any feedback on how this might be accomplished. I believe the quickest way for me to try anything out is the WPS process. I will look into that solution first but would appreciate it if anyone could give me advice/help on if I am barking up the wrong tree.

-Mike

Morning Mike: we should have this conversation on the geoserver-users list (or geotools users list).

Here is the original functionality (with user interface):

It has been packaged as a WPS process here (does WPS support CSV or Excel output?)

Andrea has also packaged it up for developers as a FeatureSource / FeatureStore wrapper:

This should mean we could create a proposal to add a GeoServer “java view” (similar to sql view) allowing us to create new feature types directly in GeoServer. I should put that our “wishlist” for good ideas needing a volunteer or sponsor.

Thanks Mike, can you try the WPS process? I am not sure if it supports Excel output (or if supporting Excel output would be easy to add).

···

Jody Garnett

On Mon, Sep 15, 2014 at 2:18 PM, Nowak, Michael J. <mnowak@anonymised.com> wrote:

I originally talked with Jody about my question at FOSS4G and I followed up by email. I am documenting this email here… My original problem/question is that I have looked at the excel plugin for geoserver. This plugin copies the names of the columns in my database. The database column names are not user friendly names and the excel plugin could allow an easy way to grab the data filtered out by CQL if I was able to change the column names through some means in geoserver.

Andrea said he would follow up with this request this week.

Jody mentioned there may be a way using the Transform Operation. I followed up by email today, and he suggested a couple of things. I am hoping to spark discussion to figure out if anyone has done what I am hoping to do or if anyone has any feedback on how this might be accomplished. I believe the quickest way for me to try anything out is the WPS process. I will look into that solution first but would appreciate it if anyone could give me advice/help on if I am barking up the wrong tree.

-Mike

Morning Mike: we should have this conversation on the geoserver-users list (or geotools users list).

Here is the original functionality (with user interface):

It has been packaged as a WPS process here (does WPS support CSV or Excel output?)

Andrea has also packaged it up for developers as a FeatureSource / FeatureStore wrapper:

This should mean we could create a proposal to add a GeoServer “java view” (similar to sql view) allowing us to create new feature types directly in GeoServer. I should put that our “wishlist” for good ideas needing a volunteer or sponsor.

Jody

On Mon, Sep 15, 2014 at 10:12 AM, Nowak, Michael J. <mnowak@anonymised.com.> wrote:

Jody,

Thank you for the geotools tutorial on Tuesday. I am also thankful for your assistance while out at FOSS4G. One of the issues we talked about in passing is if I can change the column names of the excel output format on the Excel plugin for GeoServer. You mentioned I may have some luck with the transform plugin. I’ve done some digging around and I suspect I just don’t know where to look for what you described. Where can I find documentation of the transform plugin that may be able to accept the output format of the excel format?

-Mike Nowak

Argonne National Laboratory


Want excitement?
Manually upgrade your production database.
When you want reliability, choose Perforce
Perforce version control. Predictably reliable.
http://pubads.g.doubleclick.net/gampad/clk?id=157508191&iu=/4140/ostg.clktrk


Geoserver-users mailing list
Geoserver-users@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

I will look into this now. Thank you so much for the lead. I’ll write back with an update when I have it.

-Mike

···

Thanks Mike, can you try the WPS process? I am not sure if it supports Excel output (or if supporting Excel output would be easy to add).

Jody Garnett

On Mon, Sep 15, 2014 at 2:18 PM, Nowak, Michael J. <mnowak@…6673…> wrote:

I originally talked with Jody about my question at FOSS4G and I followed up by email. I am documenting this email here… My original problem/question is that I have looked at the excel plugin for geoserver. This plugin copies the names of the columns in my database. The database column names are not user friendly names and the excel plugin could allow an easy way to grab the data filtered out by CQL if I was able to change the column names through some means in geoserver.

Andrea said he would follow up with this request this week.

Jody mentioned there may be a way using the Transform Operation. I followed up by email today, and he suggested a couple of things. I am hoping to spark discussion to figure out if anyone has done what I am hoping to do or if anyone has any feedback on how this might be accomplished. I believe the quickest way for me to try anything out is the WPS process. I will look into that solution first but would appreciate it if anyone could give me advice/help on if I am barking up the wrong tree.

-Mike

Morning Mike: we should have this conversation on the geoserver-users list (or geotools users list).

Here is the original functionality (with user interface):

It has been packaged as a WPS process here (does WPS support CSV or Excel output?)

Andrea has also packaged it up for developers as a FeatureSource / FeatureStore wrapper:

This should mean we could create a proposal to add a GeoServer “java view” (similar to sql view) allowing us to create new feature types directly in GeoServer. I should put that our “wishlist” for good ideas needing a volunteer or sponsor.

Jody

On Mon, Sep 15, 2014 at 10:12 AM, Nowak, Michael J. <mnowak@…6673…> wrote:

Jody,

Thank you for the geotools tutorial on Tuesday. I am also thankful for your assistance while out at FOSS4G. One of the issues we talked about in passing is if I can change the column names of the excel output format on the Excel plugin for GeoServer. You mentioned I may have some luck with the transform plugin. I’ve done some digging around and I suspect I just don’t know where to look for what you described. Where can I find documentation of the transform plugin that may be able to accept the output format of the excel format?

-Mike Nowak

Argonne National Laboratory


Want excitement?
Manually upgrade your production database.
When you want reliability, choose Perforce
Perforce version control. Predictably reliable.
http://pubads.g.doubleclick.net/gampad/clk?id=157508191&iu=/4140/ostg.clktrk


Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

I am digging around and with the WPS Request Builder I used vec:Transform as the process. This process allows me to change the column name of the data returned which is good news. The other piece of this is the process outputs. I do not see an output for Excel. I ran fiddler2 as a proxy to modify the http request to state the RawDataOutput MimeType as excel instead of application/json. This gives me an error “An error occurred while encoding the results of the process INVALID_CHARACTER_ERR: An invalid or illegal XML character is specified.” From the following java object “org.geoserver.wps.WPSException: An error occurred while encoding the results of the process” (complete stack below) I also noticed the wps:DataInputs contain an outputFormat which always appears as GML2. I would like to understand the difference in the two output formats. I suspect that the excel plugin isn’t configured to work with the output specification of this WPS process… Any thoughts from someone more knowledgeable than me?

org.geoserver.wps.WPSException: An error occurred while encoding the results of the process

at org.geoserver.wps.response.ExecuteProcessResponse.writeComplex(ExecuteProcessResponse.java:200)

at org.geoserver.wps.response.ExecuteProcessResponse.write(ExecuteProcessResponse.java:169)

at org.geoserver.ows.Dispatcher.response(Dispatcher.java:919)

at org.geoserver.ows.Dispatcher.handleRequestInternal(Dispatcher.java:276)

at org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:153)

at org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:48)

at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:923)

at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:852)

at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:882)

at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:789)

at javax.servlet.http.HttpServlet.service(HttpServlet.java:647)

at javax.servlet.http.HttpServlet.service(HttpServlet.java:728)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)

at org.geoserver.filters.ThreadLocalsCleanupFilter.doFilter(ThreadLocalsCleanupFilter.java:27)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)

at org.geoserver.filters.SpringDelegatingFilter$Chain.doFilter(SpringDelegatingFilter.java:74)

at org.geoserver.flow.controller.IpBlacklistFilter.doFilter(IpBlacklistFilter.java:87)

at org.geoserver.filters.SpringDelegatingFilter$Chain.doFilter(SpringDelegatingFilter.java:70)

at org.geoserver.filters.SpringDelegatingFilter.doFilter(SpringDelegatingFilter.java:45)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)

at org.geoserver.platform.AdvancedDispatchFilter.doFilter(AdvancedDispatchFilter.java:49)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)

at org.vfny.geoserver.filters.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:109)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)

at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:311)

at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:68)

at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:116)

at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:83)

at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:72)

at org.geoserver.security.filter.GeoServerCompositeFilter.doFilter(GeoServerCompositeFilter.java:91)

at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)

at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:68)

at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)

at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:72)

at org.geoserver.security.filter.GeoServerCompositeFilter.doFilter(GeoServerCompositeFilter.java:91)

at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)

at org.geoserver.security.filter.GeoServerAnonymousAuthenticationFilter.doFilter(GeoServerAnonymousAuthenticationFilter.java:53)

at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)

at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:68)

at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:150)

at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:72)

at org.geoserver.security.filter.GeoServerCompositeFilter.doFilter(GeoServerCompositeFilter.java:91)

at org.geoserver.security.filter.GeoServerBasicAuthenticationFilter.doFilter(GeoServerBasicAuthenticationFilter.java:82)

at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)

at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:68)

at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)

at org.geoserver.security.filter.GeoServerSecurityContextPersistenceFilter$1.doFilter(GeoServerSecurityContextPersistenceFilter.java:46)

at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:72)

at org.geoserver.security.filter.GeoServerCompositeFilter.doFilter(GeoServerCompositeFilter.java:91)

at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)

at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:173)

at org.geoserver.security.GeoServerSecurityFilterChainProxy.doFilter(GeoServerSecurityFilterChainProxy.java:103)

at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)

at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)

at org.geoserver.filters.LoggingFilter.doFilter(LoggingFilter.java:75)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)

at org.geoserver.filters.GZIPFilter.doFilter(GZIPFilter.java:48)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)

at org.geoserver.filters.SessionDebugFilter.doFilter(SessionDebugFilter.java:47)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)

at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)

at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)

at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)

at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)

at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)

at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)

at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)

at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)

at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1008)

at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)

at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:745)

Caused by: org.w3c.dom.DOMException: INVALID_CHARACTER_ERR: An invalid or illegal XML character is specified.

at org.apache.xerces.dom.CoreDocumentImpl.checkQName(Unknown Source)

at org.apache.xerces.dom.ElementNSImpl.setName(Unknown Source)

at org.apache.xerces.dom.ElementNSImpl.(Unknown Source)

at org.apache.xerces.dom.CoreDocumentImpl.createElementNS(Unknown Source)

at org.geotools.xml.impl.ElementEncodeExecutor.(ElementEncodeExecutor.java:63)

at org.geotools.xml.impl.ElementEncoder.encode(ElementEncoder.java:84)

at org.geotools.xml.Encoder.encode(Encoder.java:1136)

at org.geotools.xml.Encoder.encode(Encoder.java:864)

at org.geoserver.wps.ppio.WFSPPIO.encode(WFSPPIO.java:191)

at org.geoserver.wps.XMLEncoderDelegate.encode(XMLEncoderDelegate.java:30)

at org.geoserver.wps.response.ExecuteProcessResponse.writeComplex(ExecuteProcessResponse.java:196)

… 83 more

···

From: Nowak, Michael J.
Sent: Tuesday, September 16, 2014 7:48 AM
To: ‘Jody Garnett’
Cc: geoserver-users@lists.sourceforge.net
Subject: RE: [Geoserver-users] transform columns of excel output plugin

I will look into this now. Thank you so much for the lead. I’ll write back with an update when I have it.

-Mike

From: Jody Garnett [mailto:jody.garnett@…84…]
Sent: Monday, September 15, 2014 5:37 PM
To: Nowak, Michael J.
Cc: geoserver-users@lists.sourceforge.net
Subject: Re: [Geoserver-users] transform columns of excel output plugin

Thanks Mike, can you try the WPS process? I am not sure if it supports Excel output (or if supporting Excel output would be easy to add).

Jody Garnett

On Mon, Sep 15, 2014 at 2:18 PM, Nowak, Michael J. <mnowak@…6673…> wrote:

I originally talked with Jody about my question at FOSS4G and I followed up by email. I am documenting this email here… My original problem/question is that I have looked at the excel plugin for geoserver. This plugin copies the names of the columns in my database. The database column names are not user friendly names and the excel plugin could allow an easy way to grab the data filtered out by CQL if I was able to change the column names through some means in geoserver.

Andrea said he would follow up with this request this week.

Jody mentioned there may be a way using the Transform Operation. I followed up by email today, and he suggested a couple of things. I am hoping to spark discussion to figure out if anyone has done what I am hoping to do or if anyone has any feedback on how this might be accomplished. I believe the quickest way for me to try anything out is the WPS process. I will look into that solution first but would appreciate it if anyone could give me advice/help on if I am barking up the wrong tree.

-Mike

Morning Mike: we should have this conversation on the geoserver-users list (or geotools users list).

Here is the original functionality (with user interface):

It has been packaged as a WPS process here (does WPS support CSV or Excel output?)

Andrea has also packaged it up for developers as a FeatureSource / FeatureStore wrapper:

This should mean we could create a proposal to add a GeoServer “java view” (similar to sql view) allowing us to create new feature types directly in GeoServer. I should put that our “wishlist” for good ideas needing a volunteer or sponsor.

Jody

On Mon, Sep 15, 2014 at 10:12 AM, Nowak, Michael J. <mnowak@…6673…> wrote:

Jody,

Thank you for the geotools tutorial on Tuesday. I am also thankful for your assistance while out at FOSS4G. One of the issues we talked about in passing is if I can change the column names of the excel output format on the Excel plugin for GeoServer. You mentioned I may have some luck with the transform plugin. I’ve done some digging around and I suspect I just don’t know where to look for what you described. Where can I find documentation of the transform plugin that may be able to accept the output format of the excel format?

-Mike Nowak

Argonne National Laboratory


Want excitement?
Manually upgrade your production database.
When you want reliability, choose Perforce
Perforce version control. Predictably reliable.
http://pubads.g.doubleclick.net/gampad/clk?id=157508191&iu=/4140/ostg.clktrk


Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

On Tue, Sep 16, 2014 at 8:15 AM, Nowak, Michael J. <mnowak@anonymised.com> wrote:

I am digging around and with the WPS Request Builder I used
vec:Transform as the process. This process allows me to change the column
name of the data returned which is good news.

Glad it is working.

The other piece of this is the process outputs. I do not see an output for
Excel.

You may wish to look at the code used for Excel WFS output and see if it
can be adapted for WPS output. They both work with FeatureCollection so it
should be possible :slight_smile:

Look for instances of PPIO for examples of how to write a WPS input/output
format.