[Geoserver-users] Oracle Stored Procedure

It would output a table structure that would mimmic the views we use now.

This e-mail and its attachments are confidential and solely for the intended addressee(s). Do not share or use them without Fannie Mae’s approval. If received in error, contact the sender and delete them.

----- Original Message -----
From: Justin Deoliveira <jdeolive@…1671…>
To: Den Herder, Nate P
Cc: geoserver-users@lists.sourceforge.net geoserver-users@lists.sourceforge.net; Marghescu, Silviu; Berry, Megan C
Sent: Wed Jun 10 18:21:06 2009
Subject: Re: [Geoserver-users] Oracle Stored Procedure

Unfortunately this is not possible. I am not even sure what it would
look like… what does your stored procedure output? A table like
structure or a row set?

Den Herder, Nate P wrote:

Hello,

I am interested in learning more about the the interaction between
Geoserver and Oracle. I am currently the developer of an application
that marries the thematic mapping capabilities of Geoserver with an
Oracle Spatial database, but because of some recent requests I am
interested in making my application more flexible. My question is this:

Is it currently possible for Geoserver to create a featuretype using an
Oracle stored procedure rather than a table or view? If so, how can a
parameter be passed from a WMS request through to the stored procedure
to facilitate the call?

Thanks in advance for any advice.

Nathan P. Den Herder
Sr. Research Analyst
Fannie Mae - Credit Research
Washington, DC



Crystal Reports - New Free Runtime and 30 Day Trial
Check out the new simplified licensing option that enables unlimited
royalty-free distribution of the report engine for externally facing
server and web deployment.
http://p.sf.net/sfu/businessobjects



Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users


Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

Interesting. So this seems to fall into the "virtual feature type" idea that has been on the table for some time. A common request is to publish a feature type backed by an SQL query, rather than table or a view. A stored procedure is more or less along the same lines.

Andrea and I agreed more or less on an approach to code this, but unfortunately its by no means trivial, something on the order of weeks. So until someone has a mandate for this feature, it probably won't be implemented.

Hooking up parameters might be a bit tricky as well. It might be good to have an idea of what sort of things you would want to parameterize.

-Justin

Den Herder, Nate P wrote:

It would output a table structure that would mimmic the views we use now.

This e-mail and its attachments are confidential and solely for the intended addressee(s). Do not share or use them without Fannie Mae’s approval. If received in error, contact the sender and delete them.

----- Original Message -----
From: Justin Deoliveira <jdeolive@anonymised.com>
To: Den Herder, Nate P
Cc: geoserver-users@lists.sourceforge.net <geoserver-users@lists.sourceforge.net>; Marghescu, Silviu; Berry, Megan C
Sent: Wed Jun 10 18:21:06 2009
Subject: Re: [Geoserver-users] Oracle Stored Procedure

Unfortunately this is not possible. I am not even sure what it would
look like... what does your stored procedure output? A table like
structure or a row set?

Den Herder, Nate P wrote:
> Hello,
> > I am interested in learning more about the the interaction between
> Geoserver and Oracle. I am currently the developer of an application
> that marries the thematic mapping capabilities of Geoserver with an
> Oracle Spatial database, but because of some recent requests I am
> interested in making my application more flexible. My question is this:
> > Is it currently possible for Geoserver to create a featuretype using an
> Oracle stored procedure rather than a table or view? If so, how can a
> parameter be passed from a WMS request through to the stored procedure
> to facilitate the call?
> > Thanks in advance for any advice.
> > > Nathan P. Den Herder
> Sr. Research Analyst
> Fannie Mae - Credit Research
> Washington, DC
>
> ------------------------------------------------------------------------
>
> ------------------------------------------------------------------------------
> Crystal Reports - New Free Runtime and 30 Day Trial
> Check out the new simplified licensing option that enables unlimited
> royalty-free distribution of the report engine for externally facing
> server and web deployment.
> http://p.sf.net/sfu/businessobjects
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Geoserver-users mailing list
> Geoserver-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

--
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

Justin Deoliveira ha scritto:

Interesting. So this seems to fall into the "virtual feature type" idea that has been on the table for some time. A common request is to publish a feature type backed by an SQL query, rather than table or a view. A stored procedure is more or less along the same lines.

Andrea and I agreed more or less on an approach to code this, but unfortunately its by no means trivial, something on the order of weeks. So until someone has a mandate for this feature, it probably won't be implemented.

Hooking up parameters might be a bit tricky as well. It might be good to have an idea of what sort of things you would want to parameterize.

Agreed on all the line, calling a stored procedure looks like a
special case of executing a random query with parameters.

Parametrization wise, I would suggest the query (sp call) is parametrized with well known markers with default values, so
that the query can be executed also in absence of them, something
like:

SELECT * FROM TABLE WHERE ATT1 BETWEEN $minval,0$ AND $maxval,100$

assuming $ is a good choice for a marker symbol (we probably want
that to be customizable as well)

And then the WMS/WFS request could look like:

...&request=GetMap&...&dataParams=minVal:10;maxVal:20

The important thing here is that the param usage should never
alter the structure or the record set returned by the query.

Having default values is crucial because we need to run the
query an inspect the results to determine the structure
of the feature type.

Oh, this jira has some more discussion:
http://jira.codehaus.org/browse/GEOT-2123

An interesting bit we discussed is how to support further
filtering without having to do that in memory by using
sub-queries (the typical wms request adds at the very least
a bbox filter to the query).
I'm not sure that would work for stored procedures thought
(never tried, maybe it does), a quick scan over the internet
seems to suggest that support is the privilege of few
databases (sql server, not mysql, probaly not oracle nor
postgres).

Cheers
Andrea

--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.

I seem to recall that the experimental GeoTools sql-datastore supported feature types based on SQL SELECT, to support JDBC datastores that did not support views.

Rob A, what is the status of sql-datastore? I am not aware of anyone using it. I believe it is unsupported. (I also copied Jody, who might know.)

This would not solve the problem of how to pass arbitrary parameters to the stored procedure via WMS. As far as I know, even filters have to be wrapped in a SLD.

Nate, what sort of parameters do you want to pass in your WMS call? Some arbitrary input, or a fixed list of options?

Calling stored procedures also has overlap with the prototype JDBC registered functions, in which WFS filter queries containing functions are converted to SQL for database-side execution. It would be good to complete this work and include it in jdbc-ng. The problem is that filters are used to determine whether a feature is included in a response, and are not used to create the content of a feature. This would limit its utility for calling a stored procedure to create a response.

Neither of these approaches are likely to be useful to Nate in the near future.

Kind regards,
Ben.

Justin Deoliveira wrote:

Interesting. So this seems to fall into the "virtual feature type" idea that has been on the table for some time. A common request is to publish a feature type backed by an SQL query, rather than table or a view. A stored procedure is more or less along the same lines.

Andrea and I agreed more or less on an approach to code this, but unfortunately its by no means trivial, something on the order of weeks. So until someone has a mandate for this feature, it probably won't be implemented.

Hooking up parameters might be a bit tricky as well. It might be good to have an idea of what sort of things you would want to parameterize.

-Justin

Den Herder, Nate P wrote:

It would output a table structure that would mimmic the views we use now.

This e-mail and its attachments are confidential and solely for the intended addressee(s). Do not share or use them without Fannie Mae’s approval. If received in error, contact the sender and delete them.

----- Original Message -----
From: Justin Deoliveira <jdeolive@anonymised.com>
To: Den Herder, Nate P
Cc: geoserver-users@lists.sourceforge.net <geoserver-users@lists.sourceforge.net>; Marghescu, Silviu; Berry, Megan C
Sent: Wed Jun 10 18:21:06 2009
Subject: Re: [Geoserver-users] Oracle Stored Procedure

Unfortunately this is not possible. I am not even sure what it would
look like... what does your stored procedure output? A table like
structure or a row set?

Den Herder, Nate P wrote:
> Hello,
> > I am interested in learning more about the the interaction between
> Geoserver and Oracle. I am currently the developer of an application
> that marries the thematic mapping capabilities of Geoserver with an
> Oracle Spatial database, but because of some recent requests I am
> interested in making my application more flexible. My question is this:
> > Is it currently possible for Geoserver to create a featuretype using an
> Oracle stored procedure rather than a table or view? If so, how can a
> parameter be passed from a WMS request through to the stored procedure
> to facilitate the call?
> > Thanks in advance for any advice.
> > > Nathan P. Den Herder
> Sr. Research Analyst
> Fannie Mae - Credit Research
> Washington, DC
>
> ------------------------------------------------------------------------
>
> ------------------------------------------------------------------------------
> Crystal Reports - New Free Runtime and 30 Day Trial
> Check out the new simplified licensing option that enables unlimited
> royalty-free distribution of the report engine for externally facing
> server and web deployment.
> http://p.sf.net/sfu/businessobjects
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Geoserver-users mailing list
> Geoserver-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineer, CSIRO Exploration and Mining
Australian Resources Research Centre
26 Dick Perry Ave, Kensington WA 6151, Australia

Ben Caradoc-Davies ha scritto:

I seem to recall that the experimental GeoTools sql-datastore supported feature types based on SQL SELECT, to support JDBC datastores that did not support views.

Rob A, what is the status of sql-datastore? I am not aware of anyone using it. I believe it is unsupported. (I also copied Jody, who might know.)

This would not solve the problem of how to pass arbitrary parameters to the stored procedure via WMS. As far as I know, even filters have to be wrapped in a SLD.

By spec, yeah, but GeoServer has vendor parameters that allow you
to pass the same filters as a WFS request, plus human readable filters
in CQL format.
See this for an example:
http://docs.geoserver.org/1.7.x/user/tutorials/cql/cql_tutorial.html

Cheers
Andrea

--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.

Andrea Aime wrote:

Ben Caradoc-Davies ha scritto:

This would not solve the problem of how to pass arbitrary parameters to the stored procedure via WMS. As far as I know, even filters have to be wrapped in a SLD.

By spec, yeah, but GeoServer has vendor parameters that allow you
to pass the same filters as a WFS request, plus human readable filters
in CQL format.
See this for an example:
http://docs.geoserver.org/1.7.x/user/tutorials/cql/cql_tutorial.html

Sure, but should a client application need to know whether the WMS is GeoServer? Our client (the AuScope Portal) consumes WFS and WMS services, and is constrained to work within the spec, so it can consume a range of WMS or WFS implementations. Supporting non-WMS-spec magic would make the client more complicated, and defeats the point of having a spec. Supporting quirks within the spec is tricky enough.

GeoServer has some very nice extensions, but these are tricky to use in an interoperable environment.

Kind regards,

--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineer, CSIRO Exploration and Mining
Australian Resources Research Centre
26 Dick Perry Ave, Kensington WA 6151, Australia

Ben Caradoc-Davies ha scritto:

Andrea Aime wrote:

Ben Caradoc-Davies ha scritto:

This would not solve the problem of how to pass arbitrary parameters to the stored procedure via WMS. As far as I know, even filters have to be wrapped in a SLD.

By spec, yeah, but GeoServer has vendor parameters that allow you
to pass the same filters as a WFS request, plus human readable filters
in CQL format.
See this for an example:
http://docs.geoserver.org/1.7.x/user/tutorials/cql/cql_tutorial.html

Sure, but should a client application need to know whether the WMS is GeoServer? Our client (the AuScope Portal) consumes WFS and WMS services, and is constrained to work within the spec, so it can consume a range of WMS or WFS implementations. Supporting non-WMS-spec magic would make the client more complicated, and defeats the point of having a spec. Supporting quirks within the spec is tricky enough.

GeoServer has some very nice extensions, but these are tricky to use in an interoperable environment.

Sometimes one chooses interoperability for good reasons, such as
allowing changes in the infrastructure without rewriting the parts that
do communicate with the swapped out component.

But if you are building a custom application and you _need_ the extra
features or they save you enough time to stay in budget, well,
you just use them and be thankful they are there to save your day.

Cheers
Andrea

--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.

Thank you all for your responses, the conversation so far has been very
helpful. I think I may have confused the issue a little by asking about
stored procedures, and the solution outlined by Andrea below would be a
perfect match for what I need.

We have been successfully displaying statistics in thematic maps but we
have been constantly getting suggestions for a new dimension to the
data. An example might be historical months or seeing population by
males vs. females and then inside those two groups - what is the
population by age group or hair color, what is the income for each
gender, age group and hair color. I have accomplished the dynamically
regeneration of thematic maps so far by creating feature types off of
views specific to each data cut, but this has only allowed me to cut the
data by geographic region (state, county, MSA) and historical trend
(current month through 6 months of history). Creating new views for
each cut is not a scalable solution so I was thinking that the ability
to pre-calculate all my cuts and filter them using SQL would allow me to
add new statistics without creating new feature types and also reduce
the number of styles I need.

Reading the documentation I have been able to accomplish some of my
goals through filters in the styles but this also leads to the
generation of numerous SLD files (much like the numerous views).

To sum up, Andrea has outlined what I believe to be a perfect solution
both in passing some number of parameters through the GET string and
having those parameters show up in the query. There would no reason to
address the stored procedure issue if this were available.

Nate

-----Original Message-----
From: Andrea Aime [mailto:aaime@anonymised.com]
Sent: Thursday, June 11, 2009 4:01 AM
To: Justin Deoliveira
Cc: Den Herder, Nate P; Marghescu, Silviu; Berry, Megan C;
geoserver-users@lists.sourceforge.net
Subject: Re: [Geoserver-users] Oracle Stored Procedure

Justin Deoliveira ha scritto:

Interesting. So this seems to fall into the "virtual feature type"

idea

that has been on the table for some time. A common request is to

publish

a feature type backed by an SQL query, rather than table or a view. A
stored procedure is more or less along the same lines.

Andrea and I agreed more or less on an approach to code this, but
unfortunately its by no means trivial, something on the order of

weeks.

So until someone has a mandate for this feature, it probably won't be
implemented.

Hooking up parameters might be a bit tricky as well. It might be good

to

have an idea of what sort of things you would want to parameterize.

Agreed on all the line, calling a stored procedure looks like a
special case of executing a random query with parameters.

Parametrization wise, I would suggest the query (sp call) is
parametrized with well known markers with default values, so
that the query can be executed also in absence of them, something
like:

SELECT * FROM TABLE WHERE ATT1 BETWEEN $minval,0$ AND $maxval,100$

assuming $ is a good choice for a marker symbol (we probably want
that to be customizable as well)

And then the WMS/WFS request could look like:

...&request=GetMap&...&dataParams=minVal:10;maxVal:20

The important thing here is that the param usage should never
alter the structure or the record set returned by the query.

Having default values is crucial because we need to run the
query an inspect the results to determine the structure
of the feature type.

Oh, this jira has some more discussion:
http://jira.codehaus.org/browse/GEOT-2123

An interesting bit we discussed is how to support further
filtering without having to do that in memory by using
sub-queries (the typical wms request adds at the very least
a bbox filter to the query).
I'm not sure that would work for stored procedures thought
(never tried, maybe it does), a quick scan over the internet
seems to suggest that support is the privilege of few
databases (sql server, not mysql, probaly not oracle nor
postgres).

Cheers
Andrea

--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.

Sql-datastore was a placeholder Gabriel put it. It has some stuff to support in-process SQL views for app-schemas, and some date handling override fixes.

It should be deprecated in favour of adding all this stuff to the core JDBC. Geometryless is probably the only thing depending on it now, which we ales want to destroy.

Rob

-----Original Message-----
From: Caradoc-Davies, Ben (E&M, Kensington)
Sent: Thursday, 11 June 2009 6:38 PM
To: Atkinson, Rob (CLW, Lucas Heights); Jody Garnett
Cc: Justin Deoliveira; Den Herder, Nate P; Marghescu, Silviu; Berry, Megan C; geoserver-users@lists.sourceforge.net
Subject: sql-datastore [Was: Re: Oracle Stored Procedure]

I seem to recall that the experimental GeoTools sql-datastore supported
feature types based on SQL SELECT, to support JDBC datastores that did
not support views.

Rob A, what is the status of sql-datastore? I am not aware of anyone
using it. I believe it is unsupported. (I also copied Jody, who might know.)

This would not solve the problem of how to pass arbitrary parameters to
the stored procedure via WMS. As far as I know, even filters have to be
wrapped in a SLD.

Nate, what sort of parameters do you want to pass in your WMS call? Some
arbitrary input, or a fixed list of options?

Calling stored procedures also has overlap with the prototype JDBC
registered functions, in which WFS filter queries containing functions
are converted to SQL for database-side execution. It would be good to
complete this work and include it in jdbc-ng. The problem is that
filters are used to determine whether a feature is included in a
response, and are not used to create the content of a feature. This
would limit its utility for calling a stored procedure to create a response.

Neither of these approaches are likely to be useful to Nate in the near
future.

Kind regards,
Ben.

Justin Deoliveira wrote:

Interesting. So this seems to fall into the "virtual feature type" idea
that has been on the table for some time. A common request is to publish
a feature type backed by an SQL query, rather than table or a view. A
stored procedure is more or less along the same lines.

Andrea and I agreed more or less on an approach to code this, but
unfortunately its by no means trivial, something on the order of weeks.
So until someone has a mandate for this feature, it probably won't be
implemented.

Hooking up parameters might be a bit tricky as well. It might be good to
have an idea of what sort of things you would want to parameterize.

-Justin

Den Herder, Nate P wrote:

It would output a table structure that would mimmic the views we use now.

This e-mail and its attachments are confidential and solely for the
intended addressee(s). Do not share or use them without Fannie Mae's
approval. If received in error, contact the sender and delete them.

----- Original Message -----
From: Justin Deoliveira <jdeolive@anonymised.com>
To: Den Herder, Nate P
Cc: geoserver-users@lists.sourceforge.net
<geoserver-users@lists.sourceforge.net>; Marghescu, Silviu; Berry, Megan C
Sent: Wed Jun 10 18:21:06 2009
Subject: Re: [Geoserver-users] Oracle Stored Procedure

Unfortunately this is not possible. I am not even sure what it would
look like... what does your stored procedure output? A table like
structure or a row set?

Den Herder, Nate P wrote:
> Hello,
>
> I am interested in learning more about the the interaction between
> Geoserver and Oracle. I am currently the developer of an application
> that marries the thematic mapping capabilities of Geoserver with an
> Oracle Spatial database, but because of some recent requests I am
> interested in making my application more flexible. My question is this:
>
> Is it currently possible for Geoserver to create a featuretype using an
> Oracle stored procedure rather than a table or view? If so, how can a
> parameter be passed from a WMS request through to the stored procedure
> to facilitate the call?
>
> Thanks in advance for any advice.
>
>
> Nathan P. Den Herder
> Sr. Research Analyst
> Fannie Mae - Credit Research
> Washington, DC
>
>
> ------------------------------------------------------------------------
>
>
------------------------------------------------------------------------------
> Crystal Reports - New Free Runtime and 30 Day Trial
> Check out the new simplified licensing option that enables unlimited
> royalty-free distribution of the report engine for externally facing
> server and web deployment.
> http://p.sf.net/sfu/businessobjects
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Geoserver-users mailing list
> Geoserver-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineer, CSIRO Exploration and Mining
Australian Resources Research Centre
26 Dick Perry Ave, Kensington WA 6151, Australia

This really is a showstopper for us on what we are trying to do. We need to run a parametrized query to generate a dataset. We need to run a script to generate the data (each time data is requested) and put in a temp table, then a select on that temp table to return to geoserver.

Welcome to discourse @mross01

This is a very old topic from 2009 that covers several ideas. What specific idea did you wish to follow up on?

I note that some subjects in this chat were subsequently addressed by the SQL View / Parameterized Query functionality: SQL Views — GeoServer 2.27.x User Manual

I think the name is poorly chosen; as the functionality is not actually a database view - just the ability to run a query and publish the results as a feature type / layer.