[Geoserver-devel] RESTConfig Current State and Moving Forward

Hi all,

As I'd like to be able to include RESTConfig in 1.7.0 when it comes out I am trying to assess its current state and work out how much needs to happen before it makes sense to consider it for inclusion. Right now it provides configuration for datastores, coveragestores, featuretypes, coverages, and styles. Test coverage is nil. Documentation is only what exists on the wiki at http://geoserver.org/display/GEOSDOC/RESTful+Configuration+API.

The resource representations at this point are mostly auto-generated XML/JSON with a fairly flat structure that lines up pretty closely with the Struts configuration pages. This will lose some intuitiveness when the new UI is released, but that won't be a problem in the timeframe of the 1.7.x series. The configuration model is going to change pretty drastically for 2.0, however, so if the API is released as is any code based on it will be broken for 2.0. That's probably "okay" for a major release, but it seems kind of unpleasant for clients to have an API released as 'supported' and then completely rearranged a few months later.

(Sidetrack: What's so different about the new configuration system? We now have lots of neat things like different backends, easier extensibility, a distinction between a published Layer and a server-side Resource. Of these, only the Layer/Resource distinction poses a real problem, since choosing a backend should be a one-time operation (and anyway, being able to switch from a high-performance database-backed configuration persistence mechanism to a verbose XML format remotely sounds more like a DoS vulnerability than a feature to me), and we can just ignore configuration options provided by extensions (although 'extensions as first-class citizens of the configuration system' would be a nice feature). But the Layer/Resource schism means that a number of options that are currently lumped into a 'layer' as exposed by the current RESTConfig implementation would need to be separated. One alternative is to continue to use the current API and just expose one layer for each publishe layer, and silently sync up changes made to the settings there that are actually on the data side of things. But this is (a) icky and (b) doesn't allow configuring datastores remotely (since we'd need to exposed the layers as grouped by workspace) A better option would be to have /data and /publish hierarchies in the REST API for 1.7.0... but it doesn't seem like this is a great option either since we already have a project underway to line up the 1.6.x and 1.7.x versions of the REST API, indicating that we have at least one client for whom this would break compatibility.)

Okay. Given all of this it seems that there is a fair bit of work to do before RESTConfig is ready for primetime.
* Nail down the resource layout... I think what's there now (in 1.7.x) will stay (for 1.7.x); the question is whether or not any of the unimplemented features should go in. I suspect automated configuration of coordinate reference systems and layergroups are of limited utility in comparison to mass shapefile uploads, but I'm a bit out of my element when it comes to envisioning RESTConfig use cases.
* Document the representation formats. This is a bit more important now that RESTConfig obscures the difference between FeatureTypes and Coverages since the representations are not uniform at the same location in the hierarchy.
* Get some test coverage in there! I've been waffling a bit on this trying to figure out how best to build the tests as a reusable client library; but at this point I think it's best to just write some tests (and not have to wonder whether the failures are due to bugs in the client or the server).

Since RESTConfig doesn't seem likely to be stable (in the reliably unchanging behavior sense, not the non-buggy sense) over the next months, I wonder whether it is useful to include it in 1.7.x releases after all; maybe it should just stay in community until 2.0.x.

-David Winslow

Wow, a lot to digest ;).

But I think in the end I agree with you. It does not seem to be a huge win to spend a lot of time getting REST working on 1.7.x if we just have to redo a lot of it for 2.0 anyways. Especially since its just going to be something that ties us down as the api will be changing.

Cutting REST from scope for 1.7.0 would allow us to get it out the door faster and focus more time on getting 2.0 ready for prime time. The downside being a feature short for 1.7.x... which i don't think will be too bad... we still have per-layer security.

My 2c.

-Justin

David Winslow wrote:

Hi all,

As I'd like to be able to include RESTConfig in 1.7.0 when it comes out I am trying to assess its current state and work out how much needs to happen before it makes sense to consider it for inclusion. Right now it provides configuration for datastores, coveragestores, featuretypes, coverages, and styles. Test coverage is nil. Documentation is only what exists on the wiki at http://geoserver.org/display/GEOSDOC/RESTful+Configuration+API.

The resource representations at this point are mostly auto-generated XML/JSON with a fairly flat structure that lines up pretty closely with the Struts configuration pages. This will lose some intuitiveness when the new UI is released, but that won't be a problem in the timeframe of the 1.7.x series. The configuration model is going to change pretty drastically for 2.0, however, so if the API is released as is any code based on it will be broken for 2.0. That's probably "okay" for a major release, but it seems kind of unpleasant for clients to have an API released as 'supported' and then completely rearranged a few months later.

(Sidetrack: What's so different about the new configuration system? We now have lots of neat things like different backends, easier extensibility, a distinction between a published Layer and a server-side Resource. Of these, only the Layer/Resource distinction poses a real problem, since choosing a backend should be a one-time operation (and anyway, being able to switch from a high-performance database-backed configuration persistence mechanism to a verbose XML format remotely sounds more like a DoS vulnerability than a feature to me), and we can just ignore configuration options provided by extensions (although 'extensions as first-class citizens of the configuration system' would be a nice feature). But the Layer/Resource schism means that a number of options that are currently lumped into a 'layer' as exposed by the current RESTConfig implementation would need to be separated. One alternative is to continue to use the current API and just expose one layer for each publishe layer, and silently sync up changes made to the settings there that are actually on the data side of things. But this is (a) icky and (b) doesn't allow configuring datastores remotely (since we'd need to exposed the layers as grouped by workspace) A better option would be to have /data and /publish hierarchies in the REST API for 1.7.0... but it doesn't seem like this is a great option either since we already have a project underway to line up the 1.6.x and 1.7.x versions of the REST API, indicating that we have at least one client for whom this would break compatibility.)

Okay. Given all of this it seems that there is a fair bit of work to do before RESTConfig is ready for primetime.
* Nail down the resource layout... I think what's there now (in 1.7.x) will stay (for 1.7.x); the question is whether or not any of the unimplemented features should go in. I suspect automated configuration of coordinate reference systems and layergroups are of limited utility in comparison to mass shapefile uploads, but I'm a bit out of my element when it comes to envisioning RESTConfig use cases.
* Document the representation formats. This is a bit more important now that RESTConfig obscures the difference between FeatureTypes and Coverages since the representations are not uniform at the same location in the hierarchy.
* Get some test coverage in there! I've been waffling a bit on this trying to figure out how best to build the tests as a reusable client library; but at this point I think it's best to just write some tests (and not have to wonder whether the failures are due to bugs in the client or the server).

Since RESTConfig doesn't seem likely to be stable (in the reliably unchanging behavior sense, not the non-buggy sense) over the next months, I wonder whether it is useful to include it in 1.7.x releases after all; maybe it should just stay in community until 2.0.x.

-David Winslow

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

!DSPAM:4007,4862c78676581431913854!

--
Justin Deoliveira
The Open Planning Project
jdeolive@anonymised.com

Hi guys,

I almost agree with you David, and also I think RESTConfig should not be included into 1.7.x release at least in this very incomplete state, but anyway it is a valid alternative to interact with GS catalog configuration … so what I’m thinking is that maybe we can distribute it as a GeoServer 1.7.x extension or something like that. Does it make sense?

On Thu, Jun 26, 2008 at 2:01 AM, Justin Deoliveira <jdeolive@anonymised.com> wrote:

Wow, a lot to digest ;).

But I think in the end I agree with you. It does not seem to be a huge
win to spend a lot of time getting REST working on 1.7.x if we just have
to redo a lot of it for 2.0 anyways. Especially since its just going to
be something that ties us down as the api will be changing.

Cutting REST from scope for 1.7.0 would allow us to get it out the door
faster and focus more time on getting 2.0 ready for prime time. The
downside being a feature short for 1.7.x… which i don’t think will be
too bad… we still have per-layer security.

My 2c.

-Justin

David Winslow wrote:

Hi all,

As I’d like to be able to include RESTConfig in 1.7.0 when it comes out
I am trying to assess its current state and work out how much needs to
happen before it makes sense to consider it for inclusion. Right now it
provides configuration for datastores, coveragestores, featuretypes,
coverages, and styles. Test coverage is nil. Documentation is only what
exists on the wiki at
http://geoserver.org/display/GEOSDOC/RESTful+Configuration+API.

The resource representations at this point are mostly auto-generated
XML/JSON with a fairly flat structure that lines up pretty closely with
the Struts configuration pages. This will lose some intuitiveness when
the new UI is released, but that won’t be a problem in the timeframe of
the 1.7.x series. The configuration model is going to change pretty
drastically for 2.0, however, so if the API is released as is any code
based on it will be broken for 2.0. That’s probably “okay” for a major
release, but it seems kind of unpleasant for clients to have an API
released as ‘supported’ and then completely rearranged a few months later.

(Sidetrack: What’s so different about the new configuration system? We
now have lots of neat things like different backends, easier
extensibility, a distinction between a published Layer and a server-side
Resource. Of these, only the Layer/Resource distinction poses a real
problem, since choosing a backend should be a one-time operation (and
anyway, being able to switch from a high-performance database-backed
configuration persistence mechanism to a verbose XML format remotely
sounds more like a DoS vulnerability than a feature to me), and we can
just ignore configuration options provided by extensions (although
‘extensions as first-class citizens of the configuration system’ would
be a nice feature). But the Layer/Resource schism means that a number
of options that are currently lumped into a ‘layer’ as exposed by the
current RESTConfig implementation would need to be separated. One
alternative is to continue to use the current API and just expose one
layer for each publishe layer, and silently sync up changes made to the
settings there that are actually on the data side of things. But this
is (a) icky and (b) doesn’t allow configuring datastores remotely (since
we’d need to exposed the layers as grouped by workspace) A better
option would be to have /data and /publish hierarchies in the REST API
for 1.7.0.… but it doesn’t seem like this is a great option either
since we already have a project underway to line up the 1.6.x and 1.7.x
versions of the REST API, indicating that we have at least one client
for whom this would break compatibility.)

Okay. Given all of this it seems that there is a fair bit of work to do
before RESTConfig is ready for primetime.

  • Nail down the resource layout… I think what’s there now (in 1.7.x)
    will stay (for 1.7.x); the question is whether or not any of the
    unimplemented features should go in. I suspect automated configuration
    of coordinate reference systems and layergroups are of limited utility
    in comparison to mass shapefile uploads, but I’m a bit out of my element
    when it comes to envisioning RESTConfig use cases.
  • Document the representation formats. This is a bit more important now
    that RESTConfig obscures the difference between FeatureTypes and
    Coverages since the representations are not uniform at the same location
    in the hierarchy.
  • Get some test coverage in there! I’ve been waffling a bit on this
    trying to figure out how best to build the tests as a reusable client
    library; but at this point I think it’s best to just write some tests
    (and not have to wonder whether the failures are due to bugs in the
    client or the server).

Since RESTConfig doesn’t seem likely to be stable (in the reliably
unchanging behavior sense, not the non-buggy sense) over the next
months, I wonder whether it is useful to include it in 1.7.x releases
after all; maybe it should just stay in community until 2.0.x.

-David Winslow


Check out the new SourceForge.net Marketplace.
It’s the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php


Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

!DSPAM:4007,4862c78676581431913854!


Justin Deoliveira
The Open Planning Project
jdeolive@anonymised.com


Check out the new SourceForge.net Marketplace.
It’s the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php


Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

Eng. Alessio Fabiani
Vice-President /CTO GeoSolutions S.A.S.
Via Carignoni 51
55041 Camaiore (LU)
Italy

phone: +39 0584983027
fax: +39 0584983027
mob: +39 349 8227000

http://www.geo-solutions.it


Alessio Fabiani ha scritto:

Hi guys,

I almost agree with you David, and also I think RESTConfig should not be included into 1.7.x release at least in this very incomplete state, but anyway it is a valid alternative to interact with GS catalog configuration ... so what I'm thinking is that maybe we can distribute it as a GeoServer 1.7.x extension or something like that. Does it make sense?

I agree with this line of thinking. Release it as an extension with a
big "subject to change" warning. The message could be that we understand
the need of programmatic catalog manipulation and we're throwing a rope
to anybody needing it, but warn people to not rely too much on it, and
isolate their client code so that they can update when a full blown,
supported and included by default rest subsystem shows up.

Cheers
Andrea

Hi Everyone,

Is the REST API for configuration available in an experimental form
somewhere? The project I am working on creates geotiffs based on user
actions that need to be displayed through geoserver as coverages. Is this
something that is configurable through the REST API?

Thanks,
Anthony

aaime wrote:

Alessio Fabiani ha scritto:

Hi guys,

I almost agree with you David, and also I think RESTConfig should not be
included into 1.7.x release at least in this very incomplete state, but
anyway it is a valid alternative to interact with GS catalog
configuration ... so what I'm thinking is that maybe we can distribute
it as a GeoServer 1.7.x extension or something like that. Does it make
sense?

I agree with this line of thinking. Release it as an extension with a
big "subject to change" warning. The message could be that we understand
the need of programmatic catalog manipulation and we're throwing a rope
to anybody needing it, but warn people to not rely too much on it, and
isolate their client code so that they can update when a full blown,
supported and included by default rest subsystem shows up.

Cheers
Andrea

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

--
View this message in context: http://www.nabble.com/RESTConfig-Current-State-and-Moving-Forward-tp18123085p18652541.html
Sent from the GeoServer - Dev mailing list archive at Nabble.com.

Hi Anthony,

There aren't packaged binaries available for download anywhere right now, however the source is available in SVN for the ambitious (after checking out GeoServer sources, build with "mvn install -Prest,RESTConfig")

We plan on making packages available sometime after 1.7.0 is released, although the API exposed by RESTConfig will probably change a lot before 2.0.

As far as adding geotiffs via REST, I believe this is supported already.

Hope this helps,
-David Winslow

Anthony Fox wrote:

Hi Everyone,

Is the REST API for configuration available in an experimental form
somewhere? The project I am working on creates geotiffs based on user
actions that need to be displayed through geoserver as coverages. Is this
something that is configurable through the REST API?

Thanks,
Anthony

aaime wrote:
  

Alessio Fabiani ha scritto:
    

Hi guys,

I almost agree with you David, and also I think RESTConfig should not be included into 1.7.x release at least in this very incomplete state, but anyway it is a valid alternative to interact with GS catalog configuration ... so what I'm thinking is that maybe we can distribute it as a GeoServer 1.7.x extension or something like that. Does it make sense?
      

I agree with this line of thinking. Release it as an extension with a
big "subject to change" warning. The message could be that we understand
the need of programmatic catalog manipulation and we're throwing a rope
to anybody needing it, but warn people to not rely too much on it, and
isolate their client code so that they can update when a full blown,
supported and included by default rest subsystem shows up.

Cheers
Andrea

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel