[Geoserver-devel] REST API Refresh Discussion

GeoServer GeoServer Developer
1

Hi all!

Even though I’m not coming to the REST refresh sprint I’ve been doing some preparatory/research for it. I’m wondering if anyone on the list has started looking at it yet or worked out an example? I took a stab yesterday, it definitely raised some questions.

Here’s what I did:

  • Created a new top level module, rest-ng, that uses Spring MVC
  • Created a StyleController in order to recreate our styles REST endpoint.
  • Mapped this endpoint to “/restng/styles”
  • Returned a list of StyleInfo objects
  • Attempted to configure a new HttpMessageConverter to potentially handle the FreeMarker HTML responses

That’s about it. Just this raised a decent few questions and problems for me:

  • Right off the bat despite mapping my new endpoint to “/restng/styles” I found that it actually ended up being mapped to “/rest/restng/styles”. I can’t for the life of me figure out why that’s happening. I tried removing the existing rest and restconfig modules from the project, removed the REST wrapper config and still had this issue. Maybe not a big issue in practice, since we want all the end points to be under /rest presumably, but it’d certainly be nice to know WHY this happens. Something to do with the AdvancedDispatchFilter maybe? (In fact after a little digging this seems to be the “culprit”)

  • Conflicting Spring MVC contexts. I used the mvc:annotation-driven/ directive to enable Spring MVC in my module. Then I added a new HttpMessageConverter (like this https://dzone.com/articles/customizing) to customize response handling. I could see the new converter being instantiated and added to the list of converters, however during the response cycle it was nowhere to be found. It seemed like internally there was another Spring context that was being used that had the default converters. If I remove the mvc:annotation-driven directive from my context then my new converter DOES get used though. This is interesting though because the only references to Spring MVC are in community.

  • How should HTML responses be handled? Spring MVC RestControllers aren’t really meant to return HTML. AFAICT the options are use the actual ModelAndView functionality with freemarker views, which would involve separate controller methods for HTML calls, or to create a new HttpMessageConverter to handle HTML requests. Both probably have issues. Creating a new MessageConverter is a little problematic, since these are centralized and orthogonal to the actual controllers. HTML responses in the current REST framework rely a lot on the type hierarchy of the RESTlet controller that handles the request. It’s a little tougher to do it this way in Spring MVC because the response encoding happens outside the controller lifecycle.

  • This seems a little more straightforward, but I assume we want to re-use all the existing XML/JSON serialization. This will require custom MessageConverters for those.

I think that’s it for now. Sorry for the wall of text.

Cheers,
Devon

2

Thanks Devon:

···

That’s about it. Just this raised a decent few questions and problems for me:

  • Right off the bat despite mapping my new endpoint to “/restng/styles” I found that it actually ended up being mapped to “/rest/restng/styles”. I can’t for the life of me figure out why that’s happening. I tried removing the existing rest and restconfig modules from the project, removed the REST wrapper config and still had this issue. Maybe not a big issue in practice, since we want all the end points to be under /rest presumably, but it’d certainly be nice to know WHY this happens. Something to do with the AdvancedDispatchFilter maybe? (In fact after a little digging this seems to be the “culprit”)

  • Conflicting Spring MVC contexts. I used the mvc:annotation-driven/ directive to enable Spring MVC in my module. Then I added a new HttpMessageConverter (like this https://dzone.com/articles/customizing) to customize response handling. I could see the new converter being instantiated and added to the list of converters, however during the response cycle it was nowhere to be found. It seemed like internally there was another Spring context that was being used that had the default converters. If I remove the mvc:annotation-driven directive from my context then my new converter DOES get used though. This is interesting though because the only references to Spring MVC are in community.

I think the geofence-server used spring mvc - could that be providing your conflict?

  • How should HTML responses be handled? Spring MVC RestControllers aren’t really meant to return HTML. AFAICT the options are use the actual ModelAndView functionality with freemarker views, which would involve separate controller methods for HTML calls, or to create a new HttpMessageConverter to handle HTML requests. Both probably have issues. Creating a new MessageConverter is a little problematic, since these are centralized and orthogonal to the actual controllers. HTML responses in the current REST framework rely a lot on the type hierarchy of the RESTlet controller that handles the request. It’s a little tougher to do it this way in Spring MVC because the response encoding happens outside the controller lifecycle.

Is there any documentation explains to look at for these two options?

My quick search found more examples of using freemarker views (and I think we use freemarker elsewhere).

  • This seems a little more straightforward, but I assume we want to re-use all the existing XML/JSON serialization. This will require custom MessageConverters for those.

Yes, we outlined that as part of the plan (it was one of the reasons we thought this was possible).

3

Good stuff!

With regard to the advanced dispatcher stuff, indeed that is what is throwing you off. I’ve worked around this (also in a spring webmvc project) by doing the following in a servlet filter.

root=“/api”
if (servletPath != null && servletPath.startsWith(root)) {
//hack to deal with the GeoServer “advanced dispatch”, the point here is that
// we want to expose our api directly at “/api”
req = new HttpServletRequestWrapper(r) {
@Override
public String getRequestURI() {
return super.getRequestURI().replace(root, root + root);
}
};
}

There may be a better/cleaner way to do this, but this worked.

Also, if I can offer a suggestion about the endpoint name, using something like “/api” might be a bit more aesthetically pleasing than “/restng”… and more inline with how I’ve seen other modern servers expose restful apis, fwiw.

-Justin

On Wed, Mar 15, 2017 at 10:42 AM Devon Tucker <dtucker@anonymised.com> wrote:

Hi all!

Even though I’m not coming to the REST refresh sprint I’ve been doing some preparatory/research for it. I’m wondering if anyone on the list has started looking at it yet or worked out an example? I took a stab yesterday, it definitely raised some questions.

Here’s what I did:

  • Created a new top level module, rest-ng, that uses Spring MVC
  • Created a StyleController in order to recreate our styles REST endpoint.
  • Mapped this endpoint to “/restng/styles”
  • Returned a list of StyleInfo objects
  • Attempted to configure a new HttpMessageConverter to potentially handle the FreeMarker HTML responses

That’s about it. Just this raised a decent few questions and problems for me:

  • Right off the bat despite mapping my new endpoint to “/restng/styles” I found that it actually ended up being mapped to “/rest/restng/styles”. I can’t for the life of me figure out why that’s happening. I tried removing the existing rest and restconfig modules from the project, removed the REST wrapper config and still had this issue. Maybe not a big issue in practice, since we want all the end points to be under /rest presumably, but it’d certainly be nice to know WHY this happens. Something to do with the AdvancedDispatchFilter maybe? (In fact after a little digging this seems to be the “culprit”)

  • Conflicting Spring MVC contexts. I used the mvc:annotation-driven/ directive to enable Spring MVC in my module. Then I added a new HttpMessageConverter (like this https://dzone.com/articles/customizing) to customize response handling. I could see the new converter being instantiated and added to the list of converters, however during the response cycle it was nowhere to be found. It seemed like internally there was another Spring context that was being used that had the default converters. If I remove the mvc:annotation-driven directive from my context then my new converter DOES get used though. This is interesting though because the only references to Spring MVC are in community.

  • How should HTML responses be handled? Spring MVC RestControllers aren’t really meant to return HTML. AFAICT the options are use the actual ModelAndView functionality with freemarker views, which would involve separate controller methods for HTML calls, or to create a new HttpMessageConverter to handle HTML requests. Both probably have issues. Creating a new MessageConverter is a little problematic, since these are centralized and orthogonal to the actual controllers. HTML responses in the current REST framework rely a lot on the type hierarchy of the RESTlet controller that handles the request. It’s a little tougher to do it this way in Spring MVC because the response encoding happens outside the controller lifecycle.

  • This seems a little more straightforward, but I assume we want to re-use all the existing XML/JSON serialization. This will require custom MessageConverters for those.

I think that’s it for now. Sorry for the wall of text.

Cheers,
Devon

Check out the vibrant tech community on one of the world’s most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

4

I think the geofence-server used spring mvc - could that be providing your conflict?

Probably not. I don’t have any community modules enabled.

Is there any documentation explains to look at for these two options?

I’ll try to compile the results of my research. Using FreeMarker views is probably the best option, but probably lead to very, very slightly more complicated Controllers.

Yes, we outlined that as part of the plan (it was one of the reasons we thought this was possible).

It is possible. It’s just not entirely clear on the details yet. XML and JSON serialization all have small bits of customization depending on what’s being returned (see ReflectiveJSON/XMLFormat.java). Currently these customizations are part of the type hierarchy of the RESTlet resource being accessed which is where the wrinkle comes from. Response serialization in Spring MVC is fairly detached from the controller itself and more centralized. It’s something that just needs more research, in particular documenting the various bits of customization that already exist.

···

On Wed, Mar 15, 2017 at 10:57 AM, Jody Garnett <jody.garnett@anonymised.com> wrote:

Thanks Devon:

That’s about it. Just this raised a decent few questions and problems for me:

  • Right off the bat despite mapping my new endpoint to “/restng/styles” I found that it actually ended up being mapped to “/rest/restng/styles”. I can’t for the life of me figure out why that’s happening. I tried removing the existing rest and restconfig modules from the project, removed the REST wrapper config and still had this issue. Maybe not a big issue in practice, since we want all the end points to be under /rest presumably, but it’d certainly be nice to know WHY this happens. Something to do with the AdvancedDispatchFilter maybe? (In fact after a little digging this seems to be the “culprit”)

  • Conflicting Spring MVC contexts. I used the mvc:annotation-driven/ directive to enable Spring MVC in my module. Then I added a new HttpMessageConverter (like this https://dzone.com/articles/customizing) to customize response handling. I could see the new converter being instantiated and added to the list of converters, however during the response cycle it was nowhere to be found. It seemed like internally there was another Spring context that was being used that had the default converters. If I remove the mvc:annotation-driven directive from my context then my new converter DOES get used though. This is interesting though because the only references to Spring MVC are in community.

I think the geofence-server used spring mvc - could that be providing your conflict?

  • How should HTML responses be handled? Spring MVC RestControllers aren’t really meant to return HTML. AFAICT the options are use the actual ModelAndView functionality with freemarker views, which would involve separate controller methods for HTML calls, or to create a new HttpMessageConverter to handle HTML requests. Both probably have issues. Creating a new MessageConverter is a little problematic, since these are centralized and orthogonal to the actual controllers. HTML responses in the current REST framework rely a lot on the type hierarchy of the RESTlet controller that handles the request. It’s a little tougher to do it this way in Spring MVC because the response encoding happens outside the controller lifecycle.

Is there any documentation explains to look at for these two options?

My quick search found more examples of using freemarker views (and I think we use freemarker elsewhere).

  • This seems a little more straightforward, but I assume we want to re-use all the existing XML/JSON serialization. This will require custom MessageConverters for those.

Yes, we outlined that as part of the plan (it was one of the reasons we thought this was possible).

5

Thanks Justin. It’s not clear that the AdvancedDispatchFilter is actually needed outside of the REST api. When I remove it the only thing that breaks is REST. In particular the end points themselves stop working without it present. So “/rest” works, but “/rest/styles” does not. Do you know of anything else that relies on this filter? If we’re re-doing the rest API maybe it can go away entirely?

The end point name right now is not permanent, it’s just a sample for R&D.

···

On Wed, Mar 15, 2017 at 11:58 AM, Justin Deoliveira <jdeolive@anonymised.com> wrote:

Good stuff!

With regard to the advanced dispatcher stuff, indeed that is what is throwing you off. I’ve worked around this (also in a spring webmvc project) by doing the following in a servlet filter.

root=“/api”
if (servletPath != null && servletPath.startsWith(root)) {
//hack to deal with the GeoServer “advanced dispatch”, the point here is that
// we want to expose our api directly at “/api”
req = new HttpServletRequestWrapper(r) {
@Override
public String getRequestURI() {
return super.getRequestURI().replace(root, root + root);
}
};
}

There may be a better/cleaner way to do this, but this worked.

Also, if I can offer a suggestion about the endpoint name, using something like “/api” might be a bit more aesthetically pleasing than “/restng”… and more inline with how I’ve seen other modern servers expose restful apis, fwiw.

-Justin

On Wed, Mar 15, 2017 at 10:42 AM Devon Tucker <dtucker@anonymised.com> wrote:

Hi all!

Even though I’m not coming to the REST refresh sprint I’ve been doing some preparatory/research for it. I’m wondering if anyone on the list has started looking at it yet or worked out an example? I took a stab yesterday, it definitely raised some questions.

Here’s what I did:

  • Created a new top level module, rest-ng, that uses Spring MVC
  • Created a StyleController in order to recreate our styles REST endpoint.
  • Mapped this endpoint to “/restng/styles”
  • Returned a list of StyleInfo objects
  • Attempted to configure a new HttpMessageConverter to potentially handle the FreeMarker HTML responses

That’s about it. Just this raised a decent few questions and problems for me:

  • Right off the bat despite mapping my new endpoint to “/restng/styles” I found that it actually ended up being mapped to “/rest/restng/styles”. I can’t for the life of me figure out why that’s happening. I tried removing the existing rest and restconfig modules from the project, removed the REST wrapper config and still had this issue. Maybe not a big issue in practice, since we want all the end points to be under /rest presumably, but it’d certainly be nice to know WHY this happens. Something to do with the AdvancedDispatchFilter maybe? (In fact after a little digging this seems to be the “culprit”)

  • Conflicting Spring MVC contexts. I used the mvc:annotation-driven/ directive to enable Spring MVC in my module. Then I added a new HttpMessageConverter (like this https://dzone.com/articles/customizing) to customize response handling. I could see the new converter being instantiated and added to the list of converters, however during the response cycle it was nowhere to be found. It seemed like internally there was another Spring context that was being used that had the default converters. If I remove the mvc:annotation-driven directive from my context then my new converter DOES get used though. This is interesting though because the only references to Spring MVC are in community.

  • How should HTML responses be handled? Spring MVC RestControllers aren’t really meant to return HTML. AFAICT the options are use the actual ModelAndView functionality with freemarker views, which would involve separate controller methods for HTML calls, or to create a new HttpMessageConverter to handle HTML requests. Both probably have issues. Creating a new MessageConverter is a little problematic, since these are centralized and orthogonal to the actual controllers. HTML responses in the current REST framework rely a lot on the type hierarchy of the RESTlet controller that handles the request. It’s a little tougher to do it this way in Spring MVC because the response encoding happens outside the controller lifecycle.

  • This seems a little more straightforward, but I assume we want to re-use all the existing XML/JSON serialization. This will require custom MessageConverters for those.

I think that’s it for now. Sorry for the wall of text.

Cheers,
Devon

Check out the vibrant tech community on one of the world’s most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@anonymised.comsourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

6

Hi Devon,

Using FreeMarker views is probably the best option, but probably lead to very, very slightly more complicated Controllers.

In my experience, that’s right. Generally, @ResponseBody controller methods are meant for directly returning the object (in a REST-y kind of way), while the traditional ModelAndView-type (non-@anonymised.com) methods are meant for rendering the object with a view / template like Freemarker.

You could mix the concepts, e.g., by using a @ResponseBody controller method to return an object for “Accept: text/html”, and then using an HttpMessageConverter to try to render it as HTML using a template. But I think that would be inviting complexity in the wrong place – Spring treats the components as “orthogonal,” like you said. The controllers would not even be able to select an intended view, so the http message converter would have to figure it out.

Keeping them separate costs one additional method, but to me at least, it would be a more natural and clear use of the framework, and shouldn’t be any more complicated if the underlying logic is shared. It should also cause fewer surprises down the line.

See an example here: https://spring.io/blog/2013/05/11/content-negotiation-using-spring-mvc (under “Combining Data and Presentation Formats”).

One other note, while on the topic of serialization. We’ve been talking about docs autogeneration tools that are based on introspection of the framework annotations, like Swagger/Springfox – but such tools will generally not be aware of arbitrary serialization logic that happens outside of the controller. I think some more research is required to figure out how big the gap is between the actual results of an API call and what those tools will be able to introspect.

  • Matt
···

On Wed, Mar 15, 2017 at 2:58 PM, Devon Tucker <dtucker@anonymised.com> wrote:

I think the geofence-server used spring mvc - could that be providing your conflict?

Probably not. I don’t have any community modules enabled.

Is there any documentation explains to look at for these two options?

I’ll try to compile the results of my research. Using FreeMarker views is probably the best option, but probably lead to very, very slightly more complicated Controllers.

Yes, we outlined that as part of the plan (it was one of the reasons we thought this was possible).

It is possible. It’s just not entirely clear on the details yet. XML and JSON serialization all have small bits of customization depending on what’s being returned (see ReflectiveJSON/XMLFormat.java). Currently these customizations are part of the type hierarchy of the RESTlet resource being accessed which is where the wrinkle comes from. Response serialization in Spring MVC is fairly detached from the controller itself and more centralized. It’s something that just needs more research, in particular documenting the various bits of customization that already exist.

Check out the vibrant tech community on one of the world’s most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

Geoserver-devel mailing list
Geoserver-devel@anonymised.com.366…sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

On Wed, Mar 15, 2017 at 10:57 AM, Jody Garnett <jody.garnett@anonymised.com…> wrote:

Thanks Devon:

That’s about it. Just this raised a decent few questions and problems for me:

  • Right off the bat despite mapping my new endpoint to “/restng/styles” I found that it actually ended up being mapped to “/rest/restng/styles”. I can’t for the life of me figure out why that’s happening. I tried removing the existing rest and restconfig modules from the project, removed the REST wrapper config and still had this issue. Maybe not a big issue in practice, since we want all the end points to be under /rest presumably, but it’d certainly be nice to know WHY this happens. Something to do with the AdvancedDispatchFilter maybe? (In fact after a little digging this seems to be the “culprit”)

  • Conflicting Spring MVC contexts. I used the mvc:annotation-driven/ directive to enable Spring MVC in my module. Then I added a new HttpMessageConverter (like this https://dzone.com/articles/customizing) to customize response handling. I could see the new converter being instantiated and added to the list of converters, however during the response cycle it was nowhere to be found. It seemed like internally there was another Spring context that was being used that had the default converters. If I remove the mvc:annotation-driven directive from my context then my new converter DOES get used though. This is interesting though because the only references to Spring MVC are in community.

I think the geofence-server used spring mvc - could that be providing your conflict?

  • How should HTML responses be handled? Spring MVC RestControllers aren’t really meant to return HTML. AFAICT the options are use the actual ModelAndView functionality with freemarker views, which would involve separate controller methods for HTML calls, or to create a new HttpMessageConverter to handle HTML requests. Both probably have issues. Creating a new MessageConverter is a little problematic, since these are centralized and orthogonal to the actual controllers. HTML responses in the current REST framework rely a lot on the type hierarchy of the RESTlet controller that handles the request. It’s a little tougher to do it this way in Spring MVC because the response encoding happens outside the controller lifecycle.

Is there any documentation explains to look at for these two options?

My quick search found more examples of using freemarker views (and I think we use freemarker elsewhere).

  • This seems a little more straightforward, but I assume we want to re-use all the existing XML/JSON serialization. This will require custom MessageConverters for those.

Yes, we outlined that as part of the plan (it was one of the reasons we thought this was possible).

7

Hey all,

The Style endpoint is basically 100% at this point. Thanks to Torben for the help as well.

https://github.com/boundlessgeo/geoserver/tree/spring_mvc_example/src/rest-ng

I also have some brief notes written about it here:

https://github.com/geoserver/geoserver/wiki/REST-Refresh-Notes-and-Tips

Let me know if there’s any questions or comments. One thing that’s not supported right now are the Style extension end points. I still need to look at how those are handled. Many of the extensions lack their own tests for this though.

Cheers,
Devon

···

On Thu, Mar 16, 2017 at 9:46 AM, Matt Kruszewski <mkruszewski@anonymised.com> wrote:

Hi Devon,

Using FreeMarker views is probably the best option, but probably lead to very, very slightly more complicated Controllers.

In my experience, that’s right. Generally, @ResponseBody controller methods are meant for directly returning the object (in a REST-y kind of way), while the traditional ModelAndView-type (non-@anonymised.com) methods are meant for rendering the object with a view / template like Freemarker.

You could mix the concepts, e.g., by using a @ResponseBody controller method to return an object for “Accept: text/html”, and then using an HttpMessageConverter to try to render it as HTML using a template. But I think that would be inviting complexity in the wrong place – Spring treats the components as “orthogonal,” like you said. The controllers would not even be able to select an intended view, so the http message converter would have to figure it out.

Keeping them separate costs one additional method, but to me at least, it would be a more natural and clear use of the framework, and shouldn’t be any more complicated if the underlying logic is shared. It should also cause fewer surprises down the line.

See an example here: https://spring.io/blog/2013/05/11/content-negotiation-using-spring-mvc (under “Combining Data and Presentation Formats”).

One other note, while on the topic of serialization. We’ve been talking about docs autogeneration tools that are based on introspection of the framework annotations, like Swagger/Springfox – but such tools will generally not be aware of arbitrary serialization logic that happens outside of the controller. I think some more research is required to figure out how big the gap is between the actual results of an API call and what those tools will be able to introspect.

  • Matt

On Wed, Mar 15, 2017 at 2:58 PM, Devon Tucker <dtucker@anonymised.com3839…> wrote:

I think the geofence-server used spring mvc - could that be providing your conflict?

Probably not. I don’t have any community modules enabled.

Is there any documentation explains to look at for these two options?

I’ll try to compile the results of my research. Using FreeMarker views is probably the best option, but probably lead to very, very slightly more complicated Controllers.

Yes, we outlined that as part of the plan (it was one of the reasons we thought this was possible).

It is possible. It’s just not entirely clear on the details yet. XML and JSON serialization all have small bits of customization depending on what’s being returned (see ReflectiveJSON/XMLFormat.java). Currently these customizations are part of the type hierarchy of the RESTlet resource being accessed which is where the wrinkle comes from. Response serialization in Spring MVC is fairly detached from the controller itself and more centralized. It’s something that just needs more research, in particular documenting the various bits of customization that already exist.

Check out the vibrant tech community on one of the world’s most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

Geoserver-devel mailing list
Geoserver-devel@anonymised.comrge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

On Wed, Mar 15, 2017 at 10:57 AM, Jody Garnett <jody.garnett@anonymised.com> wrote:

Thanks Devon:

That’s about it. Just this raised a decent few questions and problems for me:

  • Right off the bat despite mapping my new endpoint to “/restng/styles” I found that it actually ended up being mapped to “/rest/restng/styles”. I can’t for the life of me figure out why that’s happening. I tried removing the existing rest and restconfig modules from the project, removed the REST wrapper config and still had this issue. Maybe not a big issue in practice, since we want all the end points to be under /rest presumably, but it’d certainly be nice to know WHY this happens. Something to do with the AdvancedDispatchFilter maybe? (In fact after a little digging this seems to be the “culprit”)

  • Conflicting Spring MVC contexts. I used the mvc:annotation-driven/ directive to enable Spring MVC in my module. Then I added a new HttpMessageConverter (like this https://dzone.com/articles/customizing) to customize response handling. I could see the new converter being instantiated and added to the list of converters, however during the response cycle it was nowhere to be found. It seemed like internally there was another Spring context that was being used that had the default converters. If I remove the mvc:annotation-driven directive from my context then my new converter DOES get used though. This is interesting though because the only references to Spring MVC are in community.

I think the geofence-server used spring mvc - could that be providing your conflict?

  • How should HTML responses be handled? Spring MVC RestControllers aren’t really meant to return HTML. AFAICT the options are use the actual ModelAndView functionality with freemarker views, which would involve separate controller methods for HTML calls, or to create a new HttpMessageConverter to handle HTML requests. Both probably have issues. Creating a new MessageConverter is a little problematic, since these are centralized and orthogonal to the actual controllers. HTML responses in the current REST framework rely a lot on the type hierarchy of the RESTlet controller that handles the request. It’s a little tougher to do it this way in Spring MVC because the response encoding happens outside the controller lifecycle.

Is there any documentation explains to look at for these two options?

My quick search found more examples of using freemarker views (and I think we use freemarker elsewhere).

  • This seems a little more straightforward, but I assume we want to re-use all the existing XML/JSON serialization. This will require custom MessageConverters for those.

Yes, we outlined that as part of the plan (it was one of the reasons we thought this was possible).

8

Now that the example branch is complete, we can move it over to a branch in
the geoserver repository (rather than the boundless geoserver fork), if no
one has any objections.

Let me know if there's any questions or comments. One thing that's not

supported right now are the Style extension end points. I still need to
look at how those are handled. Many of the extensions lack their own tests
for this though.

I've done a little bit of testing against a previous GeoServer release

using YSLD, and I found Style "PUT" seems quite buggy, so this seems like
an opportunity for improvement. The answer to "how those are handled" might
be pretty close to "they aren't".

Torben

9

On Thu, Mar 23, 2017 at 11:21 PM, Torben Barsballe <
tbarsballe@anonymised.com> wrote:

Now that the example branch is complete, we can move it over to a branch
in the geoserver repository (rather than the boundless geoserver fork), if
no one has any objections.

Please. As this is a community effort, it would have made sense to start
the branch in the geoserver repo directly.

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------

10

Hi Torben,

I also noticed issues with this PUT /styles/{styleName} endpoint while testing the REST API for mapbox styles. I collected my notes in this gist, but here’s the “short” version of what I found:

Any style formats that don’t support bidirectional encoding from SLD → theFormat will require you to set ?raw=true when uploading with the REST API, or else a HTTP 500 will be returned.

By default, GeoServer attempts to decode the uploaded style to SLD, then encode it back to the original format, and then write that result to the catalog. See StyleFormat::read, line 101, then ResourcePool::writeStyle, line 1947. For an SLD, that roundtrip works and accomplishes parsing, validating, and formatting of the uploaded style body. For other style formats that don’t support encoding from SLD, it results in a HTTP 500 response and nothing being written to the catalog. See CSSHandler::encode, which throws an UnsupportedOperationException at line 142.

We at least need the docs to better explain how the endpoint works, but we might want to allow GeoServer to be more flexible with uploaded styles by default.

···

On Thu, Mar 23, 2017 at 5:21 PM, Torben Barsballe <tbarsballe@anonymised.com> wrote:

Now that the example branch is complete, we can move it over to a branch in the geoserver repository (rather than the boundless geoserver fork), if no one has any objections.

Let me know if there’s any questions or comments. One thing that’s not supported right now are the Style extension end points. I still need to look at how those are handled. Many of the extensions lack their own tests for this though.

I’ve done a little bit of testing against a previous GeoServer release using YSLD, and I found Style “PUT” seems quite buggy, so this seems like an opportunity for improvement. The answer to “how those are handled” might be pretty close to “they aren’t”.

Torben

11

On Fri, Mar 24, 2017 at 4:14 PM, Matt Kruszewski <
mkruszewski@anonymised.com> wrote:

We at least need the docs to better explain how the endpoint works, but we
might want to allow GeoServer to be more flexible with uploaded styles by
default.

Yeah, at least the code should recognize a format that cannot round-trip
and don't do the reformat dance.
We also had complains about that being the default behavior, the annoying
thing is that there are tools out there relying on it, so changing
that behavior is going to hurt some upgrades... we could still do it with a
flag to get the old behavior back imho.

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------

12

The example has been tidied up, squashed+rebased, and can now be found as a branch of the main geoserver repository here: https://github.com/geoserver/geoserver/tree/rest-api-refresh

Torben

···

On Fri, Mar 24, 2017 at 8:19 AM, Andrea Aime <andrea.aime@anonymised.com…> wrote:

On Fri, Mar 24, 2017 at 4:14 PM, Matt Kruszewski <mkruszewski@anonymised.com> wrote:

We at least need the docs to better explain how the endpoint works, but we might want to allow GeoServer to be more flexible with uploaded styles by default.

Yeah, at least the code should recognize a format that cannot round-trip and don’t do the reformat dance.
We also had complains about that being the default behavior, the annoying thing is that there are tools out there relying on it, so changing
that behavior is going to hurt some upgrades… we could still do it with a flag to get the old behavior back imho.

Cheers

Andrea

–

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313

fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

13

Hello all,

I have started a spreadsheet for managing conversion of endpoints: https://docs.google.com/spreadsheets/d/1q9HV5cjMhh94zQW4H_u9SH-sUsoM0zrf4bmbdGjqtHM

This includes a full endpoint listing, and what classes handle each endpoint. This spreadsheet is also linked from the GeoServer Wiki Rest API Refresh page.
Everyone attending the sprint should have edit access; if you don’t please contact me.

Torben

···

On Fri, Mar 24, 2017 at 10:28 AM, Torben Barsballe <tbarsballe@anonymised.com> wrote:

The example has been tidied up, squashed+rebased, and can now be found as a branch of the main geoserver repository here: https://github.com/geoserver/geoserver/tree/rest-api-refresh

Torben

On Fri, Mar 24, 2017 at 8:19 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Fri, Mar 24, 2017 at 4:14 PM, Matt Kruszewski <mkruszewski@…3839…> wrote:

We at least need the docs to better explain how the endpoint works, but we might want to allow GeoServer to be more flexible with uploaded styles by default.

Yeah, at least the code should recognize a format that cannot round-trip and don’t do the reformat dance.
We also had complains about that being the default behavior, the annoying thing is that there are tools out there relying on it, so changing
that behavior is going to hurt some upgrades… we could still do it with a flag to get the old behavior back imho.

Cheers

Andrea

–

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313

fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

14

Hi Torben,
I’m having a look around, one thing that seems to be missing is the per resource xstream persister configuration,
most resources do that (unfortunately the style one is not among them).
E.g.:

https://github.com/geoserver/geoserver/blob/master/src/restconfig/src/main/java/org/geoserver/catalog/rest/LayerResource.java#L100

This is resource specific, does it mean we’ll have to create a custom subclass of XStreamXMLMessageConverter for most
catalog resources and register it in the configuration? However… programmatic configuration breaks pluggability, we’d need
to have a way to have the be looked up from the context imho.

Cheers
Andrea

···

On Fri, Mar 24, 2017 at 6:28 PM, Torben Barsballe <tbarsballe@anonymised.com> wrote:

The example has been tidied up, squashed+rebased, and can now be found as a branch of the main geoserver repository here: https://github.com/geoserver/geoserver/tree/rest-api-refresh

Torben

–

On Fri, Mar 24, 2017 at 8:19 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Fri, Mar 24, 2017 at 4:14 PM, Matt Kruszewski <mkruszewski@anonymised.com> wrote:

We at least need the docs to better explain how the endpoint works, but we might want to allow GeoServer to be more flexible with uploaded styles by default.

Yeah, at least the code should recognize a format that cannot round-trip and don’t do the reformat dance.
We also had complains about that being the default behavior, the annoying thing is that there are tools out there relying on it, so changing
that behavior is going to hurt some upgrades… we could still do it with a flag to get the old behavior back imho.

Cheers

Andrea

–

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313

fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313

fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

15

I believe we should be able to accomplish this with wrapper classes returned from the controller, similar to how the XStream list configuration is handled. (See XStreamListWrapper). Perhaps an XStreamCallbackWrapper, with an extensible method that takes the XStreamPersister and applies some logic. The converters would then check for such a wrapper, and call its methods accordingly.

Torben

···

On Sat, Mar 25, 2017 at 4:43 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

Hi Torben,
I’m having a look around, one thing that seems to be missing is the per resource xstream persister configuration,
most resources do that (unfortunately the style one is not among them).
E.g.:

https://github.com/geoserver/geoserver/blob/master/src/restconfig/src/main/java/org/geoserver/catalog/rest/LayerResource.java#L100

This is resource specific, does it mean we’ll have to create a custom subclass of XStreamXMLMessageConverter for most
catalog resources and register it in the configuration? However… programmatic configuration breaks pluggability, we’d need
to have a way to have the be looked up from the context imho.

Cheers

Andrea

On Fri, Mar 24, 2017 at 6:28 PM, Torben Barsballe <tbarsballe@anonymised.com> wrote:

The example has been tidied up, squashed+rebased, and can now be found as a branch of the main geoserver repository here: https://github.com/geoserver/geoserver/tree/rest-api-refresh

Torben

–

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313

fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

On Fri, Mar 24, 2017 at 8:19 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Fri, Mar 24, 2017 at 4:14 PM, Matt Kruszewski <mkruszewski@anonymised.com> wrote:

We at least need the docs to better explain how the endpoint works, but we might want to allow GeoServer to be more flexible with uploaded styles by default.

Yeah, at least the code should recognize a format that cannot round-trip and don’t do the reformat dance.
We also had complains about that being the default behavior, the annoying thing is that there are tools out there relying on it, so changing
that behavior is going to hurt some upgrades… we could still do it with a flag to get the old behavior back imho.

Cheers

Andrea

–

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313

fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.