[Geoserver-devel] [GEOS-9886] yaml schemas definitions aren't compatible with OpenAPI (Swagger 2.0) definitions. Generated Swagger clients don't work.

Andrea_Aime4 · February 13, 2021, 1:00pm

Hello,

About workspaces.yaml :I added the corrected yaml file and a test for

[GEOS-9886] workspaces.yaml doesn’t carry correct descriptions of REST services it presents. (as an effect, a working Swagger client cannot be generated for it)

Before creating a pull request to send the corrected geoserver/doc/en/api/1.0.0/workspaces.yaml,
I would like to find a way to put the test I did in geoserver. But it could a long work, because I don’t know how to reproduce all the steps of my own test project in a manner compatible to geoserver project rules.
And I would like to check the next point to see if it can offer a better solution too.

I see you are mentioning a Spring Boot test in the ticket, however, GeoServer does not use nor depends on Spring boot.
But maybe it could be a separate maven module that only runs tests? That one could be using whatever you like.
On the other end, using only technologies/libraries already in use in the project would help in maintenance, modules
that long term maintainers are not familiar with, risk of being kicked to the side and eventually removed.
Then again, it just takes a single developer to maintain a module, if you’re going to be the one, then the above is not a problem.

About openapi.json generation from existing REST servicesWhen adding this dependency to an application (I’ve used it’s version 1.5.3, compatible with Spring-boot 2.4.1)
org.springdoc springdoc-openapi-ui
Swagger/OpenAPI immediately makes available

http://localhost:8080/swagger-ui.html
that produces the same web interface than the one on Geoserver documentation website.

http://localhost:8080/v3/api-docs
that describes the REST services in a json format (instead of a yaml one, but I don’t know if it matters).

This path can be changed with this property :
springdoc.api-docs.path=/geoserver/rest/v3/api-docs

Would it work without having Spring-boot in the classpath?
I’ll also note the ogc-api modules are already publishing an API endpoint that renders with Swagger, without
that particular maven dependency, e.g.:
https://tb16.geo-solutions.it/geoserver/ogc/dggs/api?f=text%2Fhtml

The setup for the OGC APIs is different, because the yamls have to be adapted on the fly based on the collections
in the server, which involves reading a yaml starting template, adapting it with Java code, and re-encoding
it back in json/yaml/html.
The swagger-ui is referenced directly from the classpath:
https://github.com/geoserver/geoserver/tree/master/src/community/ogcapi/ogcapi-core/src/main/resources/swagger-ui

and published using a ClasspathPublisher:

https://github.com/geoserver/geoserver/blob/master/src/community/ogcapi/ogcapi-core/src/main/resources/applicationContext.xml#L24

If all you need is static files, then a single classpath publisher will do the trick, without requiring any new dependency.
Just a matter of placing the files in src/main/resources and add a declaration in the application context.

What worries me however is that we’re talking just about the workspace yaml file.
Is that the only file you’re going to add?
Having a directly reachable API declaration will just reinforce the idea that the API declarations are machine processable,
so it would be best if it did not contain anything that’s not working for generating clients, no?

Cheers
Andrea

···

== GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.

mlebihan · February 15, 2021, 7:38am

Hello Andrea,

Integration test running and Spring compatibiltyMy best target for integration testing is Geoserver project. But currently I don’t know of to set a test, that :

Launch a Geoserver instance, like org.geoserver.web.Start
Then, this one started, starts the Integration test ConfigurationIT on it.

It’s something that comes immediately in my test class that is ran by failsafe plugin (and not surefire) at the integration-test phase of Maven build cycle :

@SpringBootTest(classes = TestApplication.class)
public class ConfigurateurIT {

where TestApplication is a simple declaration of a main class, with a component scan :
@SpringBootApplication
@ComponentScan(basePackages={“fr”, “test”, “com”})
public class TestApplication {
}

I don’t know how to reproduce the same behavior in Geoserver, using Spring and following the Geoserver practices.
Else, it would cause the troubles you wrote for long term maintenance. I just don’t know how to perform the same test in “geoserver way” at the moment.

Spring / Spring boot usagefor Spring / Spring boot usage and OpenApi generation, you’re right, it’s the underlying dependency you see in projects :

io.swagger.core.v3 swagger-core 2.0.2

that is doing the job of publishing swagger-ui.html pages, and springdoc-openapi-ui is just an additional hat over it, I guess to offer services like setting this page on another URL with a property, such as :
springdoc.api-docs.path=/geoserver/rest/v3/api-docs

openapi.json generation

I wasn’t able to set the correct url to go to http://localhost:8080/geoserver/ogc/dggs/api?f=text%2Fhtml on my own geoserver (I had a 404 error code),

but I’ve took a look of what is inside : this part is generating a swagger-ui.html from a static openapi.json (from 2021, January 26th) coming with the last push someone did on git.

It does not generate openapi.json from the existing REST server classes, and does not generate interfaces that server classes would implement instead form that openapi.json, neither ( we should find swagger classes generated for each model class in a target/swagger directory at compile time).
When I search for the string “Access the list of zones in a given DGGS”, that is readable among the documentation on https://github.com/geoserver/geoserver/tree/master/src/community/ogcapi/ogcapi-core/src/main/resources/swagger-ui, in Geoserver sources,
I can’t find it in any other place than openapi.json : if openapi.json was used for generation or implementation, this string should be found in one annotation on a REST server class, or in generated classes in target/swagger sources.
Here, I see only openapi.json in java/main/resources

I don’t understand where openapi.json files are coming from, who is maintaining it and comiting them.
It looks to be like a manually generated file again, having the same trouble than workspaces.yaml and other yaml files : unable to prove that it depicts the real REST methods and classes used.

Regards,

Marc Le Bihan

···

On 13/02/2021 14:00, Andrea Aime wrote:

On Sat, Feb 13, 2021 at 9:28 AM Marc Le Bihan <mlebihan29@anonymised.com> wrote:

Hello,

About workspaces.yaml :I added the corrected yaml file and a test for

[GEOS-9886] workspaces.yaml doesn’t carry correct descriptions of REST services it presents. (as an effect, a working Swagger client cannot be generated for it)

Before creating a pull request to send the corrected geoserver/doc/en/api/1.0.0/workspaces.yaml,
I would like to find a way to put the test I did in geoserver. But it could a long work, because I don’t know how to reproduce all the steps of my own test project in a manner compatible to geoserver project rules.
And I would like to check the next point to see if it can offer a better solution too.

I see you are mentioning a Spring Boot test in the ticket, however, GeoServer does not use nor depends on Spring boot.
But maybe it could be a separate maven module that only runs tests? That one could be using whatever you like.
On the other end, using only technologies/libraries already in use in the project would help in maintenance, modules
that long term maintainers are not familiar with, risk of being kicked to the side and eventually removed.
Then again, it just takes a single developer to maintain a module, if you’re going to be the one, then the above is not a problem.

About openapi.json generation from existing REST servicesWhen adding this dependency to an application (I’ve used it’s version 1.5.3, compatible with Spring-boot 2.4.1)
org.springdoc springdoc-openapi-ui
Swagger/OpenAPI immediately makes available

http://localhost:8080/swagger-ui.html
that produces the same web interface than the one on Geoserver documentation website.

http://localhost:8080/v3/api-docs
that describes the REST services in a json format (instead of a yaml one, but I don’t know if it matters).

This path can be changed with this property :
springdoc.api-docs.path=/geoserver/rest/v3/api-docs

Would it work without having Spring-boot in the classpath?
I’ll also note the ogc-api modules are already publishing an API endpoint that renders with Swagger, without
that particular maven dependency, e.g.:
https://tb16.geo-solutions.it/geoserver/ogc/dggs/api?f=text%2Fhtml

The setup for the OGC APIs is different, because the yamls have to be adapted on the fly based on the collections
in the server, which involves reading a yaml starting template, adapting it with Java code, and re-encoding
it back in json/yaml/html.
The swagger-ui is referenced directly from the classpath:
https://github.com/geoserver/geoserver/tree/master/src/community/ogcapi/ogcapi-core/src/main/resources/swagger-ui

and published using a ClasspathPublisher:

https://github.com/geoserver/geoserver/blob/master/src/community/ogcapi/ogcapi-core/src/main/resources/applicationContext.xml#L24

If all you need is static files, then a single classpath publisher will do the trick, without requiring any new dependency.
Just a matter of placing the files in src/main/resources and add a declaration in the application context.

What worries me however is that we’re talking just about the workspace yaml file.
Is that the only file you’re going to add?
Having a directly reachable API declaration will just reinforce the idea that the API declarations are machine processable,
so it would be best if it did not contain anything that’s not working for generating clients, no?

Cheers
Andrea

== GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.

Andrea_Aime4 · February 15, 2021, 8:31am

Hi Marc,
checking this whole thread again, you will probably have an easier time rewriting the relevant part
of GeoServer from scratch using Spring boot, rather than successfully contributing a single line of code to GeoServer.
The helpful bits on how to expose swagger UI has been completely ignored, and turned back to become another example
on how the current GeoServer setup is broken. I have no desire to continue this conversation any further, maybe some
other core developer will be more motivated.

Andrea

···

Regards, Andrea Aime

== GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.

mlebihan · February 15, 2021, 11:53am

Hi Andrea,

I’m not willing using Spring boot : it’s only the one I used in my own project when I detected the bug and attempted a fix. Now that the fix is done, I would like to port the test from Spring-boot to the Spring of Geoserver, following the project rules, and it’s what I’m writing. This, and nothing else.

Second, I read the whole thread too, and it took time to convince you that the feature of automatic generation from rest services is needed and used in businesses and that it requires correct descriptions for that. Like wsdl of Soap services have to conform to the web-services they are presenting.

When I was first searching for the cause of the issue, I went on Stackoverflow and on XStream project, both showed me that explanations wasn’t easy to find, as Geoserver has plenty of specific way to handle REST services, and these, I can only learn them for you, as I’m actually attempting to correct the problem. For all of us.
In it’s build phase, Geoserver project doesn’t seem to use the content of yaml or openapi.json files, and it doesn’t seem to produce them too. (Or, I would to know where: it’s the goal of my last message).
This make me believe that these files are written manually, as a documentation source, but they might all be unsynchronized. And I would like to find a way to correct this.

The work (if I am lucky) isn’t so hard :

Put OpenAPI annotations on REST server classes.
Copy human written documentation from yaml in these annotation
Ask Geoserver to publish that content from it’s “target” directory classes, and not from a static file in main/resources.
Then thinks will ok for sure.

But starting here, I don’t know enough Geoserver internals to find each thing I would like to find. And understand how everything is working. This is why I’m asking. But I’m asking only to correct the problem globally and the more widely it is possible. If others parts of Geoserver dggs, wfs3 suffer the same problem that yaml files, no worry if we find the accurate way of correcting them easily.

In 2017, when the documentation was written in yaml, I’m sure that it was accurate. Surely, input and output descriptions were ok with the REST services. And because Swagger didn’t offer generation plugin/tools to create clients, it was difficult to check if what was in yaml files were strictly what the server was serving. In 2019, Swagger tool was here, but how writers of yaml documentation of 2017 would have been notified ? They were on other tasks then.
But now that the tool exist, that the problem is detected and that desynchronization make yaml files less useful, it’s time to correct them. I’m on it since 20 or 30 hours. I’m not searching to talk against Geoserver : I’m just trying to see the whole extend of the issue to find a way to correct it everywhere. Because if I take each yaml file one by one and correct them like I did for workspaces.yaml, you understand that this work will take months.

Regards,

Marc Le Bihan

Le lun. 15 févr. 2021 à 09:31, Andrea Aime <andrea.aime@anonymised.com> a écrit :

Hi Marc,
checking this whole thread again, you will probably have an easier time rewriting the relevant part
of GeoServer from scratch using Spring boot, rather than successfully contributing a single line of code to GeoServer.
The helpful bits on how to expose swagger UI has been completely ignored, and turned back to become another example
on how the current GeoServer setup is broken. I have no desire to continue this conversation any further, maybe some
other core developer will be more motivated.

Andrea

On Mon, Feb 15, 2021 at 8:38 AM Marc Le Bihan <mlebihan29@anonymised.com> wrote:

Hello Andrea,

Integration test running and Spring compatibiltyMy best target for integration testing is Geoserver project. But currently I don’t know of to set a test, that :

Launch a Geoserver instance, like org.geoserver.web.Start

Then, this one started, starts the Integration test ConfigurationIT on it.

It’s something that comes immediately in my test class that is ran by failsafe plugin (and not surefire) at the integration-test phase of Maven build cycle :

@SpringBootTest(classes = TestApplication.class)
public class ConfigurateurIT {

where TestApplication is a simple declaration of a main class, with a component scan :
@SpringBootApplication
@ComponentScan(basePackages={“fr”, “test”, “com”})
public class TestApplication {
}

I don’t know how to reproduce the same behavior in Geoserver, using Spring and following the Geoserver practices.
Else, it would cause the troubles you wrote for long term maintenance. I just don’t know how to perform the same test in “geoserver way” at the moment.

Spring / Spring boot usagefor Spring / Spring boot usage and OpenApi generation, you’re right, it’s the underlying dependency you see in projects :
io.swagger.core.v3 swagger-core 2.0.2
that is doing the job of publishing swagger-ui.html pages, and springdoc-openapi-ui is just an additional hat over it, I guess to offer services like setting this page on another URL with a property, such as :
springdoc.api-docs.path=/geoserver/rest/v3/api-docs

openapi.json generation

I wasn’t able to set the correct url to go to http://localhost:8080/geoserver/ogc/dggs/api?f=text%2Fhtml on my own geoserver (I had a 404 error code),

but I’ve took a look of what is inside : this part is generating a swagger-ui.html from a static openapi.json (from 2021, January 26th) coming with the last push someone did on git.

It does not generate openapi.json from the existing REST server classes, and does not generate interfaces that server classes would implement instead form that openapi.json, neither ( we should find swagger classes generated for each model class in a target/swagger directory at compile time).
When I search for the string “Access the list of zones in a given DGGS”, that is readable among the documentation on https://github.com/geoserver/geoserver/tree/master/src/community/ogcapi/ogcapi-core/src/main/resources/swagger-ui, in Geoserver sources,
I can’t find it in any other place than openapi.json : if openapi.json was used for generation or implementation, this string should be found in one annotation on a REST server class, or in generated classes in target/swagger sources.
Here, I see only openapi.json in java/main/resources

I don’t understand where openapi.json files are coming from, who is maintaining it and comiting them.
It looks to be like a manually generated file again, having the same trouble than workspaces.yaml and other yaml files : unable to prove that it depicts the real REST methods and classes used.

Regards,

Marc Le Bihan

On 13/02/2021 14:00, Andrea Aime wrote:

On Sat, Feb 13, 2021 at 9:28 AM Marc Le Bihan <mlebihan29@anonymised.com> wrote:

Hello,

About workspaces.yaml :I added the corrected yaml file and a test for

[GEOS-9886] workspaces.yaml doesn’t carry correct descriptions of REST services it presents. (as an effect, a working Swagger client cannot be generated for it)

Before creating a pull request to send the corrected geoserver/doc/en/api/1.0.0/workspaces.yaml,
I would like to find a way to put the test I did in geoserver. But it could a long work, because I don’t know how to reproduce all the steps of my own test project in a manner compatible to geoserver project rules.
And I would like to check the next point to see if it can offer a better solution too.

I see you are mentioning a Spring Boot test in the ticket, however, GeoServer does not use nor depends on Spring boot.
But maybe it could be a separate maven module that only runs tests? That one could be using whatever you like.
On the other end, using only technologies/libraries already in use in the project would help in maintenance, modules
that long term maintainers are not familiar with, risk of being kicked to the side and eventually removed.
Then again, it just takes a single developer to maintain a module, if you’re going to be the one, then the above is not a problem.

About openapi.json generation from existing REST servicesWhen adding this dependency to an application (I’ve used it’s version 1.5.3, compatible with Spring-boot 2.4.1)
org.springdoc springdoc-openapi-ui
Swagger/OpenAPI immediately makes available

http://localhost:8080/swagger-ui.html
that produces the same web interface than the one on Geoserver documentation website.

http://localhost:8080/v3/api-docs
that describes the REST services in a json format (instead of a yaml one, but I don’t know if it matters).

This path can be changed with this property :
springdoc.api-docs.path=/geoserver/rest/v3/api-docs

Would it work without having Spring-boot in the classpath?
I’ll also note the ogc-api modules are already publishing an API endpoint that renders with Swagger, without
that particular maven dependency, e.g.:
https://tb16.geo-solutions.it/geoserver/ogc/dggs/api?f=text%2Fhtml

The setup for the OGC APIs is different, because the yamls have to be adapted on the fly based on the collections
in the server, which involves reading a yaml starting template, adapting it with Java code, and re-encoding
it back in json/yaml/html.
The swagger-ui is referenced directly from the classpath:
https://github.com/geoserver/geoserver/tree/master/src/community/ogcapi/ogcapi-core/src/main/resources/swagger-ui

and published using a ClasspathPublisher:

https://github.com/geoserver/geoserver/blob/master/src/community/ogcapi/ogcapi-core/src/main/resources/applicationContext.xml#L24

If all you need is static files, then a single classpath publisher will do the trick, without requiring any new dependency.
Just a matter of placing the files in src/main/resources and add a declaration in the application context.

What worries me however is that we’re talking just about the workspace yaml file.
Is that the only file you’re going to add?
Having a directly reachable API declaration will just reinforce the idea that the API declarations are machine processable,
so it would be best if it did not contain anything that’s not working for generating clients, no?

Cheers
Andrea

== GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.

–

Regards, Andrea Aime

== GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.

Andrea_Aime4 · February 15, 2021, 7:04pm

Marc,
automatic generation of APIs from annotated methods is quite likely… impossible (if you want to include the schema of representations, at least).

The REST API was born around 15 years ago, based on one of the first REST books, using the first REST oriented library, Restlet (which we migrated
to Spring MVC a few years ago).
The representations of objects were (and still are) based on the support we have to store the config as XML files, with minimum variations to provide linking
between the resources (when object A refers to object B, we don’t get a nested object, but a link to the relevant resource).

That XML generation is based on XStream, which is using some level of reflection, but has a very large number of customizations, some of which
are mere aliases, others that are full programmatic replacements of the default representation, to get a nicer one. See all the converters
registered in this class: https://github.com/geoserver/geoserver/blob/master/src/main/src/main/java/org/geoserver/config/util/XStreamPersister.java

I am confident the tooling you wish to use could do wonders based on model classes that are annotated with Jackson annotations, but here, we are
light years away. Given some of the conversion bits are programmatic, there is basically no way to get the JSON schemas needed for OpenAPI
but simple code inspection. Even ignoring those bits, the configuration of XStream aliases is fully programmatic, so you’d need to first let XStreamPersister
create a configured XStream instance, and then somehow walk its configuration to generate the schemas.

As a cherry on top of the cake, JSON representation in XStream is just an afterthought, and looks pretty poor, as Gabriel has noted.

A similar situation is happening on the OGC API front, but for different reasons. There we don’t have usage of XStream, and for the simplest methods,
there are actual Jackson annotated classes. However, for some resources, the WFS output formats are used to generate GML, GeoJSON, CSV,
and all other formats that plugins provide. The structure of the response changes by the format, sometimes by the configuration of template driven outputs,
and more importantly, by the structure of the data behind the collections. All of this information can only be known at runtime.

The same goes for the API itself, the document contents can only be fully determined once the list of available collections (layers) and the list
of available pluggable output formats is known.
The yaml files you see in the code come from the OGC API - Features standard itself, and are modified on the fly based on the dynamic information
above.
Also, OGC API plugins could be modifying the structure of the documents on the fly, for example, when plugging in the Styles API, the contents
of the collection documents is enhanced (more links), when plugging in the Tiles API, the contents of the OGC API Features API itself are augmented
with new specific resources handling tiled features.

Again, I don’t see how a build time machinery can handle this type of dynamic behavior.

And oh, the original YAML files were written during the Spring MVC migration by a team composed of documentation writers, with some
help from developers that never saw Swagger before. They likely never have been correct to the point of being useful for client generation.
Back at the time an annotation based approach was also considered, and dropped, due to the dynamic behavior explained above, see
https://github.com/geoserver/geoserver/wiki/REST-API-Refresh#reference-api-documentation-swagger

Hope this helps understanding the problem you’re trying to solve better
Andrea

···

Regards, Andrea Aime

== GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.

jive · February 23, 2021, 7:31pm

Yeah they were intended as documentation and were more complete than what was previously available.

Gabe is it possible to throw out the 2.0 yaml and roll out your 3.0 version as a replacement? That is does it meet the documentation goal?

···

–
Jody Garnett

Gabriel_Roldan1 · February 23, 2021, 11:51pm

Yeah they were intended as documentation and were more complete than what was previously available.

Gabe is it possible to throw out the 2.0 yaml and roll out your 3.0 version as a replacement?

I’ll have some allocated time this or next week to donate the project as a community module.
The port of 2.0 to 3.0 API definitions is incomplete for a number of reasons:

The object model of request and responses in the 2.0 version don’t match what the REST API actually expects and spits out
The migration is painful because it often requires debugging to know exactly what the server does
For the bits I’ve migrated, only cared about the json representation, stripping off the XML and HTML declarations and examples. If it was up to me, I’d keep it that way.

That is does it meet the documentation goal?

Once the migration is finished, it would not only meet the documentation goal but also allow to generate clients in different languages. Yet the mismatch between API responses and some expected request body schemas makes the generated clients still painful to use.

My opinion is still that completing the v2 to v3 migration would make it so much easier to have a v2 rest API with clean contracts and the ability to generate working clients in about any language, which could alleviate the concern Andrea mentioned about having to maintain both API versions. I’d certainly prefer to freeze development on v1 and start out v2 with an api-first approach, it could start being a community module / separate service and see where it goes from there.

···

Gabriel Roldán