[Geoserver-devel] GeoTools / GeoServer Meeting 2018-10-16

GeoServer GeoServer Developer
1

Participants

Kevin Smith

Nuno Oliveira

Torben Barsballe

Jody Garnett

Jukka Rahkonen

Actions- Torben: update the Java 11 GSIP with links to email discussion

  • Torben: install jdk 11 on build.geoserver.org

  • Jody: add a “preflight” checklist for people attending

  • Start email list discussion on GWC Site / GS Docs and GitHub Pages

Actions From Last Meeting- Jody: to request funding from board (done)

  • All: Please promote via twitter etc (done)

  • Jody: Action schedule breakout meeting for the above time (moved to email)

Agenda- OSGeo Updates

  • Code Sprint

  • GeoServer Doc & GeoWebCache site hosting

OSGeo Updates

Code sprint approved:

  • $2000 approved, will update sprint page to list OSGeo as host

Jody commented on osgeo todo about CITE tests:

Code Sprint

Wiki:

Finances:

  • We have 2-3 sponsors now, and OSGeo hosting

  • Budget

  • Priority: airfare > accommodation > catering

Participants:

  • We are down 12 people

Andrea has been adding a bunch of useful information on the mailing lists

  • One concern: Build times

  • Need to capture discussion in an organized place

Sprint plan stages (from email), goal is to have something “shippable” at the end of each stage:

  1. build and run in JDK 11
    Everything in our stack (jaitools, jai-ext, imageio-ext, geotools, geowebcache, geoserver) builds and run without any flag added, off the classpath (it’s ok to have warnings). This will allow us to get JDK 11 builds going.

  2. Compile: Andrea has made considerable progress here, to mirror you need to check out and build locally all the dependencies

  3. Pass tests: This may require updating some dependencies
    (this can be split up)

  4. Reduce warnings from Dependency Analysis Tools
    Get as much as possible build and run without warnings.

  5. This cannot be automated, and is y we have so many people in the sprint!

  6. Module repackage
    Add automatic module descriptors, eliminate split packages in library projects

  7. Repackage

  8. make sure we can run a true module app depending on the automatic modules (idea, we could use the demo module, already depending on many of the others, and make that one a true module?).

  9. Adjust imports and the like as needed in all projects, try to collect migration scripts to help others do the same.

  10. org.opengis repackage
    Swich gt-api away from using org.opengis package, upgrade everything else to follow

  11. repackage

  12. update emf models and regenerate

How to coordinate across teams with 8 hour time difference:

  • If possible teams could work independently

  • Spring 5 upgrade could happen concurrently with logging upgrade? More generally, dependency updates between GeoTools and GeoServer

  • Making test cases work

  • Fixing

  • We need to coordinate a hand over of work from one team to the next

  • commit everything to shared branch

  • Hangout with the next person taking over

  • Use a spreadsheet to communicate status, progress

  • Use gitter to communicate early and often, stuck for 15 mins ask for help!

Actions for this week:

GeoServer Doc & GeoWebCache site hosting

The server “wedge” running this stuff is decommissioned:

  • docs.geoserver.org

  • this is too big for githubpages? (Limit is 1G, docs currently ~7G due to point versions from releases)

  • We could only publish stable, maintenance, master

  • Alternative publish to S3

  • http://geowebcache.org

  • github pages

  • need to update build box

Discussion:

Action:

  • Email lists on this discussion
2

On Tue, Oct 16, 2018 at 10:48 PM Torben Barsballe <tbarsballe@anonymised.com> wrote:

  1. build and run in JDK 11
    Everything in our stack (jaitools, jai-ext, imageio-ext, geotools, geowebcache, geoserver) builds and run without any flag added, off the classpath (it’s ok to have warnings). This will allow us to get JDK 11 builds going.

  2. Compile: Andrea has made considerable progress here, to mirror you need to check out and build locally all the dependencies

  3. Pass tests: This may require updating some dependencies
    (this can be split up)

The PRs that I’ve sent do not just compile, they pass a full “mvn clean install”, that includes tests,
if you take the time to build every bit from sources, in turn. Not sure how this got overlooked.

There is however one catch in the GeoServer one, everything builds in core and extensions, however, while the security LDAP module builds, but the tests are actually skipped because the embedded LDAP server used for tests is not starting up (I’m guessing the config files are not compatible anymore, but I’m not sure). We need an LDAP expert to fix it.

The imageio-ext and GeoWebCache pull requests instead are ready to go, they could be merged now on master. The GeoTools one had a few issues that I fixed, it’s ready to go too. Finally jai-ext goes through a full “mvn install” without any change.

  1. Reduce warnings from Dependency Analysis Tools
    Get as much as possible build and run without warnings.

  2. This cannot be automated, and is y we have so many people in the sprint!

The “fix” column advertises using “add-modules” as the solution, but there might be better ways,
such as upgrading the dependencies or changing the code we’re using. I want to try and persue those before
giving up on what looks like a band-aid.

  1. Module repackage
    Add automatic module descriptors, eliminate split packages in library projects

  2. Repackage

  3. make sure we can run a true module app depending on the automatic modules (idea, we could use the demo module, already depending on many of the others, and make that one a true module?).

  4. Adjust imports and the like as needed in all projects, try to collect migration scripts to help others do the same.

  5. org.opengis repackage
    Swich gt-api away from using org.opengis package, upgrade everything else to follow

  6. repackage

  7. update emf models and regenerate

Did anyone look at how to do this step? Newer version of Eclipse might need EMF/XSD updates and possibly changes in the product structure, picking the right version used to
generate each bit is quite messy. Last few times I did, could not get the current Eclipse version to work, and had to do a history search and get the Eclipse version that was “current”
when the EMF module in question was generated in order to modify the beans.

GeoServer Doc & GeoWebCache site hosting

The server “wedge” running this stuff is decommissioned:

  • docs.geoserver.org

  • this is too big for githubpages? (Limit is 1G, docs currently ~7G due to point versions from releases)

  • We could only publish stable, maintenance, master

Sourceforge has the zip of the HTML docs for every release, so keeping the last 3 would not be a problem

Discussion:

“everyone can edit”… but everyone already can edit the docs, I assume you’re talking about the https://docs.geoserver.org/
landing page right?

  • docs.geoserver.org has already been migrated to S3? DNS not changed over yet.

  • swagger docs don’t like being hosted statically, need to investigate and see if there is a workaround.

I am confused about this bit, the swagger UI I know it’s pure HTML/javascript, all one has to do is to point to the YAML/JSON definition
of the API?

The output of this looks better than Swagger imho.

Cheers
Andrea

==

GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.

3

Hi Andrea

Some quick responses, based on what was discussed:

Pass tests: This may require updating some dependencies
(this can be split up)

The PRs that I’ve sent do not just compile, they pass a full “mvn clean install”, that includes tests,
if you take the time to build every bit from sources, in turn. Not sure how this got overlooked.

It didn’t get overlooked so much as no one in the meeting had run those branches locally yet, and no one knew for sure if this was the case. Thanks for the clarification, this point can be removed from the plan in this case.

Discussion:

“everyone can edit”… but everyone already can edit the docs, I assume you’re talking about the https://docs.geoserver.org/
landing page right?

Yes, we were talking about the landing page (which isn’t even in version control)

  • docs.geoserver.org has already been migrated to S3? DNS not changed over yet.

  • swagger docs don’t like being hosted statically, need to investigate and see if there is a workaround.

I am confused about this bit, the swagger UI I know it’s pure HTML/javascript, all one has to do is to point to the YAML/JSON definition
of the API?

Yeah, in theory it should be able to be host the swagger docs statically, but for some reason this doesn’t work. We need to figure out why, given that this is something that ought to be trivial.

The output of this looks better than Swagger imho.

That does look nice. If there is no easy way of getting the current docs to work statically, that could be an alternative.

Cheers,
Torben

4

The PRs that I’ve sent do not just compile, they pass a full “mvn clean install”, that includes tests,
if you take the time to build every bit from sources, in turn. Not sure how this got overlooked.

It is more we did not have the information readily on hand, trying to assemble it now :slight_smile:

There is however one catch in the GeoServer one, everything builds in core and extensions, however, while the security LDAP module builds, but the tests are actually skipped because the embedded LDAP server used for tests is not starting up (I’m guessing the config files are not compatible anymore, but I’m not sure). We need an LDAP expert to fix it.

Will make a note

The imageio-ext and GeoWebCache pull requests instead are ready to go, they could be merged now on master. The GeoTools one had a few issues that I fixed, it’s ready to go too. Finally jai-ext goes through a full “mvn install” without any change.

Do we want to merge each stage to master? or work on a jdk11 branch in each repository…

  1. Reduce warnings from Dependency Analysis Tools
    Get as much as possible build and run without warnings.

  2. This cannot be automated, and is y we have so many people in the sprint!

The “fix” column advertises using “add-modules” as the solution, but there might be better ways,

such as upgrading the dependencies or changing the code we’re using. I want to try and persue those before
giving up on what looks like a band-aid.

I copied that over from before, updating dependencies, or fixing our code, is what this stage is about.

  1. org.opengis repackage
    Swich gt-api away from using org.opengis package, upgrade everything else to follow

  2. repackage

  3. update emf models and regenerate

Did anyone look at how to do this step? Newer version of Eclipse might need EMF/XSD updates and possibly changes in the product structure, picking the right version used to
generate each bit is quite messy. Last few times I did, could not get the current Eclipse version to work, and had to do a history search and get the Eclipse version that was “current”
when the EMF module in question was generated in order to modify the beans.

Have not looked yet, expect this task will fall to me as one of the few still using eclipse.

“everyone can edit”… but everyone already can edit the docs, I assume you’re talking about the https://docs.geoserver.org/
landing page right?

Yes, right now we reply on the build server running a script to unpack the docs for each release. Only a few boundless people had access to the docs.geoserver.org machine (in case that script went wrong).

The output of this looks better than Swagger imho.

It is node based, but generates static output.

5

Looked quickly, the https://docs.geoserver.org/latest/en/api/#/1.0.0/datastores.yaml URL
tries to load the YML at https://docs.geoserver.org/1.0.0/datastores.yaml while it’s at https://docs.geoserver.org/latest/en/api/1.0.0/datastores.yaml

Maybe stumped by the “#” mark? Don’t know

Cheers
Andrea

···

GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.