[Geoserver-devel] JDK 11: Spring 5 update & JDK compatibility issues

GeoServer GeoServer Developer
1

This is pretty long - there is a TL;DR with action items at the bottom.

The Spring 5 update is now largely complete. Pull request here: https://github.com/geoserver/geoserver/pull/3202

There have been substantial changes in security that could do with a thorough review, most everything else has been relatively minor bugfixes. All core tests should be passing when run on Java 11. Some extensions may still be failing.

Separately, tests fail when run on Java 8 (I think GeoTools needs to be built with Java 11 for these failures to occur). These failures seem to be caused by covariant return types introduces in Java 11; basically we are running into the issue described here: https://github.com/apache/felix/pull/114

Note: These failures appear unrelated to the Spring 5 update, they were just masked by other issues until recently.

The test failures are mostly of the form:
No reader for /Users/tbarsballe/repos/GeoServer/geoserver/src/main/target/default7340521811081519346data/watertemp with format ImageMosaic

This is ultimately caused by a JVM failure deep in file handling; typically via GeoTools NIOUtilities - that failure is of the form java.lang.NoSuchMethodError: java.nio.ByteBuffer.flip()Ljava/nio/ByteBuffer;
I’ve looked into fixing this on GeoTools - an initial attempt is on the jdk11 branch here: https://github.com/geotools/geotools/commit/5246a20249bc42ab0b047170003fddf4d36e4ec3

That wasn’t entirely successful, so I’ve attempted a more in-depth fix, currently in a PR here: https://github.com/geotools/geotools/pull/2141
Local testing has not been promising; It seems like more than just ByteBuffer is having this problem - I have most recently been seeing failures of the form java.lang.NoSuchMethodError: java.nio.CharBuffer.flip()Ljava/nio/CharBuffer;

So, it seems like we will either need a more in-depth fix, or will not be able to build JDK8 artifacts using JDK11, even if compliance level is 1.8.

TL;DR - What needs doing:

  • Review https://github.com/geoserver/geoserver/pull/3202 , especially the gs-security changes

  • Test that PR / the spring-5.1 branch with some old data dirs that have encoded passwords to verify encoding/decoding is still backwards compatible (the unit tests ought to have caught this, but I don’t quite trust them that far)

  • Fix any remaining test failures from the Spring 5.1 update (e.g. in extensions)

  • Determine a solution for JDK8 vs JDK11 artifacts, based on the information above.

Cheers,

Torben

2

This is pretty long - there is a TL;DR with action items at the bottom.

The Spring 5 update is now largely complete. Pull request here: https://github.com/geoserver/geoserver/pull/3202

I’ll have a look, although as said, I don’t know much about how the spring security integration works. I also need to help
Ian with the EMF migration.

There have been substantial changes in security that could do with a thorough review, most everything else has been relatively minor bugfixes. All core tests should be passing when run on Java 11. Some extensions may still be failing.

I see Grib and NetCDF failing, maybe due to the byte casing thing

Separately, tests fail when run on Java 8 (I think GeoTools needs to be built with Java 11 for these failures to occur). These failures seem to be caused by covariant return types introduces in Java 11; basically we are running into the issue described here: https://github.com/apache/felix/pull/114

Note: These failures appear unrelated to the Spring 5 update, they were just masked by other issues until recently.

The test failures are mostly of the form:
No reader for /Users/tbarsballe/repos/GeoServer/geoserver/src/main/target/default7340521811081519346data/watertemp with format ImageMosaic

This is ultimately caused by a JVM failure deep in file handling; typically via GeoTools NIOUtilities - that failure is of the form java.lang.NoSuchMethodError: java.nio.ByteBuffer.flip()Ljava/nio/ByteBuffer;
I’ve looked into fixing this on GeoTools - an initial attempt is on the jdk11 branch here: https://github.com/geotools/geotools/commit/5246a20249bc42ab0b047170003fddf4d36e4ec3

That wasn’t entirely successful, so I’ve attempted a more in-depth fix, currently in a PR here: https://github.com/geotools/geotools/pull/2141
Local testing has not been promising; It seems like more than just ByteBuffer is having this problem - I have most recently been seeing failures of the form java.lang.NoSuchMethodError: java.nio.CharBuffer.flip()Ljava/nio/CharBuffer;

So, it seems like we will either need a more in-depth fix, or will not be able to build JDK8 artifacts using JDK11, even if compliance level is 1.8.

I proposed from the beginning to keep on building with JDK8, as a build with JDK11 won’t warns us (afaik) of using new API by accident. This covariant type issue is even worse :slight_smile:
No matter what we do, we’ll need two separate builds, one with JDK8 and one with JDK11, with two separate local repositories, probably
with the JDK11 one not deploying to the remote repositories.

Cheers
Andrea

···

GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.