[Geoserver-devel] asm java 8 incompatibility breakout meeting

Notes form the breakout meeting, and a few email conversations (around things like release schedule).

GeoServer ASM Java 8 Incompatibility

Action:

  • Jody turn this into a technical debt document

  • All: communicate early and often (seeking a prompt decision/discussion while we are in position to act)

problem

ASM incompatible with Java 8 Byte Code

Q: Like what?

Internet research indicates ASM used to recreate dynamic proxies, so this should only happen if classes uses Java 8 byte code. Andrea researches shows the surefire way to cause this try to proxy a bean that uses Java 8 default method.

Q: Where is this happening? A: Kevin working on a GWC feature branch that causes this issue

The feature branch introduces this problem by writing a method using java 8 syntax, no change to GeoServer interfaces, no introduction of default methods etc…

Kevin has found the problem here (https://github.com/smithkm/geowebcache/commits/wmscap-refresh) in a feature branch of GWC. It actually looks like ASM will die when creating a proxy for any bean that uses Java 8 byte code. Unit tests for GWC do not use spring, so this was only found late in the game during manual testing.

a) Options use Java 8 very very carefully - run with Java 8 and test a lot?

Andrea strongly advocated for this solution, it would be a risk but possibly an acceptable risk.

The approach would be to avoid the use of any Java 8 features … in any bean that will be instaciated by spring.

that is not created by spring (implementation or interface).

Pros:

  • This would keep our current release schedule, opening master up to spring 4 migration in march for a GeoServer 2.10 timeframe.
  • No change to bi-annual release schedule
  • Users could run with “some” confidence (safety in numbers) on Java 8, would really reply on build box and user list testing

Cons:

  • Developers would be unwise to use Java 8 features (since they are risky and may accidentally break things)
  • Risk from GeoTools 15 beans or those working on custom extensions built with Java 8
  • We would still need to migrate to Spring 4 pretty much now, but would have “more time” to get it right
  • Less “pressure” means less chance to get this issue funded and resolved

b) Option Fallback to target=1.7 - don’t use Java 8 byte code anywhere

Is set compile target=1.7 byte code.

How much work? this would be a 2-3 days. Is this worth it?

update: checking now by changing jre and target=1.7 in eclipse shows:

  • 35 errors (mostly lambda and function use)

Pros:

  • This would keep our current release schedule, opening master up to spring 4 migration in march for a GeoServer 2.10 timeframe.
  • No change to bi-annual release schedule
  • Users could run with confidence on Java 8? Meeting security policies without licensing Java 7 from oracle.

Cons:

  • Developers would be unable to use Java 8 features (at least casual use would be caught)
  • Risk from GeoTools 15 beans or those working on custom extensions built with Java 8
  • We would still need to migrate to Spring 4 pretty much now, but would have “more time” to get it right
  • Less “pressure” means less chance to get this issue funded and resolved

c) Option Delay Release / Update now - to update to Spring 4.03 or greater

Migrate to Spring 4 now, delaying the GeoServer 2.9 release.

This requires us to Extend GeoServer 2.8 time line:

  • We will need to backport some work need to backport (UTF-GRID for geosolutions customer commitments, wps remote, etc…)

Checking with key stakeholders:

  • Boundless - in favour of this, willing to commit resources to see this happen
  • GeoSolutions - require a release delay of three months, in favour but will need to scramble to meet customer contracts (example backporting work).

Justin has done some checks in October:

  • servlet api upgrade (at least 3.0)
  • mock runner library does not go that high (spring has its own mock runner)
  • mechanical change, but not dropin replacing- security
  • cas, ldap, security modules in general:
  • Andrea emphasis manual testing
  • Especially manual testing for CAS (it is a rewrite, not just an update)

Want to hear back from christian

  • CAS is a risk identified above

Andrea has some experience running GeoServer 2.9 in production:

  • Building on Java 8 for 2 months, a couple 2.9 systems in preproduction testing

Jody feels this is a bit of a time bomb - but can now understand the geosolutions 3 month delay expectation.

What would a timeframe look like?

  1. Two or three weeks to get something working at all
  2. Migrate/Test problem areas (such as security cas)
  3. Go through careful release and test plan
  4. Q: could we make a test plan over the next couple of days?

What will be affected?

  • Note jdbconfig / jdbcconfig disk quota are hit by this upgrade
  • How much will geofence rest be affected? due to use of spring rest api

Pros:

  • We would update spring now … and avoid what happened to wicket
  • Developers able to use Java 8 features
  • less project risk
  • Users could run with confidence on Java 8 (Meeting security policies without licensing Java 7 from oracle).

Cons:

  • Delay to release schedule (2-3 months), would need to restore schedule or use 9 month schedule
  • can we do this quickly and safely?

On Sat, Feb 27, 2016 at 12:40 AM, Jody Garnett <jody.garnett@anonymised.com>
wrote:

Kevin has found the problem here (
https://github.com/smithkm/geowebcache/commits/wmscap-refresh) in a
feature branch of GWC. It actually looks like ASM will die when creating a
proxy for any bean that uses Java 8 byte code. Unit tests for GWC do not
use spring, so this was only found late in the game during manual testing.

Double checked what's going on here, the issue happens because all Spring
bean instantiation relies on ASM reading
the class, even to just get the constructor parameters used to instantiate
the bean.
Unfortunately when ASM reads the class, it reads everything, not sure
what's needed, and balks at reading the
bytecode of TileLayerDispatcher due to the stream usage it seems.

None of the changes in TileLayerDispatcher seemed to change actual
functionalitiy, they were more of
a replacement of imperative loops with functional equivalents, so I quickly
reverted them, and voilà,
GWC starts up again.
Note that streams are also used in GetCapabilitiesConfiguration.java, but
that does not seem to be
causing issues, because the bean in question is not Spring instantiated.

So it seems that indeed any usage of java 8 syntax will break Spring
ability to work with it. However, it
also seems that using java 8 syntax in stuff that Spring does not have to
read with ASM (e.g.
GetCapabilitiesConfiguration) is fine.
It is however not so clear how deep the rat nest goes, e.g., will ASM have
to read also the classes
of method return types? Probably not. What about constructor arguments...
hum.. yeah, those it
has since they are also spring beans themselves.

Aside, checking on the internet, people just say "if you want to use Java
8, then upgrade to Spring 4",
I haven't found anyone trying to use a mixed approach, Spring 3 + java 7
for spring managed classes + java 8 for
the rest (so no success stories to referer to in this direction I'm afraid)

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------