[Geoserver-devel] Image I/O problems

Justin_Deoliveira · March 23, 2007, 2:03am

Hi all,

I seem to stuck in a jai nightmare and cant get out. Let me tell you all about it.

Java 4 + JAI + Image I/O:
- "PNGException: codecLib error ..." when previewing nurc:mosaic

Java 5 + JAI + Image I/O:
- "sealing violation: package com.sun.media.imageioimpl.stream is sealed" on startup and GeoServer dies

Java 4 + JAI ( No Image I/O ):
- works!!
Java 5 + JAI ( No Image I/O ):
- works!!

Java 4 + JAI + Image I/O with Tomcat:
- "PNGException: codecLib error ..." when previewing nurc:mosaic

Java 5 + JAI + Image I/O with Tomcat:
Java 4 + JAI ( No Image I/O ) with Tomcat:
- works!!

Working in tomcat with image i/o and not in the binary release has me boggled.

I should also state that I was able to replicate this in two different environments, one linux and one running windows.

Anyways, I have posted the artifacts for the 1.5.0-RC3 release on sourceforge but have not made it public yet. If people could try them out to see if they can replicate any of this behaviour that would be good.

If it isn't just me I see this as a blocker for release.

-Justin

--
Justin Deoliveira
The Open Planning Project
http://topp.openplans.org

davidblasby · March 23, 2007, 5:07am

On 3/22/07, Justin Deoliveira <jdeolive@anonymised.com> wrote:
...

Java 5 + JAI + Image I/O:
- "sealing violation: package com.sun.media.imageioimpl.stream is
sealed" on startup and GeoServer dies

...

I've sent several old messages out about the sealing conflict between
the native-installed-by-JAI jars and the
compiled-from-source-and-put-in-lib JAI jars.

The problem is that both of these jars are "sealed" and it causes
issues when you pull plugins out of the
compiled-from-source-and-put-in-lib JAI jars and other classes out of
the native-jars-installed-by-JAI. Search for old messages on this
topic should shed light on the issues and how to solve them.

I dont know what the other problems are about....

dave

Andrea_Aime3 · March 23, 2007, 7:42am

David Blasby ha scritto:

On 3/22/07, Justin Deoliveira <jdeolive@anonymised.com> wrote:
...

Java 5 + JAI + Image I/O:
- "sealing violation: package com.sun.media.imageioimpl.stream is
sealed" on startup and GeoServer dies

...

I've sent several old messages out about the sealing conflict between
the native-installed-by-JAI jars and the
compiled-from-source-and-put-in-lib JAI jars.

The problem is that both of these jars are "sealed" and it causes
issues when you pull plugins out of the
compiled-from-source-and-put-in-lib JAI jars and other classes out of
the native-jars-installed-by-JAI. Search for old messages on this
topic should shed light on the issues and how to solve them.

David, many thanks for shading some light on this issue.
I thought that having native jai installed would completely override
the in-classpath one (by the extensions mechanism, jdk/jre/lib/ext)
but it seems in some cases this is not true.

In particular, given Justin tests, it seems the issue has something
to do with the way Jetty and Tomcat class loaders work, since the
issue is there in one case, but not in the other.

I'll have a look at the mailing list archives, thanks muchly.

Cheers
Andrea

Andrea_Aime3 · March 23, 2007, 8:21am

Justin Deoliveira ha scritto:

Hi all,

I seem to stuck in a jai nightmare and cant get out. Let me tell you all about it.

Java 4 + JAI + Image I/O:
   - "PNGException: codecLib error ..." when previewing nurc:mosaic

Java 5 + JAI + Image I/O:
   - "sealing violation: package com.sun.media.imageioimpl.stream is sealed" on startup and GeoServer dies

Java 4 + JAI ( No Image I/O ):
  - works!!
Java 5 + JAI ( No Image I/O ):
  - works!!

Java 4 + JAI + Image I/O with Tomcat:
  - "PNGException: codecLib error ..." when previewing nurc:mosaic

Java 5 + JAI + Image I/O with Tomcat:
Java 4 + JAI ( No Image I/O ) with Tomcat:
  - works!!

Working in tomcat with image i/o and not in the binary release has me boggled.

I should also state that I was able to replicate this in two different environments, one linux and one running windows.

Tried out on my machine, and yes, I can replicate your issues 100%.
In particular, it seems the issues occur only when we use Jetty as
a container, whilst with Tomcat, or Jetty launched from Eclipse (thus,
used an embedded, instead of as container) everything works fine.

I'm going to check with other containers too (JBoss, Jetty, eventually
Glassfish) and try to find out what's so special in Jetty container
classloading. My wild guess is that Jetty classloading breaks the
java extensions mechanism (http://java.sun.com/j2se/1.5.0/docs/guide/extensions/). I may be totally off the mark thought.

Cheers
Andrea

Andrea_Aime3 · March 23, 2007, 11:01am

Andrea Aime ha scritto:

Justin Deoliveira ha scritto:

Ok, I've done some extensive testing me too, and confirmed that indeed
Jetty classloading is broken, what's worse, it's broken by design.
They decided to do their own stuff and not to call the parent classloader before loading with their own (classloading protocol violation).
Luckily, they left a variable you can set to get back the standard classloading behaviour, and it must be set in etc/jetty.xml:

<Call class="org.mortbay.jetty.webapp.WebAppContext" name="addWebApplications">
       <Arg><Ref id="contexts"/></Arg>
       <Arg><SystemProperty name="jetty.home" default="."/>/webapps</Arg>
       <Arg><SystemProperty name="jetty.home" default="."/>/etc/webdefault.xml</Arg>
       <Arg type="boolean">True</Arg> 
       <Arg type="boolean">True</Arg> 
     </Call>

The latter "boolean" argument is false by default.

Unfortunately, it seems we're screwed anyways under jdk 1.4.2_12, and native jai components won't work anyways. There is a way to disable them using -Dcom.sun.media.imageio.disableCodecLib=true
when launching jetty.
I'll refer to this setting as DCL in the following.

Leaving out the case were you don't have native jai installed (this won't break anyways afaik), here is a brief summary:
+ tomcat 5.0.30 + j2sdk1.4.2_12 + native jai: works
+ tomcat 5.5.28 + j2sdk1.5.0_09 + native jai: works
* jetty 6.0.1 + j2sdk1.5.0_09 + native jai: sealing violation!
+ jetty 6.0.1 + cl fix + j2sdk1.5.0_09 + native jai: works
* jetty 6.0.1 + cl fix + j2sdk1.4.2_12 + native jai: codelib error!
+ jetty 6.0.1 + cl fix + DCL + j2sdk1.4.2_12 + native jai: works
* jboss 3.2.8 + j2sdk1.4.2_12 + native jai: codelib errror!
+ jboss 3.2.8 + DCL + j2sdk1.4.2_12 + native jai: works
+ jboss 3.2.8 + j2sdk1.5.0_09 + native jai: works
* jboss 4.0.5GA + j2sdk1.4.2_12 + native jai: codelib errror!
+ jboss 4.0.5GA + DCL + j2sdk1.4.2_12 + native jai: works
+ jboss 4.0.5GA + j2sdk1.5.0_09 + native jai: works

That is, 1.4.2 + codelib + webapp classloaders other than tomcat one -> boom!
I also tried out with jdk 1.4.2_13 but it does not make a difference.

I tried to test with resin too, but there are too many issues.
First the xml parsers, since they do impose their own, and our
xml config reader breaks, once that is fixed by changing our
web.xml, struts will break. I've opened GEOS-984 to track this.

So, in the end it seems we can solve the issue by either telling
people not to use jdk 1.4.2 or, if they have to, not to install
java image i/o native componentes (jai itself seems to be fine),
or to avoid playing with png images, or finally to set the -Dcom.sun.media.imageio.disableCodecLib=true property.
If we want to go an extra mile, we can check if jdk is 1.4,
if so, disable the codec lib by default unless people enabled
it explicitly (which people running tomcat will want to do).

Another minor issue is that even with DCL, the first time
we serve the mosaic the image is distorted, whilst subsequent requests
do work fine. This does not happen with jdk 1.5.

The story does not end here thought. Our binary release is still
broken, even when forcing jetty to behave, because doing so
trigger other problems, and struts ui won't start up.
Well, turns out most of these problems are there because we
have a ton of needless libraries in the main /lib, such as
spring, the db2 jdbc driver, jta and whatnot.
Everything works fine if the /lib is purged to have just the
jars a standard jetty distribution has:

23/03/2007 08.47 1.033.875 ant-1.6.4.jar
23/03/2007 08.47 112.341 commons-el-1.0.jar
23/03/2007 08.47 407.502 jasper-compiler-5.5.15.jar
23/03/2007 08.47 1.209.873 jasper-compiler-jdt-5.5.15.jar
23/03/2007 08.47 77.056 jasper-runtime-5.5.15.jar
23/03/2007 08.47 6.340 jcl104-over-slf4j-1.0.1.jar
23/03/2007 08.47 429.619 jetty-6.0.1.jar
23/03/2007 08.47 107.342 jetty-util-6.0.1.jar
23/03/2007 08.47 50.493 jsp-api-2.0.jar
23/03/2007 08.47 136.279 servlet-api-2.5-6.0.1.jar
23/03/2007 08.47 13.873 slf4j-simple-1.0.1.jar
23/03/2007 08.47 1.010.675 xercesImpl-2.6.2.jar
23/03/2007 08.47 124.724 xmlParserAPIs-2.6.2.jar

Well, this is it. We just have to decide how to deal with the
jdk 1.4.2 issues, and then we should be good to release.

Cheers
Andrea