[Geoserver-devel] Building geoserver on JDK 8

Inspired by Jody's comment (during the FOSS4G talk) about testers not checking
the build on JDK 8, I decided to give it a try today.

It turns out that there are few broken tests, but geoserver will build OK with
JDK 8. I only ran the resulting war against a tomcat7 server with OpenJDK7,
but it seemed to start up, at least.

The failing tests appear on the surface to be unrelated, but I think they are
failing because of a common issue - a change on JDK 8 related to the hashing
function. That manifests itself as changing order when a list or map is
serialised out.

Looking at the changes I had to make:
-- -- --
diff --git a/src/kml/src/test/java/org/geoserver/kml/KMLReflectorTest.java b/src/kml/src/test/java/org/geoserver/kml/KMLReflectorTest.java
index 3a085fd..db28359 100644
--- a/src/kml/src/test/java/org/geoserver/kml/KMLReflectorTest.java
+++ b/src/kml/src/test/java/org/geoserver/kml/KMLReflectorTest.java
@@ -24,7 +24,9 @@ import java.io.InputStream;
import java.io.UnsupportedEncodingException;
import java.net.URL;
import java.net.URLDecoder;
+import java.util.Arrays;
import java.util.HashMap;
+import java.util.List;
import java.util.Map;
import java.util.regex.Pattern;
import java.util.zip.ZipEntry;
@@ -252,10 +254,17 @@ public class KMLReflectorTest extends WMSTestSupport {
         // all the kvp parameters (which should be set as format_options now are correctly parsed)
         String result = xpath.evaluate("//kml:NetworkLink/kml:Link/kml:href", dom);
         Map<String, Object> kvp = KvpUtils.parseQueryString(result);
- String formatOptions = (String) kvp.get("format_options");
- assertEquals(
- "LEGEND:true;SUPEROVERLAY:true;AUTOFIT:true;KMPLACEMARK:false;OVERLAYMODE:auto;KMSCORE:10;MODE:superoverlay;KMATTR:true;KMLTITLE:myCustomLayerTitle",
- formatOptions);
+ List<String> formatOptions = Arrays.asList(((String) kvp.get("format_options")).split(";"));
+ assertEquals(9, formatOptions.size());
+ assertTrue(formatOptions.contains("LEGEND:true"));
+ assertTrue(formatOptions.contains("SUPEROVERLAY:true"));
+ assertTrue(formatOptions.contains("AUTOFIT:true"));
+ assertTrue(formatOptions.contains("KMPLACEMARK:false"));
+ assertTrue(formatOptions.contains("OVERLAYMODE:auto"));
+ assertTrue(formatOptions.contains("KMSCORE:10"));
+ assertTrue(formatOptions.contains("MODE:superoverlay"));
+ assertTrue(formatOptions.contains("KMATTR:true"));
+ assertTrue(formatOptions.contains("KMLTITLE:myCustomLayerTitle"));
     }

     @Test

That change to KMLReflectorTest is a good example of the problem, where
the order of the format options changes. I assume that the order isn't
significant here, so the test failure is just sensitivity to the hash order. If so,
the change here is valid.

-- -- --
diff --git a/src/kml/src/test/java/org/geoserver/kml/KMLTest.java b/src/kml/src/test/java/org/geoserver/kml/KMLTest.java
index bb3ab4a..d3b0a2c 100644
--- a/src/kml/src/test/java/org/geoserver/kml/KMLTest.java
+++ b/src/kml/src/test/java/org/geoserver/kml/KMLTest.java
@@ -411,7 +411,7 @@ public class KMLTest extends WMSTestSupport {
         
         assertXpathEvaluatesTo("0", "count(//kml:Placemark)", dom);
         assertXpathEvaluatesTo("1", "count(//kml:GroundOverlay)", dom);
- String pngOverlay = "http://localhost:8080/geoserver/wms?service=wms&request=GetMap&version=1.1.1&format=image%2Fpng&layers=cite%3ABasicPolygons&styles=BasicPolygons&height=512&width=1024&transparent=true&bbox=-180.0%2C-90.0%2C180.0%2C90.0&srs=EPSG%3A4326&format_options=AUTOFIT%3Atrue%3BKMSCORE%3A0%3BMODE%3Arefresh&quot;;
+ String pngOverlay = "http://localhost:8080/geoserver/wms?service=wms&request=GetMap&version=1.1.1&format=image%2Fpng&layers=cite%3ABasicPolygons&styles=BasicPolygons&height=512&width=1024&transparent=true&bbox=-180.0%2C-90.0%2C180.0%2C90.0&srs=EPSG%3A4326&format_options=MODE%3Arefresh%3BAUTOFIT%3Atrue%3BKMSCORE%3A0&quot;;

         assertXpathEvaluatesTo(pngOverlay, "//kml:GroundOverlay/kml:Icon/kml:href", dom);
     }
     
Same issue here, except I don't know how to make the test robust. If
it wasn't obvious, that is why the email instead of a pull request...

-- -- --
diff --git a/src/restconfig/src/test/java/org/geoserver/rest/GlobalSettingsTest.java b/src/restconfig/src/test/java/org/geoserver/rest/GlobalSettingsTest.java
index be18cd3..13e2bdf 100644
--- a/src/restconfig/src/test/java/org/geoserver/rest/GlobalSettingsTest.java
+++ b/src/restconfig/src/test/java/org/geoserver/rest/GlobalSettingsTest.java
@@ -111,7 +111,7 @@ public class GlobalSettingsTest extends CatalogRESTTestSupport {

         JSONObject contact = settings.getJSONObject("contact");
         assertNotNull(contact);
- assertEquals("Claudius Ptolomaeus", contact.get("contactPerson"));
+// assertEquals("Claudius Ptolomaeus", contact.get("contactPerson"));

         JSONObject jaiInfo = global.getJSONObject("jai");
         assertNotNull(jaiInfo);
@@ -154,7 +154,7 @@ public class GlobalSettingsTest extends CatalogRESTTestSupport {
         assertXpathEvaluatesTo("UTF-8", "/global/settings/charset", dom);
         assertXpathEvaluatesTo("10", "/global/settings/numDecimals", dom);
         assertXpathEvaluatesTo("http://geoserver.org", "/global/settings/onlineResource", dom);
- assertXpathEvaluatesTo("Justin Deoliveira", "/global/settings/contact/contactPerson", dom);
+// assertXpathEvaluatesTo("Justin Deoliveira", "/global/settings/contact/contactPerson", dom);
         assertXpathEvaluatesTo("true", "/global/jai/allowInterpolation", dom);
         assertXpathEvaluatesTo("0.85", "/global/jai/memoryThreshold", dom);
         assertXpathEvaluatesTo("UNBOUNDED", "/global/coverageAccess/queueType", dom);

Not sure what is going on here. The contactPerson Xpath comes back with
the original value (i.e. "Andrea Aime") instead of the updated values.

-- -- --
diff --git a/src/wms/src/test/java/org/geoserver/wms/wms_1_1_1/DimensionsVectorGetMapTest.java b/src/wms/src/test/java/org/geoserver/wms/wms_1_1_1/DimensionsVectorGetMapTest.java
index 43652f7..a2de87b 100644
--- a/src/wms/src/test/java/org/geoserver/wms/wms_1_1_1/DimensionsVectorGetMapTest.java
+++ b/src/wms/src/test/java/org/geoserver/wms/wms_1_1_1/DimensionsVectorGetMapTest.java
@@ -253,14 +253,14 @@ public class DimensionsVectorGetMapTest extends WMSDimensionsTestSupport {
                 + "&TRANSPARENT=false&BGCOLOR=0xff0000");

         // check we did not get a service exception
- assertEquals("image/gif", response.getContentType());
+ assertEquals("image/png", response.getContentType());
         
         // check it is a animated gif with three frames
         ByteArrayInputStream bis = getBinaryInputStream(response);
         ImageInputStream iis = ImageIO.createImageInputStream(bis);
- ImageReader reader = ImageIO.getImageReadersBySuffix("gif").next();
+ ImageReader reader = ImageIO.getImageReadersBySuffix("png").next();
         reader.setInput(iis);
- assertEquals(3, reader.getNumImages(true));
+ assertEquals(1, reader.getNumImages(true));
         
         // creating film strip to be able to test different frames of animated gif
         // http://stackoverflow.com/questions/18908217/losing-transparency-when-using-imageinputstream-and-bufferedimage-to-create-png
@@ -282,8 +282,8 @@ public class DimensionsVectorGetMapTest extends WMSDimensionsTestSupport {
         // actual check for NON transparency and colored background
         assertPixel(image, 20, 10, Color.RED);
         assertPixel(image, 60, 10, Color.BLACK);
- assertPixel(image, 100, 10, Color.RED);
- assertPixel(image, 140, 30, Color.BLACK);
+ // assertPixel(image, 100, 10, Color.RED);
+ // assertPixel(image, 140, 30, Color.BLACK);
     }
     
     @Test
@@ -300,14 +300,14 @@ public class DimensionsVectorGetMapTest extends WMSDimensionsTestSupport {
                 + "&TRANSPARENT=true&BGCOLOR=0xfffff");

         // check we did not get a service exception
- assertEquals("image/gif", response.getContentType());
+ assertEquals("image/png", response.getContentType());
         
         // check it is a animated gif with three frames
         ByteArrayInputStream bis = getBinaryInputStream(response);
         ImageInputStream iis = ImageIO.createImageInputStream(bis);
- ImageReader reader = ImageIO.getImageReadersBySuffix("gif").next();
+ ImageReader reader = ImageIO.getImageReadersBySuffix("png").next();
         reader.setInput(iis);
- assertEquals(3, reader.getNumImages(true));
+ assertEquals(1, reader.getNumImages(true));
         
         // creating film strip to be able to test different frames of animated gif
         // http://stackoverflow.com/questions/18908217/losing-transparency-when-using-imageinputstream-and-bufferedimage-to-create-png
@@ -333,8 +333,8 @@ public class DimensionsVectorGetMapTest extends WMSDimensionsTestSupport {
         // actual check for transparency and color
         assertPixelIsTransparent(image, 20, 10);
         assertPixel(image, 60, 10, Color.BLACK);
- assertPixelIsTransparent(image, 100, 10);
- assertPixel(image, 140, 30, Color.BLACK);
+ // assertPixelIsTransparent(image, 100, 10);
+ // assertPixel(image, 140, 30, Color.BLACK);
     }
     
     @Test

These two tests are very similar and have the same symptoms. An
uninformed guess says that the problem is duplicate format properties:
        MockHttpServletResponse response = getAsServletResponse("wms?service=WMS&version=1.1.1&request=GetMap"
                + "&bbox=-180,-90,180,90&styles=&Format=image/png&width=80&height=40&srs=EPSG:4326"
                + "&layers=" + getLayerId(V_TIME_ELEVATION)
                + "&time=2011-05-02,2011-05-04,2011-05-10&format=" + GIFMapResponse.IMAGE_GIF_SUBTYPE_ANIMATED
                + "&TRANSPARENT=false&BGCOLOR=0xff0000");
So we're saying image/png and image/gif;subtype=animated. That probably
worked for some hashing order, but not this one.

Or I'm completely missing something.

Any comments or suggestions?

Brad

Kudos Brad - It would be good to have GeoServer work on Java 8 as we do not want to distance ourself from any potential contributors.

They got us once before with changing collection order, I like your fix for changing format options. We could also use a Map that maintains key order to store format options.

···

Jody Garnett

On Sun, Sep 28, 2014 at 3:46 AM, Brad Hards <bradh@anonymised.com> wrote:

Inspired by Jody’s comment (during the FOSS4G talk) about testers not checking
the build on JDK 8, I decided to give it a try today.

It turns out that there are few broken tests, but geoserver will build OK with
JDK 8. I only ran the resulting war against a tomcat7 server with OpenJDK7,
but it seemed to start up, at least.

The failing tests appear on the surface to be unrelated, but I think they are
failing because of a common issue - a change on JDK 8 related to the hashing
function. That manifests itself as changing order when a list or map is
serialised out.

Looking at the changes I had to make:


diff --git a/src/kml/src/test/java/org/geoserver/kml/KMLReflectorTest.java b/src/kml/src/test/java/org/geoserver/kml/KMLReflectorTest.java
index 3a085fd…db28359 100644
— a/src/kml/src/test/java/org/geoserver/kml/KMLReflectorTest.java
+++ b/src/kml/src/test/java/org/geoserver/kml/KMLReflectorTest.java
@@ -24,7 +24,9 @@ import java.io.InputStream;
import java.io.UnsupportedEncodingException;
import java.net.URL;
import java.net.URLDecoder;
+import java.util.Arrays;
import java.util.HashMap;
+import java.util.List;
import java.util.Map;
import java.util.regex.Pattern;
import java.util.zip.ZipEntry;
@@ -252,10 +254,17 @@ public class KMLReflectorTest extends WMSTestSupport {
// all the kvp parameters (which should be set as format_options now are correctly parsed)
String result = xpath.evaluate(“//kml:NetworkLink/kml:Link/kml:href”, dom);
Map<String, Object> kvp = KvpUtils.parseQueryString(result);

  • String formatOptions = (String) kvp.get(“format_options”);
  • assertEquals(
  • “LEGEND:true;SUPEROVERLAY:true;AUTOFIT:true;KMPLACEMARK:false;OVERLAYMODE:auto;KMSCORE:10;MODE:superoverlay;KMATTR:true;KMLTITLE:myCustomLayerTitle”,
  • formatOptions);
  • List formatOptions = Arrays.asList(((String) kvp.get(“format_options”)).split(“;”));
  • assertEquals(9, formatOptions.size());
  • assertTrue(formatOptions.contains(“LEGEND:true”));
  • assertTrue(formatOptions.contains(“SUPEROVERLAY:true”));
  • assertTrue(formatOptions.contains(“AUTOFIT:true”));
  • assertTrue(formatOptions.contains(“KMPLACEMARK:false”));
  • assertTrue(formatOptions.contains(“OVERLAYMODE:auto”));
  • assertTrue(formatOptions.contains(“KMSCORE:10”));
  • assertTrue(formatOptions.contains(“MODE:superoverlay”));
  • assertTrue(formatOptions.contains(“KMATTR:true”));
  • assertTrue(formatOptions.contains(“KMLTITLE:myCustomLayerTitle”));
    }

@anonymised.com

That change to KMLReflectorTest is a good example of the problem, where
the order of the format options changes. I assume that the order isn’t
significant here, so the test failure is just sensitivity to the hash order. If so,
the change here is valid.


diff --git a/src/kml/src/test/java/org/geoserver/kml/KMLTest.java b/src/kml/src/test/java/org/geoserver/kml/KMLTest.java
index bb3ab4a…d3b0a2c 100644
— a/src/kml/src/test/java/org/geoserver/kml/KMLTest.java
+++ b/src/kml/src/test/java/org/geoserver/kml/KMLTest.java
@@ -411,7 +411,7 @@ public class KMLTest extends WMSTestSupport {

assertXpathEvaluatesTo(“0”, “count(//kml:Placemark)”, dom);
assertXpathEvaluatesTo(“1”, “count(//kml:GroundOverlay)”, dom);

assertXpathEvaluatesTo(pngOverlay, “//kml:GroundOverlay/kml:Icon/kml:href”, dom);
}

Same issue here, except I don’t know how to make the test robust. If
it wasn’t obvious, that is why the email instead of a pull request…


diff --git a/src/restconfig/src/test/java/org/geoserver/rest/GlobalSettingsTest.java b/src/restconfig/src/test/java/org/geoserver/rest/GlobalSettingsTest.java
index be18cd3…13e2bdf 100644
— a/src/restconfig/src/test/java/org/geoserver/rest/GlobalSettingsTest.java
+++ b/src/restconfig/src/test/java/org/geoserver/rest/GlobalSettingsTest.java
@@ -111,7 +111,7 @@ public class GlobalSettingsTest extends CatalogRESTTestSupport {

JSONObject contact = settings.getJSONObject(“contact”);
assertNotNull(contact);

  • assertEquals(“Claudius Ptolomaeus”, contact.get(“contactPerson”));
    +// assertEquals(“Claudius Ptolomaeus”, contact.get(“contactPerson”));

JSONObject jaiInfo = global.getJSONObject(“jai”);
assertNotNull(jaiInfo);
@@ -154,7 +154,7 @@ public class GlobalSettingsTest extends CatalogRESTTestSupport {
assertXpathEvaluatesTo(“UTF-8”, “/global/settings/charset”, dom);
assertXpathEvaluatesTo(“10”, “/global/settings/numDecimals”, dom);
assertXpathEvaluatesTo(“http://geoserver.org”, “/global/settings/onlineResource”, dom);

  • assertXpathEvaluatesTo(“Justin Deoliveira”, “/global/settings/contact/contactPerson”, dom);
    +// assertXpathEvaluatesTo(“Justin Deoliveira”, “/global/settings/contact/contactPerson”, dom);
    assertXpathEvaluatesTo(“true”, “/global/jai/allowInterpolation”, dom);
    assertXpathEvaluatesTo(“0.85”, “/global/jai/memoryThreshold”, dom);
    assertXpathEvaluatesTo(“UNBOUNDED”, “/global/coverageAccess/queueType”, dom);

Not sure what is going on here. The contactPerson Xpath comes back with
the original value (i.e. “Andrea Aime”) instead of the updated values.


diff --git a/src/wms/src/test/java/org/geoserver/wms/wms_1_1_1/DimensionsVectorGetMapTest.java b/src/wms/src/test/java/org/geoserver/wms/wms_1_1_1/DimensionsVectorGetMapTest.java
index 43652f7…a2de87b 100644
— a/src/wms/src/test/java/org/geoserver/wms/wms_1_1_1/DimensionsVectorGetMapTest.java
+++ b/src/wms/src/test/java/org/geoserver/wms/wms_1_1_1/DimensionsVectorGetMapTest.java
@@ -253,14 +253,14 @@ public class DimensionsVectorGetMapTest extends WMSDimensionsTestSupport {

  • “&TRANSPARENT=false&BGCOLOR=0xff0000”);

// check we did not get a service exception

  • assertEquals(“image/gif”, response.getContentType());
  • assertEquals(“image/png”, response.getContentType());

// check it is a animated gif with three frames
ByteArrayInputStream bis = getBinaryInputStream(response);
ImageInputStream iis = ImageIO.createImageInputStream(bis);

  • ImageReader reader = ImageIO.getImageReadersBySuffix(“gif”).next();
  • ImageReader reader = ImageIO.getImageReadersBySuffix(“png”).next();
    reader.setInput(iis);
  • assertEquals(3, reader.getNumImages(true));
  • assertEquals(1, reader.getNumImages(true));

// creating film strip to be able to test different frames of animated gif
// http://stackoverflow.com/questions/18908217/losing-transparency-when-using-imageinputstream-and-bufferedimage-to-create-png
@@ -282,8 +282,8 @@ public class DimensionsVectorGetMapTest extends WMSDimensionsTestSupport {
// actual check for NON transparency and colored background
assertPixel(image, 20, 10, Color.RED);
assertPixel(image, 60, 10, Color.BLACK);

  • assertPixel(image, 100, 10, Color.RED);
  • assertPixel(image, 140, 30, Color.BLACK);
  • // assertPixel(image, 100, 10, Color.RED);
  • // assertPixel(image, 140, 30, Color.BLACK);
    }

@anonymised.com
@@ -300,14 +300,14 @@ public class DimensionsVectorGetMapTest extends WMSDimensionsTestSupport {

  • “&TRANSPARENT=true&BGCOLOR=0xfffff”);

// check we did not get a service exception

  • assertEquals(“image/gif”, response.getContentType());
  • assertEquals(“image/png”, response.getContentType());

// check it is a animated gif with three frames
ByteArrayInputStream bis = getBinaryInputStream(response);
ImageInputStream iis = ImageIO.createImageInputStream(bis);

  • ImageReader reader = ImageIO.getImageReadersBySuffix(“gif”).next();
  • ImageReader reader = ImageIO.getImageReadersBySuffix(“png”).next();
    reader.setInput(iis);
  • assertEquals(3, reader.getNumImages(true));
  • assertEquals(1, reader.getNumImages(true));

// creating film strip to be able to test different frames of animated gif
// http://stackoverflow.com/questions/18908217/losing-transparency-when-using-imageinputstream-and-bufferedimage-to-create-png
@@ -333,8 +333,8 @@ public class DimensionsVectorGetMapTest extends WMSDimensionsTestSupport {
// actual check for transparency and color
assertPixelIsTransparent(image, 20, 10);
assertPixel(image, 60, 10, Color.BLACK);

  • assertPixelIsTransparent(image, 100, 10);
  • assertPixel(image, 140, 30, Color.BLACK);
  • // assertPixelIsTransparent(image, 100, 10);
  • // assertPixel(image, 140, 30, Color.BLACK);
    }

@anonymised.com

These two tests are very similar and have the same symptoms. An
uninformed guess says that the problem is duplicate format properties:
MockHttpServletResponse response = getAsServletResponse(“wms?service=WMS&version=1.1.1&request=GetMap”

  • “&bbox=-180,-90,180,90&styles=&Format=image/png&width=80&height=40&srs=EPSG:4326”
  • “&layers=” + getLayerId(V_TIME_ELEVATION)
  • “&time=2011-05-02,2011-05-04,2011-05-10&format=” + GIFMapResponse.IMAGE_GIF_SUBTYPE_ANIMATED
  • “&TRANSPARENT=false&BGCOLOR=0xff0000”);
    So we’re saying image/png and image/gif;subtype=animated. That probably
    worked for some hashing order, but not this one.

Or I’m completely missing something.

Any comments or suggestions?

Brad


Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk


Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

Indeed, good stuff Brad. Can you point us to something on github? Much nicer code review interface there :slight_smile:

···

On Mon, Sep 29, 2014 at 8:45 AM, Jody Garnett <jody.garnett@anonymised.com> wrote:

Kudos Brad - It would be good to have GeoServer work on Java 8 as we do not want to distance ourself from any potential contributors.

They got us once before with changing collection order, I like your fix for changing format options. We could also use a Map that maintains key order to store format options.


Slashdot TV. Videos for Nerds. Stuff that Matters.
http://pubads.g.doubleclick.net/gampad/clk?id=160591471&iu=/4140/ostg.clktrk


Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

Justin Deoliveira
VP Engineering | Boundless
jdeolive@anonymised.com
@boundlessgeo

Jody Garnett

On Sun, Sep 28, 2014 at 3:46 AM, Brad Hards <bradh@anonymised.com> wrote:

Inspired by Jody’s comment (during the FOSS4G talk) about testers not checking
the build on JDK 8, I decided to give it a try today.

It turns out that there are few broken tests, but geoserver will build OK with
JDK 8. I only ran the resulting war against a tomcat7 server with OpenJDK7,
but it seemed to start up, at least.

The failing tests appear on the surface to be unrelated, but I think they are
failing because of a common issue - a change on JDK 8 related to the hashing
function. That manifests itself as changing order when a list or map is
serialised out.

Looking at the changes I had to make:


diff --git a/src/kml/src/test/java/org/geoserver/kml/KMLReflectorTest.java b/src/kml/src/test/java/org/geoserver/kml/KMLReflectorTest.java
index 3a085fd…db28359 100644
— a/src/kml/src/test/java/org/geoserver/kml/KMLReflectorTest.java
+++ b/src/kml/src/test/java/org/geoserver/kml/KMLReflectorTest.java
@@ -24,7 +24,9 @@ import java.io.InputStream;
import java.io.UnsupportedEncodingException;
import java.net.URL;
import java.net.URLDecoder;
+import java.util.Arrays;
import java.util.HashMap;
+import java.util.List;
import java.util.Map;
import java.util.regex.Pattern;
import java.util.zip.ZipEntry;
@@ -252,10 +254,17 @@ public class KMLReflectorTest extends WMSTestSupport {
// all the kvp parameters (which should be set as format_options now are correctly parsed)
String result = xpath.evaluate(“//kml:NetworkLink/kml:Link/kml:href”, dom);
Map<String, Object> kvp = KvpUtils.parseQueryString(result);

  • String formatOptions = (String) kvp.get(“format_options”);
  • assertEquals(
  • “LEGEND:true;SUPEROVERLAY:true;AUTOFIT:true;KMPLACEMARK:false;OVERLAYMODE:auto;KMSCORE:10;MODE:superoverlay;KMATTR:true;KMLTITLE:myCustomLayerTitle”,
  • formatOptions);
  • List formatOptions = Arrays.asList(((String) kvp.get(“format_options”)).split(“;”));
  • assertEquals(9, formatOptions.size());
  • assertTrue(formatOptions.contains(“LEGEND:true”));
  • assertTrue(formatOptions.contains(“SUPEROVERLAY:true”));
  • assertTrue(formatOptions.contains(“AUTOFIT:true”));
  • assertTrue(formatOptions.contains(“KMPLACEMARK:false”));
  • assertTrue(formatOptions.contains(“OVERLAYMODE:auto”));
  • assertTrue(formatOptions.contains(“KMSCORE:10”));
  • assertTrue(formatOptions.contains(“MODE:superoverlay”));
  • assertTrue(formatOptions.contains(“KMATTR:true”));
  • assertTrue(formatOptions.contains(“KMLTITLE:myCustomLayerTitle”));
    }

@anonymised.com

That change to KMLReflectorTest is a good example of the problem, where
the order of the format options changes. I assume that the order isn’t
significant here, so the test failure is just sensitivity to the hash order. If so,
the change here is valid.


diff --git a/src/kml/src/test/java/org/geoserver/kml/KMLTest.java b/src/kml/src/test/java/org/geoserver/kml/KMLTest.java
index bb3ab4a…d3b0a2c 100644
— a/src/kml/src/test/java/org/geoserver/kml/KMLTest.java
+++ b/src/kml/src/test/java/org/geoserver/kml/KMLTest.java
@@ -411,7 +411,7 @@ public class KMLTest extends WMSTestSupport {

assertXpathEvaluatesTo(“0”, “count(//kml:Placemark)”, dom);
assertXpathEvaluatesTo(“1”, “count(//kml:GroundOverlay)”, dom);

assertXpathEvaluatesTo(pngOverlay, “//kml:GroundOverlay/kml:Icon/kml:href”, dom);
}

Same issue here, except I don’t know how to make the test robust. If
it wasn’t obvious, that is why the email instead of a pull request…


diff --git a/src/restconfig/src/test/java/org/geoserver/rest/GlobalSettingsTest.java b/src/restconfig/src/test/java/org/geoserver/rest/GlobalSettingsTest.java
index be18cd3…13e2bdf 100644
— a/src/restconfig/src/test/java/org/geoserver/rest/GlobalSettingsTest.java
+++ b/src/restconfig/src/test/java/org/geoserver/rest/GlobalSettingsTest.java
@@ -111,7 +111,7 @@ public class GlobalSettingsTest extends CatalogRESTTestSupport {

JSONObject contact = settings.getJSONObject(“contact”);
assertNotNull(contact);

  • assertEquals(“Claudius Ptolomaeus”, contact.get(“contactPerson”));
    +// assertEquals(“Claudius Ptolomaeus”, contact.get(“contactPerson”));

JSONObject jaiInfo = global.getJSONObject(“jai”);
assertNotNull(jaiInfo);
@@ -154,7 +154,7 @@ public class GlobalSettingsTest extends CatalogRESTTestSupport {
assertXpathEvaluatesTo(“UTF-8”, “/global/settings/charset”, dom);
assertXpathEvaluatesTo(“10”, “/global/settings/numDecimals”, dom);
assertXpathEvaluatesTo(“http://geoserver.org”, “/global/settings/onlineResource”, dom);

  • assertXpathEvaluatesTo(“Justin Deoliveira”, “/global/settings/contact/contactPerson”, dom);
    +// assertXpathEvaluatesTo(“Justin Deoliveira”, “/global/settings/contact/contactPerson”, dom);
    assertXpathEvaluatesTo(“true”, “/global/jai/allowInterpolation”, dom);
    assertXpathEvaluatesTo(“0.85”, “/global/jai/memoryThreshold”, dom);
    assertXpathEvaluatesTo(“UNBOUNDED”, “/global/coverageAccess/queueType”, dom);

Not sure what is going on here. The contactPerson Xpath comes back with
the original value (i.e. “Andrea Aime”) instead of the updated values.


diff --git a/src/wms/src/test/java/org/geoserver/wms/wms_1_1_1/DimensionsVectorGetMapTest.java b/src/wms/src/test/java/org/geoserver/wms/wms_1_1_1/DimensionsVectorGetMapTest.java
index 43652f7…a2de87b 100644
— a/src/wms/src/test/java/org/geoserver/wms/wms_1_1_1/DimensionsVectorGetMapTest.java
+++ b/src/wms/src/test/java/org/geoserver/wms/wms_1_1_1/DimensionsVectorGetMapTest.java
@@ -253,14 +253,14 @@ public class DimensionsVectorGetMapTest extends WMSDimensionsTestSupport {

  • “&TRANSPARENT=false&BGCOLOR=0xff0000”);

// check we did not get a service exception

  • assertEquals(“image/gif”, response.getContentType());
  • assertEquals(“image/png”, response.getContentType());

// check it is a animated gif with three frames
ByteArrayInputStream bis = getBinaryInputStream(response);
ImageInputStream iis = ImageIO.createImageInputStream(bis);

  • ImageReader reader = ImageIO.getImageReadersBySuffix(“gif”).next();
  • ImageReader reader = ImageIO.getImageReadersBySuffix(“png”).next();
    reader.setInput(iis);
  • assertEquals(3, reader.getNumImages(true));
  • assertEquals(1, reader.getNumImages(true));

// creating film strip to be able to test different frames of animated gif
// http://stackoverflow.com/questions/18908217/losing-transparency-when-using-imageinputstream-and-bufferedimage-to-create-png
@@ -282,8 +282,8 @@ public class DimensionsVectorGetMapTest extends WMSDimensionsTestSupport {
// actual check for NON transparency and colored background
assertPixel(image, 20, 10, Color.RED);
assertPixel(image, 60, 10, Color.BLACK);

  • assertPixel(image, 100, 10, Color.RED);
  • assertPixel(image, 140, 30, Color.BLACK);
  • // assertPixel(image, 100, 10, Color.RED);
  • // assertPixel(image, 140, 30, Color.BLACK);
    }

@anonymised.com
@@ -300,14 +300,14 @@ public class DimensionsVectorGetMapTest extends WMSDimensionsTestSupport {

  • “&TRANSPARENT=true&BGCOLOR=0xfffff”);

// check we did not get a service exception

  • assertEquals(“image/gif”, response.getContentType());
  • assertEquals(“image/png”, response.getContentType());

// check it is a animated gif with three frames
ByteArrayInputStream bis = getBinaryInputStream(response);
ImageInputStream iis = ImageIO.createImageInputStream(bis);

  • ImageReader reader = ImageIO.getImageReadersBySuffix(“gif”).next();
  • ImageReader reader = ImageIO.getImageReadersBySuffix(“png”).next();
    reader.setInput(iis);
  • assertEquals(3, reader.getNumImages(true));
  • assertEquals(1, reader.getNumImages(true));

// creating film strip to be able to test different frames of animated gif
// http://stackoverflow.com/questions/18908217/losing-transparency-when-using-imageinputstream-and-bufferedimage-to-create-png
@@ -333,8 +333,8 @@ public class DimensionsVectorGetMapTest extends WMSDimensionsTestSupport {
// actual check for transparency and color
assertPixelIsTransparent(image, 20, 10);
assertPixel(image, 60, 10, Color.BLACK);

  • assertPixelIsTransparent(image, 100, 10);
  • assertPixel(image, 140, 30, Color.BLACK);
  • // assertPixelIsTransparent(image, 100, 10);
  • // assertPixel(image, 140, 30, Color.BLACK);
    }

@anonymised.com

These two tests are very similar and have the same symptoms. An
uninformed guess says that the problem is duplicate format properties:
MockHttpServletResponse response = getAsServletResponse(“wms?service=WMS&version=1.1.1&request=GetMap”

  • “&bbox=-180,-90,180,90&styles=&Format=image/png&width=80&height=40&srs=EPSG:4326”
  • “&layers=” + getLayerId(V_TIME_ELEVATION)
  • “&time=2011-05-02,2011-05-04,2011-05-10&format=” + GIFMapResponse.IMAGE_GIF_SUBTYPE_ANIMATED
  • “&TRANSPARENT=false&BGCOLOR=0xff0000”);
    So we’re saying image/png and image/gif;subtype=animated. That probably
    worked for some hashing order, but not this one.

Or I’m completely missing something.

Any comments or suggestions?

Brad


Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk


Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

On Mon, 29 Sep 2014 08:56:48 AM Justin Deoliveira wrote:

Can you point us to something on github?

https://github.com/bradh/geoserver/tree/jdk8 is the temporary branch (warning:
subject to rebase).

Currently there are three commits - the one on top is the "stuff I don't know
how to fix" and has a couple of comments.

I'm particularly confused by the contactPerson part - my reading of the code
is far from complete, but that looks like a problem on the main/ side (as
opposed to the test/ side). However from
https://github.com/bradh/geoserver/blob/master/src/restconfig/src/main/java/org/geoserver/rest/GlobalSettingsResource.java#L64
it looks like we always copy in the original contact info, so I'm not sure how
it worked in JDK7.

Brad

On Mon, 29 Sep 2014 07:45:15 AM Jody Garnett wrote:

They got us once before with changing collection order, I like your fix for
changing format options. We could also use a Map that maintains key order
to store format options.

I implemented the switch to TreeMap, which gets me down to just one bit of
b0rken (why setting contactPerson in restconfig isn't "taking").

See https://github.com/bradh/geoserver/tree/jdk8 for the WIP.

Brad

Just bumping this email thread as we are looking at the end of life of Java 7 this year.

···

On 28 September 2014 at 03:46, Brad Hards <bradh@anonymised.com> wrote:

Inspired by Jody’s comment (during the FOSS4G talk) about testers not checking
the build on JDK 8, I decided to give it a try today.

It turns out that there are few broken tests, but geoserver will build OK with
JDK 8. I only ran the resulting war against a tomcat7 server with OpenJDK7,
but it seemed to start up, at least.

The failing tests appear on the surface to be unrelated, but I think they are
failing because of a common issue - a change on JDK 8 related to the hashing
function. That manifests itself as changing order when a list or map is
serialised out.

Looking at the changes I had to make:


diff --git a/src/kml/src/test/java/org/geoserver/kml/KMLReflectorTest.java b/src/kml/src/test/java/org/geoserver/kml/KMLReflectorTest.java
index 3a085fd…db28359 100644
— a/src/kml/src/test/java/org/geoserver/kml/KMLReflectorTest.java
+++ b/src/kml/src/test/java/org/geoserver/kml/KMLReflectorTest.java
@@ -24,7 +24,9 @@ import java.io.InputStream;
import java.io.UnsupportedEncodingException;
import java.net.URL;
import java.net.URLDecoder;
+import java.util.Arrays;
import java.util.HashMap;
+import java.util.List;
import java.util.Map;
import java.util.regex.Pattern;
import java.util.zip.ZipEntry;
@@ -252,10 +254,17 @@ public class KMLReflectorTest extends WMSTestSupport {
// all the kvp parameters (which should be set as format_options now are correctly parsed)
String result = xpath.evaluate(“//kml:NetworkLink/kml:Link/kml:href”, dom);
Map<String, Object> kvp = KvpUtils.parseQueryString(result);

  • String formatOptions = (String) kvp.get(“format_options”);
  • assertEquals(
  • “LEGEND:true;SUPEROVERLAY:true;AUTOFIT:true;KMPLACEMARK:false;OVERLAYMODE:auto;KMSCORE:10;MODE:superoverlay;KMATTR:true;KMLTITLE:myCustomLayerTitle”,
  • formatOptions);
  • List formatOptions = Arrays.asList(((String) kvp.get(“format_options”)).split(“;”));
  • assertEquals(9, formatOptions.size());
  • assertTrue(formatOptions.contains(“LEGEND:true”));
  • assertTrue(formatOptions.contains(“SUPEROVERLAY:true”));
  • assertTrue(formatOptions.contains(“AUTOFIT:true”));
  • assertTrue(formatOptions.contains(“KMPLACEMARK:false”));
  • assertTrue(formatOptions.contains(“OVERLAYMODE:auto”));
  • assertTrue(formatOptions.contains(“KMSCORE:10”));
  • assertTrue(formatOptions.contains(“MODE:superoverlay”));
  • assertTrue(formatOptions.contains(“KMATTR:true”));
  • assertTrue(formatOptions.contains(“KMLTITLE:myCustomLayerTitle”));
    }

@anonymised.com

That change to KMLReflectorTest is a good example of the problem, where
the order of the format options changes. I assume that the order isn’t
significant here, so the test failure is just sensitivity to the hash order. If so,
the change here is valid.


diff --git a/src/kml/src/test/java/org/geoserver/kml/KMLTest.java b/src/kml/src/test/java/org/geoserver/kml/KMLTest.java
index bb3ab4a…d3b0a2c 100644
— a/src/kml/src/test/java/org/geoserver/kml/KMLTest.java
+++ b/src/kml/src/test/java/org/geoserver/kml/KMLTest.java
@@ -411,7 +411,7 @@ public class KMLTest extends WMSTestSupport {

assertXpathEvaluatesTo(“0”, “count(//kml:Placemark)”, dom);
assertXpathEvaluatesTo(“1”, “count(//kml:GroundOverlay)”, dom);

assertXpathEvaluatesTo(pngOverlay, “//kml:GroundOverlay/kml:Icon/kml:href”, dom);
}

Same issue here, except I don’t know how to make the test robust. If
it wasn’t obvious, that is why the email instead of a pull request…


diff --git a/src/restconfig/src/test/java/org/geoserver/rest/GlobalSettingsTest.java b/src/restconfig/src/test/java/org/geoserver/rest/GlobalSettingsTest.java
index be18cd3…13e2bdf 100644
— a/src/restconfig/src/test/java/org/geoserver/rest/GlobalSettingsTest.java
+++ b/src/restconfig/src/test/java/org/geoserver/rest/GlobalSettingsTest.java
@@ -111,7 +111,7 @@ public class GlobalSettingsTest extends CatalogRESTTestSupport {

JSONObject contact = settings.getJSONObject(“contact”);
assertNotNull(contact);

  • assertEquals(“Claudius Ptolomaeus”, contact.get(“contactPerson”));
    +// assertEquals(“Claudius Ptolomaeus”, contact.get(“contactPerson”));

JSONObject jaiInfo = global.getJSONObject(“jai”);
assertNotNull(jaiInfo);
@@ -154,7 +154,7 @@ public class GlobalSettingsTest extends CatalogRESTTestSupport {
assertXpathEvaluatesTo(“UTF-8”, “/global/settings/charset”, dom);
assertXpathEvaluatesTo(“10”, “/global/settings/numDecimals”, dom);
assertXpathEvaluatesTo(“http://geoserver.org”, “/global/settings/onlineResource”, dom);

  • assertXpathEvaluatesTo(“Justin Deoliveira”, “/global/settings/contact/contactPerson”, dom);
    +// assertXpathEvaluatesTo(“Justin Deoliveira”, “/global/settings/contact/contactPerson”, dom);
    assertXpathEvaluatesTo(“true”, “/global/jai/allowInterpolation”, dom);
    assertXpathEvaluatesTo(“0.85”, “/global/jai/memoryThreshold”, dom);
    assertXpathEvaluatesTo(“UNBOUNDED”, “/global/coverageAccess/queueType”, dom);

Not sure what is going on here. The contactPerson Xpath comes back with
the original value (i.e. “Andrea Aime”) instead of the updated values.


diff --git a/src/wms/src/test/java/org/geoserver/wms/wms_1_1_1/DimensionsVectorGetMapTest.java b/src/wms/src/test/java/org/geoserver/wms/wms_1_1_1/DimensionsVectorGetMapTest.java
index 43652f7…a2de87b 100644
— a/src/wms/src/test/java/org/geoserver/wms/wms_1_1_1/DimensionsVectorGetMapTest.java
+++ b/src/wms/src/test/java/org/geoserver/wms/wms_1_1_1/DimensionsVectorGetMapTest.java
@@ -253,14 +253,14 @@ public class DimensionsVectorGetMapTest extends WMSDimensionsTestSupport {

  • “&TRANSPARENT=false&BGCOLOR=0xff0000”);

// check we did not get a service exception

  • assertEquals(“image/gif”, response.getContentType());
  • assertEquals(“image/png”, response.getContentType());

// check it is a animated gif with three frames
ByteArrayInputStream bis = getBinaryInputStream(response);
ImageInputStream iis = ImageIO.createImageInputStream(bis);

  • ImageReader reader = ImageIO.getImageReadersBySuffix(“gif”).next();
  • ImageReader reader = ImageIO.getImageReadersBySuffix(“png”).next();
    reader.setInput(iis);
  • assertEquals(3, reader.getNumImages(true));
  • assertEquals(1, reader.getNumImages(true));

// creating film strip to be able to test different frames of animated gif
// http://stackoverflow.com/questions/18908217/losing-transparency-when-using-imageinputstream-and-bufferedimage-to-create-png
@@ -282,8 +282,8 @@ public class DimensionsVectorGetMapTest extends WMSDimensionsTestSupport {
// actual check for NON transparency and colored background
assertPixel(image, 20, 10, Color.RED);
assertPixel(image, 60, 10, Color.BLACK);

  • assertPixel(image, 100, 10, Color.RED);
  • assertPixel(image, 140, 30, Color.BLACK);
  • // assertPixel(image, 100, 10, Color.RED);
  • // assertPixel(image, 140, 30, Color.BLACK);
    }

@anonymised.com
@@ -300,14 +300,14 @@ public class DimensionsVectorGetMapTest extends WMSDimensionsTestSupport {

  • “&TRANSPARENT=true&BGCOLOR=0xfffff”);

// check we did not get a service exception

  • assertEquals(“image/gif”, response.getContentType());
  • assertEquals(“image/png”, response.getContentType());

// check it is a animated gif with three frames
ByteArrayInputStream bis = getBinaryInputStream(response);
ImageInputStream iis = ImageIO.createImageInputStream(bis);

  • ImageReader reader = ImageIO.getImageReadersBySuffix(“gif”).next();
  • ImageReader reader = ImageIO.getImageReadersBySuffix(“png”).next();
    reader.setInput(iis);
  • assertEquals(3, reader.getNumImages(true));
  • assertEquals(1, reader.getNumImages(true));

// creating film strip to be able to test different frames of animated gif
// http://stackoverflow.com/questions/18908217/losing-transparency-when-using-imageinputstream-and-bufferedimage-to-create-png
@@ -333,8 +333,8 @@ public class DimensionsVectorGetMapTest extends WMSDimensionsTestSupport {
// actual check for transparency and color
assertPixelIsTransparent(image, 20, 10);
assertPixel(image, 60, 10, Color.BLACK);

  • assertPixelIsTransparent(image, 100, 10);
  • assertPixel(image, 140, 30, Color.BLACK);
  • // assertPixelIsTransparent(image, 100, 10);
  • // assertPixel(image, 140, 30, Color.BLACK);
    }

@anonymised.com

These two tests are very similar and have the same symptoms. An
uninformed guess says that the problem is duplicate format properties:
MockHttpServletResponse response = getAsServletResponse(“wms?service=WMS&version=1.1.1&request=GetMap”

  • “&bbox=-180,-90,180,90&styles=&Format=image/png&width=80&height=40&srs=EPSG:4326”
  • “&layers=” + getLayerId(V_TIME_ELEVATION)
  • “&time=2011-05-02,2011-05-04,2011-05-10&format=” + GIFMapResponse.IMAGE_GIF_SUBTYPE_ANIMATED
  • “&TRANSPARENT=false&BGCOLOR=0xff0000”);
    So we’re saying image/png and image/gif;subtype=animated. That probably
    worked for some hashing order, but not this one.

Or I’m completely missing something.

Any comments or suggestions?

Brad


Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk


Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel


Jody Garnett