[Geoserver-devel] [geoserver-scm] [11835] trunk/src/main/src/main/java/org/geoserver/catalog/impl/CatalogImpl.java: GEOS-2780: CatalogImpl validation too aggressive, breaks the build

Ben, I would appreciate being asked to review these changes before they are committed. I realize my changes broke the build, so please when this happens, just back out my changes, or make the changes locally and post a patch for review, rather than re-write my changes.

-Justin

bencaradocdavies@anonymised.com wrote:

Revision
    11835 <http://fisheye.codehaus.org/changelog/geoserver/?cs=11835&gt;
Author
    bencaradocdavies
Date
    2009-03-25 02:40:40 -0500 (Wed, 25 Mar 2009)

      Log Message

GEOS-2780 <http://jira.codehaus.org/secure/ViewIssue.jspa?key=GEOS-2780&gt;: CatalogImpl validation too aggressive, breaks the build

      Modified Paths

    * trunk/src/main/src/main/java/org/geoserver/catalog/impl/CatalogImpl.java
      <#trunksrcmainsrcmainjavaorggeoservercatalogimplCatalogImpljava>

      Diff

        Modified:
        trunk/src/main/src/main/java/org/geoserver/catalog/impl/CatalogImpl.java
        (11834 => 11835)

--- trunk/src/main/src/main/java/org/geoserver/catalog/impl/CatalogImpl.java 2009-03-25 00:20:53 UTC (rev 11834)
+++ trunk/src/main/src/main/java/org/geoserver/catalog/impl/CatalogImpl.java 2009-03-25 07:40:40 UTC (rev 11835)
@@ -116,6 +116,7 @@
         }
          validate(store);
+ validateNew(store);
         resolve(store);
         stores.put(store.getClass(), store);
         added(store);
@@ -128,7 +129,9 @@
         if ( store.getWorkspace() == null ) {
             throw new IllegalArgumentException( "Store must be part of a workspace");
         }
+ }
         + void validateNew(StoreInfo store) {
         WorkspaceInfo workspace = store.getWorkspace();
         if ( getStoreByName( workspace, store.getName(), StoreInfo.class ) != null ) {
             String msg = "Store '"+ store.getName() +"' already exists in workspace '"+workspace.getName()+"'";
@@ -324,6 +327,7 @@
         }
                  validate(resource);
+ validateNew(resource);
         resolve(resource);
         resources.put(resource.getClass(), resource);
         added(resource);
@@ -339,7 +343,9 @@
         if ( resource.getNamespace() == null ) {
             throw new IllegalArgumentException( "Resource must be part of a namespace");
         }
+ }
         + void validateNew(ResourceInfo resource) {
         StoreInfo store = resource.getStore();
         if ( getResourceByStore( store, resource.getName(), ResourceInfo.class) != null ) {
             String msg = "Resource named '"+resource.getName()+"' already exists in store: '"+ store.getName()+"'";
@@ -614,6 +620,7 @@
     // Layer methods
     public void add(LayerInfo layer) {
         validate(layer);
+ validateNew(layer);
         resolve(layer);
                  if ( layer.getType() == null ) {
@@ -640,14 +647,17 @@
         if ( layer.getResource() == null ) {
             throw new NullPointerException( "Layer resource must not be null" );
         }
- if ( getLayerByName( layer.getName() ) != null ) {
- throw new IllegalArgumentException( "Layer named '" + layer.getName() + "', already exists.");
- }
         //(JD): not sure if default style should be mandatory
         //if ( layer.getDefaultStyle() == null ){
         // throw new NullPointerException( "Layer default style must not be null" );
         //}
     }
+ + void validateNew( LayerInfo layer ) {
+ if ( getLayerByName( layer.getName() ) != null ) {
+ throw new IllegalArgumentException( "Layer named '" + layer.getName() + "', already exists.");
+ }
+ }
          public void remove(LayerInfo layer) {
         //ensure no references to the layer
@@ -893,6 +903,7 @@
      public void add(NamespaceInfo namespace) {
         validate(namespace);
+ validateNew(namespace);
         resolve(namespace);
                  synchronized (namespaces) {
@@ -912,7 +923,9 @@
         if ( namespace.getURI() == null ) {
             throw new NullPointerException( "Namespace uri must not be null");
         }
- + }
+
+ void validateNew(NamespaceInfo namespace) {
         if ( getNamespaceByPrefix( namespace.getPrefix() ) != null ) {
             throw new IllegalArgumentException( "Namespace with prefix '" + namespace.getPrefix() + "' already exists.");
         }
@@ -1084,6 +1097,7 @@
      public void add(StyleInfo style) {
         validate(style);
+ validateNew(style);
         resolve(style);
         styles.add(style);
         added(style);
@@ -1096,7 +1110,9 @@
         if ( style.getFilename() == null ) {
             throw new NullPointerException( "Style fileName must not be null");
         }
+ }
         + void validateNew( StyleInfo style ) {
         if ( getStyleByName( style.getName() ) != null ) {
             throw new IllegalArgumentException( "Style named '" + style.getName() +"' already exists.");
         }

------------------------------------------------------------------------

To unsubscribe from this list please visit:

http://xircles.codehaus.org/manage_email

--
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

Sure, Justin, that would be my usual preference, and will be my usual practice in future. Unfortunately, the last few commits contained several changes, each of which broke the build in different ways. There was more than one committer, and the commits were being added to a broken build. The build was broken for the whole working day in my time zone. The main module cannot be excluded from the build. I needed to be up to date because I needed to:

(1) synchronise a GeoTools external updated by Rini (app-schema-test uses GeoTools test-data configuration files),

(2) commit app-schema-test so Andrea can see it (Andrea I still owe you an email response: will do so when I commit), and

(3) diagnose the failures to see if any were caused or exacerbated by my GeoTools XML commit.

It was not obvious to me whether backing out your CatalogImpl change would fix the build (and it would not have, as wms was also broken). The failures in main were straightforward; the changes were minor and localised, and fixed the problem in that module. I made an extra effort to preserve the functionality you added, rather than just discard it. I realised I would be stepping on your toes, so I did the full Jira issue workflow, and sent you an email to make sure you new about it.

I get mixed messages from the GeoServer/GeoTools community. On one hand I submit patches and am told that I should step up and fix the build if I can, but when I do so, I get complaints.

I do not mean to preach to the opengeo team, for whom I have only the highest respect. For the benefit of the more junior developers whom I know are watching, here are some principles my teams follow to ensure successful continuous integration for test driven development:

(1) Do not break the build. This can be achieved by running the unit tests for the module, or even better, the whole project.

(2) If you do break the build, stop what you are doing and fix the build. This means that you must monitor the build until is has succeeded after your most recent commit. No committing and then going home, or going on holiday. This becomes more important with global distributed teams.

(3) If the build is broken by someone else and not fixed promptly, everyone stops what they are doing and fixes the build. The build-breaker owes a forfeit (at my current site, provision of muffins for the team).

(4) Do not commit when the build is broken, except to fix the build. If you commit new code to a broken build, you will not know if your new code is also broken, or causes further integration failures.

As I am sure we are all aware, failure to maintain continuous integration leads to huge problems disentangling the causes of unit test failures. I have been there before. This community in general does a great job of keeping the build working. All credit to the opengeo team.

Justin, I appreciate the huge effort you and Andrea have been making, particularly in the adoption of GSIP 34. You guys are more likely to break the build just because you are such prolific committers! I am happy to help where I can, but if you want me to do so, you must accept that in fixing the build I may need to make small changes that do not conform to your vision. If this is the case, you are free to revert or amend these changes.

There, I feel better now. :slight_smile:

Kind regards,
Ben.

Justin Deoliveira wrote:

Ben, I would appreciate being asked to review these changes before they are committed. I realize my changes broke the build, so please when this happens, just back out my changes, or make the changes locally and post a patch for review, rather than re-write my changes.

-Justin

bencaradocdavies@anonymised.com wrote:

Revision
    11835 <http://fisheye.codehaus.org/changelog/geoserver/?cs=11835&gt;
Author
    bencaradocdavies
Date
    2009-03-25 02:40:40 -0500 (Wed, 25 Mar 2009)

      Log Message

GEOS-2780 <http://jira.codehaus.org/secure/ViewIssue.jspa?key=GEOS-2780&gt;: CatalogImpl validation too aggressive, breaks the build

      Modified Paths

    * trunk/src/main/src/main/java/org/geoserver/catalog/impl/CatalogImpl.java
      <#trunksrcmainsrcmainjavaorggeoservercatalogimplCatalogImpljava>

      Diff

        Modified:
        trunk/src/main/src/main/java/org/geoserver/catalog/impl/CatalogImpl.java
        (11834 => 11835)

--- trunk/src/main/src/main/java/org/geoserver/catalog/impl/CatalogImpl.java 2009-03-25 00:20:53 UTC (rev 11834)
+++ trunk/src/main/src/main/java/org/geoserver/catalog/impl/CatalogImpl.java 2009-03-25 07:40:40 UTC (rev 11835)
@@ -116,6 +116,7 @@
         }
          validate(store);
+ validateNew(store);
         resolve(store);
         stores.put(store.getClass(), store);
         added(store);
@@ -128,7 +129,9 @@
         if ( store.getWorkspace() == null ) {
             throw new IllegalArgumentException( "Store must be part of a workspace");
         }
+ }
         + void validateNew(StoreInfo store) {
         WorkspaceInfo workspace = store.getWorkspace();
         if ( getStoreByName( workspace, store.getName(), StoreInfo.class ) != null ) {
             String msg = "Store '"+ store.getName() +"' already exists in workspace '"+workspace.getName()+"'";
@@ -324,6 +327,7 @@
         }
                  validate(resource);
+ validateNew(resource);
         resolve(resource);
         resources.put(resource.getClass(), resource);
         added(resource);
@@ -339,7 +343,9 @@
         if ( resource.getNamespace() == null ) {
             throw new IllegalArgumentException( "Resource must be part of a namespace");
         }
+ }
         + void validateNew(ResourceInfo resource) {
         StoreInfo store = resource.getStore();
         if ( getResourceByStore( store, resource.getName(), ResourceInfo.class) != null ) {
             String msg = "Resource named '"+resource.getName()+"' already exists in store: '"+ store.getName()+"'";
@@ -614,6 +620,7 @@
     // Layer methods
     public void add(LayerInfo layer) {
         validate(layer);
+ validateNew(layer);
         resolve(layer);
                  if ( layer.getType() == null ) {
@@ -640,14 +647,17 @@
         if ( layer.getResource() == null ) {
             throw new NullPointerException( "Layer resource must not be null" );
         }
- if ( getLayerByName( layer.getName() ) != null ) {
- throw new IllegalArgumentException( "Layer named '" + layer.getName() + "', already exists.");
- }
         //(JD): not sure if default style should be mandatory
         //if ( layer.getDefaultStyle() == null ){
         // throw new NullPointerException( "Layer default style must not be null" );
         //}
     }
+ + void validateNew( LayerInfo layer ) {
+ if ( getLayerByName( layer.getName() ) != null ) {
+ throw new IllegalArgumentException( "Layer named '" + layer.getName() + "', already exists.");
+ }
+ }
          public void remove(LayerInfo layer) {
         //ensure no references to the layer
@@ -893,6 +903,7 @@
      public void add(NamespaceInfo namespace) {
         validate(namespace);
+ validateNew(namespace);
         resolve(namespace);
                  synchronized (namespaces) {
@@ -912,7 +923,9 @@
         if ( namespace.getURI() == null ) {
             throw new NullPointerException( "Namespace uri must not be null");
         }
- + }
+
+ void validateNew(NamespaceInfo namespace) {
         if ( getNamespaceByPrefix( namespace.getPrefix() ) != null ) {
             throw new IllegalArgumentException( "Namespace with prefix '" + namespace.getPrefix() + "' already exists.");
         }
@@ -1084,6 +1097,7 @@
      public void add(StyleInfo style) {
         validate(style);
+ validateNew(style);
         resolve(style);
         styles.add(style);
         added(style);
@@ -1096,7 +1110,9 @@
         if ( style.getFilename() == null ) {
             throw new NullPointerException( "Style fileName must not be null");
         }
+ }
         + void validateNew( StyleInfo style ) {
         if ( getStyleByName( style.getName() ) != null ) {
             throw new IllegalArgumentException( "Style named '" + style.getName() +"' already exists.");
         }

------------------------------------------------------------------------

To unsubscribe from this list please visit:

http://xircles.codehaus.org/manage_email

--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineer, CSIRO Exploration and Mining
Australian Resources Research Centre
26 Dick Perry Ave, Kensington WA 6151, Australia

I don't really require the manifesto... i classify these under common practice on any open source project I have worked on. And indeed today I put half my day toward fixing the build.

I fully accept responsibility for breaking the build, and I assure you that i am 99% of the time anal about making sure my commits never break anything. But I will fully admit, I got sloppy with this one mostly due to the volume of commits lately.

That said I stand by my initial statements in that I thought your fixing was premature. I realize it is painful, but for the 4 years i have been a geotools and geoserver committer the policy has never been to just blindly change someones last commit to fix the build. It potentially can be a lot of hassle for that person. To some extent they deserve it for breaking the build on the first place but often the case is that they are doing more work in the areas of the source that you will change. Which means when they update it is conflict city and a bunch of hassle to resolve. It is much nicer to just back the changes out.

The unofficial policy I have always followed is to either out the change, or fix it locally and wait for them to respond. I know this requires patience, but it is usually the courtesy of a day is given. I realize being in different time zones makes that hard but still. If after a full day there has been no response, then by all means, fix the build as you see fit. But it was literally the first thing i did today, so it would have gotten done.

But this is just my opinion. Others may feel differently.

Ben Caradoc-Davies wrote:

Sure, Justin, that would be my usual preference, and will be my usual practice in future. Unfortunately, the last few commits contained several changes, each of which broke the build in different ways. There was more than one committer, and the commits were being added to a broken build. The build was broken for the whole working day in my time zone. The main module cannot be excluded from the build. I needed to be up to date because I needed to:

(1) synchronise a GeoTools external updated by Rini (app-schema-test uses GeoTools test-data configuration files),

(2) commit app-schema-test so Andrea can see it (Andrea I still owe you an email response: will do so when I commit), and

(3) diagnose the failures to see if any were caused or exacerbated by my GeoTools XML commit.

It was not obvious to me whether backing out your CatalogImpl change would fix the build (and it would not have, as wms was also broken). The failures in main were straightforward; the changes were minor and localised, and fixed the problem in that module. I made an extra effort to preserve the functionality you added, rather than just discard it. I realised I would be stepping on your toes, so I did the full Jira issue workflow, and sent you an email to make sure you new about it.

I get mixed messages from the GeoServer/GeoTools community. On one hand I submit patches and am told that I should step up and fix the build if I can, but when I do so, I get complaints.

I do not mean to preach to the opengeo team, for whom I have only the highest respect. For the benefit of the more junior developers whom I know are watching, here are some principles my teams follow to ensure successful continuous integration for test driven development:

(1) Do not break the build. This can be achieved by running the unit tests for the module, or even better, the whole project.

(2) If you do break the build, stop what you are doing and fix the build. This means that you must monitor the build until is has succeeded after your most recent commit. No committing and then going home, or going on holiday. This becomes more important with global distributed teams.

(3) If the build is broken by someone else and not fixed promptly, everyone stops what they are doing and fixes the build. The build-breaker owes a forfeit (at my current site, provision of muffins for the team).

(4) Do not commit when the build is broken, except to fix the build. If you commit new code to a broken build, you will not know if your new code is also broken, or causes further integration failures.

As I am sure we are all aware, failure to maintain continuous integration leads to huge problems disentangling the causes of unit test failures. I have been there before. This community in general does a great job of keeping the build working. All credit to the opengeo team.

Justin, I appreciate the huge effort you and Andrea have been making, particularly in the adoption of GSIP 34. You guys are more likely to break the build just because you are such prolific committers! I am happy to help where I can, but if you want me to do so, you must accept that in fixing the build I may need to make small changes that do not conform to your vision. If this is the case, you are free to revert or amend these changes.

There, I feel better now. :slight_smile:

Kind regards,
Ben.

Justin Deoliveira wrote:

Ben, I would appreciate being asked to review these changes before they are committed. I realize my changes broke the build, so please when this happens, just back out my changes, or make the changes locally and post a patch for review, rather than re-write my changes.

-Justin

bencaradocdavies@anonymised.com wrote:

Revision
    11835 <http://fisheye.codehaus.org/changelog/geoserver/?cs=11835&gt;
Author
    bencaradocdavies
Date
    2009-03-25 02:40:40 -0500 (Wed, 25 Mar 2009)

      Log Message

GEOS-2780 <http://jira.codehaus.org/secure/ViewIssue.jspa?key=GEOS-2780&gt;: CatalogImpl validation too aggressive, breaks the build

      Modified Paths

    * trunk/src/main/src/main/java/org/geoserver/catalog/impl/CatalogImpl.java
      <#trunksrcmainsrcmainjavaorggeoservercatalogimplCatalogImpljava>

      Diff

        Modified:
        trunk/src/main/src/main/java/org/geoserver/catalog/impl/CatalogImpl.java
        (11834 => 11835)

--- trunk/src/main/src/main/java/org/geoserver/catalog/impl/CatalogImpl.java 2009-03-25 00:20:53 UTC (rev 11834)
+++ trunk/src/main/src/main/java/org/geoserver/catalog/impl/CatalogImpl.java 2009-03-25 07:40:40 UTC (rev 11835)
@@ -116,6 +116,7 @@
         }
          validate(store);
+ validateNew(store);
         resolve(store);
         stores.put(store.getClass(), store);
         added(store);
@@ -128,7 +129,9 @@
         if ( store.getWorkspace() == null ) {
             throw new IllegalArgumentException( "Store must be part of a workspace");
         }
+ }
         + void validateNew(StoreInfo store) {
         WorkspaceInfo workspace = store.getWorkspace();
         if ( getStoreByName( workspace, store.getName(), StoreInfo.class ) != null ) {
             String msg = "Store '"+ store.getName() +"' already exists in workspace '"+workspace.getName()+"'";
@@ -324,6 +327,7 @@
         }
                  validate(resource);
+ validateNew(resource);
         resolve(resource);
         resources.put(resource.getClass(), resource);
         added(resource);
@@ -339,7 +343,9 @@
         if ( resource.getNamespace() == null ) {
             throw new IllegalArgumentException( "Resource must be part of a namespace");
         }
+ }
         + void validateNew(ResourceInfo resource) {
         StoreInfo store = resource.getStore();
         if ( getResourceByStore( store, resource.getName(), ResourceInfo.class) != null ) {
             String msg = "Resource named '"+resource.getName()+"' already exists in store: '"+ store.getName()+"'";
@@ -614,6 +620,7 @@
     // Layer methods
     public void add(LayerInfo layer) {
         validate(layer);
+ validateNew(layer);
         resolve(layer);
                  if ( layer.getType() == null ) {
@@ -640,14 +647,17 @@
         if ( layer.getResource() == null ) {
             throw new NullPointerException( "Layer resource must not be null" );
         }
- if ( getLayerByName( layer.getName() ) != null ) {
- throw new IllegalArgumentException( "Layer named '" + layer.getName() + "', already exists.");
- }
         //(JD): not sure if default style should be mandatory
         //if ( layer.getDefaultStyle() == null ){
         // throw new NullPointerException( "Layer default style must not be null" );
         //}
     }
+ + void validateNew( LayerInfo layer ) {
+ if ( getLayerByName( layer.getName() ) != null ) {
+ throw new IllegalArgumentException( "Layer named '" + layer.getName() + "', already exists.");
+ }
+ }
          public void remove(LayerInfo layer) {
         //ensure no references to the layer
@@ -893,6 +903,7 @@
      public void add(NamespaceInfo namespace) {
         validate(namespace);
+ validateNew(namespace);
         resolve(namespace);
                  synchronized (namespaces) {
@@ -912,7 +923,9 @@
         if ( namespace.getURI() == null ) {
             throw new NullPointerException( "Namespace uri must not be null");
         }
- + }
+
+ void validateNew(NamespaceInfo namespace) {
         if ( getNamespaceByPrefix( namespace.getPrefix() ) != null ) {
             throw new IllegalArgumentException( "Namespace with prefix '" + namespace.getPrefix() + "' already exists.");
         }
@@ -1084,6 +1097,7 @@
      public void add(StyleInfo style) {
         validate(style);
+ validateNew(style);
         resolve(style);
         styles.add(style);
         added(style);
@@ -1096,7 +1110,9 @@
         if ( style.getFilename() == null ) {
             throw new NullPointerException( "Style fileName must not be null");
         }
+ }
         + void validateNew( StyleInfo style ) {
         if ( getStyleByName( style.getName() ) != null ) {
             throw new IllegalArgumentException( "Style named '" + style.getName() +"' already exists.");
         }

------------------------------------------------------------------------

To unsubscribe from this list please visit:

http://xircles.codehaus.org/manage_email

--
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

Justin Deoliveira wrote:

The unofficial policy I have always followed is to either out the change, or fix it locally and wait for them to respond. I know this requires patience, but it is usually the courtesy of a day is given.

That seems a fair and workable policy. I will follow it. Would you like to add it to the developer manual? GeoTools has a bit more on committer responsibilities, but neither project has anything on fixing-the-build etiquette. And these things matter.

Thanks for your patience.

Kind regards,

--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineer, CSIRO Exploration and Mining
Australian Resources Research Centre
26 Dick Perry Ave, Kensington WA 6151, Australia