[Geoserver-devel] Importer code is starting to rot

GeoServer GeoServer Developer
,
1

Hi,
today for unrelated causes (see another mail soon) I’ve tried to build the importer module on trunk,
finding that:

  • it does not compile
  • when made to compile, has tests that do not pass anymore

Long story short, the rest of the code is moving and importer is getting left behind.
I know I’m being a pest, but any chance we can push and turn it into an extension, so that it
gets compiled and tested at every build?

Cheers
Andrea

PS: the issues I’ve seen are both related to image mosaic changes, one is that IM dropped
dependencies towards jdbc-postgis, the other one is a test failure that I did not investigate,
but are related to image mosaics with time:

Failed tests: testTimeMosaic(org.geoserver.importer.mosaic.ImporterMosaicTest): Expecting to find matches for Xpath //wms:Layer[wms:Name = ‘importer7531593112498765111data’]/wms:Dimension[@name = ‘time’]
testTimeMosaicAuto(org.geoserver.importer.mosaic.ImporterMosaicTest): Expecting to find matches for Xpath //wms:Layer[wms:Name = ‘importer4229144263967260784data’]/wms:Dimension[@name = ‘time’]

–

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

2

Indeed, i found a couple of these issues last week and fixed them, wasn’t aware of the mosaic issue though.

I definitely appreciate your patience with the importer module, but I a little confused about what our extension policy is these days. One of the requirements states usage by at least 3 users? What sort of usage does that imply? Does simply trying out the module once qualify?

If we simply want to build the module and run the tests it might be a little less heavy handed to simply set up a build job for it (or perhaps state that nightly community modules must pass tests).

That said my reservation in pushing on its status in the past has been due to Ian’s work on a client for the rest api and not wanting to release that api until its stable. I think that work is starting to stabilize now ian? Any chance you point us at those sources so folks can try it out?

···

So, how is this for a plan:

  1. Get an update from Ian
  2. Add the module to the community nightlies (and run the tests)
  3. If all goes well we start working on the module status? Maybe a blog post about it might help to drum up some interest.

On Sat, Nov 2, 2013 at 7:37 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

Hi,
today for unrelated causes (see another mail soon) I’ve tried to build the importer module on trunk,
finding that:

  • it does not compile
  • when made to compile, has tests that do not pass anymore

Long story short, the rest of the code is moving and importer is getting left behind.
I know I’m being a pest, but any chance we can push and turn it into an extension, so that it
gets compiled and tested at every build?

Cheers
Andrea

PS: the issues I’ve seen are both related to image mosaic changes, one is that IM dropped
dependencies towards jdbc-postgis, the other one is a test failure that I did not investigate,
but are related to image mosaics with time:

Failed tests: testTimeMosaic(org.geoserver.importer.mosaic.ImporterMosaicTest): Expecting to find matches for Xpath //wms:Layer[wms:Name = ‘importer7531593112498765111data’]/wms:Dimension[@name = ‘time’]
testTimeMosaicAuto(org.geoserver.importer.mosaic.ImporterMosaicTest): Expecting to find matches for Xpath //wms:Layer[wms:Name = ‘importer4229144263967260784data’]/wms:Dimension[@name = ‘time’]

–

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

Android is increasing in popularity, but the open development platform that
developers love is also attractive to malware creators. Download this white
paper to learn more about secure code signing practices that can help keep
Android apps secure.
http://pubads.g.doubleclick.net/gampad/clk?id=65839951&iu=/4140/ostg.clktrk

Geoserver-devel mailing list
Geoserver-devel@anonymised.comt
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

–

Justin Deoliveira
Vice President, Engineering | Boundless
jdeolive@anonymised.com
@j_deolive

3

On Mon, Nov 4, 2013 at 3:57 PM, Justin Deoliveira <jdeolive@anonymised.com

wrote:

Indeed, i found a couple of these issues last week and fixed them, wasn't
aware of the mosaic issue though.

I definitely appreciate your patience with the importer module, but I a
little confused about what our extension policy is these days. One of the
requirements states usage by at least 3 users? What sort of usage does that
imply? Does simply trying out the module once qualify?

Should be at least 3 people having interest in it, hard to get anyone to
truly use a module that is not supported nor tested.
I believe the criteria was there to avoid putting into extension modules
that nobody cares about.
That said, isn't the importer already being used by many (all?) of the
OpenGeo Suite users? I thought the "3 users" criteria
was in the clear already with this module.

If we simply want to build the module and run the tests it might be a
little less heavy handed to simply set up a build job for it (or perhaps
state that nightly community modules must pass tests).

Yes, that would also be a way. Generally speaking, I prefer the current
setup, it offers a "quid pro quo" setup: someone gets charged to be
maintainer, but
in exchange, other devs breaking his/her module need to fix the breakages.
Modules (and poeole behind them) imho need incentive to get out of
community land.

That said my reservation in pushing on its status in the past has been due
to Ian's work on a client for the rest api and not wanting to release that
api until its stable. I think that work is starting to stabilize now ian?
Any chance you point us at those sources so folks can try it out?

So, how is this for a plan:

1. Get an update from Ian
2. Add the module to the community nightlies (and run the tests)
3. If all goes well we start working on the module status? Maybe a blog
post about it might help to drum up some interest.

Works for me.

Cheers
Andrea

--

Our support, Your Success! Visit http://opensdi.geo-solutions.it for more
information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

4

Sorry for the delay, busy.

The python client is here: https://github.com/ischneider/gsimporter
Please don’t consider this the permanent home :slight_smile:

My 2.3 branch of the community importer is here: https://github.com/ischneider/geoserver/tree/importer-2.3.x
This contains a number of fixes. Apologies for not having time for branch maintenance.

I have a laundry list of things that should be revisited, modified, or added :slight_smile:
Best place to put that for discussion?

Regards,

···

On Mon, Nov 4, 2013 at 8:10 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Mon, Nov 4, 2013 at 3:57 PM, Justin Deoliveira <jdeolive@anonymised.com> wrote:

–

Ian Schneider
Software Engineer | Boundless
ischneider@anonymised.com

Indeed, i found a couple of these issues last week and fixed them, wasn’t aware of the mosaic issue though.

I definitely appreciate your patience with the importer module, but I a little confused about what our extension policy is these days. One of the requirements states usage by at least 3 users? What sort of usage does that imply? Does simply trying out the module once qualify?

Should be at least 3 people having interest in it, hard to get anyone to truly use a module that is not supported nor tested.
I believe the criteria was there to avoid putting into extension modules that nobody cares about.
That said, isn’t the importer already being used by many (all?) of the OpenGeo Suite users? I thought the “3 users” criteria
was in the clear already with this module.

If we simply want to build the module and run the tests it might be a little less heavy handed to simply set up a build job for it (or perhaps state that nightly community modules must pass tests).

Yes, that would also be a way. Generally speaking, I prefer the current setup, it offers a “quid pro quo” setup: someone gets charged to be maintainer, but
in exchange, other devs breaking his/her module need to fix the breakages.
Modules (and poeole behind them) imho need incentive to get out of community land.

That said my reservation in pushing on its status in the past has been due to Ian’s work on a client for the rest api and not wanting to release that api until its stable. I think that work is starting to stabilize now ian? Any chance you point us at those sources so folks can try it out?

Works for me.

Cheers

Andrea

–

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

So, how is this for a plan:

  1. Get an update from Ian
  2. Add the module to the community nightlies (and run the tests)
  3. If all goes well we start working on the module status? Maybe a blog post about it might help to drum up some interest.
5

On Wed, Nov 6, 2013 at 6:10 PM, Ian Schneider
<ischneider@anonymised.com>wrote:

Sorry for the delay, busy.

The python client is here: https://github.com/ischneider/gsimporter
Please don't consider this the permanent home :slight_smile:

My 2.3 branch of the community importer is here:
https://github.com/ischneider/geoserver/tree/importer-2.3.x
This contains a number of fixes. Apologies for not having time for branch
maintenance.

I have a laundry list of things that should be revisited, modified, or
added :slight_smile:
Best place to put that for discussion?

I'd say, let's discuss it here on geoserver-devel?

Cheers
Andrea

--

Our support, Your Success! Visit http://opensdi.geo-solutions.it for more
information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

6

On Wed, Nov 6, 2013 at 10:14 AM, Andrea Aime
<andrea.aime@anonymised.com>wrote:

On Wed, Nov 6, 2013 at 6:10 PM, Ian Schneider <ischneider@anonymised.com
> wrote:

Sorry for the delay, busy.

The python client is here: https://github.com/ischneider/gsimporter
Please don't consider this the permanent home :slight_smile:

My 2.3 branch of the community importer is here:
https://github.com/ischneider/geoserver/tree/importer-2.3.x
This contains a number of fixes. Apologies for not having time for branch
maintenance.

I have a laundry list of things that should be revisited, modified, or
added :slight_smile:
Best place to put that for discussion?

I'd say, let's discuss it here on geoserver-devel?

Agree we can discuss here and then throw the outcome into the relevant
issue trackers.

Cheers
Andrea

--

Our support, Your Success! Visit http://opensdi.geo-solutions.it for more
information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

--
*Justin Deoliveira*
Vice President, Engineering | Boundless
jdeolive@anonymised.com
@j_deolive <https://twitter.com/j_deolive&gt;