[Geoserver-devel] GeoServer Meeting 2016-06-28

GeoServer GeoServer Developer
,
1

GeoTools / GeoServer Meeting 2016-06-28

Attending

Andrea Aime
Torben Barsballe
Simone Giannecchini
Kevin Smith - welcome back
Jody Garnett
Alessio Fabiani

Apologies

Ben Caradoc-Davies
Brad Hards

Agenda

  • JPEG or PNG output format pull requests ready
  • Backporting layer group specific services
  • FOSS4G code sprint
  • GeoTools 15.1/GeoServer 2.9.1 release manager
  • GWC / GeoServer remote execution vulnerability (Restlet 1.0)
  • BugFix Minisprint?
  • GeoTools developer guide not deploying
  • ImageMosaic refactor - take 1 (and 2 as well)
  • Roadmap Discussion

Actions

  • Reach out to QGis community for SLD export.
  • Try to organize mini code sprints [Simone]

Actions from last meeting

  • Jody: encourage updates to docs/jira to record Simone as GeoServer project officer
  • Alessandro and Andrea: release 14.4 / 2.8.4 [DONE]

JPEG or PNG output format pull requests ready

Support format=image/vnd.jpeg-png to dynamically choose appropriate compression:

The “image/vnd.jpeg-png” will also be implemented in MapServer 7.2.

Backporting layer group specific services

We have layer specific services, but don’t work for layer groups. Code pretty similar to layer specific services, extending it to layer groups.
https://osgeo-org.atlassian.net/browse/GEOS-7463
https://github.com/geoserver/geoserver/pull/1539

Backport to 2.9 and 2.8? Unlikely to harm stability?

FOSS4G code sprint

Ian started thread about code sprint on ml

Participants please sign up on wiki page: https://wiki.osgeo.org/wiki/FOSS4G_2016_Code_Sprint#GeoTools.2FGeoServer

Ideas:

  • Looking to pair with a QGis developer and make SLD export towards GeoServer actually work.
  • JAI replacement, spending time to outline what the API would look like.
  • Java 8 update for GeoTools.

GeoTools 15.1/GeoServer 2.9.1 release manager

Thanks to Alessandro and Andrea for 14.4 release.

And the winner is… Devon (with backup from Jody).

GWC / GeoServer remote execution vulnerability (Restlet 1.0)

Upgrade postgres driver (not really needed but trivial to do).

Restlet 1.0

Q: What is it going to take to do the update? When restlet 1.0 → restlet 2.0 had a J2SE / J2EE split (and we use a bit of both).

Consider setting up a code sprint on this (as the change over is likely freeze the code-base topic). Consider asking GeoSoutions to host … or running remote code/sprint.

action: take this to email

Q: The vunerability is based on a specific format, can we avoid using this for a quick fix?

BugFix Minisprint

Bug reports getting out of control … last three months we have gathered 45 new tickets.

Around 3/4 of these reports are coming non-developers, indeed most of the reports that are not getting solved are coming from non-developers.

https://osgeo-org.atlassian.net/secure/ConfigureReport.jspa?projectOrFilterId=project-10000&periodName=daily&daysprevious=90&cumulative=true&versionLabels=major&selectedProjectId=10000&reportKey=com.atlassian.jira.plugin.system.reports%3Acreatedvsresolved-report&Next=Next

Need some warning to get resoruces avaialble?

Alternative idea

  • make this recurrent 1 day a month, some lose coordination
  • some prep can be used to sort / cull issues prior

Chart reporting reporters distribution:

https://osgeo-org.atlassian.net/secure/ConfigureReport.jspa?projectOrFilterId=filter-10801&statistictype=reporter&selectedProjectId=10000&reportKey=com.atlassian.jira.plugin.system.reports%3Apie-report&Next=Next

GeoTools developer guide not deploying

http://docs.geotools.org/latest/developer/conventions/code/style.html

This looks correct now, thanks Chris.

ImageMosaic refactor - take 1 (and 2 as well)

https://github.com/geotools/geotools/wiki/Refactor-ImageMosaic-Index-and-Catalog-management-for-improved-extensibility

  • Consider a vote of 1 (Simone as module maintainer)
  • No proposal for take 2, but writing something still helps on email

https://s3.amazonaws.com/archive.travis-ci.org/jobs/140140350/log.txt

[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /home/travis/build/geotools/geotools/modules/unsupported/coveragetools/src/main/java/org/geotools/utils/imagepyramid/PyramidBuilder.java:[591,64] error: ImageMosaicConfigHandler(CatalogBuilderConfiguration,ImageMosaicEventHandlers) is not public in ImageMosaicConfigHandler; cannot be accessed from outside package
[ERROR] /home/travis/build/geotools/geotools/modules/unsupported/coveragetools/src/main/java/org/geotools/utils/imagemosaic/CommandLineCatalogBuilderRunner.java:[164,56] error: ImageMosaicConfigHandler(CatalogBuilderConfiguration,ImageMosaicEventHandlers) is not public in ImageMosaicConfigHandler; cannot be accessed from outside package
[INFO] 2 errors

Roadmap Discussion

Like to prep a few for foss4g milestone.

Style Editor

  • Take the css style editor as the default style editor (for css, sld, ysld, etc…)
  • Consider preview timeout cutout
  • consider moving tabs to the top
  • wish: consider a basemap or group layer background?
  • Reduce SLD tab to a link to GetStyles REQUEST may be useful?
  • Will close one ticket! https://osgeo-org.atlassian.net/browse/GEOS-3000

GeoTools + Java 8

  • Talk with Kevin about ideas here
  • streams
  • predicates
  • maybe optionals
  • factor out functional interfaces

Version hell:

  • Guava → migrate to Java 8 (conflict with ElasticGeo and GeoMesa). Consder removing as.
2

Hi all,

Looks like there is plenty going on, I just wanted to add some info re: GeoMesa and Guava.

Since GeoMesa code ends up being deployed in GeoServer, Hadoop, and other environments, Guava is a provided dependency. We try to use Guava features which have remained consistent between Guava versions (ranging from 11 to 17 or so).

From talking to Tyler on GeoMesa’s Gitter, we figured out that as GeoMesa moves to GT 15/GS 2.9, GeoMesa would need to update the Joda time dependency to 2.8 or 2.9.

Let me know if there’s a better practice or thing I can do to help out, etc.

Cheers,

Jim

···

On 06/28/2016 12:34 PM, Jody Garnett wrote:

Version hell:

  • Guava → migrate to Java 8 (conflict with ElasticGeo and GeoMesa). Consder removing as.
3

As per meeting notes I would like to know if removing Guava is something we can do over the course of 2.10 development.

So does Joda time use Guava as well?

···

On 28 June 2016 at 12:31, Jim Hughes <jnh5y@…1612…> wrote:

Hi all,

Looks like there is plenty going on, I just wanted to add some info re: GeoMesa and Guava.

Since GeoMesa code ends up being deployed in GeoServer, Hadoop, and other environments, Guava is a provided dependency. We try to use Guava features which have remained consistent between Guava versions (ranging from 11 to 17 or so).

From talking to Tyler on GeoMesa’s Gitter, we figured out that as GeoMesa moves to GT 15/GS 2.9, GeoMesa would need to update the Joda time dependency to 2.8 or 2.9.

Let me know if there’s a better practice or thing I can do to help out, etc.

Cheers,

Jim

On 06/28/2016 12:34 PM, Jody Garnett wrote:

Version hell:

  • Guava → migrate to Java 8 (conflict with ElasticGeo and GeoMesa). Consder removing as.

Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape

Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

–
Jody Garnett

4

Hi Jody,

Thanks for the clarification; I didn’t get it that the line was a suggestion to remove Guava.

From a cursory look, it appears that GeoTools has few dependencies on Guava. For GeoServer, it seems that GWC and other pieces want to be able to have a Guava cache. I don’t think there’s a Java 8 analog for those caching options…

For my money, I’d be happy if GeoServer picked a version of Guava (or a range of versions) to aim for. Some of the Guava functionality has stayed the same between versions, and modules which can leverage that might work out better together. (That is, there’d be some utility in using more common Guava functionality rather than a feature which is only in one version.)

Do you have more details about the dependency issues with GeoMesa? (I do see where the ElasticGeo instructions have you ripping out Guava, etc.)

Joda doesn’t use Guava. Bringing up GeoServer and GeoMesa dependencies made me think of the Joda updates.

Cheers,

Jim

···

On 06/28/2016 06:13 PM, Jody Garnett wrote:

As per meeting notes I would like to know if removing Guava is something we can do over the course of 2.10 development.

So does Joda time use Guava as well?

–
Jody Garnett

On 28 June 2016 at 12:31, Jim Hughes <jnh5y@anonymised.com> wrote:

Hi all,

Looks like there is plenty going on, I just wanted to add some info re: GeoMesa and Guava.

Since GeoMesa code ends up being deployed in GeoServer, Hadoop, and other environments, Guava is a provided dependency. We try to use Guava features which have remained consistent between Guava versions (ranging from 11 to 17 or so).

From talking to Tyler on GeoMesa’s Gitter, we figured out that as GeoMesa moves to GT 15/GS 2.9, GeoMesa would need to update the Joda time dependency to 2.8 or 2.9.

Let me know if there’s a better practice or thing I can do to help out, etc.

Cheers,

Jim

On 06/28/2016 12:34 PM, Jody Garnett wrote:

Version hell:

  • Guava → migrate to Java 8 (conflict with ElasticGeo and GeoMesa). Consder removing as.

Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape

Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

5

Hi Jim

One thing we were looking at was trying to make sure that GeoMesa (using Guava 17 by way of Accumulo) and ElasticGeo (using Guava 18 by way of Elasticsearch) wouldn’t cause conflicts if both extensions were installed. One suggestion was to try and have each extension run in its own class loader. However, if we want to have a different version of Guava for these two dependencies, then we could not have Guava in core GeoServer (else it would conflict with one of the extensions).

Torben

···

On Tue, Jun 28, 2016 at 3:43 PM, Jim Hughes <jnh5y@anonymised.com.1612…> wrote:

Hi Jody,

Thanks for the clarification; I didn’t get it that the line was a suggestion to remove Guava.

From a cursory look, it appears that GeoTools has few dependencies on Guava. For GeoServer, it seems that GWC and other pieces want to be able to have a Guava cache. I don’t think there’s a Java 8 analog for those caching options…

For my money, I’d be happy if GeoServer picked a version of Guava (or a range of versions) to aim for. Some of the Guava functionality has stayed the same between versions, and modules which can leverage that might work out better together. (That is, there’d be some utility in using more common Guava functionality rather than a feature which is only in one version.)

Do you have more details about the dependency issues with GeoMesa? (I do see where the ElasticGeo instructions have you ripping out Guava, etc.)

Joda doesn’t use Guava. Bringing up GeoServer and GeoMesa dependencies made me think of the Joda updates.

Cheers,

Jim

On 06/28/2016 06:13 PM, Jody Garnett wrote:

As per meeting notes I would like to know if removing Guava is something we can do over the course of 2.10 development.

So does Joda time use Guava as well?

Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape

Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

–
Jody Garnett

On 28 June 2016 at 12:31, Jim Hughes <jnh5y@anonymised.com> wrote:

Hi all,

Looks like there is plenty going on, I just wanted to add some info re: GeoMesa and Guava.

Since GeoMesa code ends up being deployed in GeoServer, Hadoop, and other environments, Guava is a provided dependency. We try to use Guava features which have remained consistent between Guava versions (ranging from 11 to 17 or so).

From talking to Tyler on GeoMesa’s Gitter, we figured out that as GeoMesa moves to GT 15/GS 2.9, GeoMesa would need to update the Joda time dependency to 2.8 or 2.9.

Let me know if there’s a better practice or thing I can do to help out, etc.

Cheers,

Jim

On 06/28/2016 12:34 PM, Jody Garnett wrote:

Version hell:

  • Guava → migrate to Java 8 (conflict with ElasticGeo and GeoMesa). Consder removing as.

Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape

Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

6

Hi Torben,

Makes sense. While it isn’t a full-up integration test, I was able to build and run most of the GeoMesa unit tests with Guava 18. There was only one slightly issue; in one module, there was a method used which is not present in Guava 18.

I think if we removed that from GeoMesa, we’d be able to work with Guava 11-18. The GeoMesa datastore jars/zips/extensions targeting GeoServer 2.8.x do not include Guava since Guava 17 is part of the GeoServer distribution.

Anyhow, having classloaders per extension might be necessary at some point, but I’d like to try and help avoid it;).

Cheers,

Jim

···

On 06/28/2016 07:51 PM, Torben Barsballe wrote:

Hi Jim

One thing we were looking at was trying to make sure that GeoMesa (using Guava 17 by way of Accumulo) and ElasticGeo (using Guava 18 by way of Elasticsearch) wouldn’t cause conflicts if both extensions were installed. One suggestion was to try and have each extension run in its own class loader. However, if we want to have a different version of Guava for these two dependencies, then we could not have Guava in core GeoServer (else it would conflict with one of the extensions).

Torben

On Tue, Jun 28, 2016 at 3:43 PM, Jim Hughes <jnh5y@anonymised.com> wrote:

Hi Jody,

Thanks for the clarification; I didn’t get it that the line was a suggestion to remove Guava.

From a cursory look, it appears that GeoTools has few dependencies on Guava. For GeoServer, it seems that GWC and other pieces want to be able to have a Guava cache. I don’t think there’s a Java 8 analog for those caching options…

For my money, I’d be happy if GeoServer picked a version of Guava (or a range of versions) to aim for. Some of the Guava functionality has stayed the same between versions, and modules which can leverage that might work out better together. (That is, there’d be some utility in using more common Guava functionality rather than a feature which is only in one version.)

Do you have more details about the dependency issues with GeoMesa? (I do see where the ElasticGeo instructions have you ripping out Guava, etc.)

Joda doesn’t use Guava. Bringing up GeoServer and GeoMesa dependencies made me think of the Joda updates.

Cheers,

Jim

On 06/28/2016 06:13 PM, Jody Garnett wrote:

As per meeting notes I would like to know if removing Guava is something we can do over the course of 2.10 development.

So does Joda time use Guava as well?

–
Jody Garnett

On 28 June 2016 at 12:31, Jim Hughes <jnh5y@anonymised.com> wrote:

Hi all,

Looks like there is plenty going on, I just wanted to add some info re: GeoMesa and Guava.

Since GeoMesa code ends up being deployed in GeoServer, Hadoop, and other environments, Guava is a provided dependency. We try to use Guava features which have remained consistent between Guava versions (ranging from 11 to 17 or so).

From talking to Tyler on GeoMesa’s Gitter, we figured out that as GeoMesa moves to GT 15/GS 2.9, GeoMesa would need to update the Joda time dependency to 2.8 or 2.9.

Let me know if there’s a better practice or thing I can do to help out, etc.

Cheers,

Jim

On 06/28/2016 12:34 PM, Jody Garnett wrote:

Version hell:

  • Guava → migrate to Java 8 (conflict with ElasticGeo and GeoMesa). Consder removing as.

Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape

Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape

Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

7

For GeoTools we use ObjectPool for caching, perhaps that is an alternative to Guava cache.

···

On 28 June 2016 at 15:43, Jim Hughes <jnh5y@…1612…> wrote:

Hi Jody,

Thanks for the clarification; I didn’t get it that the line was a suggestion to remove Guava.

From a cursory look, it appears that GeoTools has few dependencies on Guava. For GeoServer, it seems that GWC and other pieces want to be able to have a Guava cache. I don’t think there’s a Java 8 analog for those caching options…

For my money, I’d be happy if GeoServer picked a version of Guava (or a range of versions) to aim for. Some of the Guava functionality has stayed the same between versions, and modules which can leverage that might work out better together. (That is, there’d be some utility in using more common Guava functionality rather than a feature which is only in one version.)

Do you have more details about the dependency issues with GeoMesa? (I do see where the ElasticGeo instructions have you ripping out Guava, etc.)

Joda doesn’t use Guava. Bringing up GeoServer and GeoMesa dependencies made me think of the Joda updates.

Cheers,

Jim

On 06/28/2016 06:13 PM, Jody Garnett wrote:

As per meeting notes I would like to know if removing Guava is something we can do over the course of 2.10 development.

So does Joda time use Guava as well?

–
Jody Garnett

–
Jody Garnett

On 28 June 2016 at 12:31, Jim Hughes <jnh5y@anonymised.com> wrote:

Hi all,

Looks like there is plenty going on, I just wanted to add some info re: GeoMesa and Guava.

Since GeoMesa code ends up being deployed in GeoServer, Hadoop, and other environments, Guava is a provided dependency. We try to use Guava features which have remained consistent between Guava versions (ranging from 11 to 17 or so).

From talking to Tyler on GeoMesa’s Gitter, we figured out that as GeoMesa moves to GT 15/GS 2.9, GeoMesa would need to update the Joda time dependency to 2.8 or 2.9.

Let me know if there’s a better practice or thing I can do to help out, etc.

Cheers,

Jim

On 06/28/2016 12:34 PM, Jody Garnett wrote:

Version hell:

  • Guava → migrate to Java 8 (conflict with ElasticGeo and GeoMesa). Consder removing as.

Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape

Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

8

On Wed, Jun 29, 2016 at 7:03 PM, Jody Garnett <jody.garnett@anonymised.com>
wrote:

For GeoTools we use ObjectPool for caching, perhaps that is an alternative
to Guava cache.

You mean the commons-pool cache? That does not seem like a suitable
replacement from
performance and flexibility point of view, the one we have is pretty old...
unless maybe
commons-pool 2 is much better, but we'd also have to upgrade commons-dbcp
as a result,
possibly something else...

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------