[Geoserver-devel] Asking include GEOS-9321 (json nested limit fix) into 2.16.0

GeoServer GeoServer Developer
1

Hi community,

Recently I sent a PR for fixing GEOS-9321 (json nested limit fix) on master. This fix includes json-lib version upgrade to a forked and fixed new release, since the hard-coded limit resides into this library code. Currently new Json-lib Jar is released into Geosolutions maven server, but the plan is having it on OSGeo maven servers as next step.

This is an important fix for complex features usage and for downstream projects, and it is isolated to WFS GeoJSON output max nested objects support. No format, logic or behavior changes.

PR:

https://github.com/geoserver/geoserver/pull/3736

JIRA issue:
https://osgeo-org.atlassian.net/browse/GEOS-9321

So I’m asking permission for including this fix into 2.16.0 release branch.

Thanks.

Regards,

Fernando Mino

==

GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.

==

Fernando Mino

Software Engineer

GeoSolutions S.A.S.

Via di Montramito 3/A

55054 Massarosa (LU)

Italy

phone: +39 0584 962313

fax: +39 0584 1660272

http://www.geo-solutions.it

http://twitter.com/geosolutions_it

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.

2

Hi all,
adding a bit more info about this one, this nested JSON limit issue and how to handle has been discussed in this ML:
http://osgeo-org.1560.x6.nabble.com/JSON-Nesting-too-deep-exception-due-to-hard-coded-limit-td5409280.html

As described in the linked thread, this is not a GeoServer limitation, it is a json-lib limitation, in GeoServer we only need to modify the pom.xml.
We forked the lib and implemented the fix discussed on the thread, and make the necessary artifact available for Maven.

That say, we need this fix badly on a few downstream projects that will depend on GeoServer 2.16.0.
Since this is a very isolated bug fix, which doesn’t change any API, he would like to include it on 2.16.0.

Any objection?

Cheers,
Nuno Oliveira

On Fri, 2019-09-06 at 15:28 +0200, fernando.mino@anonymised.com wrote:

Hi community,

Recently I sent a PR for fixing GEOS-9321 (json nested limit fix) on master. This fix includes json-lib version upgrade to a forked and fixed new release, since the hard-coded limit resides into this library code. Currently new Json-lib Jar is released into Geosolutions maven server, but the plan is having it on OSGeo maven servers as next step.

This is an important fix for complex features usage and for downstream projects, and it is isolated to WFS GeoJSON output max nested objects support. No format, logic or behavior changes.

PR:

https://github.com/geoserver/geoserver/pull/3736

JIRA issue:
https://osgeo-org.atlassian.net/browse/GEOS-9321

So I’m asking permission for including this fix into 2.16.0 release branch.

Thanks.

Regards,

Fernando Mino

==

GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.

==

Fernando Mino

Software Engineer

GeoSolutions S.A.S.

Via di Montramito 3/A

55054 Massarosa (LU)

Italy

phone: +39 0584 962313

fax: +39 0584 1660272

http://www.geo-solutions.it

http://twitter.com/geosolutions_it

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.

_______________________________________________
Geoserver-devel mailing list
[Geoserver-devel@anonymised.comsts.sourceforge.net](mailto:Geoserver-devel@lists.sourceforge.net)
[https://lists.sourceforge.net/lists/listinfo/geoserver-devel](https://lists.sourceforge.net/lists/listinfo/geoserver-devel)


-- 
Regards,
Nuno Oliveira
==
GeoServer Professional Services from the
experts! 
Visit http://goo.gl/it488V for more information.
==

Nuno Miguel Carvalho Oliveira
@nmcoliveira
Software Engineer

GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
Italy
phone: +39 0584 962313
fax:      +39 0584 1660272

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

Con riferimento alla normativa sul trattamento dei dati 
personali (Reg. UE 2016/679 - Regolamento generale sulla 
protezione dei dati “GDPR”), si precisa che ogni 
circostanza inerente alla presente email (il suo contenuto, 
gli eventuali allegati, etc.) è un dato la cui conoscenza 
è riservata al/i solo/i destinatario/i indicati dallo 
scrivente. Se il messaggio Le è giunto per errore, è 
tenuta/o a cancellarlo, ogni altra operazione è illecita. 
Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to 
which it is addressed and may contain information that 
is privileged, confidential or otherwise protected from 
disclosure. We remind that - as provided by European 
Regulation 2016/679 “GDPR” - copying, dissemination or 
use of this e-mail or the information herein by anyone 
other than the intended recipient is prohibited. If you 
have received this email by mistake, please notify 
us immediately by telephone or e-mail.
3

We discussed this internally already and I am fine with this approach.

We also need to talk about long term approach to maintain the JSON lib we are talking about.

···

Regards,
Simone Giannecchini

GeoServer Professional Services from the experts!
Visit http://bit.ly/gs-services for more information.

Ing. Simone Giannecchini
@simogeo
Founder/Director

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 333 8128928

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.

4

Seems fine to me as it is just a pom change.

+1

Ian

On Fri, 6 Sep 2019, 14:29 fernando.mino@anonymised.com, <fernando.mino@anonymised.com> wrote:

Hi community,

Recently I sent a PR for fixing GEOS-9321 (json nested limit fix) on master. This fix includes json-lib version upgrade to a forked and fixed new release, since the hard-coded limit resides into this library code. Currently new Json-lib Jar is released into Geosolutions maven server, but the plan is having it on OSGeo maven servers as next step.

This is an important fix for complex features usage and for downstream projects, and it is isolated to WFS GeoJSON output max nested objects support. No format, logic or behavior changes.

PR:

https://github.com/geoserver/geoserver/pull/3736

JIRA issue:
https://osgeo-org.atlassian.net/browse/GEOS-9321

So I’m asking permission for including this fix into 2.16.0 release branch.

Thanks.

Regards,

Fernando Mino

==

GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.

==

Fernando Mino

Software Engineer

GeoSolutions S.A.S.

Via di Montramito 3/A

55054 Massarosa (LU)

Italy

phone: +39 0584 962313

fax: +39 0584 1660272

http://www.geo-solutions.it

http://twitter.com/geosolutions_it

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.

Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

5

Should we fork the JSON lib into the geoserver repository?

I note that forked jars are a little bit of a bother to track for GeoMesa (and other eclipse projects) so I would like to confirm that this is just a GeoServer dependency.

···

–
Jody Garnett

6

Should we fork the JSON lib into the geoserver repository?

I note that forked jars are a little bit of a bother to track for GeoMesa (and other eclipse projects) so I would like to confirm that this is just a GeoServer dependency.

Yes, to the best of my knowledge it’s just a GeoServer dependency. Quick check (you can propose and run more thorough ones):

geoserver (master) $ git grep json-lib
src/community/script/groovy/pom.xml: net.sf.json-lib
src/community/script/groovy/pom.xml: json-lib
src/community/security/keycloak/pom.xml: net.sf.json-lib
src/community/security/keycloak/pom.xml: json-lib
src/community/security/oauth2-geonode/pom.xml: net.sf.json-lib
src/community/security/oauth2-geonode/pom.xml: json-lib
src/community/security/oauth2-github/pom.xml: net.sf.json-lib
src/community/security/oauth2-github/pom.xml: json-lib
src/community/security/oauth2-openid-connect/pom.xml: net.sf.json-lib
src/community/security/oauth2-openid-connect/pom.xml: json-lib
src/extension/sldService/pom.xml:
src/main/pom.xml: net.sf.json-lib
src/main/pom.xml: json-lib
src/pom.xml: net.sf.json-lib
src/pom.xml: json-lib
src/release/ext-authkey.xml: json-lib*
src/wfs/pom.xml: net.sf.json-lib
src/wfs/pom.xml: json-lib

geotools (master) $ git grep json-lib
(nothing)

And oh, +1. At some point we’ll have to make a code sprint to upgrade all our json processing to a more
modern library (we have like 3 different approaches, or more, spread around) but I believe that will require
a code sprint.

Cheers
Andrea

···

GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.

7

We have a fork with the fix here: https://github.com/geosolutions-it/Json-lib/tree/limit-fix-2.4
No issue in copying it once more over to GeoServer

Cheers
Andrea

···

Regards, Andrea Aime == GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.

8

Dear all,
thank you for the feedback, related issue here:
https://osgeo-org.atlassian.net/browse/GEOS-9321

Pull requests created and reviewed:

I will merged once Travis is happy :slight_smile:

Best regards,
Nuno Oliveira

···

GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.