[Geoserver-devel] Convention for nested kvp inside of format_options

GeoServer GeoServer Developer
1

Hi folks,

I am wondering if there is a convention for encoding key value pairs inside of a single format_options kvp? Basically I want to add a new kml format option, but I want that option to have multiple options within it. The structure would look something like this:

format_options:
   foo: 
      opt1: val1
      opt2: val2

I am wondering if there is any precedent for encoding this type of info in a way that doesn’t break the format_options parser. If there is I’ll use that, if not I was thinking maybe using @ and , … something like:

format_options=foo:opt1@anonymised.com,opt2@anonymised.com;

Any thoughts? Suggestions welcome :slight_smile:

Thanks.

-Justin

2

Hi Justin,
I cannot remember any case like that. Does it really have to be structured that way, like,
wouldn’it it be possible to just add N keys instead of a structured one?
Worries:

  • The new separator would introduce a potential backwards compatibility issue (if legit part of any value, it would have to be espaped)
  • The KVP protocol would become more complex, with the risk of moving towards WPS like complexity (Execute does not have a KVP binding anymore in WPS 2.0 because of that)
    If they keys are not predictable, thinking out loud, what about supporting a JSON encoded document as an alternative syntax, at that point?
format_options={"foo"{"opt1"="val1","opt2"="val2"}}

Cheers

Andrea

···

On Thu, Apr 13, 2017 at 1:46 PM, Justin Deoliveira <jdeolive@anonymised.com> wrote:

Hi folks,

I am wondering if there is a convention for encoding key value pairs inside of a single format_options kvp? Basically I want to add a new kml format option, but I want that option to have multiple options within it. The structure would look something like this:

format_options:
   foo: 
      opt1: val1
      opt2: val2

I am wondering if there is any precedent for encoding this type of info in a way that doesn’t break the format_options parser. If there is I’ll use that, if not I was thinking maybe using @ and , … something like:

format_options=foo:opt1@anonymised.com,opt2@anonymised.com;

Any thoughts? Suggestions welcome :slight_smile:

Thanks.

-Justin

Check out the vibrant tech community on one of the world’s most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

Geoserver-devel mailing list
Geoserver-devel@anonymised.com.366…sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

–

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313

fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

3

Hey Andrea,

Thanks for the input. Adding multiple keys at the top level would certainly be an option, I just figured that since there are already quite a few kml format options keys that having it encapsulated into a single one would be a little nicer, and more aesthetically pleasing.

To be clear I wasn’t proposing any change to format options syntax or parser, just to see if someone had done this sort of thing before. My thinking was that as long I chose delimiters that aren’t currently used (ie not one of “&”, “=“, “:”, or “;”) then it should be pretty uninvasive and not introduce any compatibility issues.

Having the ability to use a JSON object would definitely be nice. From what I understand of the current parser that would break it, but it seems like it should be doable without too many changes. Perhaps I’ll explore that option.

Anyways I’ve got a few avenues to explore here so I’ll do that. I’ll see what makes most sense in the end and we can revisit this when I submit the pull request for the change. Actually I’ll probably spark up some discussion before that just to make sure my approach jives ok with folks. For context I am working on GEOS-8033.

-Justin

···

On Thu, Apr 13, 2017 at 1:46 PM, Justin Deoliveira <jdeolive@anonymised.com> wrote:

Hi folks,

I am wondering if there is a convention for encoding key value pairs inside of a single format_options kvp? Basically I want to add a new kml format option, but I want that option to have multiple options within it. The structure would look something like this:

format_options:
   foo: 
      opt1: val1
      opt2: val2

I am wondering if there is any precedent for encoding this type of info in a way that doesn’t break the format_options parser. If there is I’ll use that, if not I was thinking maybe using @ and , … something like:

format_options=foo:opt1@anonymised.com,opt2@anonymised.com;

Any thoughts? Suggestions welcome :slight_smile:

Thanks.

-Justin

Check out the vibrant tech community on one of the world’s most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

–

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313

fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

4

On Thu, Apr 13, 2017 at 3:17 PM, Justin Deoliveira <jdeolive@anonymised.com>
wrote:

Hey Andrea,

Thanks for the input. Adding multiple keys at the top level would
certainly be an option, I just figured that since there are already quite a
few kml format options keys that having it encapsulated into a single one
would be a little nicer, and more aesthetically pleasing.

To be clear I wasn’t proposing any change to format options syntax or
parser, just to see if someone had done this sort of thing before. My
thinking was that as long I chose delimiters that aren’t currently used (ie
not one of “&”, “=“, “:”, or “;”) then it should be pretty uninvasive and
not introduce any compatibility issues.

I'm not so sure, the format options parser is used for more than just
format options:

wfs/src/main/java/applicationContext.xml: <bean
id="wfsFormatOptionsKvpParser"
class="org.geoserver.ows.kvp.FormatOptionsKvpParser"/>
wms/src/main/java/applicationContext.xml: <bean
id="wmsFormatOptionsKvpParser"
class="org.geoserver.ows.kvp.FormatOptionsKvpParser"/>
wms/src/main/java/applicationContext.xml: <bean
id="wmsEnviromentKvpParser"
class="org.geoserver.ows.kvp.FormatOptionsKvpParser">
wms/src/main/java/applicationContext.xml: <bean
id="wmsLegendOptionsKvpParser"
class="org.geoserver.ows.kvp.FormatOptionsKvpParser">

While I agree you can guess by looking at the existing format options what
chars are not used (at least in the official GeoServer code base), I would
not try to make the
same guess about env vars (anything goes in there, imho). If you really
want to go there, I'd add a parameter to limit the nested options
properties to just the
real format options parsers (the wms and wfs ones).

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------

5

I see, so this would mean adding my option at the top level (rather than inside of format_options) and having it use the same format_options syntax. That would work.

I am not really following here… so if I am going to add a new format option (for whatever output format not necessarily kml) are you saying it’s only safe to use a very limited character set for it’s value space? Like only alphanumerics and no symbols?

···

On Thu, Apr 13, 2017 at 3:17 PM, Justin Deoliveira <jdeolive@anonymised.com.403…> wrote:

Hey Andrea,

Thanks for the input. Adding multiple keys at the top level would certainly be an option, I just figured that since there are already quite a few kml format options keys that having it encapsulated into a single one would be a little nicer, and more aesthetically pleasing.

To be clear I wasn’t proposing any change to format options syntax or parser, just to see if someone had done this sort of thing before. My thinking was that as long I chose delimiters that aren’t currently used (ie not one of “&”, “=“, “:”, or “;”) then it should be pretty uninvasive and not introduce any compatibility issues.

I’m not so sure, the format options parser is used for more than just format options:

wfs/src/main/java/applicationContext.xml:
wms/src/main/java/applicationContext.xml:

wms/src/main/java/applicationContext.xml:
wms/src/main/java/applicationContext.xml:

While I agree you can guess by looking at the existing format options what chars are not used (at least in the official GeoServer code base), I would not try to make the
same guess about env vars (anything goes in there, imho). If you really want to go there, I’d add a parameter to limit the nested options properties to just the
real format options parsers (the wms and wfs ones).

Cheers

Andrea

–

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313

fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

6

On Thu, Apr 13, 2017 at 3:41 PM, Justin Deoliveira <jdeolive@anonymised.com>
wrote:

I am not really following here… so if I am going to add a new format
option (for whatever output format not necessarily kml) are you saying it’s
only safe to use a very limited character set for it’s value space? Like
only alphanumerics and no symbols?

From your initial proposal I thought the splitting of the inner values was

done inside FormatOptionsKvpParser, for any options new and old, using a
well known (but new) separator, like @.
If you instead post-parse the top level option in the KML handling code
then there is no problem

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------

7

My apologies for being unclear, yes I was proposing the second option.

···

On Thu, Apr 13, 2017 at 3:41 PM, Justin Deoliveira <jdeolive@anonymised.com.403…> wrote:

I am not really following here… so if I am going to add a new format option (for whatever output format not necessarily kml) are you saying it’s only safe to use a very limited character set for it’s value space? Like only alphanumerics and no symbols?

From your initial proposal I thought the splitting of the inner values was done inside FormatOptionsKvpParser, for any options new and old, using a well known (but new) separator, like @.
If you instead post-parse the top level option in the KML handling code then there is no problem

Cheers

Andrea

–

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313

fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.