[Geoserver-devel] Properly advertising the charset in text resposes

GeoServer GeoServer Developer
1

Hi,
I’m looking at an issue where browsers are not correctly displaying documents
such as csv or json because we are not declaring in which charset the document
is being returned.

Now, the HTTP spec encourages servers to setup the charset in the content type header
for this purpose, see:
http://www.w3.org/International/O-HTTP-charset

GeoServer wise, we have a target charset in our SettingsInfo object, and the response
are diligently using it when wrapping the output stream provided by Response.write
into a Writer… however pretty much nothing besides the exception handlers
remembers to set it in the HTTP headers.
The XML responses are sort of an exception/gray area, in that while they are not
setting the HTTP header, they are declaring the charset in the XML header.

HTTP headers wise we have two options, we can either extend the mime type
we are returning (as suggested by the above link) and say something like
“text/csv; charset=UTF-8” instead of just “text/csv”, or be programmatic
like the exception handlers, and call ServletResponse.setCharacterEncoding
http://docs.oracle.com/javaee/6/api/javax/servlet/ServletResponse.html#setCharacterEncoding(java.lang.String)

Personally I would prefer the latter. Now… how to make Response commmunicate
the charset it’s using? It’s likely going to be the one we have in the Settings, but
in some cases it might not (custom responses, or standards mandating the usage
of a particular charset for a given format) and in other cases the response is just
binary, so we should not set it.

My suggestion would be to extend org.geoserver.ows.Response and add the following method:

/**

  • Returns the charset for this response, the Dispatcher will set it in the ServletResponse.
  • The default implementation returns null, in this case no encoding will be set.
  • Subclasses returning text documents should override taking into account SettingsInfo.getCharset()
  • as well as the specific encoding requirements of the returned format.
    */

public String getCharset()
return null;
}

Opinions?

Cheers
Andrea

–

==

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

==

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

2

Works for me.

···

On Mon, Oct 6, 2014 at 3:09 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

Hi,
I’m looking at an issue where browsers are not correctly displaying documents
such as csv or json because we are not declaring in which charset the document
is being returned.

Now, the HTTP spec encourages servers to setup the charset in the content type header
for this purpose, see:
http://www.w3.org/International/O-HTTP-charset

GeoServer wise, we have a target charset in our SettingsInfo object, and the response
are diligently using it when wrapping the output stream provided by Response.write
into a Writer… however pretty much nothing besides the exception handlers
remembers to set it in the HTTP headers.
The XML responses are sort of an exception/gray area, in that while they are not
setting the HTTP header, they are declaring the charset in the XML header.

HTTP headers wise we have two options, we can either extend the mime type
we are returning (as suggested by the above link) and say something like
“text/csv; charset=UTF-8” instead of just “text/csv”, or be programmatic
like the exception handlers, and call ServletResponse.setCharacterEncoding
http://docs.oracle.com/javaee/6/api/javax/servlet/ServletResponse.html#setCharacterEncoding(java.lang.String)

Personally I would prefer the latter. Now… how to make Response commmunicate
the charset it’s using? It’s likely going to be the one we have in the Settings, but
in some cases it might not (custom responses, or standards mandating the usage
of a particular charset for a given format) and in other cases the response is just
binary, so we should not set it.

My suggestion would be to extend org.geoserver.ows.Response and add the following method:

/**

  • Returns the charset for this response, the Dispatcher will set it in the ServletResponse.
  • The default implementation returns null, in this case no encoding will be set.
  • Subclasses returning text documents should override taking into account SettingsInfo.getCharset()
  • as well as the specific encoding requirements of the returned format.
    */

public String getCharset()
return null;
}

Opinions?

Cheers
Andrea

–

==

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

==

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

Slashdot TV. Videos for Nerds. Stuff that Matters.
http://pubads.g.doubleclick.net/gampad/clk?id=160591471&iu=/4140/ostg.clktrk

Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

–

Justin Deoliveira
VP Engineering | Boundless
jdeolive@anonymised.com
@boundlessgeo

3

My suggestion would be to extend org.geoserver.ows.Response and add the
following method:

/**
* Returns the charset for this response, the Dispatcher will set it in
the ServletResponse.
* The default implementation returns <code>null</code>, in this case no
encoding will be set.
* Subclasses returning text documents should override taking into account
SettingsInfo.getCharset()
* as well as the specific encoding requirements of the returned format.
*/
public String getCharset()
   return null;
}

Opinions?

Interesting problem. Is it worth returning a Java Charset there - and
providing a utility class to handle mapping from Java Charset to
ServletResponse, mime type suffix (or XML header snippet) away from
response implementor?

class CharsetUtil {
   /** used by Dispatcher to match Charset to Servlet response */
   String toServletResponse( Charset charset);
   String toMimeType( String mimeType, Charset charset);
   String toXMLEncoding( Charset charset);
}
--
Jody

4

On Mon, Oct 6, 2014 at 5:46 PM, Jody Garnett <jody.garnett@anonymised.com> wrote:

My suggestion would be to extend org.geoserver.ows.Response and add the

following method:

/**
* Returns the charset for this response, the Dispatcher will set it in
the ServletResponse.
* The default implementation returns <code>null</code>, in this case no
encoding will be set.
* Subclasses returning text documents should override taking into
account SettingsInfo.getCharset()
* as well as the specific encoding requirements of the returned format.
*/
public String getCharset()
   return null;
}

Opinions?

Interesting problem. Is it worth returning a Java Charset there - and
providing a utility class to handle mapping from Java Charset to
ServletResponse, mime type suffix (or XML header snippet) away from
response implementor?

class CharsetUtil {
   /** used by Dispatcher to match Charset to Servlet response */
   String toServletResponse( Charset charset);
   String toMimeType( String mimeType, Charset charset);
   String toXMLEncoding( Charset charset);
}

I would not go there... for the simple reason that it would break
uniformity (and to restore it we'd
have to make much more extensive changes), right now we are trading
them as String, not Charset.
I also don't see a reason for having the three above
methods, the charset name is the same for all (they are standardized, not a
java thing, as far as I know).

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------

5

A String is fine for consistency. I think there are some restrictions around using the String to lookup a Charset for encoding.

Jody

···

Jody Garnett

On Mon, Oct 6, 2014 at 8:55 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Mon, Oct 6, 2014 at 5:46 PM, Jody Garnett <jody.garnett@anonymised.com> wrote:

I would not go there… for the simple reason that it would break uniformity (and to restore it we’d
have to make much more extensive changes), right now we are trading
them as String, not Charset.
I also don’t see a reason for having the three above
methods, the charset name is the same for all (they are standardized, not a java thing, as far as I know).

Cheers

Andrea

–

==

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

==

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

My suggestion would be to extend org.geoserver.ows.Response and add the following method:

/**

  • Returns the charset for this response, the Dispatcher will set it in the ServletResponse.
  • The default implementation returns null, in this case no encoding will be set.
  • Subclasses returning text documents should override taking into account SettingsInfo.getCharset()
  • as well as the specific encoding requirements of the returned format.
    */

public String getCharset()
return null;
}

Opinions?

Interesting problem. Is it worth returning a Java Charset there - and providing a utility class to handle mapping from Java Charset to ServletResponse, mime type suffix (or XML header snippet) away from response implementor?

class CharsetUtil {
/** used by Dispatcher to match Charset to Servlet response */
String toServletResponse( Charset charset);
String toMimeType( String mimeType, Charset charset);
String toXMLEncoding( Charset charset);
}

6

On Mon, Oct 6, 2014 at 6:28 PM, Jody Garnett <jody.garnett@anonymised.com> wrote:

A String is fine for consistency. I think there are some restrictions
around using the String to lookup a Charset for encoding.

Nothing comes to mind... could you make an example?

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------

7

The javadocs provide a grammar - http://docs.oracle.com/javase/7/docs/api/java/nio/charset/Charset.html

···

Jody Garnett

On Mon, Oct 6, 2014 at 9:30 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Mon, Oct 6, 2014 at 6:28 PM, Jody Garnett <jody.garnett@anonymised.com> wrote:

A String is fine for consistency. I think there are some restrictions around using the String to lookup a Charset for encoding.

Nothing comes to mind… could you make an example?

Cheers

Andrea

–

==

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

==

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

8

On Mon, Oct 6, 2014 at 6:35 PM, Jody Garnett <jody.garnett@anonymised.com> wrote:

The javadocs provide a grammar -
http://docs.oracle.com/javase/7/docs/api/java/nio/charset/Charset.html

Ah, I see. HTTP/XML is somewhat more restrictive than a grammar, you have
to respect a whitelist in order
for the clients to understand the response (and generally speaking, I'd
guess that in order to get
good compliance, one should not get too fancy with this one either):

http://www.iana.org/assignments/character-sets/character-sets.xhtml

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------

9

And oh, now I’ve realized at least the reason for your first proposal… you want an enumeration/codelist
Which is fine in theory, but there are two issues in practice:

  • It only work as long as the Java version is up to date, and has all the charsets you need (the grammar is one thing, but if you pass a valid but unrecognized charset name, it will throw an exception)
  • The usage of Charset, assuming it works, would be solving technical debt (we have been using a string for at least 8 years), which I don’t believe is a requirement when adding new features or fixing bugs… next code sprint maybe?

Cheers
Andrea

···

On Mon, Oct 6, 2014 at 6:50 PM, Andrea Aime <andrea.aime@anonymised.com> wrote:

–

==

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

==

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

On Mon, Oct 6, 2014 at 6:35 PM, Jody Garnett <jody.garnett@anonymised.com> wrote:

The javadocs provide a grammar - http://docs.oracle.com/javase/7/docs/api/java/nio/charset/Charset.html

Ah, I see. HTTP/XML is somewhat more restrictive than a grammar, you have to respect a whitelist in order
for the clients to understand the response (and generally speaking, I’d guess that in order to get
good compliance, one should not get too fancy with this one either):

http://www.iana.org/assignments/character-sets/character-sets.xhtml

Cheers

Andrea

–

==

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

==

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

10

I think we have a few spots where we work with a String, and use Charset to ensure the String is valid.

How about a compromise - use a String and issue a warning if the String is not a valid Charset?

Jody

···

Jody Garnett

On Mon, Oct 6, 2014 at 10:05 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

And oh, now I’ve realized at least the reason for your first proposal… you want an enumeration/codelist
Which is fine in theory, but there are two issues in practice:

  • It only work as long as the Java version is up to date, and has all the charsets you need (the grammar is one thing, but if you pass a valid but unrecognized charset name, it will throw an exception)
  • The usage of Charset, assuming it works, would be solving technical debt (we have been using a string for at least 8 years), which I don’t believe is a requirement when adding new features or fixing bugs… next code sprint maybe?

Cheers

Andrea

On Mon, Oct 6, 2014 at 6:50 PM, Andrea Aime <andrea.aime@anonymised.com> wrote:

–

==

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

==

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

On Mon, Oct 6, 2014 at 6:35 PM, Jody Garnett <jody.garnett@anonymised.com> wrote:

The javadocs provide a grammar - http://docs.oracle.com/javase/7/docs/api/java/nio/charset/Charset.html

Ah, I see. HTTP/XML is somewhat more restrictive than a grammar, you have to respect a whitelist in order
for the clients to understand the response (and generally speaking, I’d guess that in order to get
good compliance, one should not get too fancy with this one either):

http://www.iana.org/assignments/character-sets/character-sets.xhtml

Cheers

Andrea

–

==

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

==

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

11

On Mon, Oct 6, 2014 at 7:33 PM, Jody Garnett <jody.garnett@anonymised.com> wrote:

I think we have a few spots where we work with a String, and use Charset
to ensure the String is valid.

How about a compromise - use a String and issue a warning if the String is
not a valid Charset?

Works for me... so considering only the code path of this new extension,
the Dispatcher
would make a check for the charset name coming from the response before
encoding it?
Hopefully Charset has good caching, and besides, we are already looking it
up when
wrapping the servlet output stream into a Writer

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------