[GeoNetwork-users] Character encoding of WMS Capabilities XML

Hi,

first of all, thanks a lot for all the interesting discussions on this
list, to me they are very helpful!

But I have still an issue with character encoding of WMS capabilities
XML. I can't find any way how to resolve it.

In Geonetwork, when Geonetwork gets the capabilities XML of a WMS
(adding a WMS layer to the map or adding a WMS online resource to
metadata), special spanish characters are not shown correctly.

In other words, characters like ó are not shown when requesting the
capabilities XML like this:
http://mygeoportal.org/geonetwork/proxy?url=http%3A%2F%2Fmygeoportal.com%3A80%2Fgeoserver%2Fservices%2Fows%3FSERVICE%3DWMS%26REQUEST%3DGetCapabilities,
firebug informs for response header: Content-Type
text/xml;charset=ISO-8859-1

But the characters in the XML-File are shown correctly in the browser
requesting the capabilities XML using an URL like:
http://mygeoportal.org/geoserver/services/ows?SERVICE=WMS&REQUEST=GetCapabilities
- in this case firebug informs for response header: Content-Type text/xml

I'm running Geonetwork 3.0.2 and Geoserver 2.8 as Tomcat8 servlets with
OpenJDK7 under Debian, using Apache HTTPD server with AJP/1.3 connectors
(workers).

Some details about server configuration:
Debian system locale is "es_EC.UTF-8".
Geonetwork postgresql db is UTF-8.
In Apache httpd server config the line AddDefaultCharset utf-8 is
discommented.
Tomcat8: in /etc/tomcat8/web.xml the build in filter
"setCharacterEncodingFilter" is set to UTF-8 per default, also discommented.
Java System properties in /etc/default/tomcat8 set to
JAVA_OPTS="-Djava.awt.headless=true -Dfile.encoding=UTF-8" (i also tried
UTF8),
and in /etc/tomcat8/server.xml both parameters URIEncoding="UTF-8"
useBodyEncodingForURI="true" are added to connectors tag.
The Geoserver character encoding is set to UTF-8 in web interface.

Any idea what is missing or wrong with this configuration?
Why is the server-response header of the capabilities XML charset
ISO-8859-1 and not UTF-8 or nothing when using
http:/…/geonetwork/proxy?url=…

Thanks a lot in advance for any help!

All the best,
Jan

Hi,

Could there be an issue about your map server ?
Check that your getCapabilities request has

   1. Content-Type:
   text/xml;charset=UTF_8

If no info is provided, default is Latin1.

Genetwork proxy does the encoding stuff there
https://github.com/geonetwork/core-geonetwork/blob/develop/core/src/main/java/org/geonetwork/http/proxy/HttpProxyServlet.java#L249

It's not about java or tomcat config, just the proxy thing.

On Thu, Oct 15, 2015 at 12:31 AM, Jan Schorn <js@anonymised.com> wrote:

Hi,

first of all, thanks a lot for all the interesting discussions on this
list, to me they are very helpful!

But I have still an issue with character encoding of WMS capabilities
XML. I can't find any way how to resolve it.

In Geonetwork, when Geonetwork gets the capabilities XML of a WMS
(adding a WMS layer to the map or adding a WMS online resource to
metadata), special spanish characters are not shown correctly.

In other words, characters like ó are not shown when requesting the
capabilities XML like this:

http://mygeoportal.org/geonetwork/proxy?url=http%3A%2F%2Fmygeoportal.com%3A80%2Fgeoserver%2Fservices%2Fows%3FSERVICE%3DWMS%26REQUEST%3DGetCapabilities
,
firebug informs for response header: Content-Type
text/xml;charset=ISO-8859-1

But the characters in the XML-File are shown correctly in the browser
requesting the capabilities XML using an URL like:

http://mygeoportal.org/geoserver/services/ows?SERVICE=WMS&REQUEST=GetCapabilities
- in this case firebug informs for response header: Content-Type text/xml

I'm running Geonetwork 3.0.2 and Geoserver 2.8 as Tomcat8 servlets with
OpenJDK7 under Debian, using Apache HTTPD server with AJP/1.3 connectors
(workers).

Some details about server configuration:
Debian system locale is "es_EC.UTF-8".
Geonetwork postgresql db is UTF-8.
In Apache httpd server config the line AddDefaultCharset utf-8 is
discommented.
Tomcat8: in /etc/tomcat8/web.xml the build in filter
"setCharacterEncodingFilter" is set to UTF-8 per default, also
discommented.
Java System properties in /etc/default/tomcat8 set to
JAVA_OPTS="-Djava.awt.headless=true -Dfile.encoding=UTF-8" (i also tried
UTF8),
and in /etc/tomcat8/server.xml both parameters URIEncoding="UTF-8"
useBodyEncodingForURI="true" are added to connectors tag.
The Geoserver character encoding is set to UTF-8 in web interface.

Any idea what is missing or wrong with this configuration?
Why is the server-response header of the capabilities XML charset
ISO-8859-1 and not UTF-8 or nothing when using
http:/…/geonetwork/proxy?url=…

Thanks a lot in advance for any help!

All the best,
Jan

------------------------------------------------------------------------------
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net
geonetwork-users List Signup and Options
GeoNetwork OpenSource is maintained at
GeoNetwork - Geographic Metadata Catalog download | SourceForge.net

--
*camptocamp*
INNOVATIVE SOLUTIONS
BY OPEN SOURCE EXPERTS

*Florent Gravin*
0479444492

I think you just need to change the output encoding directly into your
geoserver.
http://docs.geoserver.org/latest/en/user/webadmin/server/globalsettings.html

On Thu, Oct 15, 2015 at 9:18 AM, Florent Gravin <
florent.gravin@anonymised.com> wrote:

Hi,

Could there be an issue about your map server ?
Check that your getCapabilities request has

   1. Content-Type:
   text/xml;charset=UTF_8

If no info is provided, default is Latin1.

Genetwork proxy does the encoding stuff there

https://github.com/geonetwork/core-geonetwork/blob/develop/core/src/main/java/org/geonetwork/http/proxy/HttpProxyServlet.java#L249

It's not about java or tomcat config, just the proxy thing.

On Thu, Oct 15, 2015 at 12:31 AM, Jan Schorn <js@anonymised.com> wrote:

Hi,

first of all, thanks a lot for all the interesting discussions on this
list, to me they are very helpful!

But I have still an issue with character encoding of WMS capabilities
XML. I can't find any way how to resolve it.

In Geonetwork, when Geonetwork gets the capabilities XML of a WMS
(adding a WMS layer to the map or adding a WMS online resource to
metadata), special spanish characters are not shown correctly.

In other words, characters like ó are not shown when requesting the
capabilities XML like this:

http://mygeoportal.org/geonetwork/proxy?url=http%3A%2F%2Fmygeoportal.com%3A80%2Fgeoserver%2Fservices%2Fows%3FSERVICE%3DWMS%26REQUEST%3DGetCapabilities
,
firebug informs for response header: Content-Type
text/xml;charset=ISO-8859-1

But the characters in the XML-File are shown correctly in the browser
requesting the capabilities XML using an URL like:

http://mygeoportal.org/geoserver/services/ows?SERVICE=WMS&REQUEST=GetCapabilities
- in this case firebug informs for response header: Content-Type text/xml

I'm running Geonetwork 3.0.2 and Geoserver 2.8 as Tomcat8 servlets with
OpenJDK7 under Debian, using Apache HTTPD server with AJP/1.3 connectors
(workers).

Some details about server configuration:
Debian system locale is "es_EC.UTF-8".
Geonetwork postgresql db is UTF-8.
In Apache httpd server config the line AddDefaultCharset utf-8 is
discommented.
Tomcat8: in /etc/tomcat8/web.xml the build in filter
"setCharacterEncodingFilter" is set to UTF-8 per default, also
discommented.
Java System properties in /etc/default/tomcat8 set to
JAVA_OPTS="-Djava.awt.headless=true -Dfile.encoding=UTF-8" (i also tried
UTF8),
and in /etc/tomcat8/server.xml both parameters URIEncoding="UTF-8"
useBodyEncodingForURI="true" are added to connectors tag.
The Geoserver character encoding is set to UTF-8 in web interface.

Any idea what is missing or wrong with this configuration?
Why is the server-response header of the capabilities XML charset
ISO-8859-1 and not UTF-8 or nothing when using
http:/…/geonetwork/proxy?url=…

Thanks a lot in advance for any help!

All the best,
Jan

------------------------------------------------------------------------------
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net
geonetwork-users List Signup and Options
GeoNetwork OpenSource is maintained at
GeoNetwork - Geographic Metadata Catalog download | SourceForge.net

--
*camptocamp*
INNOVATIVE SOLUTIONS
BY OPEN SOURCE EXPERTS

*Florent Gravin*
0479444492

--
*camptocamp*
INNOVATIVE SOLUTIONS
BY OPEN SOURCE EXPERTS

*Florent Gravin*
0479444492

Hi Florent

Thanks a lot for the hints!

Finally I found a fix for my situation, but i think it's quite a dirty
one...

The geoserver character encoding was already set to UTF-8 in global
options. The spanish special characters in the WMS capabilities XML are
shown fine in the browser or when connecting with QGIS. But not in
Geonetwork.

Then I tried to set geoserver character encoding to latin1 like Hoddih
did, but didn't worked for me.

Firebug states that the content-type in geoserver response header is
text/xml - but the charset=utf-8 is missing (there is no charset
information).
The first line of recieved capabilities XML starts with <?xml
version="1.0" encoding="UTF-8">.

So I changed the line 249 in the proxy script you mencioned to
String responseBody =
IOUtils.toString(httpResponse.getEntity().getContent(), "UTF-8").trim();

This works, the special characters in WMS capabilities XML from
geoserver are shown fine in geonetwork now.

But I think proxy-script only works in utf-8 now, so i would appreciate
any help to do this in a better way.

El 15/10/15 a las 02:21, Florent Gravin escribió:

I think you just need to change the output encoding directly into your
geoserver.
http://docs.geoserver.org/latest/en/user/webadmin/server/globalsettings.html

On Thu, Oct 15, 2015 at 9:18 AM, Florent Gravin
<florent.gravin@anonymised.com <mailto:florent.gravin@anonymised.com>>
wrote:

    Hi,

    Could there be an issue about your map server ?
    Check that your getCapabilities request has

     1.
        Content-Type:
        text/xml;charset=UTF_8

    If no info is provided, default is Latin1.

    Genetwork proxy does the encoding stuff there
    https://github.com/geonetwork/core-geonetwork/blob/develop/core/src/main/java/org/geonetwork/http/proxy/HttpProxyServlet.java#L249

    It's not about java or tomcat config, just the proxy thing.

    On Thu, Oct 15, 2015 at 12:31 AM, Jan Schorn <js@anonymised.com
    <mailto:js@anonymised.com>> wrote:

        Hi,

        first of all, thanks a lot for all the interesting discussions
        on this
        list, to me they are very helpful!

        But I have still an issue with character encoding of WMS
        capabilities
        XML. I can't find any way how to resolve it.

        In Geonetwork, when Geonetwork gets the capabilities XML of a WMS
        (adding a WMS layer to the map or adding a WMS online resource to
        metadata), special spanish characters are not shown correctly.

        In other words, characters like ó are not shown when requesting the
        capabilities XML like this:
        http://mygeoportal.org/geonetwork/proxy?url=http%3A%2F%2Fmygeoportal.com%3A80%2Fgeoserver%2Fservices%2Fows%3FSERVICE%3DWMS%26REQUEST%3DGetCapabilities,
        firebug informs for response header: Content-Type
        text/xml;charset=ISO-8859-1

        But the characters in the XML-File are shown correctly in the
        browser
        requesting the capabilities XML using an URL like:
        http://mygeoportal.org/geoserver/services/ows?SERVICE=WMS&REQUEST=GetCapabilities
        - in this case firebug informs for response header: Content-Type
        text/xml

        I'm running Geonetwork 3.0.2 and Geoserver 2.8 as Tomcat8
        servlets with
        OpenJDK7 under Debian, using Apache HTTPD server with AJP/1.3
        connectors
        (workers).

        Some details about server configuration:
        Debian system locale is "es_EC.UTF-8".
        Geonetwork postgresql db is UTF-8.
        In Apache httpd server config the line AddDefaultCharset utf-8 is
        discommented.
        Tomcat8: in /etc/tomcat8/web.xml the build in filter
        "setCharacterEncodingFilter" is set to UTF-8 per default, also
        discommented.
        Java System properties in /etc/default/tomcat8 set to
        JAVA_OPTS="-Djava.awt.headless=true -Dfile.encoding=UTF-8" (i
        also tried
        UTF8),
        and in /etc/tomcat8/server.xml both parameters URIEncoding="UTF-8"
        useBodyEncodingForURI="true" are added to connectors tag.
        The Geoserver character encoding is set to UTF-8 in web interface.

        Any idea what is missing or wrong with this configuration?
        Why is the server-response header of the capabilities XML charset
        ISO-8859-1 and not UTF-8 or nothing when using
        http:/…/geonetwork/proxy?url=…

        Thanks a lot in advance for any help!

        All the best,
        Jan

        ------------------------------------------------------------------------------
        _______________________________________________
        GeoNetwork-users mailing list
        GeoNetwork-users@lists.sourceforge.net
        <mailto:GeoNetwork-users@lists.sourceforge.net>
        https://lists.sourceforge.net/lists/listinfo/geonetwork-users
        GeoNetwork OpenSource is maintained at
        http://sourceforge.net/projects/geonetwork

    --
    *camptocamp*
    INNOVATIVE SOLUTIONS
    BY OPEN SOURCE EXPERTS

    *Florent Gravin*
    0479444492

--
*camptocamp*
INNOVATIVE SOLUTIONS
BY OPEN SOURCE EXPERTS

*Florent Gravin*
0479444492

Well,

If geoserver send content in UTF8 and do not say it in Content-Type header,
then i guess this is an issue from geoserver, as the default, if not
specified, should be latin1.
You also need to consider the encoding of your data sources (shp or
postigs).

On Thu, Oct 15, 2015 at 7:01 PM, Jan Schorn <js@anonymised.com> wrote:

Hi Florent

Thanks a lot for the hints!

Finally I found a fix for my situation, but i think it's quite a dirty
one...

The geoserver character encoding was already set to UTF-8 in global
options. The spanish special characters in the WMS capabilities XML are
shown fine in the browser or when connecting with QGIS. But not in
Geonetwork.

Then I tried to set geoserver character encoding to latin1 like Hoddih
did, but didn't worked for me.

Firebug states that the content-type in geoserver response header is
text/xml - but the charset=utf-8 is missing (there is no charset
information).
The first line of recieved capabilities XML starts with <?xml
version="1.0" encoding="UTF-8">.

So I changed the line 249 in the proxy script you mencioned to
String responseBody =
IOUtils.toString(httpResponse.getEntity().getContent(), "UTF-8").trim();

This works, the special characters in WMS capabilities XML from
geoserver are shown fine in geonetwork now.

But I think proxy-script only works in utf-8 now, so i would appreciate
any help to do this in a better way.

El 15/10/15 a las 02:21, Florent Gravin escribió:
> I think you just need to change the output encoding directly into your
> geoserver.
>
http://docs.geoserver.org/latest/en/user/webadmin/server/globalsettings.html
>
> On Thu, Oct 15, 2015 at 9:18 AM, Florent Gravin
> <florent.gravin@anonymised.com <mailto:florent.gravin@anonymised.com>>
> wrote:
>
> Hi,
>
> Could there be an issue about your map server ?
> Check that your getCapabilities request has
>
> 1.
> Content-Type:
> text/xml;charset=UTF_8
>
> If no info is provided, default is Latin1.
>
> Genetwork proxy does the encoding stuff there
>
https://github.com/geonetwork/core-geonetwork/blob/develop/core/src/main/java/org/geonetwork/http/proxy/HttpProxyServlet.java#L249
>
> It's not about java or tomcat config, just the proxy thing.
>
> On Thu, Oct 15, 2015 at 12:31 AM, Jan Schorn <js@anonymised.com
> <mailto:js@anonymised.com>> wrote:
>
> Hi,
>
> first of all, thanks a lot for all the interesting discussions
> on this
> list, to me they are very helpful!
>
> But I have still an issue with character encoding of WMS
> capabilities
> XML. I can't find any way how to resolve it.
>
> In Geonetwork, when Geonetwork gets the capabilities XML of a WMS
> (adding a WMS layer to the map or adding a WMS online resource to
> metadata), special spanish characters are not shown correctly.
>
> In other words, characters like ó are not shown when requesting
the
> capabilities XML like this:
>
http://mygeoportal.org/geonetwork/proxy?url=http%3A%2F%2Fmygeoportal.com%3A80%2Fgeoserver%2Fservices%2Fows%3FSERVICE%3DWMS%26REQUEST%3DGetCapabilities
,
> firebug informs for response header: Content-Type
> text/xml;charset=ISO-8859-1
>
> But the characters in the XML-File are shown correctly in the
> browser
> requesting the capabilities XML using an URL like:
>
http://mygeoportal.org/geoserver/services/ows?SERVICE=WMS&REQUEST=GetCapabilities
> - in this case firebug informs for response header: Content-Type
> text/xml
>
> I'm running Geonetwork 3.0.2 and Geoserver 2.8 as Tomcat8
> servlets with
> OpenJDK7 under Debian, using Apache HTTPD server with AJP/1.3
> connectors
> (workers).
>
> Some details about server configuration:
> Debian system locale is "es_EC.UTF-8".
> Geonetwork postgresql db is UTF-8.
> In Apache httpd server config the line AddDefaultCharset utf-8 is
> discommented.
> Tomcat8: in /etc/tomcat8/web.xml the build in filter
> "setCharacterEncodingFilter" is set to UTF-8 per default, also
> discommented.
> Java System properties in /etc/default/tomcat8 set to
> JAVA_OPTS="-Djava.awt.headless=true -Dfile.encoding=UTF-8" (i
> also tried
> UTF8),
> and in /etc/tomcat8/server.xml both parameters
URIEncoding="UTF-8"
> useBodyEncodingForURI="true" are added to connectors tag.
> The Geoserver character encoding is set to UTF-8 in web
interface.
>
> Any idea what is missing or wrong with this configuration?
> Why is the server-response header of the capabilities XML charset
> ISO-8859-1 and not UTF-8 or nothing when using
> http:/…/geonetwork/proxy?url=…
>
>
> Thanks a lot in advance for any help!
>
> All the best,
> Jan
>
>
------------------------------------------------------------------------------
> _______________________________________________
> GeoNetwork-users mailing list
> GeoNetwork-users@lists.sourceforge.net
> <mailto:GeoNetwork-users@lists.sourceforge.net>
> geonetwork-users List Signup and Options
> GeoNetwork OpenSource is maintained at
> GeoNetwork - Geographic Metadata Catalog download | SourceForge.net
>
>
>
>
> --
> *camptocamp*
> INNOVATIVE SOLUTIONS
> BY OPEN SOURCE EXPERTS
>
> *Florent Gravin*
> 0479444492
>
>
>
>
> --
> *camptocamp*
> INNOVATIVE SOLUTIONS
> BY OPEN SOURCE EXPERTS
>
> *Florent Gravin*
> 0479444492

--
*camptocamp*
INNOVATIVE SOLUTIONS
BY OPEN SOURCE EXPERTS

*Florent Gravin*
0479444492