Thanks a world Ben
Your help worth a million for me.
The 2nd link did the trick. Adding URIEncoding=“UTF-8” or useBodyEncodingForURI =“true” to server.xml was the right solution for Tomcat 7.
Also, upgrading to Tomcat 8 solves everything without a need to add above codes. According to 2nd link, “In Tomcat 8 starting with 8.0.0 (8.0.0-RC3, to be specific), default value (strict compliance is off) of URIEncoding is now UTF-8.”
Thanks again
From: Ben Caradoc-Davies <ben@anonymised.com…>
To: Saka Royban sakaroyban@anonymised.com; GeoServer Mailing List List geoserver-users@lists.sourceforge.net
Sent: Saturday, August 6, 2016 9:48 AM
Subject: Re: [Geoserver-users] GeoServer doesn’t understand non-english Characters in cql filter
GeoServer expects URL-encoded parameters as UTF-8 regardless of the host
operating system character set. This is correct.
Some languages can encode characters in different ways in Unicode, for
example using Unicode combining characters. This can affect comparisons.
Could your data have different bytes to your filter literal? What is the
exact byte sequence stored in your shapefile for the feature with the
property with value ‘سامی’? That is, does it have the same UTF-8 byte
sequence? Do you see the same bytes in a WFS response?
What is your servlet container? Tomcat or the embedded Jetty? There have
been reports of CQL encoding issues with some Tomcat configurations.
Quite old but may still be valid:
http://osgeo-org.1560.x6.nabble.com/CQL-Filter-Syntax-td3806188.html
https://sourceforge.net/p/geoserver/mailman/message/21111455/
One reported fix for Tomcat is to set URIEncoding=“UTF-8” on your
in server.xml, but I have not tried this myself:
https://wiki.apache.org/tomcat/FAQ/CharacterEncoding
Kind regards,
Ben.
On 06/08/16 15:26, Saka Royban wrote:
Thanks a lot Ben for your helpSo, if CQL-filter can handle non-ASCII characters, it is misunderstanding of me. I’ve may done a step wrongly.
- Yes. My shapefile store is declared as UTF-8 charset in GeoServer. Plain wfs requests returns correct data (=correct non-english characters). In fact, there is no problem without CQL-filter. To add more, my shapefile is an output of ArcGIS which in its recent versions only exports to UTF-8. (There is .cpg file which has been set to UTF-8, after export.)
- My system encoding is Windows-1256 (the encoding that is used by default in word editors). My webpage is declared in UTF-8. So, my javascript codes must be in UTF-8 (I’ve also set encoding in notepad++ to UTF-8. If i change that to Win-1256, i see some dummy characters which proves me that my filter string is encoded in UTF-8 correctly)
- OpenLayers sends following string as part of URL to server:CQL_FILTER=NAME%20%3D%20%27%D8%B3%D8%A7%D9%85%DB%8C%27Using online decoders, i get back correct filter string.
- About log file, i tried different encoding in Notepad++ but unfortunately i can’t see the filter string correctly. My encodings are restricted to two: UTF-8 and Win-1256. Filter string isn’t displayed correctly in none of them.
I’ve also tried another shapefile in Win-1256 encoding. Changing meta tag of webpage to chareset=windows-1256, writing againg filter string but in this case, i can’t get a good result as well.(Note: I noticed that Openlayers encodes filter in win-1256 exactly as it was in UTF-8. Is it right?)
Thanks again for your help and patience
From: Ben Caradoc-Davies <ben@anonymised.com>
To: Saka Royban <sakaroyban@anonymised.com>; GeoServer Mailing List List <geoserver-users@lists.sourceforge.net>
Sent: Saturday, August 6, 2016 1:29 AM
Subject: Re: [Geoserver-users] GeoServer doesn’t understand non-english Characters in cql filter
Yes, non-ASCII characters are supported in CQL and XML filters.
Successful filtering also requires correct configuration of your data
source.
What charset is declared in your GeoServer shapefile data store
configuration? Is it UTF-8?
http://docs.geoserver.org/latest/en/user/data/vector/shapefile.html
What is your platform encoding (the charset of your operating system)?
Log files are written in the encoding of your platform, so if your
platform cannot encode these characters, placeholders characters will be
written, even when filtering is working correctly.
Kind regards,
Ben.
On 06/08/16 05:09, Saka Royban wrote:
Hi all have a shapefile loaded in GeoServer 2.8 and accessed by OpenLayers 2.13. This layer is in UTF-8 encoding and has some english and non-english records.
When i use english fields in cql-filter - as follows, but no differnce between WMS and WFS request- GeoServer filters data very well and returns the result. layer1 = new ol.layer.Image({
title: ‘Ostan’,
source: new ol.source.ImageWMS({
url: ‘http://localhost:8080/geoserver/wms’,
params: {
LAYERS: ‘ws:layer1’,
VERSION: ‘1.1.1’,
CQL_FILTER: “NAME_ENG = ‘sommy’”
}
})
});But if i try to use a field which contains non-english data in cql-filter, GeoServer returns no data. My webpage has following meta, as well: When i check geoserver log, i see some messy characters instead of my non-english string in cql-filter.
|
| |
| Using Firebug, in Parameteres section, everything is correct. I see "CQL-Filter: Name = ‘سامی’ " which is right for my language. But in Geoserver log, i see some dummy characters instead of abovementioned characters
Can anyone confirm that using characters other than ASCII is possible in CQL-Filter?
Regards |
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users
–
Ben Caradoc-Davies <ben@anonymised.com>
Director
Transient Software Limited <http://transient.nz/>
New Zealand