panos wrote:
Hi Chris….
Hey, I'm ccing the devel list and Artie, who has done the bulk of the geoserver encoding issues, I imagine he's come in to some of these before. http://jira.codehaus.org/browse/GEOS-323 and http://jira.codehaus.org/browse/GEOS-258 have the research he did. Perhaps we didn't properly apply GEOS-323
Using this code:
public class Test {
public static void main(String args) {
System.out.println(sun.io.Converters.getDefaultEncodingName());
System.out.println(System.getProperty("file.encoding"));
}
}
I found out that my default encoding is Cp1253. I am using Java 1.4.2_09.
So my problems with the encoding in Geoserver could mainly exist due to the
use of Streams in different places in Geoserver.In details the problems are the following:
1.
The logger is not showing some of the characters correctly. Maybe (just an
idea) this could be resolved with the use of a handler in the initLogging in
Geoserver.java.e.g. handler.setEncoding(some encoding);
I realize that this is not really an issue. It's not so important to
internationalize the log file. However I could use it to find out easily
where the messing with the characters occurs.
This I think we can do easily enough. Is it ok if we just use the same charset that you set in Config -> Server? That makes things easy for us. I can play with it and try to send you a patched version.
2.
The post in Geoserver is not working with Greek parameters. This could be
happening due to the use of the wfsdispatchNUMBERtmp file in the post
process.The wfsdispatchNUMBERtmp is not creating the right post. For example this is
a sample of a wfsdispatchNUMBERtmp:<wfs:GetFeature service="WFS" version="1.0.0"
outputFormat="GML2"
xmlns:gr="http://www.gr.com"
xmlns:wfs="http://www.opengis.net/wfs"
xmlns:ogc="http://www.opengis.net/ogc"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.opengis.net/wfs
http://schemas.opengis.net/wfs/1.0.0/WFS-basic.xsd">
<wfs:Query typeName="gr:nom">
<ogc:Filter>
<PropertyIsEqualTo>
<PropertyName>NOM_DES</PropertyName>
*<Literal>???</Literal>*
</PropertyIsEqualTo>
</ogc:Filter>
</wfs:Query>
</wfs:GetFeature>
So Geoserver cannot read the correct <Litera>l and the query returns empty
results.This maybe has to do with the way the WfsDispatcher.java is creating the tmp
file in doPost (BufferedOutputStream). The " //JD: GEOS-323, Adding char
encoding support" it doesn't seem to do anything….
This one is easy enough to check to see if it's the problem. Instead of submitting your request to http://localhost:8080/geoserver/wfs
submit it directly to http://localhost:8080/geoserver/wfs/GetFeature
This will bypass that little dispatcher hack, just holding it all in memory. If things work that way let us know, and then we obviously need to fix the dispatcher.
And just to confirm, you are using a GeoServer 1.3, at least later than RC5? Ideally GeoServer 1.3.0. 1.2 definitely does not have these improvements.
3.
The results in XML are not encoding the parameters correctly. For example:
<?xml version="1.0" encoding="windows-1253" ?>
-<http://127.0.0.1:8080/geoserver/wfs?Request=GetFeature&typename=gr:nom&Filter=<Filter><PropertyIsEqualTo><PropertyName>gr:nom/NOM_DES</PropertyName><Literal>ΚΙΛΚΗΣ</Literal></PropertyIsEqualTo></>
<wfs:FeatureCollection xmlns:wfs="http://www.opengis.net/wfs" xmlns:gr="
http://www.gr.com" xmlns:gml="http://www.opengis.net/gml" xmlns:xsi="
http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="
http://www.gr.com
http://127.0.0.1:8080/geoserver/wfs/DescribeFeatureType?typeName=gr:nom
http://www.opengis.net/wfs
http://127.0.0.1:8080/geoserver/schemas/wfs/1.0.0/WFS-basic.xsd">-<http://127.0.0.1:8080/geoserver/wfs?Request=GetFeature&typename=gr:nom&Filter=<Filter><PropertyIsEqualTo><PropertyName>gr:nom/NOM_DES</PropertyName><Literal>ΚΙΛΚΗΣ</Literal></PropertyIsEqualTo></>
<gml:Box srsName="http://www.opengis.net/gml/srs/epsg.xml#2100"><gml:coordinates decimal="." cs="," ts="">352695.125,4515963 437060.09375
,4577358</gml:coordinates></gml:Box>
</gml:boundedBy>
-<http://127.0.0.1:8080/geoserver/wfs?Request=GetFeature&typename=gr:nom&Filter=<Filter><PropertyIsEqualTo><PropertyName>gr:nom/NOM_DES</PropertyName><Literal>ΚΙΛΚΗΣ</Literal></PropertyIsEqualTo></>
<gml:featureMember>-<http://127.0.0.1:8080/geoserver/wfs?Request=GetFeature&typename=gr:nom&Filter=<Filter><PropertyIsEqualTo><PropertyName>gr:nom/NOM_DES</PropertyName><Literal>ΚΙΛΚΗΣ</Literal></PropertyIsEqualTo></>
<gr:nom fid="nom.6">-<http://127.0.0.1:8080/geoserver/wfs?Request=GetFeature&typename=gr:nom&Filter=<Filter><PropertyIsEqualTo><PropertyName>gr:nom/NOM_DES</PropertyName><Literal>ΚΙΛΚΗΣ</Literal></PropertyIsEqualTo></>
<gml:MultiPolygon srsName="http://www.opengis.net/gml/srs/epsg.xml#2100">+<http://127.0.0.1:8080/geoserver/wfs?Request=GetFeature&typename=gr:nom&Filter=<Filter><PropertyIsEqualTo><PropertyName>gr:nom/NOM_DES</PropertyName><Literal>ΚΙΛΚΗΣ</Literal></PropertyIsEqualTo></>
<gml:polygonMember>-<http://127.0.0.1:8080/geoserver/wfs?Request=GetFeature&typename=gr:nom&Filter=<Filter><PropertyIsEqualTo><PropertyName>gr:nom/NOM_DES</PropertyName><Literal>ΚΙΛΚΗΣ</Literal></PropertyIsEqualTo></>
<gml:outerBoundaryIs><gml:coordinates decimal="." cs="," ts="">398918,4576097
398927.59375,............etc
.......398882.3125,4575989 398892.59375,4576011 398900.3125,4576035
398907,4576059 398918,4576097</gml:coordinates></gml:LinearRing>
</gml:outerBoundaryIs>
</gml:Polygon>
</gml:polygonMember>
</gml:MultiPolygon>
</gr:the_geom>
<gr:AREA>2.52439054145313E9</gr:AREA>
<gr:PERIMETER>317524.16733</gr:PERIMETER>
<gr:NOMOI_>7</gr:NOMOI_>
<gr:NOMOI_ID>6</gr:NOMOI_ID>
<gr:NOM91>57</gr:NOM91>
<gr:PERIF>02</gr:PERIF>
*<**gr:NOM_DES**>*ÊÉËÊÇÓ*</**gr:NOM_DES**>** *
</gr:nom>
</gml:featureMember>
</wfs:FeatureCollection>
Even if I change the encoding in xml (or in every place in Geoserver) the
result in NOM_DES is always *ÊÉËÊÇÓ *which is wrong. It seems like the code
is internally somehow converting utf-8 encoding characters into ANSI (at
least the same result).I haven't figure out yet, if it is a thing of the FeatureResponse.java,
XmlOutputStream.java or even GetFeatureResults.java.
Which datastore are you using? If it's still shapefile that can be the issue. If you tell me what encoding you want, I can just hardcode your encoding in directly (I still haven't debugged why my one that allows you to select didn't work). And then we can test for sure.
(Artie, I started getting shapefile dbf working again for 1.3.x, I'll try to finish it up soon, you can test too, panos couldn't get it working, but it may have easily been a bug on my end).
I am not really a programmer and I don't know if the previous remarks are
all correct. I also don't know if you think that these issues are important
enough so as to do something about it. This depends on your priorities.
We definitely want to support multiple encodings, though tend to rely on others, since it's hard for us to really test these things.
At least you could tell me, as an experienced programmer that you are, if
these problems really have to do with my default encoding, and the Streams
that the Geoserver is using.In that case, should I try to replace them with Writers and encoding info?
Hopefully Artie will respond, as he's thought about this stuff a lot.
Finally I will try to setup Geoserver in a different machine to see what
happens (maybe with different default encoding).
Let us know (go ahead and just email the devel list)
In general my interest with Geoserver has to do with the implementation of
the OGC standards - developing applications and not so much in programming.
I am doing a Phd on these maters and it would be great opportunity to
present in Greece (scientific community) what really are these standards and
how interpolation works with real examples.
Cool, hopefully we can support you in that effort. If/when you get results do be sure to put something up in one of the user experience sections: http://www.geotools.org/display/GEOTOOLS/DataStore+Walkthrough
best regards,
Chris
Thanks
Panos Tz.
--
Chris Holmes
The Open Planning Project
thoughts at: http://cholmes.wordpress.com