[Geoserver-devel] More WFS axis issues

GeoServer GeoServer Developer
,
1

Hi,
I'm still playing with axis order issues and finding
other issues.

So far I've fixed the postgis sample data set, and I have
tests that do specify BBOX in "urn:x-ogc:def:crs:EPSG:6.11.2:4326"
work fine (the server flips the envelope before throwing it
at the datastore, which is working in lon/lat order).

Unfortunately this does not work with the following query
(notice the is no SRS specified):

<wfs:Query typeName="sf:PrimitiveGeoFeature">
             <ogc:Filter>
                <ogc:BBOX>
                   <ogc:PropertyName>sf:pointProperty</ogc:PropertyName>
                   <gml:Envelope>
                      <gml:lowerCorner>37.0 -2.5</gml:lowerCorner>
                      <gml:upperCorner>42.0 2.5</gml:upperCorner>
                   </gml:Envelope>
                </ogc:BBOX>
             </ogc:Filter>
          </wfs:Query>
       </wfs:GetFeature>

Where the test assertion is:
"If the @srsName of the supplied literal geometry or gml:Envelope is unspecified, it shall be handled as if it corresponded to the advertised default CRS."

Now, the capabilities document for that layer states:

<FeatureType>
<Name>sf:PrimitiveGeoFeature</Name>
<Title>PrimitiveGeoFeatureType</Title>
<Abstract>Generated from sf</Abstract>
−
  <ows:Keywords>
<ows:Keyword>sf PrimitiveGeoFeature</ows:Keyword>
</ows:Keywords>
<DefaultSRS>urn:x-ogc:def:crs:EPSG:6.11.2:4326</DefaultSRS>
−
  <OutputFormats>
<Format>text/gml; subtype=gml/3.1.1</Format>
</OutputFormats>
−
  <ows:WGS84BoundingBox>
<ows:LowerCorner>-180.0 -90.0</ows:LowerCorner>
<ows:UpperCorner>180.0 90.0</ows:UpperCorner>
</ows:WGS84BoundingBox>
</FeatureType>
</FeatureTypeList>

Sigh, how crazy is this... Now, the cite test suite is
somewhat right, in fact we're advertising the default
srs with lat/lon order, I'll check if using EPSG:4326 changes
anything. I really doubt it anyways, the cite test expects
a certain result from that request, so it'll always assume
the data is really in that SRS, but for sure with this
configuration we are the one at loss.

It seems the only way to have the cite tests pass for good
is to have the datastore use lat/lon axis ordering, but yet
I'm wondering, would this mean that the only way to have
GeoServer be WFS 1.1 compliant is to have it handle all
data in latitude/longitude order? This is crazy, no
datastore and no storage besides GML uses that order, which
means all datastores would have to apply a transfomation
on whatever data they load before passing it to the upper
levels!

Would stating EPSG:4326 in the capabilities save our bacon
in terms of stating we are compliant?
Really, the OGC attitude about axis order is starting to
get on my nerves, the real world data is using longitude/latitude
people, get out of that crystal ball!

Cheers
Andrea

2

Andrea Aime ha scritto:
...

It seems the only way to have the cite tests pass for good
is to have the datastore use lat/lon axis ordering, but yet
I'm wondering, would this mean that the only way to have
GeoServer be WFS 1.1 compliant is to have it handle all
data in latitude/longitude order? This is crazy, no
datastore and no storage besides GML uses that order, which
means all datastores would have to apply a transfomation
on whatever data they load before passing it to the upper
levels!

Oh, and btw, we'll have to change our configuration subsystem
to store a full srsName instead of just an integer, otherwise
people won't have a way to state their datastore is working
natively in lat/lon order (a GML3 one would need that).

Cheers
Andrea

3

Hi Andrea,

I think the wfs 1.1 cite tests require us to use the urn syntax.
Actually I had to fight to keep the "epsg:" around as legal. One of the
tests actually was asserting that requesting epsg:4326 should throw an
exception... talk about crazyness.

I feel your pain on this issue and like you I am unsure how to proceed.
I dont think we should go changing datastores... i mean if this is a
silly cite requirement that is not useful in real world, then it
deserves a silly cite hack to make it work.

Andrea Aime wrote:

Hi,
I'm still playing with axis order issues and finding
other issues.

So far I've fixed the postgis sample data set, and I have
tests that do specify BBOX in "urn:x-ogc:def:crs:EPSG:6.11.2:4326"
work fine (the server flips the envelope before throwing it
at the datastore, which is working in lon/lat order).

Unfortunately this does not work with the following query
(notice the is no SRS specified):

<wfs:Query typeName="sf:PrimitiveGeoFeature">
             <ogc:Filter>
                <ogc:BBOX>
                   <ogc:PropertyName>sf:pointProperty</ogc:PropertyName>
                   <gml:Envelope>
                      <gml:lowerCorner>37.0 -2.5</gml:lowerCorner>
                      <gml:upperCorner>42.0 2.5</gml:upperCorner>
                   </gml:Envelope>
                </ogc:BBOX>
             </ogc:Filter>
          </wfs:Query>
       </wfs:GetFeature>

Where the test assertion is:
"If the @srsName of the supplied literal geometry or gml:Envelope is
unspecified, it shall be handled as if it corresponded to the advertised
default CRS."

Now, the capabilities document for that layer states:

<FeatureType>
<Name>sf:PrimitiveGeoFeature</Name>
<Title>PrimitiveGeoFeatureType</Title>
<Abstract>Generated from sf</Abstract>
−
  <ows:Keywords>
<ows:Keyword>sf PrimitiveGeoFeature</ows:Keyword>
</ows:Keywords>
<DefaultSRS>urn:x-ogc:def:crs:EPSG:6.11.2:4326</DefaultSRS>
−
  <OutputFormats>
<Format>text/gml; subtype=gml/3.1.1</Format>
</OutputFormats>
−
  <ows:WGS84BoundingBox>
<ows:LowerCorner>-180.0 -90.0</ows:LowerCorner>
<ows:UpperCorner>180.0 90.0</ows:UpperCorner>
</ows:WGS84BoundingBox>
</FeatureType>
</FeatureTypeList>

Sigh, how crazy is this... Now, the cite test suite is
somewhat right, in fact we're advertising the default
srs with lat/lon order, I'll check if using EPSG:4326 changes
anything. I really doubt it anyways, the cite test expects
a certain result from that request, so it'll always assume
the data is really in that SRS, but for sure with this
configuration we are the one at loss.

It seems the only way to have the cite tests pass for good
is to have the datastore use lat/lon axis ordering, but yet
I'm wondering, would this mean that the only way to have
GeoServer be WFS 1.1 compliant is to have it handle all
data in latitude/longitude order? This is crazy, no
datastore and no storage besides GML uses that order, which
means all datastores would have to apply a transfomation
on whatever data they load before passing it to the upper
levels!

Would stating EPSG:4326 in the capabilities save our bacon
in terms of stating we are compliant?
Really, the OGC attitude about axis order is starting to
get on my nerves, the real world data is using longitude/latitude
people, get out of that crystal ball!

Cheers
Andrea

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

!DSPAM:4007,46e66a4c168565210051143!

--
Justin Deoliveira
The Open Planning Project
http://topp.openplans.org

4

Justin Deoliveira ha scritto:

Hi Andrea,

I think the wfs 1.1 cite tests require us to use the urn syntax.
Actually I had to fight to keep the "epsg:" around as legal. One of the
tests actually was asserting that requesting epsg:4326 should throw an
exception... talk about crazyness.

I feel your pain on this issue and like you I am unsure how to proceed.
I dont think we should go changing datastores... i mean if this is a
silly cite requirement that is not useful in real world, then it
deserves a silly cite hack to make it work.

Well, it's not sure a hack to me. I mean, WFS 1.1 introduced that
issue and we have to handle it even in production configuration, which
means datastores using lon/lat and us serving and dealing with lat/lon.
In fact there are some italian users claiming GeoServer 1.5 is not WFS 1.0 compliant because we return data in lon/lat format, and I cannot
blame them completely because the WFS 1.0 standard did not state clearly
which the axis order was.

I've dumped some options to tackle this here:
http://jira.codehaus.org/browse/GEOS-1325

Cheers
Andrea

5

One argument of OGC guys is that developpers are using (longitude,
latitude) axis order for a few decades while geographers, pilots,
scientists, etc. are using (latitude, longitude) axis order for a few
centuries. OGC said (quoting from WMS specification):

    "The use of geographic CRSs with axis order of longitude before
     latitude differs from historical convention. Users in the
     international aviation and marine sectors may expect latitude
     to be before longitude, and a different coordinate display may
     have safety implications, especially in an emergency response
     situation."

I believe that the axis order issue is currently a pain because CRS
handling has been added relatively late in the Geotools development
process, and often not in the generic way. We are feeling frustration
against OGC specifications while actually our own implementations share
the blame.

I'm may sound simplist, but I believe that we are hurt by the fact that
many GeoTools modules handle AffineTransform badly. Sequences of "if ..
else" statements around AffineTransform are usually bad signs. There is
no short-term solution I can suggest. In long-term, since we can not ask
to every developpers to master AffineTransform, some duty CRS tasks can
be handled by appropriate abstract base class. When handled properly, we
really don't need to bother about axis order at all (except providing a
default order for DataStores without CRS information, like Shapefile
without .prj as mentioned by Andrea - but again an abstract base class
could handle the dirty tasks given a minimal set of hints).

So I have no short term solution to offer. But in (very?) long term, if
I could manage to get time for that, I could offer (if peoples wish) to
see if I can improve CRS handling in AbstractDataStore, in order to make
CRS handling easier for subclasses, like I'm trying to do with
ReferencedCanvas2D in the "go" module (building on J2D renderer
experience - which didn't suffered from axis order problems - hopefully
in a less convolved way).

  Martin

6

Martin Desruisseaux ha scritto:

One argument of OGC guys is that developpers are using (longitude,
latitude) axis order for a few decades while geographers, pilots,
scientists, etc. are using (latitude, longitude) axis order for a few
centuries. OGC said (quoting from WMS specification):

    "The use of geographic CRSs with axis order of longitude before
     latitude differs from historical convention. Users in the
     international aviation and marine sectors may expect latitude
     to be before longitude, and a different coordinate display may
     have safety implications, especially in an emergency response
     situation."

I believe that the axis order issue is currently a pain because CRS
handling has been added relatively late in the Geotools development
process, and often not in the generic way. We are feeling frustration
against OGC specifications while actually our own implementations share
the blame.

I disagree. We are simply aligned with the major vendors in my opinion.
ESRI, Autodesk and whatnot are sharing the same axis ordering as us.

I'm may sound simplist, but I believe that we are hurt by the fact that
many GeoTools modules handle AffineTransform badly. Sequences of "if ..
else" statements around AffineTransform are usually bad signs. There is
no short-term solution I can suggest. In long-term, since we can not ask
to every developpers to master AffineTransform, some duty CRS tasks can
be handled by appropriate abstract base class. When handled properly, we
really don't need to bother about axis order at all (except providing a
default order for DataStores without CRS information, like Shapefile
without .prj as mentioned by Andrea - but again an abstract base class
could handle the dirty tasks given a minimal set of hints).

WMS 1.1 is lon/lat
oriented. Shapefiles and postgis are lon/lat oriented. Oracle is
lon/lat oriented. MySql too. MIF files as well.
Most shapefiles do not have a .prj, and when they have, the axis ordering is usually not specified, but I still have to find one that
does not display correctly with the current gt2 (meaning it's lon/lat).

So I have no short term solution to offer. But in (very?) long term, if
I could manage to get time for that, I could offer (if peoples wish) to
see if I can improve CRS handling in AbstractDataStore, in order to make
CRS handling easier for subclasses, like I'm trying to do with
ReferencedCanvas2D in the "go" module (building on J2D renderer
experience - which didn't suffered from axis order problems - hopefully
in a less convolved way).

Well, besides query in my opinion datastores should serve data in the
native format, and that native format is usually lon/lat.
The real problem comes from the fact we usually can't trust the CRS
information provided in datastores:
* in shapefile, it's either missing, or lacking the TOWGS params and
   the axis order -> the default behaviour in GeoServer is to ignore
   it and have the user specify one, and all the CRS the user can
   specify are lon/lat because that's how the data user have is
   (if data in the wild was lat/lon we would have that ordering
    by default)
* in postgis there is WKT all right, but missing TOWGS information,
   so we just ignore it and CRS.decode() whatever crs number we find.
   Again, we get an lon/lat one by default in Geoserver (we flip axis)
   and it's all right because if you look at the proj strings in
   Postgis, they all are lon/lat
* in Oracle happens the same, with the aggravation that the WKT
   information specified is just plain wrong (wrong syntax, wrong
   transformation name). We thus do the same we do in Postgis
The list could go on. To tell you the truth, if we could use
the CRS information in the datastores as is, there would be no
problem, because in WFS and WMS the reprojection code is there
and working (in both cases, there is out of the datastore
reprojection because ds aren't doing it by themselves, but
as you can see this does not save us from troubles).
The query code is broken too, because it
expects the bbox and geometries to be specified in the same
CRS as the geometry storage, but again I have a workaround to
be applied before the transformation.

This is still not enough because I have no way in GeoServer to
specify that a datastore has a lat/lon CRS unless the datastore
has the native CRS properly set, something which is in general
false. And I cannot default to have lat/lon CRS in GeoServer
for the same reason, because I would pick up the wrong defaults
compared to the actual data storage.

I have detailed some course of action here, to try and keep
backwards compatibility and still be able to handle this
situation: http://jira.codehaus.org/browse/GEOS-1325

Cheers
Andrea

7

I have a proposal below, but I'm not sure if it was already suggested or
if I just misunderstood...

Le mardi 11 septembre 2007 à 19:59 +0200, Andrea Aime a écrit :

WMS 1.1 is lon/lat
oriented. Shapefiles and postgis are lon/lat oriented. Oracle is
lon/lat oriented. MySql too. MIF files as well.
Most shapefiles do not have a .prj, and when they have, the axis
ordering is usually not specified, but I still have to find one that
does not display correctly with the current gt2 (meaning it's lon/lat).

I do not contest that. But if the CRS characteristics are well known
(long,lat order in those DataStores and inside JTS libraries, lat,long
order in other parts), then we are not supposed to have any trouble. The
blame may be in the referencing module lacking convenience methods, but
OGC may not be to blame for aligning themself on centuries of practice.
They are just enforcing what "EPSG:4326" means for data exchange, not
how to represents data internally.

Well, besides query in my opinion datastores should serve data in the
native format, and that native format is usually lon/lat.

I agree with that.

* in postgis there is WKT all right, but missing TOWGS information,
   so we just ignore it and CRS.decode() whatever crs number we find.
   Again, we get an lon/lat one by default in Geoserver (we flip axis)
   and it's all right because if you look at the proj strings in
   Postgis, they all are lon/lat

Taking the PostGIS case as an example, what about the following?

* Scan the whole GeoTools code base, looking for "CRS.decode(String)"
  method calls.

* For each "CRS.decode(String)", if the call is performed from a
  DataStore that expect (lon,lat) axis order in all cases like PostGIS,
  replace "CRS.decode(String)" by "CRS.decode(String, true)".

* Remove the "force xy axis order=true" hint. Most of GeoTools should
  use the (lat,lon) except the DataStore that explicitly ask for (lon,
  lat) order.

* If the CRS is specified as a fully formed CoordinateReferenceSystem
  object and a particular DataStore need to enforce that the axis order
  is (lon,lat), then we need more support from the referencing module.
  The referencing module is already perfoming partially this task for
  CRS obtained from an authority code; we could generalize it to fully
  formed CRS objects.

  Martin

8

Martin Desruisseaux ha scritto:

I have a proposal below, but I'm not sure if it was already suggested or
if I just misunderstood...

I hear you, but the situation is not so simple. In both three examples
what I'm reporting is the "common practice", but there nothing structural in it.

What I mean is that nothing prevents you from having a shapefile, postgis table, or oracle layer to be in lat/lon orientation.
What I'm referring to is just the common practice.

Unfortunately, storages are not helping us because axis
orientation is not declared most of the time, so I need to
rely on the user to tell me what's the real native axis orientation
most of the time.

Look, for example, at how 4326 is defined in postgis:

select * from spatial_ref_sys where srid=4326;
  srid | auth_name | auth_srid |srtext proj4text
  4326 | EPSG | 4326 | GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],TOWGS84[0,0,0,0,0,0,0],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.01745329251994328,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4326"]] | +proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs

The only hint that axis orientation is lon lat is in the proj4
params, the wkt is useless. .prj files are usually in the same boat.

Cheers
Andrea

9

Le mercredi 12 septembre 2007 à 10:56 +0200, Andrea Aime a écrit :

The only hint that axis orientation is lon lat is in the proj4
params, the wkt is useless. .prj files are usually in the same boat.

When the AXIS[...] elements are not explicitly defined in a WKT, they
are assumed (lon,lat) or (x,y). This part is in the OGC specification,
so we are probably safe with this assumption. Maybe we are missing some
way to substitute the CoordinateSystem of a CoordinateReferenceSystem? I
mean something like that:

CoordinateReferenceSystem theUserCRS = ...
CoordinateReferenceSystem theDatastoreCRS = ...
CoordinateReferenceSystem userWithModifiedAxisOrderCRS =
     theUserCRS.useSameAxisOrderThan(theDatastoreCRS.getCoordinateSystem());

Probably as a Factory method rather than a CRS method, but the idea is
there.

  Martin