[Geoserver-devel] [Geotools-devel] gt-property migration issues - resolvePropertyNames

Your suggested fix seems to have gotten rid of most of the bounding box problems (Although there are some other tests that assume the first geometry is the default one that will have to be changed). However, I am a bit concerned about how reliable this change is. As you can see below, the dataset has 3 different geometry columns. Each of these columns is for at least one line. Given this, I am surprised that setting the point column to be the default actually works.

The other question would be is how often do we actually have multiple geometries in a single feature type? If this is somewhat comen, then som proper support might be a good idea. If this rarely occurs, then it might be advisable to change the test data so it isn’t using multiple geometry features in one feature type.

This is the dataset used for the WFS tests:

_=description:String,name:
String,surfaceProperty:Polygon:srid=4326,pointProperty:Point:srid=4326,curveProperty:LineString:srid=4326,intProperty:Integer,uriProperty:java.net.URI,measurand:String,dateTimeProperty:java.sql.Timestamp,dateProperty:java.sql.Date,decimalProperty:Double,booleanProperty:Boolean
PrimitiveGeoFeature.f001=description-f001|name-f001||POINT(39.73245 2.00342)||155|http://www.opengeospatial.org/|12765||2006-10-25|5.03|true
PrimitiveGeoFeature.f002=description-f002|name-f002||POINT(59.41276 0.22601)||154|http://www.opengeospatial.org/|12769||2006-10-23|4.02|false
PrimitiveGeoFeature.f003=description-f003|name-f003|||LINESTRING(46.074 9.799,46.652 10.466,47.114 11.021)|180||672.1||2006-09-01|12.92|true
PrimitiveGeoFeature.f008=description-f008|name-f008|POLYGON((45.174 30.899,45.652 30.466,45.891 30.466,45.174 30.899))|||300||783.5|2006-06-27 22:08:00-07|2006-12-12|18.92|true
PrimitiveGeoFeature.f015=|||POINT(34.94 -10.52)||-900||2.4|||7.9|

···

On Mon, Dec 22, 2014 at 3:26 PM, Jody Garnett <jgarnett@anonymised.com> wrote:

Good debugging Torben, you have hit on an oddness in the WFS specification. Much of the specification is written using a geometry expression against a feature … and then it becomes a bit murky. As an web / xml specification it was written with the idea we would explore all the attributes (each expressed as an XML snippet). Since we are boiling this down into Filter and SQL queries we like the idea of a specific geometry to test against for speed / indexing / sanity.

So the “correct” thing to do would be to look at the feature type definition and determine which attributes are “geometry” attributes and test against both.
The “quick” thing to do is guess a default geometry (often the first one unless the user says different) and only test against that one.

As you can see AbstractDataStore and ContentDataStore made a different choice!

Aside: The WFS Specification is split into sections, the filter stuff is factored out so it can be used in a couple specifications (such as WFS and SLD).

From the WFS Tutorial[1]: This is known as the “bounding box” or BBOX. The BBOX allows us to ask for only such Features that are contained (or partially contained) inside a box of the coordinates we specify. The form of the bbox query is “bbox=a1,b1,a2,b2” where a, b, c, and d refer to coordinates.

The Filter [2] specification actually fixed things up to be clear this time: BBOX operator

The fes:BBOX element is defined as a convenient and more compact way of encoding the very common bounding box constraint based on the gml:Envelope geometry. It is equivalent to the spatial operation fes:Notfes:Disjoint … </fes:Disjoint></fes:Not> meaning that the fes:BBOX operator shall identify all geometries that spatially interact with the box. If there is only one argument (i.e. one child element specified for the BBOX operator, the calling service shall apply the operator to the geometric values of all the spatial properties of the resource. In this case, the operator shall evaluate to true if all tested spatial property values fulfill the spatial relationship implied by the operator. Otherwise, the BBOX operator shall evaluate the specified arguments (i.e. the two child elements) and test whether their geometric values satisfy the implied spatial relationship.

I also note that Filter 2.0.2 matches our GeoTools understanding of Filter - i.e. no longer expects one of the expressions to be a propertyName.

Links:
[1] http://www.ogcnetwork.net/wfstutorial

[2] http://docs.opengeospatial.org/is/09-026r2/09-026r2.html#60 ← finally html welcome to 1999 OGC you made it!

To return the matter at hand, we may be able to mark which geometry is the default in the property file, and enable these tests to pass while still maintaining the ContendDataStore design decisions (focused on being quick and clear).

The DataUtilities method createType has the following javadoc examples:

Examples:

name:“”,age:0,geom:Geometry,centroid:Point,url:java.io.URL

id:String,polygonProperty:Polygon:srid=32615

identifier:UUID,location:Point,*area:MultiPolygon,created:Date

uuid:UUID,name:String,description:String,time:java.sql.Timestamp

See how the first one has both geom and centroid attributes? The javadocs also indicate you can use a * to mark one of these as a default geometry.

The following would mark centroid as the default geometry: name:“”,age:0,geom:Geometry,*centroid:Point,url:java.io.URL

Making a similar change to the dataset used for your failed geoserver-devel test would allow it to pass.


Jody Garnett
Senior Software Engineer | Boundless

On 22 December 2014 at 14:12, Torben Barsballe <tbarsballe@anonymised.com> wrote:

While attempting to migrate geoserver to use gt-property-ng, I have ran into a bit of an issue with the ContentFeatureSource.resolvePropertyNames() method, when it is called by ContentFeatureCollection.size().

Geoserver is running a BBox query against a FeatureCollection containing multiple geometries. expression1 of the BBox is a blank attribute, expression2 is the BBox.

When using gt-property and AbstractFeatureStore, expression1 remains blank throught execution, and the BBox query is tested against each geometry in the featureCollection

When using gt-property-ng and ContentFeatureStore, the resolvePropertyNames() method is called, and results in expression1 being set to the first geometry. Then, when ContentFeatureCollection.size() runs, the BBox query is only tested against this first geometry, and fails to return any results.

It seems that the implementation of DefaultGeometrySimpleFeaturePropertyAccessorFactory is only intended to return a single geometry.
Upon analysis of the code, I feel that there are two places this could probably be fixed:

  1. Change DefaultGeometrySimpleFeaturePropertyAccessorFactory.get() to handle multiple geometries properly, such that it tryAccessor should set value to an AttributeDescriptor of “” when xpath is “”. However, I am not sure exactly what this class interacts with, and it looks like it is doing what it should be doing right now, so I am hesitant to change anything.

Looking at how accessor caching and accessor factories are handled, it is also possible that the Accessor caching implementation in AttributeExpressionImpl.evaluate() is returning the wrong type of accessor (DefaultGeometrySimpleFeaturePropertyAccessorFactory) but this is not being caught because the value of “target” being tested against is null (causing the wrong accessor to (incorectly) return true for accessor.canHandle();

  1. Skip the resolvePropertyNames in ContentFeatureSource.getCount(). However, this seems to set some values that are required by later calls, and may require more significant rewriting.

Alternatively, I could add a special check prior to resolvePropertyNames such that it is not called in this specific circumstance (Blank xpath, etc.) but this seems a bit hacky.

For now, I will continue to compare the new implementation with the implementation used by AbstractFeatureSource to see if I can come up with any ideas.

Torben Barsballe

On Tue, Dec 23, 2014 at 1:49 AM, Torben Barsballe <
tbarsballe@anonymised.com> wrote:

Your suggested fix seems to have gotten rid of most of the bounding box
problems (Although there are some other tests that assume the first
geometry is the default one that will have to be changed). However, I am a
bit concerned about how reliable this change is. As you can see below, the
dataset has 3 different geometry columns. Each of these columns is <null>
for at least one line. Given this, I am surprised that setting the point
column to be the default actually works.

The other question would be is how often do we actually have multiple
geometries in a single feature type? If this is somewhat comen, then som
proper support might be a good idea. If this rarely occurs, then it might
be advisable to change the test data so it isn't using multiple geometry
features in one feature type.

Multiple geometries are not so uncommon (e.g., polygon plus label point, or
polygon plus label line), plus, this feature type is coming from the WFS
1.1.0 CITE test for OGC compliance, so yeah,
we should handle it properly

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

Please, notice that GeoSolutions will be closed for seasonal holidays
from December the 24th to January the 6th

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------

The functionality to read and compare against multiple geometry features exists, it is merely being ignored/bypassed due to the resolvePropertyNames method in ContentFeatureSource. This method is intended to resolve propertyName references to simple attribute names against the schema of the feature source; eg. “gml:name” becomes simply “name”. However this method also turns a blank property name into the default geometry name (I am not sure this is intentional or not).

The WFS BBox request uses a query with two expression - An AttributeExpression with a blank name and a BBox. When this query is put through resolvePropertyNames() it makes the blank AttributeExpression refer to the default geometry.

Then, when ContentFeatureSource creates a FilteringFeatureReader, it only checks against the default geometry. Note that the AbstractFeatureSource implementation simply did not have an equivalent of resolvePropertyNames, resulting in the blank name never being set to the default geometry.

I am working on a “fix” that makes the resolvePropertyNames method return a blank property name when it is given a blank property name by altering the PropertyNameResolvingVisitor.visit() method. This fixed the WFS test cases in question, although may have caused some other test failures in geotools.

I am not sure that this is the right way to fix this. Based on the implementation of Query/Filter/Expression, I am thinking that maybe setting the first expression in the Query to “null” when the Query is first created might also work (I am going to try this next…)?

Torben Barsballe

···

On Tue, Dec 23, 2014 at 11:00 AM, Jody Garnett <jgarnett@anonymised.com> wrote:

It is not very common, occasionally you will get the same feature represented multiple times:

  • say a city represented as an urban area polygon and a point location for city hall.
  • or the same feature represented at different level of details

In these cases often the feature has the same bounding box for the geometries.

We can ask at the next Skype meeting, or hope others join this conversation.

If you like you can try and prep a patch to allow ContentDataStore to match the expected Filter 2.0.2 functionality.

Jody

On Mon, Dec 22, 2014 at 4:51 PM Torben Barsballe <tbarsballe@anonymised.com839…> wrote:

Your suggested fix seems to have gotten rid of most of the bounding box problems (Although there are some other tests that assume the first geometry is the default one that will have to be changed). However, I am a bit concerned about how reliable this change is. As you can see below, the dataset has 3 different geometry columns. Each of these columns is for at least one line. Given this, I am surprised that setting the point column to be the default actually works.

The other question would be is how often do we actually have multiple geometries in a single feature type? If this is somewhat comen, then som proper support might be a good idea. If this rarely occurs, then it might be advisable to change the test data so it isn’t using multiple geometry features in one feature type.

This is the dataset used for the WFS tests:

_=description:String,name:
String,surfaceProperty:Polygon:srid=4326,pointProperty:Point:srid=4326,curveProperty:LineString:srid=4326,intProperty:Integer,uriProperty:java.net.URI,measurand:String,dateTimeProperty:java.sql.Timestamp,dateProperty:java.sql.Date,decimalProperty:Double,booleanProperty:Boolean
PrimitiveGeoFeature.f001=description-f001|name-f001||POINT(39.73245 2.00342)||155|http://www.opengeospatial.org/|12765||2006-10-25|5.03|true
PrimitiveGeoFeature.f002=description-f002|name-f002||POINT(59.41276 0.22601)||154|http://www.opengeospatial.org/|12769||2006-10-23|4.02|false
PrimitiveGeoFeature.f003=description-f003|name-f003|||LINESTRING(46.074 9.799,46.652 10.466,47.114 11.021)|180||672.1||2006-09-01|12.92|true
PrimitiveGeoFeature.f008=description-f008|name-f008|POLYGON((45.174 30.899,45.652 30.466,45.891 30.466,45.174 30.899))|||300||783.5|2006-06-27 22:08:00-07|2006-12-12|18.92|true
PrimitiveGeoFeature.f015=|||POINT(34.94 -10.52)||-900||2.4|||7.9|


Dive into the World of Parallel Programming! The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@anonymised.comsourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

On Mon, Dec 22, 2014 at 3:26 PM, Jody Garnett <jgarnett@anonymised.com> wrote:

Good debugging Torben, you have hit on an oddness in the WFS specification. Much of the specification is written using a geometry expression against a feature … and then it becomes a bit murky. As an web / xml specification it was written with the idea we would explore all the attributes (each expressed as an XML snippet). Since we are boiling this down into Filter and SQL queries we like the idea of a specific geometry to test against for speed / indexing / sanity.

So the “correct” thing to do would be to look at the feature type definition and determine which attributes are “geometry” attributes and test against both.
The “quick” thing to do is guess a default geometry (often the first one unless the user says different) and only test against that one.

As you can see AbstractDataStore and ContentDataStore made a different choice!

Aside: The WFS Specification is split into sections, the filter stuff is factored out so it can be used in a couple specifications (such as WFS and SLD).

From the WFS Tutorial[1]: This is known as the “bounding box” or BBOX. The BBOX allows us to ask for only such Features that are contained (or partially contained) inside a box of the coordinates we specify. The form of the bbox query is “bbox=a1,b1,a2,b2” where a, b, c, and d refer to coordinates.

The Filter [2] specification actually fixed things up to be clear this time: BBOX operator

The fes:BBOX element is defined as a convenient and more compact way of encoding the very common bounding box constraint based on the gml:Envelope geometry. It is equivalent to the spatial operation fes:Notfes:Disjoint … </fes:Disjoint></fes:Not> meaning that the fes:BBOX operator shall identify all geometries that spatially interact with the box. If there is only one argument (i.e. one child element specified for the BBOX operator, the calling service shall apply the operator to the geometric values of all the spatial properties of the resource. In this case, the operator shall evaluate to true if all tested spatial property values fulfill the spatial relationship implied by the operator. Otherwise, the BBOX operator shall evaluate the specified arguments (i.e. the two child elements) and test whether their geometric values satisfy the implied spatial relationship.

I also note that Filter 2.0.2 matches our GeoTools understanding of Filter - i.e. no longer expects one of the expressions to be a propertyName.

Links:
[1] http://www.ogcnetwork.net/wfstutorial

[2] http://docs.opengeospatial.org/is/09-026r2/09-026r2.html#60 ← finally html welcome to 1999 OGC you made it!

To return the matter at hand, we may be able to mark which geometry is the default in the property file, and enable these tests to pass while still maintaining the ContendDataStore design decisions (focused on being quick and clear).

The DataUtilities method createType has the following javadoc examples:

Examples:

name:“”,age:0,geom:Geometry,centroid:Point,url:java.io.URL

id:String,polygonProperty:Polygon:srid=32615

identifier:UUID,location:Point,*area:MultiPolygon,created:Date

uuid:UUID,name:String,description:String,time:java.sql.Timestamp

See how the first one has both geom and centroid attributes? The javadocs also indicate you can use a * to mark one of these as a default geometry.

The following would mark centroid as the default geometry: name:“”,age:0,geom:Geometry,*centroid:Point,url:java.io.URL

Making a similar change to the dataset used for your failed geoserver-devel test would allow it to pass.


Jody Garnett
Senior Software Engineer | Boundless

On 22 December 2014 at 14:12, Torben Barsballe <tbarsballe@anonymised.com> wrote:

While attempting to migrate geoserver to use gt-property-ng, I have ran into a bit of an issue with the ContentFeatureSource.resolvePropertyNames() method, when it is called by ContentFeatureCollection.size().

Geoserver is running a BBox query against a FeatureCollection containing multiple geometries. expression1 of the BBox is a blank attribute, expression2 is the BBox.

When using gt-property and AbstractFeatureStore, expression1 remains blank throught execution, and the BBox query is tested against each geometry in the featureCollection

When using gt-property-ng and ContentFeatureStore, the resolvePropertyNames() method is called, and results in expression1 being set to the first geometry. Then, when ContentFeatureCollection.size() runs, the BBox query is only tested against this first geometry, and fails to return any results.

It seems that the implementation of DefaultGeometrySimpleFeaturePropertyAccessorFactory is only intended to return a single geometry.
Upon analysis of the code, I feel that there are two places this could probably be fixed:

  1. Change DefaultGeometrySimpleFeaturePropertyAccessorFactory.get() to handle multiple geometries properly, such that it tryAccessor should set value to an AttributeDescriptor of “” when xpath is “”. However, I am not sure exactly what this class interacts with, and it looks like it is doing what it should be doing right now, so I am hesitant to change anything.

Looking at how accessor caching and accessor factories are handled, it is also possible that the Accessor caching implementation in AttributeExpressionImpl.evaluate() is returning the wrong type of accessor (DefaultGeometrySimpleFeaturePropertyAccessorFactory) but this is not being caught because the value of “target” being tested against is null (causing the wrong accessor to (incorectly) return true for accessor.canHandle();

  1. Skip the resolvePropertyNames in ContentFeatureSource.getCount(). However, this seems to set some values that are required by later calls, and may require more significant rewriting.

Alternatively, I could add a special check prior to resolvePropertyNames such that it is not called in this specific circumstance (Blank xpath, etc.) but this seems a bit hacky.

For now, I will continue to compare the new implementation with the implementation used by AbstractFeatureSource to see if I can come up with any ideas.

Torben Barsballe

Further looking into how things work, it looks like:

  1. WFS BBox is expected to test the BBox against all geometries by default.

  2. The BBox filter implementaion in geotools is intended to test the bbox against a single geometry

It seems that the current (AbstractFeatureSource) implementation gets around this limitation by not providing a property name for the other geometry, and later testing against all geometries.

Given this, I have a question:

  1. Is a blank (i.e. attPath=“”) AttributeExpressionImpl intended to match all geometries?
    a. If so, the accept() method of this class and related visitors should be changed to reflect this.
    b. If not, is there any way of defining an Expression that matches all geometries? If this is the case, the WFS XML parser should be updated to use this when creating a geotools BBox filter with Query.getFilter

Torben Barsballe

···

On Tue, Dec 23, 2014 at 1:17 PM, Torben Barsballe <tbarsballe@anonymised.com> wrote:

The functionality to read and compare against multiple geometry features exists, it is merely being ignored/bypassed due to the resolvePropertyNames method in ContentFeatureSource. This method is intended to resolve propertyName references to simple attribute names against the schema of the feature source; eg. “gml:name” becomes simply “name”. However this method also turns a blank property name into the default geometry name (I am not sure this is intentional or not).

The WFS BBox request uses a query with two expression - An AttributeExpression with a blank name and a BBox. When this query is put through resolvePropertyNames() it makes the blank AttributeExpression refer to the default geometry.

Then, when ContentFeatureSource creates a FilteringFeatureReader, it only checks against the default geometry. Note that the AbstractFeatureSource implementation simply did not have an equivalent of resolvePropertyNames, resulting in the blank name never being set to the default geometry.

I am working on a “fix” that makes the resolvePropertyNames method return a blank property name when it is given a blank property name by altering the PropertyNameResolvingVisitor.visit() method. This fixed the WFS test cases in question, although may have caused some other test failures in geotools.

I am not sure that this is the right way to fix this. Based on the implementation of Query/Filter/Expression, I am thinking that maybe setting the first expression in the Query to “null” when the Query is first created might also work (I am going to try this next…)?

Torben Barsballe

On Tue, Dec 23, 2014 at 11:00 AM, Jody Garnett <jgarnett@anonymised.com…> wrote:

It is not very common, occasionally you will get the same feature represented multiple times:

  • say a city represented as an urban area polygon and a point location for city hall.
  • or the same feature represented at different level of details

In these cases often the feature has the same bounding box for the geometries.

We can ask at the next Skype meeting, or hope others join this conversation.

If you like you can try and prep a patch to allow ContentDataStore to match the expected Filter 2.0.2 functionality.

Jody

On Mon, Dec 22, 2014 at 4:51 PM Torben Barsballe <tbarsballe@anonymised.com> wrote:

Your suggested fix seems to have gotten rid of most of the bounding box problems (Although there are some other tests that assume the first geometry is the default one that will have to be changed). However, I am a bit concerned about how reliable this change is. As you can see below, the dataset has 3 different geometry columns. Each of these columns is for at least one line. Given this, I am surprised that setting the point column to be the default actually works.

The other question would be is how often do we actually have multiple geometries in a single feature type? If this is somewhat comen, then som proper support might be a good idea. If this rarely occurs, then it might be advisable to change the test data so it isn’t using multiple geometry features in one feature type.

This is the dataset used for the WFS tests:

_=description:String,name:
String,surfaceProperty:Polygon:srid=4326,pointProperty:Point:srid=4326,curveProperty:LineString:srid=4326,intProperty:Integer,uriProperty:java.net.URI,measurand:String,dateTimeProperty:java.sql.Timestamp,dateProperty:java.sql.Date,decimalProperty:Double,booleanProperty:Boolean
PrimitiveGeoFeature.f001=description-f001|name-f001||POINT(39.73245 2.00342)||155|http://www.opengeospatial.org/|12765||2006-10-25|5.03|true
PrimitiveGeoFeature.f002=description-f002|name-f002||POINT(59.41276 0.22601)||154|http://www.opengeospatial.org/|12769||2006-10-23|4.02|false
PrimitiveGeoFeature.f003=description-f003|name-f003|||LINESTRING(46.074 9.799,46.652 10.466,47.114 11.021)|180||672.1||2006-09-01|12.92|true
PrimitiveGeoFeature.f008=description-f008|name-f008|POLYGON((45.174 30.899,45.652 30.466,45.891 30.466,45.174 30.899))|||300||783.5|2006-06-27 22:08:00-07|2006-12-12|18.92|true
PrimitiveGeoFeature.f015=|||POINT(34.94 -10.52)||-900||2.4|||7.9|


Dive into the World of Parallel Programming! The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@anonymised.comsourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

On Mon, Dec 22, 2014 at 3:26 PM, Jody Garnett <jgarnett@anonymised.com> wrote:

Good debugging Torben, you have hit on an oddness in the WFS specification. Much of the specification is written using a geometry expression against a feature … and then it becomes a bit murky. As an web / xml specification it was written with the idea we would explore all the attributes (each expressed as an XML snippet). Since we are boiling this down into Filter and SQL queries we like the idea of a specific geometry to test against for speed / indexing / sanity.

So the “correct” thing to do would be to look at the feature type definition and determine which attributes are “geometry” attributes and test against both.
The “quick” thing to do is guess a default geometry (often the first one unless the user says different) and only test against that one.

As you can see AbstractDataStore and ContentDataStore made a different choice!

Aside: The WFS Specification is split into sections, the filter stuff is factored out so it can be used in a couple specifications (such as WFS and SLD).

From the WFS Tutorial[1]: This is known as the “bounding box” or BBOX. The BBOX allows us to ask for only such Features that are contained (or partially contained) inside a box of the coordinates we specify. The form of the bbox query is “bbox=a1,b1,a2,b2” where a, b, c, and d refer to coordinates.

The Filter [2] specification actually fixed things up to be clear this time: BBOX operator

The fes:BBOX element is defined as a convenient and more compact way of encoding the very common bounding box constraint based on the gml:Envelope geometry. It is equivalent to the spatial operation fes:Notfes:Disjoint … </fes:Disjoint></fes:Not> meaning that the fes:BBOX operator shall identify all geometries that spatially interact with the box. If there is only one argument (i.e. one child element specified for the BBOX operator, the calling service shall apply the operator to the geometric values of all the spatial properties of the resource. In this case, the operator shall evaluate to true if all tested spatial property values fulfill the spatial relationship implied by the operator. Otherwise, the BBOX operator shall evaluate the specified arguments (i.e. the two child elements) and test whether their geometric values satisfy the implied spatial relationship.

I also note that Filter 2.0.2 matches our GeoTools understanding of Filter - i.e. no longer expects one of the expressions to be a propertyName.

Links:
[1] http://www.ogcnetwork.net/wfstutorial

[2] http://docs.opengeospatial.org/is/09-026r2/09-026r2.html#60 ← finally html welcome to 1999 OGC you made it!

To return the matter at hand, we may be able to mark which geometry is the default in the property file, and enable these tests to pass while still maintaining the ContendDataStore design decisions (focused on being quick and clear).

The DataUtilities method createType has the following javadoc examples:

Examples:

name:“”,age:0,geom:Geometry,centroid:Point,url:java.io.URL

id:String,polygonProperty:Polygon:srid=32615

identifier:UUID,location:Point,*area:MultiPolygon,created:Date

uuid:UUID,name:String,description:String,time:java.sql.Timestamp

See how the first one has both geom and centroid attributes? The javadocs also indicate you can use a * to mark one of these as a default geometry.

The following would mark centroid as the default geometry: name:“”,age:0,geom:Geometry,*centroid:Point,url:java.io.URL

Making a similar change to the dataset used for your failed geoserver-devel test would allow it to pass.


Jody Garnett
Senior Software Engineer | Boundless

On 22 December 2014 at 14:12, Torben Barsballe <tbarsballe@anonymised.com> wrote:

While attempting to migrate geoserver to use gt-property-ng, I have ran into a bit of an issue with the ContentFeatureSource.resolvePropertyNames() method, when it is called by ContentFeatureCollection.size().

Geoserver is running a BBox query against a FeatureCollection containing multiple geometries. expression1 of the BBox is a blank attribute, expression2 is the BBox.

When using gt-property and AbstractFeatureStore, expression1 remains blank throught execution, and the BBox query is tested against each geometry in the featureCollection

When using gt-property-ng and ContentFeatureStore, the resolvePropertyNames() method is called, and results in expression1 being set to the first geometry. Then, when ContentFeatureCollection.size() runs, the BBox query is only tested against this first geometry, and fails to return any results.

It seems that the implementation of DefaultGeometrySimpleFeaturePropertyAccessorFactory is only intended to return a single geometry.
Upon analysis of the code, I feel that there are two places this could probably be fixed:

  1. Change DefaultGeometrySimpleFeaturePropertyAccessorFactory.get() to handle multiple geometries properly, such that it tryAccessor should set value to an AttributeDescriptor of “” when xpath is “”. However, I am not sure exactly what this class interacts with, and it looks like it is doing what it should be doing right now, so I am hesitant to change anything.

Looking at how accessor caching and accessor factories are handled, it is also possible that the Accessor caching implementation in AttributeExpressionImpl.evaluate() is returning the wrong type of accessor (DefaultGeometrySimpleFeaturePropertyAccessorFactory) but this is not being caught because the value of “target” being tested against is null (causing the wrong accessor to (incorectly) return true for accessor.canHandle();

  1. Skip the resolvePropertyNames in ContentFeatureSource.getCount(). However, this seems to set some values that are required by later calls, and may require more significant rewriting.

Alternatively, I could add a special check prior to resolvePropertyNames such that it is not called in this specific circumstance (Blank xpath, etc.) but this seems a bit hacky.

For now, I will continue to compare the new implementation with the implementation used by AbstractFeatureSource to see if I can come up with any ideas.

Torben Barsballe

The only geotools test failure I got was in the shapefile data store implementation:
There was a test of a BBox query, where the query specified only non-geometry attributes in its property list. According to the test implementation, this was intended to return a nonzero number of features (presumably because the blank expression in the filter was intended to be translated into the default geometry and then added to the schema during retyping). With my change, the blank expression is not converted into the default geometry. Then, the retyped scema only contains non-geometry attribures, and the BBox query has no geometries to test against, therefore returning no results.

This seems like a bit of an edge case. Do we actually expect to return a BBox match if no geometry properties are included in our query?

I’ll look into JDBCDataStore, but based on the tests it seems to be working fine after this fix (perhaps the tests are insufficient?)

Torben

···

On Tue, Dec 23, 2014 at 3:05 PM, Jody Garnett <jgarnett@anonymised.com> wrote:

So we are into the PropertyName expression - which surprisingly does not take a property name - instead it takes an XPath expressions. So the question is not so much about using null, but ensuring we correctly handle an empty XPath expression.

I expect ContentFeatureSource returns the default geometry in an effort to make JDBCDataStore implementation go faster. JDBCDataStore was the first production use of ContentDataStore so some optimizations may of been made.

I like the idea of resolvePropertyNames returning a blank property name (since that is accurate). The only failures I would expect would be in the JDBCDataStore implementation - if we find that JDBCDataStore really wants to restrict itself to the default geometry (for performance rit can handle that when encoding SQL.

There is a jdbc-notes.txt in the code base which describes JDBCDataStore using an encodeGeometryColumn method. Perhaps we can use this to catch an empty string being passed in.

Jody


Jody Garnett
Senior Software Engineer | Boundless

On 23 December 2014 at 13:17, Torben Barsballe <tbarsballe@anonymised.com> wrote:

The functionality to read and compare against multiple geometry features exists, it is merely being ignored/bypassed due to the resolvePropertyNames method in ContentFeatureSource. This method is intended to resolve propertyName references to simple attribute names against the schema of the feature source; eg. “gml:name” becomes simply “name”. However this method also turns a blank property name into the default geometry name (I am not sure this is intentional or not).

The WFS BBox request uses a query with two expression - An AttributeExpression with a blank name and a BBox. When this query is put through resolvePropertyNames() it makes the blank AttributeExpression refer to the default geometry.

Then, when ContentFeatureSource creates a FilteringFeatureReader, it only checks against the default geometry. Note that the AbstractFeatureSource implementation simply did not have an equivalent of resolvePropertyNames, resulting in the blank name never being set to the default geometry.

I am working on a “fix” that makes the resolvePropertyNames method return a blank property name when it is given a blank property name by altering the PropertyNameResolvingVisitor.visit() method. This fixed the WFS test cases in question, although may have caused some other test failures in geotools.

I am not sure that this is the right way to fix this. Based on the implementation of Query/Filter/Expression, I am thinking that maybe setting the first expression in the Query to “null” when the Query is first created might also work (I am going to try this next…)?

Torben Barsballe


Dive into the World of Parallel Programming! The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net


GeoTools-Devel mailing list
GeoTools-Devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel

On Tue, Dec 23, 2014 at 11:00 AM, Jody Garnett <jgarnett@anonymised.com> wrote:

It is not very common, occasionally you will get the same feature represented multiple times:

  • say a city represented as an urban area polygon and a point location for city hall.
  • or the same feature represented at different level of details

In these cases often the feature has the same bounding box for the geometries.

We can ask at the next Skype meeting, or hope others join this conversation.

If you like you can try and prep a patch to allow ContentDataStore to match the expected Filter 2.0.2 functionality.

Jody

On Mon, Dec 22, 2014 at 4:51 PM Torben Barsballe <tbarsballe@anonymised.com> wrote:

Your suggested fix seems to have gotten rid of most of the bounding box problems (Although there are some other tests that assume the first geometry is the default one that will have to be changed). However, I am a bit concerned about how reliable this change is. As you can see below, the dataset has 3 different geometry columns. Each of these columns is for at least one line. Given this, I am surprised that setting the point column to be the default actually works.

The other question would be is how often do we actually have multiple geometries in a single feature type? If this is somewhat comen, then som proper support might be a good idea. If this rarely occurs, then it might be advisable to change the test data so it isn’t using multiple geometry features in one feature type.

This is the dataset used for the WFS tests:

_=description:String,name:
String,surfaceProperty:Polygon:srid=4326,pointProperty:Point:srid=4326,curveProperty:LineString:srid=4326,intProperty:Integer,uriProperty:java.net.URI,measurand:String,dateTimeProperty:java.sql.Timestamp,dateProperty:java.sql.Date,decimalProperty:Double,booleanProperty:Boolean
PrimitiveGeoFeature.f001=description-f001|name-f001||POINT(39.73245 2.00342)||155|http://www.opengeospatial.org/|12765||2006-10-25|5.03|true
PrimitiveGeoFeature.f002=description-f002|name-f002||POINT(59.41276 0.22601)||154|http://www.opengeospatial.org/|12769||2006-10-23|4.02|false
PrimitiveGeoFeature.f003=description-f003|name-f003|||LINESTRING(46.074 9.799,46.652 10.466,47.114 11.021)|180||672.1||2006-09-01|12.92|true
PrimitiveGeoFeature.f008=description-f008|name-f008|POLYGON((45.174 30.899,45.652 30.466,45.891 30.466,45.174 30.899))|||300||783.5|2006-06-27 22:08:00-07|2006-12-12|18.92|true
PrimitiveGeoFeature.f015=|||POINT(34.94 -10.52)||-900||2.4|||7.9|


Dive into the World of Parallel Programming! The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@anonymised.comsourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

On Mon, Dec 22, 2014 at 3:26 PM, Jody Garnett <jgarnett@anonymised.com.3839…> wrote:

Good debugging Torben, you have hit on an oddness in the WFS specification. Much of the specification is written using a geometry expression against a feature … and then it becomes a bit murky. As an web / xml specification it was written with the idea we would explore all the attributes (each expressed as an XML snippet). Since we are boiling this down into Filter and SQL queries we like the idea of a specific geometry to test against for speed / indexing / sanity.

So the “correct” thing to do would be to look at the feature type definition and determine which attributes are “geometry” attributes and test against both.
The “quick” thing to do is guess a default geometry (often the first one unless the user says different) and only test against that one.

As you can see AbstractDataStore and ContentDataStore made a different choice!

Aside: The WFS Specification is split into sections, the filter stuff is factored out so it can be used in a couple specifications (such as WFS and SLD).

From the WFS Tutorial[1]: This is known as the “bounding box” or BBOX. The BBOX allows us to ask for only such Features that are contained (or partially contained) inside a box of the coordinates we specify. The form of the bbox query is “bbox=a1,b1,a2,b2” where a, b, c, and d refer to coordinates.

The Filter [2] specification actually fixed things up to be clear this time: BBOX operator

The fes:BBOX element is defined as a convenient and more compact way of encoding the very common bounding box constraint based on the gml:Envelope geometry. It is equivalent to the spatial operation fes:Notfes:Disjoint … </fes:Disjoint></fes:Not> meaning that the fes:BBOX operator shall identify all geometries that spatially interact with the box. If there is only one argument (i.e. one child element specified for the BBOX operator, the calling service shall apply the operator to the geometric values of all the spatial properties of the resource. In this case, the operator shall evaluate to true if all tested spatial property values fulfill the spatial relationship implied by the operator. Otherwise, the BBOX operator shall evaluate the specified arguments (i.e. the two child elements) and test whether their geometric values satisfy the implied spatial relationship.

I also note that Filter 2.0.2 matches our GeoTools understanding of Filter - i.e. no longer expects one of the expressions to be a propertyName.

Links:
[1] http://www.ogcnetwork.net/wfstutorial

[2] http://docs.opengeospatial.org/is/09-026r2/09-026r2.html#60 ← finally html welcome to 1999 OGC you made it!

To return the matter at hand, we may be able to mark which geometry is the default in the property file, and enable these tests to pass while still maintaining the ContendDataStore design decisions (focused on being quick and clear).

The DataUtilities method createType has the following javadoc examples:

Examples:

name:“”,age:0,geom:Geometry,centroid:Point,url:java.io.URL

id:String,polygonProperty:Polygon:srid=32615

identifier:UUID,location:Point,*area:MultiPolygon,created:Date

uuid:UUID,name:String,description:String,time:java.sql.Timestamp

See how the first one has both geom and centroid attributes? The javadocs also indicate you can use a * to mark one of these as a default geometry.

The following would mark centroid as the default geometry: name:“”,age:0,geom:Geometry,*centroid:Point,url:java.io.URL

Making a similar change to the dataset used for your failed geoserver-devel test would allow it to pass.


Jody Garnett
Senior Software Engineer | Boundless

On 22 December 2014 at 14:12, Torben Barsballe <tbarsballe@anonymised.com> wrote:

While attempting to migrate geoserver to use gt-property-ng, I have ran into a bit of an issue with the ContentFeatureSource.resolvePropertyNames() method, when it is called by ContentFeatureCollection.size().

Geoserver is running a BBox query against a FeatureCollection containing multiple geometries. expression1 of the BBox is a blank attribute, expression2 is the BBox.

When using gt-property and AbstractFeatureStore, expression1 remains blank throught execution, and the BBox query is tested against each geometry in the featureCollection

When using gt-property-ng and ContentFeatureStore, the resolvePropertyNames() method is called, and results in expression1 being set to the first geometry. Then, when ContentFeatureCollection.size() runs, the BBox query is only tested against this first geometry, and fails to return any results.

It seems that the implementation of DefaultGeometrySimpleFeaturePropertyAccessorFactory is only intended to return a single geometry.
Upon analysis of the code, I feel that there are two places this could probably be fixed:

  1. Change DefaultGeometrySimpleFeaturePropertyAccessorFactory.get() to handle multiple geometries properly, such that it tryAccessor should set value to an AttributeDescriptor of “” when xpath is “”. However, I am not sure exactly what this class interacts with, and it looks like it is doing what it should be doing right now, so I am hesitant to change anything.

Looking at how accessor caching and accessor factories are handled, it is also possible that the Accessor caching implementation in AttributeExpressionImpl.evaluate() is returning the wrong type of accessor (DefaultGeometrySimpleFeaturePropertyAccessorFactory) but this is not being caught because the value of “target” being tested against is null (causing the wrong accessor to (incorectly) return true for accessor.canHandle();

  1. Skip the resolvePropertyNames in ContentFeatureSource.getCount(). However, this seems to set some values that are required by later calls, and may require more significant rewriting.

Alternatively, I could add a special check prior to resolvePropertyNames such that it is not called in this specific circumstance (Blank xpath, etc.) but this seems a bit hacky.

For now, I will continue to compare the new implementation with the implementation used by AbstractFeatureSource to see if I can come up with any ideas.

Torben Barsballe

The method ShapefileFeatureSource.getReadSchema(Query) that looks for the attributes required for filtering relied on a blank attPath giving the default geometry. I have added a fix that includes all geometry attributes if a geometry filter is used.

This does bring up another question:

If we have a BBox query that does include a geometry property, should we still be testing the BBox against all geometries (Assuming expression1 of the BBox filter is blank), or should we test against only included geometry properties?

Current changes are here: https://github.com/geotools/geotools/pull/659

···

On Tue, Dec 23, 2014 at 4:31 PM, Jody Garnett <jgarnett@anonymised.com> wrote:

Yeah we do, it is kind of like doing an SQL query against one attribute, and then only returning the attributes you plan to use.

There should be a method in the shapefile code that looks at what attributes are needed for filtering and ensures they are fetched and parsed (even if the value is thrown away after being used for filter (and not returned).


Jody Garnett
Senior Software Engineer | Boundless

On 23 December 2014 at 15:37, Torben Barsballe <tbarsballe@anonymised.com> wrote:

The only geotools test failure I got was in the shapefile data store implementation:
There was a test of a BBox query, where the query specified only non-geometry attributes in its property list. According to the test implementation, this was intended to return a nonzero number of features (presumably because the blank expression in the filter was intended to be translated into the default geometry and then added to the schema during retyping). With my change, the blank expression is not converted into the default geometry. Then, the retyped scema only contains non-geometry attribures, and the BBox query has no geometries to test against, therefore returning no results.

This seems like a bit of an edge case. Do we actually expect to return a BBox match if no geometry properties are included in our query?

I’ll look into JDBCDataStore, but based on the tests it seems to be working fine after this fix (perhaps the tests are insufficient?)

Torben


Dive into the World of Parallel Programming! The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net


GeoTools-Devel mailing list
GeoTools-Devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel

On Tue, Dec 23, 2014 at 3:05 PM, Jody Garnett <jgarnett@anonymised.com> wrote:

So we are into the PropertyName expression - which surprisingly does not take a property name - instead it takes an XPath expressions. So the question is not so much about using null, but ensuring we correctly handle an empty XPath expression.

I expect ContentFeatureSource returns the default geometry in an effort to make JDBCDataStore implementation go faster. JDBCDataStore was the first production use of ContentDataStore so some optimizations may of been made.

I like the idea of resolvePropertyNames returning a blank property name (since that is accurate). The only failures I would expect would be in the JDBCDataStore implementation - if we find that JDBCDataStore really wants to restrict itself to the default geometry (for performance rit can handle that when encoding SQL.

There is a jdbc-notes.txt in the code base which describes JDBCDataStore using an encodeGeometryColumn method. Perhaps we can use this to catch an empty string being passed in.

Jody


Jody Garnett
Senior Software Engineer | Boundless

On 23 December 2014 at 13:17, Torben Barsballe <tbarsballe@anonymised.com> wrote:

The functionality to read and compare against multiple geometry features exists, it is merely being ignored/bypassed due to the resolvePropertyNames method in ContentFeatureSource. This method is intended to resolve propertyName references to simple attribute names against the schema of the feature source; eg. “gml:name” becomes simply “name”. However this method also turns a blank property name into the default geometry name (I am not sure this is intentional or not).

The WFS BBox request uses a query with two expression - An AttributeExpression with a blank name and a BBox. When this query is put through resolvePropertyNames() it makes the blank AttributeExpression refer to the default geometry.

Then, when ContentFeatureSource creates a FilteringFeatureReader, it only checks against the default geometry. Note that the AbstractFeatureSource implementation simply did not have an equivalent of resolvePropertyNames, resulting in the blank name never being set to the default geometry.

I am working on a “fix” that makes the resolvePropertyNames method return a blank property name when it is given a blank property name by altering the PropertyNameResolvingVisitor.visit() method. This fixed the WFS test cases in question, although may have caused some other test failures in geotools.

I am not sure that this is the right way to fix this. Based on the implementation of Query/Filter/Expression, I am thinking that maybe setting the first expression in the Query to “null” when the Query is first created might also work (I am going to try this next…)?

Torben Barsballe


Dive into the World of Parallel Programming! The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net


GeoTools-Devel mailing list
GeoTools-Devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel

On Tue, Dec 23, 2014 at 11:00 AM, Jody Garnett <jgarnett@anonymised.com…3839…> wrote:

It is not very common, occasionally you will get the same feature represented multiple times:

  • say a city represented as an urban area polygon and a point location for city hall.
  • or the same feature represented at different level of details

In these cases often the feature has the same bounding box for the geometries.

We can ask at the next Skype meeting, or hope others join this conversation.

If you like you can try and prep a patch to allow ContentDataStore to match the expected Filter 2.0.2 functionality.

Jody

On Mon, Dec 22, 2014 at 4:51 PM Torben Barsballe <tbarsballe@anonymised.com> wrote:

Your suggested fix seems to have gotten rid of most of the bounding box problems (Although there are some other tests that assume the first geometry is the default one that will have to be changed). However, I am a bit concerned about how reliable this change is. As you can see below, the dataset has 3 different geometry columns. Each of these columns is for at least one line. Given this, I am surprised that setting the point column to be the default actually works.

The other question would be is how often do we actually have multiple geometries in a single feature type? If this is somewhat comen, then som proper support might be a good idea. If this rarely occurs, then it might be advisable to change the test data so it isn’t using multiple geometry features in one feature type.

This is the dataset used for the WFS tests:

_=description:String,name:
String,surfaceProperty:Polygon:srid=4326,pointProperty:Point:srid=4326,curveProperty:LineString:srid=4326,intProperty:Integer,uriProperty:java.net.URI,measurand:String,dateTimeProperty:java.sql.Timestamp,dateProperty:java.sql.Date,decimalProperty:Double,booleanProperty:Boolean
PrimitiveGeoFeature.f001=description-f001|name-f001||POINT(39.73245 2.00342)||155|http://www.opengeospatial.org/|12765||2006-10-25|5.03|true
PrimitiveGeoFeature.f002=description-f002|name-f002||POINT(59.41276 0.22601)||154|http://www.opengeospatial.org/|12769||2006-10-23|4.02|false
PrimitiveGeoFeature.f003=description-f003|name-f003|||LINESTRING(46.074 9.799,46.652 10.466,47.114 11.021)|180||672.1||2006-09-01|12.92|true
PrimitiveGeoFeature.f008=description-f008|name-f008|POLYGON((45.174 30.899,45.652 30.466,45.891 30.466,45.174 30.899))|||300||783.5|2006-06-27 22:08:00-07|2006-12-12|18.92|true
PrimitiveGeoFeature.f015=|||POINT(34.94 -10.52)||-900||2.4|||7.9|


Dive into the World of Parallel Programming! The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@anonymised.comsourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

On Mon, Dec 22, 2014 at 3:26 PM, Jody Garnett <jgarnett@anonymised.com> wrote:

Good debugging Torben, you have hit on an oddness in the WFS specification. Much of the specification is written using a geometry expression against a feature … and then it becomes a bit murky. As an web / xml specification it was written with the idea we would explore all the attributes (each expressed as an XML snippet). Since we are boiling this down into Filter and SQL queries we like the idea of a specific geometry to test against for speed / indexing / sanity.

So the “correct” thing to do would be to look at the feature type definition and determine which attributes are “geometry” attributes and test against both.
The “quick” thing to do is guess a default geometry (often the first one unless the user says different) and only test against that one.

As you can see AbstractDataStore and ContentDataStore made a different choice!

Aside: The WFS Specification is split into sections, the filter stuff is factored out so it can be used in a couple specifications (such as WFS and SLD).

From the WFS Tutorial[1]: This is known as the “bounding box” or BBOX. The BBOX allows us to ask for only such Features that are contained (or partially contained) inside a box of the coordinates we specify. The form of the bbox query is “bbox=a1,b1,a2,b2” where a, b, c, and d refer to coordinates.

The Filter [2] specification actually fixed things up to be clear this time: BBOX operator

The fes:BBOX element is defined as a convenient and more compact way of encoding the very common bounding box constraint based on the gml:Envelope geometry. It is equivalent to the spatial operation fes:Notfes:Disjoint … </fes:Disjoint></fes:Not> meaning that the fes:BBOX operator shall identify all geometries that spatially interact with the box. If there is only one argument (i.e. one child element specified for the BBOX operator, the calling service shall apply the operator to the geometric values of all the spatial properties of the resource. In this case, the operator shall evaluate to true if all tested spatial property values fulfill the spatial relationship implied by the operator. Otherwise, the BBOX operator shall evaluate the specified arguments (i.e. the two child elements) and test whether their geometric values satisfy the implied spatial relationship.

I also note that Filter 2.0.2 matches our GeoTools understanding of Filter - i.e. no longer expects one of the expressions to be a propertyName.

Links:
[1] http://www.ogcnetwork.net/wfstutorial

[2] http://docs.opengeospatial.org/is/09-026r2/09-026r2.html#60 ← finally html welcome to 1999 OGC you made it!

To return the matter at hand, we may be able to mark which geometry is the default in the property file, and enable these tests to pass while still maintaining the ContendDataStore design decisions (focused on being quick and clear).

The DataUtilities method createType has the following javadoc examples:

Examples:

name:“”,age:0,geom:Geometry,centroid:Point,url:java.io.URL

id:String,polygonProperty:Polygon:srid=32615

identifier:UUID,location:Point,*area:MultiPolygon,created:Date

uuid:UUID,name:String,description:String,time:java.sql.Timestamp

See how the first one has both geom and centroid attributes? The javadocs also indicate you can use a * to mark one of these as a default geometry.

The following would mark centroid as the default geometry: name:“”,age:0,geom:Geometry,*centroid:Point,url:java.io.URL

Making a similar change to the dataset used for your failed geoserver-devel test would allow it to pass.


Jody Garnett
Senior Software Engineer | Boundless

On 22 December 2014 at 14:12, Torben Barsballe <tbarsballe@anonymised.com> wrote:

While attempting to migrate geoserver to use gt-property-ng, I have ran into a bit of an issue with the ContentFeatureSource.resolvePropertyNames() method, when it is called by ContentFeatureCollection.size().

Geoserver is running a BBox query against a FeatureCollection containing multiple geometries. expression1 of the BBox is a blank attribute, expression2 is the BBox.

When using gt-property and AbstractFeatureStore, expression1 remains blank throught execution, and the BBox query is tested against each geometry in the featureCollection

When using gt-property-ng and ContentFeatureStore, the resolvePropertyNames() method is called, and results in expression1 being set to the first geometry. Then, when ContentFeatureCollection.size() runs, the BBox query is only tested against this first geometry, and fails to return any results.

It seems that the implementation of DefaultGeometrySimpleFeaturePropertyAccessorFactory is only intended to return a single geometry.
Upon analysis of the code, I feel that there are two places this could probably be fixed:

  1. Change DefaultGeometrySimpleFeaturePropertyAccessorFactory.get() to handle multiple geometries properly, such that it tryAccessor should set value to an AttributeDescriptor of “” when xpath is “”. However, I am not sure exactly what this class interacts with, and it looks like it is doing what it should be doing right now, so I am hesitant to change anything.

Looking at how accessor caching and accessor factories are handled, it is also possible that the Accessor caching implementation in AttributeExpressionImpl.evaluate() is returning the wrong type of accessor (DefaultGeometrySimpleFeaturePropertyAccessorFactory) but this is not being caught because the value of “target” being tested against is null (causing the wrong accessor to (incorectly) return true for accessor.canHandle();

  1. Skip the resolvePropertyNames in ContentFeatureSource.getCount(). However, this seems to set some values that are required by later calls, and may require more significant rewriting.

Alternatively, I could add a special check prior to resolvePropertyNames such that it is not called in this specific circumstance (Blank xpath, etc.) but this seems a bit hacky.

For now, I will continue to compare the new implementation with the implementation used by AbstractFeatureSource to see if I can come up with any ideas.

Torben Barsballe

On Wed, Dec 24, 2014 at 12:37 AM, Torben Barsballe <
tbarsballe@anonymised.com> wrote:

The only geotools test failure I got was in the shapefile data store
implementation:
There was a test of a BBox query, where the query specified only
non-geometry attributes in its property list. According to the test
implementation, this was intended to return a nonzero number of features
(presumably because the blank expression in the filter was intended to be
translated into the default geometry and then added to the schema during
retyping). With my change, the blank expression is not converted into the
default geometry. Then, the retyped scema only contains non-geometry
attribures, and the BBox query has no geometries to test against, therefore
returning no results.

This seems like a bit of an edge case. Do we actually expect to return a
BBox match if no geometry properties are included in our query?

Generally speaking, filtering should be done against all the necessary
attributes, and then you have to trim them down to the requested ones.
So, the correct path is:
* Get a list of all attributes needed, the requested ones, and the ones
needed for in-memory filtering
* Perform the native and in memory filtering
* Trim down the attributes to the requested ones via retyping

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

Please, notice that GeoSolutions will be closed for seasonal holidays
from December the 24th to January the 6th

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------

On Tue, Dec 23, 2014 at 11:38 PM, Torben Barsballe <
tbarsballe@anonymised.com> wrote:

Further looking into how things work, it looks like:

1. WFS BBox is expected to test the BBox against all geometries by default.

2. The BBox filter implementaion in geotools is intended to test the bbox
against a single geometry

It seems that the current (AbstractFeatureSource) implementation gets
around this limitation by not providing a property name for the other
geometry, and later testing against all geometries.

Given this, I have a question:

1. Is a blank (i.e. attPath="") AttributeExpressionImpl intended to match
all geometries?
    a. If so, the accept() method of this class and related visitors
should be changed to reflect this.
    b. If not, is there any way of defining an Expression that matches all
geometries? If this is the case, the WFS XML parser should be updated to
use this when creating a geotools BBox filter with Query.getFilter

The black attribute is what you get when a WFS request uses
BBOX=x1,y1,x2,y2 in a GET style GetFeature request, since the name of the
attribute is unspecified.
The WFS spec does not say to which geometry it applies, just that it's a
shorthand for the XML version.
Unfortunately in the XML version the propertyName parameter is mandatory,
so basically we don't have any direction from the spec.

So, given we don't know, we have to base ourselves on the requirements for
compliance certification, embodied by the CITE tests.
I cannot tell you exactly what the CITE tests do, but for sure we pass them
using the JDBCDataStore implementation, so, I would
mimick their behavior (and make sure any chance to the base classes does
not make the GeoServer WFS CITE tests fail)

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

Please, notice that GeoSolutions will be closed for seasonal holidays
from December the 24th to January the 6th

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------

Ah, just found this in the WFS 2.0 spec:

BBOX: This is a convenient and more compact way of encoding the
BBOX constraint based on the gml:Envelope.
The BBOX operator shall identify all features whose geometries
spatially intersect the envelope.

The sentence makes me think all geometries should be taken into account

Cheers
Andrea

···

On Wed, Dec 31, 2014 at 10:28 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Tue, Dec 23, 2014 at 11:38 PM, Torben Barsballe <tbarsballe@anonymised.com> wrote:

Further looking into how things work, it looks like:

  1. WFS BBox is expected to test the BBox against all geometries by default.

  2. The BBox filter implementaion in geotools is intended to test the bbox against a single geometry

It seems that the current (AbstractFeatureSource) implementation gets around this limitation by not providing a property name for the other geometry, and later testing against all geometries.

Given this, I have a question:

  1. Is a blank (i.e. attPath=“”) AttributeExpressionImpl intended to match all geometries?
    a. If so, the accept() method of this class and related visitors should be changed to reflect this.
    b. If not, is there any way of defining an Expression that matches all geometries? If this is the case, the WFS XML parser should be updated to use this when creating a geotools BBox filter with Query.getFilter

The black attribute is what you get when a WFS request uses BBOX=x1,y1,x2,y2 in a GET style GetFeature request, since the name of the attribute is unspecified.
The WFS spec does not say to which geometry it applies, just that it’s a shorthand for the XML version.
Unfortunately in the XML version the propertyName parameter is mandatory, so basically we don’t have any direction from the spec.

So, given we don’t know, we have to base ourselves on the requirements for compliance certification, embodied by the CITE tests.
I cannot tell you exactly what the CITE tests do, but for sure we pass them using the JDBCDataStore implementation, so, I would
mimick their behavior (and make sure any chance to the base classes does not make the GeoServer WFS CITE tests fail)

Cheers

Andrea

==

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

Please, notice that GeoSolutions will be closed for seasonal holidays
from December the 24th to January the 6th

==

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


==

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

Please, notice that GeoSolutions will be closed for seasonal holidays
from December the 24th to January the 6th

==

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.