[Geoserver-users] WFS MaxFeatures & count - can a client know?

Hi,

The count of features received with resulttype=results should match the count in resulttype=hits. That also makes most sense for me because that way the client can make a test and re-think and adjust the request before sending the real GetFeature. However, often it would be useful to know also the total count of features in the service but that is missing from the WFS standard. In practice it means that for example GDAL when it reads the layer summary with ogrinfo and –so switch http://www.gdal.org/ogrinfo.html through WFS driver http://www.gdal.org/ogr/drv_wfs.html the utility actually reads the whole feature type. Or tries to read, it may hit the server side maxfeatures naturally.

-Jukka-

···

Jonathan Moules

Hi Jukka,

I see what the difference is after some testing. Using the exact same URL but only changing the WFS version:

http://wppgeog3/geoserver/ows?service=wfs&version=2.0.0&request=getfeature&typename=Public_Data_DB:OS_ADDPOINT_WSHIRE&resulttype=hits

WFS 1.0.0 - 0 (wrong)

WFS 1.1.0 - 7500 (correct)

WFS 2.0.0 - 0 (wrong)

Looking in the spec, 1.0.0 doesn’t seem to have “resulttype” (but Geoserver tries anyway). It’s in both 1.1.0 and 2.0.0 though.

So looks like a bug, reported as http://jira.codehaus.org/browse/GEOS-6145

=====

One thought - shouldn’t resulttype=hits return the actual number of features that satisfy the user-query, not the number that satisfy the query after GeoServer has added the maxfeatures to it?

So if the limit is 7500, and my database contains 10,000,000, shouldn’t it return 10,000,000?

The specs don’t say clearly one way of the other which it should be.

Cheers,

Jonathan

On 8 November 2013 15:13, Rahkonen Jukka <jukka.rahkonen@anonymised.com6…> wrote:

Hi Jonathan,

This is another WFS but like this it should work

Client side maxfeatures=10, numberOfFeatures=10
http://hip.latuviitta.org/cgi-bin/tinyows?service=wfs&version=1.1.0&maxfeatures=10&request=getfeature&typename=lv:mtk_tos_viiva&resulttype=hits

No client side maxfeatures, numberOfFeatures=100000, probably the server side limit

http://hip.latuviitta.org/cgi-bin/tinyows?service=wfs&version=1.1.0&max&request=getfeature&typename=lv:mtk_tos_viiva&resulttype=hits

-Jukka Rahkonen-

Jonathan Moules

Hi Jukka,

Ok. It just seems like the sort of thing that would be very important for the client to know so it can tell the user “Server returned 1000 features but the server will only ever return 1000 features at a time so this may not be the entire dataset”.

Testing your suggestion, whenever I do that I always get back the exact same:

<wfs:FeatureCollection xmlns:OS_Rasters=“http://www.warwickshire.gov.uk/os_rasters” xmlns:ogc=“http://www.opengis.net/ogc” xmlns:Secure_Community=“http://www.warwickshire.gov.uk/Secure_Community” xmlns:OS_Vector=“http://www.warwickshire.gov.uk/os_vector” xmlns:wfs=“http://www.opengis.net/wfs” xmlns:ows=“http://www.opengis.net/ows” xmlns:xlink=“http://www.w3.org/1999/xlink” xmlns:Misc_Rasters=“http://www.warwickshire.gov.uk/misc_rasters” xmlns:gml=“http://www.opengis.net/gml” xmlns:Public_Data_DB=“http://www.warwickshire.gov.uk/public_data_db” xmlns:xsi=“http://www.w3.org/2001/XMLSchema-instance” numberOfFeatures=“0” timeStamp=“2013-11-08T14:55:55.783Z” xsi:schemaLocation=“http://www.opengis.net/wfs http://schemas.opengis.net/wfs/1.1.0/wfs.xsd”/>

Note it’s claimed 0 features. Despite the fact the request does return 7500 features (maximum). A bug? I’ve not used that before.

Cheers,

Jonathan

On 8 November 2013 14:34, Rahkonen Jukka <jukka.rahkonen@anonymised.com> wrote:

Hi,

Not directly as far as I know, but by sending GetFeature without filters and with resulttype=hits it should be possible to do a good guess.

-Jukka Rahkonen-

Jonathan Moules wrote:

Hi List,

Relatively simple question, is it possible to a client to discover what the maximum number of results are that the GeoServer will return to a WFS request if the user has set “Maximum number of features”?

It doesn’t seem to be specified in the GetCapabilities explicitly. Closest I can see is the WFS 2.0 "ows:Constraint name=“CountDefault”

Does 1.0 have any way to find it out?

Thanks,

Jonathan

This transmission is intended for the named addressee(s) only and may contain sensitive or protectively marked material up to RESTRICTED and should be handled accordingly. Unless you are the named addressee (or authorised to receive it for the addressee) you may not copy or use it, or disclose it to anyone else. If you have received this transmission in error please notify the sender immediately. All email traffic sent to or from us, including without limitation all GCSX traffic, may be subject to recording and/or monitoring in accordance with relevant legislation.

This transmission is intended for the named addressee(s) only and may contain sensitive or protectively marked material up to RESTRICTED and should be handled accordingly. Unless you are the named addressee (or authorised to receive it for the addressee) you may not copy or use it, or disclose it to anyone else. If you have received this transmission in error please notify the sender immediately. All email traffic sent to or from us, including without limitation all GCSX traffic, may be subject to recording and/or monitoring in accordance with relevant legislation.


November Webinars for C, C++, Fortran Developers
Accelerate application performance with scalable programming models. Explore
techniques for threading, error checking, porting, and tuning. Get the most
from the latest Intel processors and coprocessors. See abstracts and register
http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk


Geoserver-users mailing list
Geoserver-users@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

This transmission is intended for the named addressee(s) only and may contain sensitive or protectively marked material up to RESTRICTED and should be handled accordingly. Unless you are the named addressee (or authorised to receive it for the addressee) you may not copy or use it, or disclose it to anyone else. If you have received this transmission in error please notify the sender immediately. All email traffic sent to or from us, including without limitation all GCSX traffic, may be subject to recording and/or monitoring in accordance with relevant legislation.

Selon Rahkonen Jukka <jukka.rahkonen@anonymised.com>:

Hi,

The count of features received with resulttype=results should match the count
in resulttype=hits. That also makes most sense for me because that way the
client can make a test and re-think and adjust the request before sending the
real GetFeature. However, often it would be useful to know also the total
count of features in the service but that is missing from the WFS standard.
In practice it means that for example GDAL when it reads the layer summary
with ogrinfo and -so switch ogrinfo — GDAL documentation through WFS
driver http://www.gdal.org/ogr/drv_wfs.html the utility actually reads the
whole feature type. Or tries to read, it may hit the server side maxfeatures
naturally.

Actually, this is not the exact reason. For the summary of the feature count,
the OGR WFS driver will issue a resulttype=hits request (but I now realize that
it might be lower than the total number of features in the layer, but as there's
no way to know it, that's OK). The reason for it to issue a full feature
download is to get the layer extent, due to the fact that the advertized one in
GetCapabilities is sometimes not reliable. I might end up changing this
non-friendly network bandwith behaviour some day...

Even

-Jukka-

Jonathan Moules

Hi Jukka,
I see what the difference is after some testing. Using the exact same URL but
only changing the WFS version:

http://wppgeog3/geoserver/ows?service=wfs&version=2.0.0&request=getfeature&typename=Public_Data_DB:OS_ADDPOINT_WSHIRE&resulttype=hits

WFS 1.0.0 - 0 (wrong)
WFS 1.1.0 - 7500 (correct)
WFS 2.0.0 - 0 (wrong)

Looking in the spec, 1.0.0 doesn't seem to have "resulttype" (but Geoserver
tries anyway). It's in both 1.1.0 and 2.0.0 though.
So looks like a bug, reported as http://jira.codehaus.org/browse/GEOS-6145

=====

One thought - shouldn't resulttype=hits return the *actual* number of
features that satisfy the user-query, not the number that satisfy the query
after GeoServer has added the maxfeatures to it?

So if the limit is 7500, and my database contains 10,000,000, shouldn't it
return 10,000,000?

The specs don't say clearly one way of the other which it should be.

Cheers,

Jonathan

On 8 November 2013 15:13, Rahkonen Jukka
<jukka.rahkonen@anonymised.com<mailto:jukka.rahkonen@anonymised.com>> wrote:
Hi Jonathan,

This is another WFS but like this it should work
Client side maxfeatures=10, numberOfFeatures=10

http://hip.latuviitta.org/cgi-bin/tinyows?service=wfs&version=1.1.0&maxfeatures=10&request=getfeature&typename=lv:mtk_tos_viiva&resulttype=hits

No client side maxfeatures, numberOfFeatures=100000, probably the server side
limit

http://hip.latuviitta.org/cgi-bin/tinyows?service=wfs&version=1.1.0&max&request=getfeature&typename=lv:mtk_tos_viiva&resulttype=hits

-Jukka Rahkonen-

Jonathan Moules

Hi Jukka,
Ok. It just seems like the sort of thing that would be very important for the
client to know so it can tell the user "Server returned 1000 features but the
server will only ever return 1000 features at a time so this may not be the
entire dataset".

Testing your suggestion, whenever I do that I always get back the exact same:

<wfs:FeatureCollection
xmlns:OS_Rasters="http://www.warwickshire.gov.uk/os_rasters&quot;
xmlns:ogc="http://www.opengis.net/ogc&quot;
xmlns:Secure_Community="http://www.warwickshire.gov.uk/Secure_Community&quot;
xmlns:OS_Vector="http://www.warwickshire.gov.uk/os_vector&quot;
xmlns:wfs="http://www.opengis.net/wfs&quot; xmlns:ows="http://www.opengis.net/ows&quot;
xmlns:xlink="http://www.w3.org/1999/xlink&quot;
xmlns:Misc_Rasters="http://www.warwickshire.gov.uk/misc_rasters&quot;
xmlns:gml="http://www.opengis.net/gml&quot;
xmlns:Public_Data_DB="http://www.warwickshire.gov.uk/public_data_db&quot;
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance&quot; numberOfFeatures="0"
timeStamp="2013-11-08T14:55:55.783Z"
xsi:schemaLocation="http://www.opengis.net/wfs
http://schemas.opengis.net/wfs/1.1.0/wfs.xsd&quot;/&gt;

Note it's claimed 0 features. Despite the fact the request does return 7500
features (maximum). A bug? I've not used that before.

Cheers,
Jonathan

On 8 November 2013 14:34, Rahkonen Jukka
<jukka.rahkonen@anonymised.com<mailto:jukka.rahkonen@anonymised.com>> wrote:
Hi,

Not directly as far as I know, but by sending GetFeature without filters and
with resulttype=hits it should be possible to do a good guess.

-Jukka Rahkonen-

Jonathan Moules wrote:

Hi List,
Relatively simple question, is it possible to a client to discover what the
maximum number of results are that the GeoServer will return to a WFS request
if the user has set "Maximum number of features"?
It doesn't seem to be specified in the GetCapabilities explicitly. Closest I
can see is the WFS 2.0 "ows:Constraint name="CountDefault"

Does 1.0 have any way to find it out?

Thanks,
Jonathan

This transmission is intended for the named addressee(s) only and may contain
sensitive or protectively marked material up to RESTRICTED and should be
handled accordingly. Unless you are the named addressee (or authorised to
receive it for the addressee) you may not copy or use it, or disclose it to
anyone else. If you have received this transmission in error please notify
the sender immediately. All email traffic sent to or from us, including
without limitation all GCSX traffic, may be subject to recording and/or
monitoring in accordance with relevant legislation.

This transmission is intended for the named addressee(s) only and may contain
sensitive or protectively marked material up to RESTRICTED and should be
handled accordingly. Unless you are the named addressee (or authorised to
receive it for the addressee) you may not copy or use it, or disclose it to
anyone else. If you have received this transmission in error please notify
the sender immediately. All email traffic sent to or from us, including
without limitation all GCSX traffic, may be subject to recording and/or
monitoring in accordance with relevant legislation.

------------------------------------------------------------------------------
November Webinars for C, C++, Fortran Developers
Accelerate application performance with scalable programming models. Explore
techniques for threading, error checking, porting, and tuning. Get the most
from the latest Intel processors and coprocessors. See abstracts and register
http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk
_______________________________________________
Geoserver-users mailing list

Geoserver-users@lists.sourceforge.net<mailto:Geoserver-users@anonymised.comrge.net>

https://lists.sourceforge.net/lists/listinfo/geoserver-users

This transmission is intended for the named addressee(s) only and may contain
sensitive or protectively marked material up to RESTRICTED and should be
handled accordingly. Unless you are the named addressee (or authorised to
receive it for the addressee) you may not copy or use it, or disclose it to
anyone else. If you have received this transmission in error please notify
the sender immediately. All email traffic sent to or from us, including
without limitation all GCSX traffic, may be subject to recording and/or
monitoring in accordance with relevant legislation.

Hi Jukka,
Well it seems I’m a bit of a muppet. :slight_smile:

As noted, 1.0.0 returns "numberOfFeatures=“0"” (but the spec says it shouldn’t even exist).

And 1.1.0 works fine too.

2.0.0 does exactly what I want and isn’t buggy! numberReturned is 0 because it’s supposed to be!. I rather foolishly missed that there is a “numberMatched” response:

numberMatched=“253245” numberReturned=“0”

And this is ignoring the WFS maxFeatures. Which is good.

That also makes most sense for me because that way the client can make a test and re-think and adjust the request before sending the real GetFeature.

Actually the 2.0.0 spec says you can’t rely on numberMatched being the number of results:

The numberMatched parameter shall be used in a response document to report the total number of features or values, of the types requested in an operation, that are in the result set. This value does not necessarily have to match the number of feature or values actually presented in the response document (see 7.7.4.3).

This transmission is intended for the named addressee(s) only and may contain sensitive or protectively marked material up to RESTRICTED and should be handled accordingly. Unless you are the named addressee (or authorised to receive it for the addressee) you may not copy or use it, or disclose it to anyone else. If you have received this transmission in error please notify the sender immediately. All email traffic sent to or from us, including without limitation all GCSX traffic, may be subject to recording and/or monitoring in accordance with relevant legislation.

···

So I’ll go amend my bug.

Seems wiser minds were on it with WFS 2.0.0. But as ever they also decided to rename things. :slight_smile:

Cheers,
Jonathan

On 8 November 2013 15:53, Rahkonen Jukka <jukka.rahkonen@anonymised.com> wrote:

Hi,

The count of features received with resulttype=results should match the count in resulttype=hits. That also makes most sense for me because that way the client can make a test and re-think and adjust the request before sending the real GetFeature. However, often it would be useful to know also the total count of features in the service but that is missing from the WFS standard. In practice it means that for example GDAL when it reads the layer summary with ogrinfo and –so switch http://www.gdal.org/ogrinfo.html through WFS driver http://www.gdal.org/ogr/drv_wfs.html the utility actually reads the whole feature type. Or tries to read, it may hit the server side maxfeatures naturally.

-Jukka-

Jonathan Moules

Hi Jukka,

I see what the difference is after some testing. Using the exact same URL but only changing the WFS version:

http://wppgeog3/geoserver/ows?service=wfs&version=2.0.0&request=getfeature&typename=Public_Data_DB:OS_ADDPOINT_WSHIRE&resulttype=hits

WFS 1.0.0 - 0 (wrong)

WFS 1.1.0 - 7500 (correct)

WFS 2.0.0 - 0 (wrong)

Looking in the spec, 1.0.0 doesn’t seem to have “resulttype” (but Geoserver tries anyway). It’s in both 1.1.0 and 2.0.0 though.

So looks like a bug, reported as http://jira.codehaus.org/browse/GEOS-6145

=====

One thought - shouldn’t resulttype=hits return the actual number of features that satisfy the user-query, not the number that satisfy the query after GeoServer has added the maxfeatures to it?

So if the limit is 7500, and my database contains 10,000,000, shouldn’t it return 10,000,000?

The specs don’t say clearly one way of the other which it should be.

Cheers,

Jonathan

On 8 November 2013 15:13, Rahkonen Jukka <jukka.rahkonen@anonymised.com6…> wrote:

Hi Jonathan,

This is another WFS but like this it should work

Client side maxfeatures=10, numberOfFeatures=10
http://hip.latuviitta.org/cgi-bin/tinyows?service=wfs&version=1.1.0&maxfeatures=10&request=getfeature&typename=lv:mtk_tos_viiva&resulttype=hits

No client side maxfeatures, numberOfFeatures=100000, probably the server side limit

http://hip.latuviitta.org/cgi-bin/tinyows?service=wfs&version=1.1.0&max&request=getfeature&typename=lv:mtk_tos_viiva&resulttype=hits

-Jukka Rahkonen-

Jonathan Moules

Hi Jukka,

Ok. It just seems like the sort of thing that would be very important for the client to know so it can tell the user “Server returned 1000 features but the server will only ever return 1000 features at a time so this may not be the entire dataset”.

Testing your suggestion, whenever I do that I always get back the exact same:

<wfs:FeatureCollection xmlns:OS_Rasters=“http://www.warwickshire.gov.uk/os_rasters” xmlns:ogc=“http://www.opengis.net/ogc” xmlns:Secure_Community=“http://www.warwickshire.gov.uk/Secure_Community” xmlns:OS_Vector=“http://www.warwickshire.gov.uk/os_vector” xmlns:wfs=“http://www.opengis.net/wfs” xmlns:ows=“http://www.opengis.net/ows” xmlns:xlink=“http://www.w3.org/1999/xlink” xmlns:Misc_Rasters=“http://www.warwickshire.gov.uk/misc_rasters” xmlns:gml=“http://www.opengis.net/gml” xmlns:Public_Data_DB=“http://www.warwickshire.gov.uk/public_data_db” xmlns:xsi=“http://www.w3.org/2001/XMLSchema-instance” numberOfFeatures=“0” timeStamp=“2013-11-08T14:55:55.783Z” xsi:schemaLocation=“http://www.opengis.net/wfs http://schemas.opengis.net/wfs/1.1.0/wfs.xsd”/>

Note it’s claimed 0 features. Despite the fact the request does return 7500 features (maximum). A bug? I’ve not used that before.

Cheers,

Jonathan

On 8 November 2013 14:34, Rahkonen Jukka <jukka.rahkonen@anonymised.com6…> wrote:

Hi,

Not directly as far as I know, but by sending GetFeature without filters and with resulttype=hits it should be possible to do a good guess.

-Jukka Rahkonen-

Jonathan Moules wrote:

Hi List,

Relatively simple question, is it possible to a client to discover what the maximum number of results are that the GeoServer will return to a WFS request if the user has set “Maximum number of features”?

It doesn’t seem to be specified in the GetCapabilities explicitly. Closest I can see is the WFS 2.0 "ows:Constraint name=“CountDefault”

Does 1.0 have any way to find it out?

Thanks,

Jonathan

This transmission is intended for the named addressee(s) only and may contain sensitive or protectively marked material up to RESTRICTED and should be handled accordingly. Unless you are the named addressee (or authorised to receive it for the addressee) you may not copy or use it, or disclose it to anyone else. If you have received this transmission in error please notify the sender immediately. All email traffic sent to or from us, including without limitation all GCSX traffic, may be subject to recording and/or monitoring in accordance with relevant legislation.

This transmission is intended for the named addressee(s) only and may contain sensitive or protectively marked material up to RESTRICTED and should be handled accordingly. Unless you are the named addressee (or authorised to receive it for the addressee) you may not copy or use it, or disclose it to anyone else. If you have received this transmission in error please notify the sender immediately. All email traffic sent to or from us, including without limitation all GCSX traffic, may be subject to recording and/or monitoring in accordance with relevant legislation.


November Webinars for C, C++, Fortran Developers
Accelerate application performance with scalable programming models. Explore
techniques for threading, error checking, porting, and tuning. Get the most
from the latest Intel processors and coprocessors. See abstracts and register
http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk


Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

This transmission is intended for the named addressee(s) only and may contain sensitive or protectively marked material up to RESTRICTED and should be handled accordingly. Unless you are the named addressee (or authorised to receive it for the addressee) you may not copy or use it, or disclose it to anyone else. If you have received this transmission in error please notify the sender immediately. All email traffic sent to or from us, including without limitation all GCSX traffic, may be subject to recording and/or monitoring in accordance with relevant legislation.


November Webinars for C, C++, Fortran Developers
Accelerate application performance with scalable programming models. Explore
techniques for threading, error checking, porting, and tuning. Get the most
from the latest Intel processors and coprocessors. See abstracts and register
http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk


Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users