[Geoserver-users] Problems in paging WFS on Geoserver 2.5

Hi

The data store is a “Directory of spatial files (shapefiles)”

Med vänlig hälsning

Anders Erlandsson

Systemförvaltare

Mobil: 073-398 03 70

···

Hi,

one question, what data store is providing this data?

Cheers

Andrea

On Wed, Nov 12, 2014 at 2:46 PM, Rahkonen Jukka (Tike) <[hidden email]> wrote:

Hi,

Something makes me think that this is a bug and it has to do with sorting.

Without SortBy the first features which are returned have fids 1, 10, 100, 1000, 1001, 101,

https://extmap.sundsvall.se/geoserver/wfs?service=WFS&version=2.0.0&request=GetFeature&typeNames=SundsvallsKommun:DetaljplanGallande_yta&count=10&startindex=0

When startindex is 1000 the first fid is .999 and when the resultset is ordered with this logic there are no higher fids than 999 in the data. Fid:1000 was already the 4th record.

I do not know if this makes sense or not, just trying to figure what could make Geoserver to behave like I see it behaves.

-Jukka-

Rahkonen Jukka wrote:

Corrent one with ”count” makes no difference

https://extmap.sundsvall.se/geoserver/wfs?service=WFS&version=2.0.0&request=GetFeature&typeNames=SundsvallsKommun:DetaljplanGallande_yta&count=50&StartIndex=1000&sortby=URSPRUNG

-Jukka-

Lähettäjä: Rahkonen Jukka (Tike)
Lähetetty: 12. marraskuuta 2014 15:28
Vastaanottaja: ‘sbkkae’
Kopio: ‘GeoServer Mailing List List’
Aihe: Re: [Geoserver-users] Problems in paging WFS on Geoserver 2.5

Hi,

I hoped too that it would be so easy but it was not.

Also typename → typenames in WFS 2.0.0 but here with KVP Geoserver is flexible and both typename and typenames is accepted. Another thing that I am remembering is that paging is not reliable without sorting, if my memory serves me right. Then there should be SortBy with some unique field included but perhaps that is needed only if there are some filters involved as well.

Anyway, this does not work either.

https://extmap.sundsvall.se/geoserver/wfs?service=WFS&version=2.0.0&request=GetFeature&typeNames=SundsvallsKommun:DetaljplanGallande_yta&maxfeatures=50&StartIndex=1000&sortby=URSPRUNG

I can also see that number of returned features is not correct, it shows always the total count

numberMatched=“1024” numberReturned=“1024”

-Jukka Rahkonen-

Andrea Aime wrote:

Hi,

if memory serves me right, WFS 2.0 changed the name of the paging paramters compared to 1.1.0, they are now startIndex and count.

Please search for the WFS 2.0 spec on the OGC site to confirm.

Cheers

Andrea

On Wed, Nov 12, 2014 at 1:42 PM, sbkkae <[hidden email]> wrote:

Hi
I’ve got a dataset set up as a layer, containing 1024 features.
When calling WFS-service for the layer without paging all objects get
returned:
https://extmap.sundsvall.se/geoserver/wfs?service=WFS&version=2.0.0&request=GetFeature&typeName=SundsvallsKommun:DetaljplanGallande_yta&outputFormat=json

When paging these features using WFS it returns only the 1000 first:
https://extmap.sundsvall.se/geoserver/wfs?service=WFS&version=2.0.0&request=GetFeature&typeName=SundsvallsKommun:DetaljplanGallande_yta&outputFormat=json&maxFeatures=50&startIndex=1000

Anyone who have had the same problem?

Regards
Anders Erlandsson
Sundsvalls Kommun
Sweden


View this message in context: http://osgeo-org.1560.x6.nabble.com/Problems-in-paging-WFS-on-Geoserver-2-5-tp5172562.html
Sent from the GeoServer - User mailing list archive at Nabble.com.


Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://pubads.g.doubleclick.net/gampad/clk?id=154624111&iu=/4140/ostg.clktrk


Geoserver-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geoserver-users

==

GeoServer Professional Services from the experts! Visit

http://goo.gl/NWWaa2 for more information.

==

Ing. Andrea Aime

@geowolf

Technical Lead

GeoSolutions S.A.S.

Via Poggio alle Viti 1187

55054 Massarosa (LU)

Italy

phone: <a href=“tel:%2B39%200584%20962313” value=“+390584962313” target=“_blank”>+39 0584 962313

fax: <a href=“tel:%2B39%200584%201660272” value=“+3905841660272” target=“_blank”>+39 0584 1660272

mob: <a href=“tel:%2B39%20%C2%A0339%208844549” value=“+393398844549” target=“_blank”>+39 339 8844549

http://www.geo-solutions.it

http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.



Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://pubads.g.doubleclick.net/gampad/clk?id=154624111&iu=/4140/ostg.clktrk


Geoserver-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geoserver-users

==

GeoServer Professional Services from the experts! Visit

http://goo.gl/NWWaa2 for more information.

==

Ing. Andrea Aime

@geowolf

Technical Lead

GeoSolutions S.A.S.

Via Poggio alle Viti 1187

55054 Massarosa (LU)

Italy

phone: +39 0584 962313

fax: +39 0584 1660272

mob: +39 339 8844549

http://www.geo-solutions.it

http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.



Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://pubads.g.doubleclick.net/gampad/clk?id=154624111&iu=/4140/ostg.clktrk


Geoserver-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geoserver-users


If you reply to this email, your message will be added to the discussion below:

http://osgeo-org.1560.x6.nabble.com/Problems-in-paging-WFS-on-Geoserver-2-5-tp5172562p5172593.html

To unsubscribe from Problems in paging WFS on Geoserver 2.5, click here.
NAML

Hello,

I downloaded the plans as shape-zip and debugged against current master.

The problem with “strange” fids seems to be due to the fact that if you provide an offset (startIndex) but no sorting a natural sort is added.
If I alter the query (in Eclipse with Expressions) and remove the sorting I get the “correct” fids.
https://github.com/geotools/geotools/blob/master/modules/library/data/src/main/java/org/geotools/data/store/ContentFeatureSource.java#L550-L555

The number of returned features seems to be wrong because of this line:
https://github.com/geoserver/geoserver/blob/master/src/wfs/src/main/java/org/geoserver/wfs/GetFeature.java#L517
Changing it to count = maxFeatures; fixed the response.
Then the numberReturned is correct and the correct links for previous and next are generated for GML3OutputFormat (next is wrong now).
The totalFeatures or numberMatched is still 1024 though.
Perhaps it is the block above that should be changed to handle the maxFeatures case?
https://github.com/geoserver/geoserver/blob/master/src/wfs/src/main/java/org/geoserver/wfs/GetFeature.java#L512-L514

/Olle

···

On Wed, Nov 12, 2014 at 3:17 PM, sbkkae <anders.erlandsson@anonymised.com> wrote:

Hi

The data store is a “Directory of spatial files (shapefiles)”

Med vänlig hälsning

Anders Erlandsson

Systemförvaltare

Mobil: 073-398 03 70

Från: geowolf [via OSGeo.org] [mailto:ml-node+[hidden email]]
Skickat: den 12 november 2014 15:13
Till: Erlandsson Anders
Ämne: Re: Problems in paging WFS on Geoserver 2.5

Hi,

one question, what data store is providing this data?

Cheers

Andrea

On Wed, Nov 12, 2014 at 2:46 PM, Rahkonen Jukka (Tike) <[hidden email]> wrote:

Hi,

Something makes me think that this is a bug and it has to do with sorting.

Without SortBy the first features which are returned have fids 1, 10, 100, 1000, 1001, 101,

https://extmap.sundsvall.se/geoserver/wfs?service=WFS&version=2.0.0&request=GetFeature&typeNames=SundsvallsKommun:DetaljplanGallande_yta&count=10&startindex=0

When startindex is 1000 the first fid is .999 and when the resultset is ordered with this logic there are no higher fids than 999 in the data. Fid:1000 was already the 4th record.

I do not know if this makes sense or not, just trying to figure what could make Geoserver to behave like I see it behaves.

-Jukka-

Rahkonen Jukka wrote:

Corrent one with ”count” makes no difference

https://extmap.sundsvall.se/geoserver/wfs?service=WFS&version=2.0.0&request=GetFeature&typeNames=SundsvallsKommun:DetaljplanGallande_yta&count=50&StartIndex=1000&sortby=URSPRUNG

-Jukka-

Lähettäjä: Rahkonen Jukka (Tike)
Lähetetty: 12. marraskuuta 2014 15:28
Vastaanottaja: ‘sbkkae’
Kopio: ‘GeoServer Mailing List List’
Aihe: Re: [Geoserver-users] Problems in paging WFS on Geoserver 2.5

Hi,

I hoped too that it would be so easy but it was not.

Also typename → typenames in WFS 2.0.0 but here with KVP Geoserver is flexible and both typename and typenames is accepted. Another thing that I am remembering is that paging is not reliable without sorting, if my memory serves me right. Then there should be SortBy with some unique field included but perhaps that is needed only if there are some filters involved as well.

Anyway, this does not work either.

https://extmap.sundsvall.se/geoserver/wfs?service=WFS&version=2.0.0&request=GetFeature&typeNames=SundsvallsKommun:DetaljplanGallande_yta&maxfeatures=50&StartIndex=1000&sortby=URSPRUNG

I can also see that number of returned features is not correct, it shows always the total count

numberMatched=“1024” numberReturned=“1024”

-Jukka Rahkonen-

Andrea Aime wrote:

Hi,

if memory serves me right, WFS 2.0 changed the name of the paging paramters compared to 1.1.0, they are now startIndex and count.

Please search for the WFS 2.0 spec on the OGC site to confirm.

Cheers

Andrea

On Wed, Nov 12, 2014 at 1:42 PM, sbkkae <[hidden email]> wrote:

Hi
I’ve got a dataset set up as a layer, containing 1024 features.
When calling WFS-service for the layer without paging all objects get
returned:
https://extmap.sundsvall.se/geoserver/wfs?service=WFS&version=2.0.0&request=GetFeature&typeName=SundsvallsKommun:DetaljplanGallande_yta&outputFormat=json

When paging these features using WFS it returns only the 1000 first:
https://extmap.sundsvall.se/geoserver/wfs?service=WFS&version=2.0.0&request=GetFeature&typeName=SundsvallsKommun:DetaljplanGallande_yta&outputFormat=json&maxFeatures=50&startIndex=1000

Anyone who have had the same problem?

Regards
Anders Erlandsson
Sundsvalls Kommun
Sweden


View this message in context: http://osgeo-org.1560.x6.nabble.com/Problems-in-paging-WFS-on-Geoserver-2-5-tp5172562.html
Sent from the GeoServer - User mailing list archive at Nabble.com.


Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://pubads.g.doubleclick.net/gampad/clk?id=154624111&iu=/4140/ostg.clktrk


Geoserver-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geoserver-users

==

GeoServer Professional Services from the experts! Visit

http://goo.gl/NWWaa2 for more information.

==

Ing. Andrea Aime

@geowolf

Technical Lead

GeoSolutions S.A.S.

Via Poggio alle Viti 1187

55054 Massarosa (LU)

Italy

phone: <a href=“tel:%2B39%200584%20962313” value=“+390584962313” target=“_blank”>+39 0584 962313

fax: <a href=“tel:%2B39%200584%201660272” value=“+3905841660272” target=“_blank”>+39 0584 1660272

mob: <a href=“tel:%2B39%20%C2%A0339%208844549” value=“+393398844549” target=“_blank”>+39 339 8844549

http://www.geo-solutions.it

http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.



Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://pubads.g.doubleclick.net/gampad/clk?id=154624111&iu=/4140/ostg.clktrk


Geoserver-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geoserver-users

==

GeoServer Professional Services from the experts! Visit

http://goo.gl/NWWaa2 for more information.

==

Ing. Andrea Aime

@geowolf

Technical Lead

GeoSolutions S.A.S.

Via Poggio alle Viti 1187

55054 Massarosa (LU)

Italy

phone: +39 0584 962313

fax: +39 0584 1660272

mob: +39 339 8844549

http://www.geo-solutions.it

http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.



Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://pubads.g.doubleclick.net/gampad/clk?id=154624111&iu=/4140/ostg.clktrk


Geoserver-users mailing list

[hidden email]
https://lists.sourceforge.net/lists/listinfo/geoserver-users


If you reply to this email, your message will be added to the discussion below:

http://osgeo-org.1560.x6.nabble.com/Problems-in-paging-WFS-on-Geoserver-2-5-tp5172562p5172593.html

To unsubscribe from Problems in paging WFS on Geoserver 2.5, click here.
NAML


View this message in context: SV: Problems in paging WFS on Geoserver 2.5

Sent from the GeoServer - User mailing list archive at Nabble.com.


Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://pubads.g.doubleclick.net/gampad/clk?id=154624111&iu=/4140/ostg.clktrk


Geoserver-users mailing list
Geoserver-users@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users