[Geoserver-devel] wfs and paging

GeoServer GeoServer Developer
1

Hi all,

As most know the next version of the wfs spec (2.0) brings paging into the mix which just boils down to the parameters startIndex and count on a GetFeature request. While it looks quite promising that wfs 2.0 will becoming to geoserver in the next few weeks / months, we have a client currently that requires paging functionality via WFS. So I have put together a patch that implements paging for all wfs versions.

http://jira.codehaus.org/browse/GEOS-4485

Feedback welcome.

Obviously I won’t be submitting this patch to the stable branch before 2.1 goes out the door but I would be curious to hear how people feel about backporting it post 2.1.

Thanks.

-Justin

–
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

2

On Wed, Apr 13, 2011 at 3:55 AM, Justin Deoliveira <jdeolive@anonymised.com> wrote:

Hi all,
As most know the next version of the wfs spec (2.0) brings paging into the
mix which just boils down to the parameters startIndex and count on a
GetFeature request. While it looks quite promising that wfs 2.0 will
becoming to geoserver in the next few weeks / months, we have a client
currently that requires paging functionality via WFS. So I have put together
a patch that implements paging for all wfs versions.
http://jira.codehaus.org/browse/GEOS-4485
Feedback welcome.

Hi Justin,
I did not review deeply, just had a quick walk over the patch sources, but it
looks pretty much like what I was expecting (one tricky part that we might
want to double check, maybe with some more tests, it's the wfs 1.0 feature
counting).

So I'm wondering about count/maxFeatures. Are they basically going to be
aliases with the same meaning?
So that one can do
...&startIndex=10&maxFeatures=5&...
or
...&startIndex=10&count=5&...
and the result will be the same?

I'm also wondering about paging stability.
The current setup will return incoherent results if someone deletes
or updates a feature while we page. Not complaining, doing a full dump
of the results just for the sake of paging in a stable way would be a massive
overhead, just wondering if the problem had been considered.

Also curious about sorting... I remember something like sorting by default
on the feature id while paging, but my memories of it are foggy, we discussed
this with Gabriel a loong time ago (when the paging machinery was added to
GeoTools).

Cheers
Andrea

--
-------------------------------------------------------
Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 333 8128928

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

-------------------------------------------------------

3

Hi Andrea, thanks for the feedback, comments inline.

On Wed, Apr 13, 2011 at 12:54 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Wed, Apr 13, 2011 at 3:55 AM, Justin Deoliveira <jdeolive@anonymised.com> wrote:

Hi all,
As most know the next version of the wfs spec (2.0) brings paging into the
mix which just boils down to the parameters startIndex and count on a
GetFeature request. While it looks quite promising that wfs 2.0 will
becoming to geoserver in the next few weeks / months, we have a client
currently that requires paging functionality via WFS. So I have put together
a patch that implements paging for all wfs versions.
http://jira.codehaus.org/browse/GEOS-4485
Feedback welcome.

Hi Justin,
I did not review deeply, just had a quick walk over the patch sources, but it
looks pretty much like what I was expecting (one tricky part that we might
want to double check, maybe with some more tests, it’s the wfs 1.0 feature
counting).

Sure thing… can you be more specific about what cases to test? Most of the test cases there are wfs 1.0 with a mix of startIndex and maxFeatures. I tried to catch most of the boundary cases but probably missed some.

So I’m wondering about count/maxFeatures. Are they basically going to be
aliases with the same meaning?
So that one can do
…&startIndex=10&maxFeatures=5&…
or
…&startIndex=10&count=5&…
and the result will be the same?

That is the general idea yeah. The patch as written should do that for the GET case. For the POST case count is not yet handled… i was going to put that off until wfs 2.0 starts.

I’m also wondering about paging stability.
The current setup will return incoherent results if someone deletes
or updates a feature while we page. Not complaining, doing a full dump
of the results just for the sake of paging in a stable way would be a massive
overhead, just wondering if the problem had been considered.

Yeah I did consider this after reading the spec because technically this is a requirement of spec on the server. But seems quite unrealistic to me since as you say we would have to maintain some sort of cache or something on the server. It would make the WFS GetFeature a stateful operation… so I imagine we would need so start creating sessions to track the request or something. All in all I think just having this be a “limitation” of the implementation is probably going to suffice in 99% of cases.

Also it seems that WMS GetMap already supports paging… and this is more or less following suite what it is doing.

Also curious about sorting… I remember something like sorting by default
on the feature id while paging, but my memories of it are foggy, we discussed
this with Gabriel a loong time ago (when the paging machinery was added to
GeoTools).

As I understand things how they are implemented now using startIndex requires the underlying datastore to be able to do sorting. And when the client does not specify an explicit attribute to sort on this means doing a natural sort (feature id). If you look at ContentFeatureSource.getReader(Query) you will see a check there.

Which more or less means that paging can only be used with jdbc datastores. Which personally I think is fine since they are the only ones that can really do it efficiently. Thoughts on that?

Cheers
Andrea

–

Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 333 8128928

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

–
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

4

On 4/13/11 8:35 AM, Justin Deoliveira wrote:

Hi Andrea, thanks for the feedback, comments inline.

On Wed, Apr 13, 2011 at 12:54 AM, Andrea Aime
<andrea.aime@anonymised.com <mailto:andrea.aime@anonymised.com>> wrote:

    On Wed, Apr 13, 2011 at 3:55 AM, Justin Deoliveira
    <jdeolive@anonymised.com <mailto:jdeolive@anonymised.com>> wrote:
     > Hi all,
     > As most know the next version of the wfs spec (2.0) brings paging
    into the
     > mix which just boils down to the parameters startIndex and count on a
     > GetFeature request. While it looks quite promising that wfs 2.0 will
     > becoming to geoserver in the next few weeks / months, we have a
    client
     > currently that requires paging functionality via WFS. So I have
    put together
     > a patch that implements paging for all wfs versions.
     > http://jira.codehaus.org/browse/GEOS-4485
     > Feedback welcome.

    Hi Justin,
    I did not review deeply, just had a quick walk over the patch
    sources, but it
    looks pretty much like what I was expecting (one tricky part that we
    might
    want to double check, maybe with some more tests, it's the wfs 1.0
    feature
    counting).

Sure thing... can you be more specific about what cases to test? Most of
the test cases there are wfs 1.0 with a mix of startIndex and
maxFeatures. I tried to catch most of the boundary cases but probably
missed some.

    So I'm wondering about count/maxFeatures. Are they basically going to be
    aliases with the same meaning?
    So that one can do
    ...&startIndex=10&maxFeatures=5&...
    or
    ...&startIndex=10&count=5&...
    and the result will be the same?

That is the general idea yeah. The patch as written should do that for
the GET case. For the POST case count is not yet handled... i was going
to put that off until wfs 2.0 starts.

    I'm also wondering about paging stability.
    The current setup will return incoherent results if someone deletes
    or updates a feature while we page. Not complaining, doing a full dump
    of the results just for the sake of paging in a stable way would be
    a massive
    overhead, just wondering if the problem had been considered.

Yeah I did consider this after reading the spec because technically this
is a requirement of spec on the server. But seems quite unrealistic to
me since as you say we would have to maintain some sort of cache or
something on the server. It would make the WFS GetFeature a stateful
operation... so I imagine we would need so start creating sessions to
track the request or something. All in all I think just having this be a
"limitation" of the implementation is probably going to suffice in 99%
of cases.

WFS 2.0 describes both paging with and without transactional consistency. It's not a requirement to be consistent there even (PagingIsTransactionSafe can be true or false), right?

Also it seems that WMS GetMap already supports paging... and this is
more or less following suite what it is doing.

    Also curious about sorting... I remember something like sorting by
    default
    on the feature id while paging, but my memories of it are foggy, we
    discussed
    this with Gabriel a loong time ago (when the paging machinery was
    added to
    GeoTools).

As I understand things how they are implemented now using startIndex
requires the underlying datastore to be able to do sorting. And when the
client does not specify an explicit attribute to sort on this means
doing a natural sort (feature id). If you look at
ContentFeatureSource.getReader(Query) you will see a check there.

Which more or less means that paging can only be used with jdbc
datastores. Which personally I think is fine since they are the only
ones that can really do it efficiently. Thoughts on that?

Would it be possible (later) to do sorting for all stores? I understand it would be inefficient, but it's a bigger drawback (in my opinion) to have a feature be store specific. As the client has no way to know about these distinctions.

    Cheers
    Andrea

    --
    -------------------------------------------------------
    Ing. Andrea Aime
    GeoSolutions S.A.S.
    Tech lead

    Via Poggio alle Viti 1187
    55054 Massarosa (LU)
    Italy

    phone: +39 0584 962313
    fax: +39 0584 962313
    mob: +39 333 8128928

    http://www.geo-solutions.it
    http://geo-solutions.blogspot.com/
    http://www.youtube.com/user/GeoSolutionsIT
    http://www.linkedin.com/in/andreaaime
    http://twitter.com/geowolf

    -------------------------------------------------------

--
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

------------------------------------------------------------------------------
Forrester Wave Report - Recovery time is now measured in hours and minutes
not days. Key insights are discussed in the 2010 Forrester Wave Report as
part of an in-depth evaluation of disaster recovery service providers.
Forrester found the best-in-class provider in terms of services and vision.
Read this report now! http://p.sf.net/sfu/ibm-webcastpromo

_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

--
Tim Schaub
OpenGeo - http://opengeo.org
Expert service straight from the developers.

5

On Thu, Apr 14, 2011 at 2:20 AM, Tim Schaub <tschaub@anonymised.com> wrote:

WFS 2.0 describes both paging with and without transactional
consistency. It's not a requirement to be consistent there even
(PagingIsTransactionSafe can be true or false), right?

Also it seems that WMS GetMap already supports paging... and this is
more or less following suite what it is doing.

Also curious about sorting\.\.\. I remember something like sorting by
default
on the feature id while paging, but my memories of it are foggy, we
discussed
this with Gabriel a loong time ago \(when the paging machinery was
added to
GeoTools\)\.

As I understand things how they are implemented now using startIndex
requires the underlying datastore to be able to do sorting. And when the
client does not specify an explicit attribute to sort on this means
doing a natural sort (feature id). If you look at
ContentFeatureSource.getReader(Query) you will see a check there.

Which more or less means that paging can only be used with jdbc
datastores. Which personally I think is fine since they are the only
ones that can really do it efficiently. Thoughts on that?

Would it be possible (later) to do sorting for all stores? I understand
it would be inefficient, but it's a bigger drawback (in my opinion) to
have a feature be store specific. As the client has no way to know
about these distinctions.

A trivial way to implement sorting is to do it in memory. The day someone
decides to sort on that 2GB shapefile boom!
We'd need a secondary storage sort approach (mergesort), writing one would
require quite a bit of time and the assurance everything in the feature is
serializable.
Another approach would be to have a hsql db, dump everything in it, and
make it do the sort.

In both cases it is to be noted that we first have to sort the large collection
and then extract the page of data we need, so in the case of the large shapefile
you'd have to first sort everything in order to get your 50 records page.
Not efficient... (a db could do that efficiently only if it has a
index on the sorting
column, something that is normally available for the primary key)

Cheers
Andrea

--
-------------------------------------------------------
Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 333 8128928

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

-------------------------------------------------------

6

On Thu, Apr 14, 2011 at 12:29 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Thu, Apr 14, 2011 at 2:20 AM, Tim Schaub <tschaub@anonymised.com> wrote:

WFS 2.0 describes both paging with and without transactional
consistency. It’s not a requirement to be consistent there even
(PagingIsTransactionSafe can be true or false), right?

Also it seems that WMS GetMap already supports paging… and this is
more or less following suite what it is doing.

Also curious about sorting… I remember something like sorting by
default
on the feature id while paging, but my memories of it are foggy, we
discussed
this with Gabriel a loong time ago (when the paging machinery was
added to
GeoTools).

As I understand things how they are implemented now using startIndex
requires the underlying datastore to be able to do sorting. And when the
client does not specify an explicit attribute to sort on this means
doing a natural sort (feature id). If you look at
ContentFeatureSource.getReader(Query) you will see a check there.

Which more or less means that paging can only be used with jdbc
datastores. Which personally I think is fine since they are the only
ones that can really do it efficiently. Thoughts on that?

Would it be possible (later) to do sorting for all stores? I understand
it would be inefficient, but it’s a bigger drawback (in my opinion) to
have a feature be store specific. As the client has no way to know
about these distinctions.

A trivial way to implement sorting is to do it in memory. The day someone
decides to sort on that 2GB shapefile boom!
We’d need a secondary storage sort approach (mergesort), writing one would
require quite a bit of time and the assurance everything in the feature is
serializable.
Another approach would be to have a hsql db, dump everything in it, and
make it do the sort.

In both cases it is to be noted that we first have to sort the large collection
and then extract the page of data we need, so in the case of the large shapefile
you’d have to first sort everything in order to get your 50 records page.
Not efficient… (a db could do that efficiently only if it has a
index on the sorting
column, something that is normally available for the primary key)

A silly idea. What if we didn’t sort in these cases. We just applied the offset with a wrapper that skipped the first n entries. And the limitation is that there no notion of stable at all unless you are using a database rather than throwing an exception if your not using a database. It would allow those sources that do provide a stable iteration order (but not necessarily capable of native sorting) to work as well.

Not sure which one is better… fail fast or try to do something. I sort of draw a parallel to transactions in which again unless you are using a database you can’t really do transactions well… but we at least try with the other data stores.

Cheers
Andrea

–

Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 333 8128928

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

Benefiting from Server Virtualization: Beyond Initial Workload
Consolidation – Increasing the use of server virtualization is a top
priority.Virtualization can reduce costs, simplify management, and improve
application availability and disaster protection. Learn more about boosting
the value of server virtualization. http://p.sf.net/sfu/vmware-sfdev2dev

Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

–
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

7

On 4/14/11 12:29 AM, Andrea Aime wrote:

On Thu, Apr 14, 2011 at 2:20 AM, Tim Schaub<tschaub@anonymised.com> wrote:

WFS 2.0 describes both paging with and without transactional
consistency. It's not a requirement to be consistent there even
(PagingIsTransactionSafe can be true or false), right?

Also it seems that WMS GetMap already supports paging... and this is
more or less following suite what it is doing.

     Also curious about sorting... I remember something like sorting by
     default
     on the feature id while paging, but my memories of it are foggy, we
     discussed
     this with Gabriel a loong time ago (when the paging machinery was
     added to
     GeoTools).

As I understand things how they are implemented now using startIndex
requires the underlying datastore to be able to do sorting. And when the
client does not specify an explicit attribute to sort on this means
doing a natural sort (feature id). If you look at
ContentFeatureSource.getReader(Query) you will see a check there.

Which more or less means that paging can only be used with jdbc
datastores. Which personally I think is fine since they are the only
ones that can really do it efficiently. Thoughts on that?

Would it be possible (later) to do sorting for all stores? I understand
it would be inefficient, but it's a bigger drawback (in my opinion) to
have a feature be store specific. As the client has no way to know
about these distinctions.

Sorry, I meant to ask about *paging* on all stores.

A trivial way to implement sorting is to do it in memory. The day someone
decides to sort on that 2GB shapefile boom!
We'd need a secondary storage sort approach (mergesort), writing one would
require quite a bit of time and the assurance everything in the feature is
serializable.
Another approach would be to have a hsql db, dump everything in it, and
make it do the sort.

In both cases it is to be noted that we first have to sort the large collection
and then extract the page of data we need, so in the case of the large shapefile
you'd have to first sort everything in order to get your 50 records page.
Not efficient... (a db could do that efficiently only if it has a
index on the sorting
column, something that is normally available for the primary key)

Cheers
Andrea

--
Tim Schaub
OpenGeo - http://opengeo.org
Expert service straight from the developers.

8

On 4/14/11 10:51 AM, Tim Schaub wrote:

On 4/14/11 12:29 AM, Andrea Aime wrote:

On Thu, Apr 14, 2011 at 2:20 AM, Tim Schaub<tschaub@anonymised.com> wrote:

WFS 2.0 describes both paging with and without transactional
consistency. It's not a requirement to be consistent there even
(PagingIsTransactionSafe can be true or false), right?

Also it seems that WMS GetMap already supports paging... and this is
more or less following suite what it is doing.

Also curious about sorting... I remember something like sorting by
default
on the feature id while paging, but my memories of it are foggy, we
discussed
this with Gabriel a loong time ago (when the paging machinery was
added to
GeoTools).

As I understand things how they are implemented now using startIndex
requires the underlying datastore to be able to do sorting. And when
the
client does not specify an explicit attribute to sort on this means
doing a natural sort (feature id). If you look at
ContentFeatureSource.getReader(Query) you will see a check there.

Which more or less means that paging can only be used with jdbc
datastores. Which personally I think is fine since they are the only
ones that can really do it efficiently. Thoughts on that?

Would it be possible (later) to do sorting for all stores? I understand
it would be inefficient, but it's a bigger drawback (in my opinion) to
have a feature be store specific. As the client has no way to know
about these distinctions.

Sorry, I meant to ask about *paging* on all stores.

And I should add, I know it is less efficient for the server to do paging for stores that don't support it. But in my mind, paging is about easing the burden on the client.

A response set with a ton of records may be a lot for the server to handle, but resources are typically more limited on the client. Paging manually would only add minor work on the server (as I imagine it), but it is a huge relief in terms of the burden on the client.

A trivial way to implement sorting is to do it in memory. The day someone
decides to sort on that 2GB shapefile boom!
We'd need a secondary storage sort approach (mergesort), writing one
would
require quite a bit of time and the assurance everything in the
feature is
serializable.
Another approach would be to have a hsql db, dump everything in it, and
make it do the sort.

In both cases it is to be noted that we first have to sort the large
collection
and then extract the page of data we need, so in the case of the large
shapefile
you'd have to first sort everything in order to get your 50 records page.
Not efficient... (a db could do that efficiently only if it has a
index on the sorting
column, something that is normally available for the primary key)

Cheers
Andrea

--
Tim Schaub
OpenGeo - http://opengeo.org
Expert service straight from the developers.

9

On Thu, Apr 14, 2011 at 10:51 AM, Tim Schaub <tschaub@anonymised.com.1501…> wrote:

On 4/14/11 12:29 AM, Andrea Aime wrote:

On Thu, Apr 14, 2011 at 2:20 AM, Tim Schaub<tschaub@anonymised.com> wrote:

WFS 2.0 describes both paging with and without transactional
consistency. It’s not a requirement to be consistent there even
(PagingIsTransactionSafe can be true or false), right?

Also it seems that WMS GetMap already supports paging… and this is
more or less following suite what it is doing.

Also curious about sorting… I remember something like sorting by
default
on the feature id while paging, but my memories of it are foggy, we
discussed
this with Gabriel a loong time ago (when the paging machinery was
added to
GeoTools).

As I understand things how they are implemented now using startIndex
requires the underlying datastore to be able to do sorting. And when the
client does not specify an explicit attribute to sort on this means
doing a natural sort (feature id). If you look at
ContentFeatureSource.getReader(Query) you will see a check there.

Which more or less means that paging can only be used with jdbc
datastores. Which personally I think is fine since they are the only
ones that can really do it efficiently. Thoughts on that?

Would it be possible (later) to do sorting for all stores? I understand
it would be inefficient, but it’s a bigger drawback (in my opinion) to
have a feature be store specific. As the client has no way to know
about these distinctions.

Sorry, I meant to ask about paging on all stores.

Right, but to do paging consistently implicitly requires sorting since you can’t rely 100% of the time on a data source returning features in a consistent ordering. Or am I missing something?

A trivial way to implement sorting is to do it in memory. The day someone
decides to sort on that 2GB shapefile boom!
We’d need a secondary storage sort approach (mergesort), writing one would
require quite a bit of time and the assurance everything in the feature is
serializable.
Another approach would be to have a hsql db, dump everything in it, and
make it do the sort.

In both cases it is to be noted that we first have to sort the large collection
and then extract the page of data we need, so in the case of the large shapefile
you’d have to first sort everything in order to get your 50 records page.
Not efficient… (a db could do that efficiently only if it has a
index on the sorting
column, something that is normally available for the primary key)

Cheers
Andrea

–

Tim Schaub
OpenGeo - http://opengeo.org
Expert service straight from the developers.

Benefiting from Server Virtualization: Beyond Initial Workload
Consolidation – Increasing the use of server virtualization is a top
priority.Virtualization can reduce costs, simplify management, and improve
application availability and disaster protection. Learn more about boosting
the value of server virtualization. http://p.sf.net/sfu/vmware-sfdev2dev

Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

–
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

10

On Thu, Apr 14, 2011 at 6:54 PM, Tim Schaub <tschaub@anonymised.com> wrote:

Sorry, I meant to ask about paging on all stores.

And I should add, I know it is less efficient for the server to do
paging for stores that don’t support it. But in my mind, paging is
about easing the burden on the client.

A response set with a ton of records may be a lot for the server to
handle, but resources are typically more limited on the client. Paging
manually would only add minor work on the server (as I imagine it), but
it is a huge relief in terms of the burden on the client.

Nevertheless we have to be very careful. We have received funding to avoid
counting records in WFS responses, because that takes a ton of time
if you have a 50 millions records in a Oracle database.
Yet that is a quite light operation compared to paging and sorting.

You should consider three orders of magnitude:

  • the size that a javascript client can handle in a reasonable time
  • the size that a server can handle in a reasonable time
  • the size that a large public administrator is mandated to handle, one
    that would bring to his knees even a server if not properly handled

So I’m ok with allow administrators to give full sorting/paging support to
whatever store, but we have to be careful that this will be either
always a light operation, otherwise we have to provide the admin the option
to turn it off to safeguard the server stability

Cheers
Andrea

–

Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 333 8128928

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

11

On Thu, Apr 14, 2011 at 7:23 PM, Justin Deoliveira <jdeolive@anonymised.com.1501…> wrote:

Sorry, I meant to ask about paging on all stores.

Right, but to do paging consistently implicitly requires sorting since you can’t rely 100% of the time on a data source returning features in a consistent ordering. Or am I missing something?

Justin, you’re right, the store is otherwise free to return the data in whatever
order it pleases. This the case in which the data to be returned is read along
a spatial index, while you access someone makes a transaction, changes
features, the index gets modified and rebalanced and as a result you get
whole blocks of features returned in a different order (if you’re lucky it’s only
just one that moves, but that cannot be guaranteed).

Sorting ensures that only the features modified move in the list instead.

However as you said we might have to give up sorting on the less complete
data sources in order to give some sort of paging support. The sources
I’m thinking about (shapefiles, files in general) are not suites for WFS-T anyways,
so we may try to be more lenient.
I guess the store should advertise “non stable paging” as opposed to “stable
paging” that requires a sort to keep it straigth

Cheers
Andrea

–

Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 333 8128928

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf