[Geoserver-devel] The annoying bug of Oracle not using spatial indexes when presented with two bboxes

Hi,
as followers of this mailing list know, we already had some discussions about Oracle
spatial (non enterprise edition) becoming very slow when presented with two or-ed bbox queries.
The issue has been discussed on this mailing list, and a suggestion has been provided
to disabled advanced projection handling as a work around.

The issue keeps on being presented in Jira though:
https://osgeo-org.atlassian.net/browse/GEOS-6813
https://osgeo-org.atlassian.net/browse/GEOS-6831
https://osgeo-org.atlassian.net/browse/GEOS-7015

I wish we had a better answer than “drop a significant bit of GeoServer functionality”
or “get an actual spatial db, dude”, but I cannot find one.

Generally speaking, why are we doing queries with 2 bboxes?
That happens any time the original request crosses or touches the dateline, regardless
of the output projection.

First, there is the general renderer functionaly that expands a bit the original bbox, to load geometries
whose footprint does not fit the request, but whose symbolizer is big enough to do. This we
cannot power off, it’s the basic functionality that makes tiled maps work (otherwise we’d get
cut symbols all the time).

Then, we have map wrapping, that is, the functionality that gives us continuous maps at the
dateline in geographic and mercator maps.
This could be turned off, and I’m planning to add a GUI option to do so, but it won’t solve the problem,
because there is plenty of projections that are including the dateline as part of their map
contents in a continuous way:

  • All stereographics
  • Many Lambert conic (any of those that has a central meridian close enough to the dateline)
  • Lambert azymuthal
  • Orthographic
  • Any geographic whose prime meridian is not Greenwhich (poeple involved in pacific mapping
    have the center of the map at 150° if memory serves me right)
    And so on… there is a whole range of projections in which the dateline is not at one edge
    of the map, but in the middle of it, and when we cross it, we need to generate two or-ed
    bboxes against the original data that might be in wgs84 (or whatever other projection).

So… we could avoid generating the two or-ed bboxes, but I believe only by making a special
case for WGS84 when the map wrapping is turned off (using a different datum and requiring
datum reprojection exposes a lot of funkiness, that’s one case in which we cannot avoid
the two bboxes as the dateline shifts in the two ellipsoids).
It might be a reasonable compromise, as most complainers are not involved in sophisticated mapping anyways.

And then, document for Oracle users that they should either disable map wrapping, advanced
projection handling altoghether, get Oracle spatial enterprise, or… “dude, get a spatial db that works”.

Another suggestion that came out, to be applied at the db level, it to turn the query with
the two bboxes into a union, e.g.:

SELECT BOUNDARY as BOUNDARY FROM OWNER.MYTABLE WHERE SDO_FILTER(BOUNDARY, :1 , ‘mask=anyinteract querytype=WINDOW’) = ‘TRUE’)
UNION
SELECT BOUNDARY as BOUNDARY FROM OWNER.MYTABLE WHERE SDO_FILTER(BOUNDARY, :2 , ‘mask=anyinteract querytype=WINDOW’) = ‘TRUE’ )

This transformation would work if we find two or-ed spatial filters at the top of the query, something
like:
filter1 and filter and filter3 and (spatial1 or spatial or …)

However… this would require quite some effort in the JDBCDataStore, to allow the dialect to
take a Query and split it into a Query array that we are then going to encode as a union
(and some fun handling aggregated queries this way, such as count, max/min and so on)

Given there is no funding whatsoever for this (I guess people finances got already depleted buying Oracle?)
is there anyone with a brilliant idea to solve this issue on the cheap (aka, something that would fit
in spare time?), or has resources to work on it? Of course funding would be nice too,
but not really expecting to find it on the devel list :wink:

Cheers
Andrea

···

==

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

==

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


On Sat, May 9, 2015 at 12:06 PM, Andrea Aime <andrea.aime@anonymised.com>
wrote:

Hi,
as followers of this mailing list know, we already had some discussions
about Oracle
spatial (non enterprise edition) becoming very slow when presented with
two or-ed bbox queries.
The issue has been discussed on this mailing list, and a suggestion has
been provided
to disabled advanced projection handling as a work around.

The issue keeps on being presented in Jira though:
https://osgeo-org.atlassian.net/browse/GEOS-6813
https://osgeo-org.atlassian.net/browse/GEOS-6831
https://osgeo-org.atlassian.net/browse/GEOS-7015

We might have another one piling up:
https://osgeo-org.atlassian.net/browse/GEOS-7021

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------

Added a way to configure wrapping and projection handling from GUI, and made sure
that we don’t end up doing two bbox queries for 4326 over 4326 and 3857 over 4326
when wrapping is disabled.

For all other cases, the two bbox query is correct, so as far as I’m concerned,
Oracle users will have either disable wrapping, if they don’t need it
and it’s a simple case, or put the money where the mouth is, and either do it and donate it, or
sponsor a fix to perform a rewrite to a union query for that case.

Review welcomed:

https://github.com/geotools/geotools/pull/844

https://github.com/geoserver/geoserver/pull/1071

Cheers
Andrea

···

On Sun, May 17, 2015 at 10:08 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Sat, May 9, 2015 at 12:06 PM, Andrea Aime <andrea.aime@anonymised.com> wrote:

Hi,
as followers of this mailing list know, we already had some discussions about Oracle
spatial (non enterprise edition) becoming very slow when presented with two or-ed bbox queries.
The issue has been discussed on this mailing list, and a suggestion has been provided
to disabled advanced projection handling as a work around.

The issue keeps on being presented in Jira though:
https://osgeo-org.atlassian.net/browse/GEOS-6813
https://osgeo-org.atlassian.net/browse/GEOS-6831
https://osgeo-org.atlassian.net/browse/GEOS-7015

We might have another one piling up: https://osgeo-org.atlassian.net/browse/GEOS-7021

Cheers

Andrea

==

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

==

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


==

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

==

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.