[Geoserver-users] Date comparisons with OGC filter and CQL filter

Hi list,

I am getting some problems using OGC filter and CQL filter
on a Oracle DATE type through ARCSDE-Oracle.
The database table is a view which contains an attribute OBS_DATE_TIME
I am using Geoserver 2.0.1 WFS with an ESRI ARCSDE-Oracle 9.3 data store.

Operator PropertyIsGreaterThan (>) and PropertyIsLessThan (<)
work OK but PropertyIsLessThanEqualTo (<=) and
PropertyIsGreaterThanEqualTo (>=) do not work, that is the
<= and >= filter had no effect and returned 'True'

This works - I get the expected 2 results

******Bilgola and year 2010 only filter ******

<wfs:GetFeature service="WFS" version="1.1.0"
  xmlns:test="http://www.metoc.gov.au/test&quot;
  xmlns:wfs="http://www.opengis.net/wfs&quot;
  xmlns:ogc="http://www.opengis.net/ogc&quot;
  xmlns:gml="http://www.opengis.net/gml&quot;
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance&quot;
  xsi:schemaLocation="http://www.opengis.net/wfs
                      http://schemas.opengis.net/wfs/1.1.0/wfs.xsd&quot;&gt;
  <wfs:Query typeName="test:TEST.BEACH_TEMPS">
  <ogc:Filter>
<ogc:And>

<ogc:PropertyIsEqualTo escapeChar="\" singleChar="_" wildCard="%">
    <ogc:PropertyName>test:LOCATION</ogc:PropertyName>
     <ogc:Literal>Bilgola</ogc:Literal>
  </ogc:PropertyIsEqualTo>

<ogc:PropertyIsGreaterThan>
  <ogc:PropertyName>test:OBS_DATE_TIME</ogc:PropertyName>
  <ogc:Function name="dateParse">
   <ogc:Literal>yyyy-MM-dd HH:mm:ss</ogc:Literal>
   <ogc:Literal>2009-12-31 23:59:59</ogc:Literal>
  </ogc:Function>
</ogc:PropertyIsGreaterThan>

<ogc:PropertyIsLessThan>
  <ogc:PropertyName>test:OBS_DATE_TIME</ogc:PropertyName>
  <ogc:Function name="dateParse">
   <ogc:Literal>yyyy-MM-dd HH:mm:ss</ogc:Literal>
   <ogc:Literal>2011-01-01 00:00:00</ogc:Literal>
  </ogc:Function>
</ogc:PropertyIsLessThan>

</ogc:And>
</ogc:Filter>
</wfs:Query>
</wfs:GetFeature>

but these don't,

<ogc:PropertyIsLessThanEqualTo>and <ogc:PropertyIsGreaterThanEqualTo>

This filter has no effect and just get 3 results i.e all results for 'Bilgola'.

I notice that the DATE string returned by Geoserver has a random 1/1000
second values appended to them and a time zone. For example a date is stored
in Oracle as:

2010-11-18 09:09:00

but is returned in GML as ISO8601 to 1/1000 sec precision with a timezone

2010-11-18T09:09:00.005+11:00

and in CSV output as ISO8601 to 1/1000 sec precsion WITHOUT a timezone

2010-11-18T09:09:00.926

Perhaps the comparison IsEqual (=) bit is not working because of
the random adding of the 1/1000 seconds and this in turn causes
the >= and <= to not work?

I feel the addition of the random 1/1000 value in the output is not correct
as its different to the time which was stored in Oracle. Oracle doesn't
store fractional seconds in a DATE type . Quoting from the Oracle
documentation at http://download.oracle.com/docs/cd/B19306_01/server.102/b14225/ch4datetime.htm
it says:

"The TIMESTAMP datatype is an extension of the DATE datatype. It stores year,
month, day, hour, minute, and second values. It also stores fractional seconds,
which are not stored by the DATE datatype."

So actually the GML and CSV returned are more like an Oracle TIMESTAMP
type.

I also tried to implement date comparsion using the cql_filter and the 'AFTER' operator
as follows but this didn't work:

http://server.metoc.gov.au:8080/geoserver/wfs?request=GetFeature&typeName=TEST:TEST.BEACH_TEMPS&cql_Filter=LOCATION%3D'Bilgola'%20AND%20OBS_DATE_TIME%20AFTER%202011-01-01T00:01:00Z&version=1.1.0&outputFormat=csv

This gave back 0 results and the following error msg. in the geoserver.log:

2011-04-11 13:39:31,083 WARN [data.ArcSDEQuery] - Error fetching row for TEST.BEACH_TEMPS[
Filter: [[ LOCATION = Bilgola ] AND [ OBS_DATE_TIME > Sat Jan 01 00:01:00 EST 2011 ]]
where clause sent: (TEST.BEACH_TEMPS.LOCATION = 'Bilgola' AND TEST.BEACH_TEMPS.OBS_DATE_TIME > 'Sat Jan 01 00:01:00 EST 2011')
geometry filter:Filter.INCLUDE
org.geotools.arcsde.ArcSdeException: [SDE error -51][Error desc=DATABASE LEVEL ERROR OCCURRED.][Extended desc=
]

Looks like Oracle was not liking the SQL sent to it.

Andrew Walsh

I also tried to implement date comparsion using the cql_filter and the 'AFTER'
operator
as follows but this didn't work:

http://server.metoc.gov.au:8080/geoserver/wfs?request=GetFeature&typeName=TEST:TEST.BEACH_TEMPS&cql_Filter=LOCATION%3D'Bilgola'%20AND%20OBS_DATE_TIME%20AFTER%202011-01-01T00:01:00Z&version=1.1.0&outputFormat=csv

This gave back 0 results and the following error msg. in the geoserver.log:

2011-04-11 13:39:31,083 WARN [data.ArcSDEQuery] - Error fetching row for
TEST.BEACH_TEMPS[
Filter: [[ LOCATION = Bilgola ] AND [ OBS_DATE_TIME > Sat Jan 01 00:01:00 EST
2011 ]]
where clause sent: (TEST.BEACH_TEMPS.LOCATION = 'Bilgola' AND
TEST.BEACH_TEMPS.OBS_DATE_TIME > 'Sat Jan 01 00:01:00 EST 2011')
geometry filter:Filter.INCLUDE
org.geotools.arcsde.ArcSdeException: [SDE error -51][Error desc=DATABASE LEVEL
ERROR OCCURRED.][Extended desc=
]

Looks like Oracle was not liking the SQL sent to it.

Indeed it is not, and yeah it looks like a bug in the ArcSDE GeoTools
plugin. How should the where clause look like and what type is the
column?

Gabriel.

Andrew Walsh

------------------------------------------------------------------------------
Xperia(TM) PLAY
It's a major breakthrough. An authentic gaming
smartphone on the nation's most reliable network.
And it wants your games.
http://p.sf.net/sfu/verizon-sfdev
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
Gabriel Roldan
groldan@anonymised.com
Expert service straight from the developers

On Mon, 2011-04-11 at 20:29 +1000, Andrew Walsh wrote:

Hi Gabriel,

Had to reply to you from my gmail account and I couldn't reply from
my work email for next few days. So this reply may not make it
to the geoserver-users list as its from my gmail address
which isn't registered with the users list. Anyway your reply
to me should capture my response to the list.

To answer your question the type of column is Oracle DATE.
A WHERE clause should compare a date to a date and look like:

WHERE TEST.BEACH_TEMPS.LOCATION = 'Bilgola' AND
TEST.BEACH_TEMPS.OBS_DATE_TIME >
to_date('2011-01-01 00:01:00','YYYY-MM-DD HH24:MI:SS')'

Problem seems that it is comparing a date to a string like
'Sat Jan 01 00:01:00 EST 2011'.

yup. I'm not sure if the ArcSDE Java API has something that abstract out
date/time handling from the underlying RDBMS details, or I should be
checking what the underlying RDBMS is and then format the where clause
appropriately. All that assuming what's getting down to the DataStore is
properly converted to the appropriate Java type, which I'm not sure
either.
In any case, would you be so kind of opening a JIRA issue for the
GeoTools ArcSDE plugin at [1]?
That way we could keep the discussion tied to the issue. Are you in
possition of debugging and contributing a patch back? I can't say for
sure when I'll have the spare time to dedicate to it but would be glad
of reviewing a patch.

Cheers,
Gabriel

Andrew

On 4/11/11, Gabriel Roldán <groldan@anonymised.com> wrote:
>
>>
>> I also tried to implement date comparsion using the cql_filter and the
>> 'AFTER'
>> operator
>> as follows but this didn't work:
>>
>> http://server.metoc.gov.au:8080/geoserver/wfs?request=GetFeature&typeName=TEST:TEST.BEACH_TEMPS&cql_Filter=LOCATION%3D'Bilgola'%20AND%20OBS_DATE_TIME%20AFTER%202011-01-01T00:01:00Z&version=1.1.0&outputFormat=csv
>>
>> This gave back 0 results and the following error msg. in the
>> geoserver.log:
>>
>> 2011-04-11 13:39:31,083 WARN [data.ArcSDEQuery] - Error fetching row for
>> TEST.BEACH_TEMPS[
>> Filter: [[ LOCATION = Bilgola ] AND [ OBS_DATE_TIME > Sat Jan 01 00:01:00
>> EST
>> 2011 ]]
>> where clause sent: (TEST.BEACH_TEMPS.LOCATION = 'Bilgola' AND
>> TEST.BEACH_TEMPS.OBS_DATE_TIME > 'Sat Jan 01 00:01:00 EST 2011')
>> geometry filter:Filter.INCLUDE
>> org.geotools.arcsde.ArcSdeException: [SDE error -51][Error desc=DATABASE
>> LEVEL
>> ERROR OCCURRED.][Extended desc=
>> ]
>>
>> Looks like Oracle was not liking the SQL sent to it.
> Indeed it is not, and yeah it looks like a bug in the ArcSDE GeoTools
> plugin. How should the where clause look like and what type is the
> column?
>
> Gabriel.
>>
>> Andrew Walsh
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Xperia(TM) PLAY
>> It's a major breakthrough. An authentic gaming
>> smartphone on the nation's most reliable network.
>> And it wants your games.
>> http://p.sf.net/sfu/verizon-sfdev
>> _______________________________________________
>> Geoserver-users mailing list
>> Geoserver-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/geoserver-users
>
> --
> Gabriel Roldan
> groldan@anonymised.com
> Expert service straight from the developers
>
>

--
Gabriel Roldan
groldan@anonymised.com
Expert service straight from the developers