[GeoNetwork-devel] CSW : regression between 2.6.4 and 2.6.5 snapshot ?

Sylvain_GRELLET · November 4, 2012, 12:48pm

Dear list,

We have deployed a 2.6.5 shapshot to test the upgrade to the new version of lucene but we ran into 3 major issues with the CSW. All with queries that ran with a 2.6.4 a seemed compatible with the OGC specs.
Our revision is : https://github.com/geonetwork/core-geonetwork/commit/76fcaf6078022c94444f1fb68f115d2f656c39e3

GetRecords: issue with literal containing “/”
example :

<?xml version="1.0"?>

<csw:GetRecords xmlns:csw=“http://www.opengis.net/cat/csw/2.0.2”
xmlns:gmd=“http://www.isotc211.org/2005/gmd” service=“CSW”
version=“2.0.2” resultType=“results”>
<csw:Query typeNames=“csw:Record”>
csw:ElementNamedc:title</csw:ElementName>
<csw:Constraint version=“1.1.0”>

apiso:ResourceIdentifier
http://www.sandre.eaufrance.fr/?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:html

</csw:Constraint>
</csw:Query>
</csw:GetRecords>

If we replace “/” by the joker (ex : “$”) in the content of the Literal tag, search works (with : http:$$www.sandre.eaufrance.fr$?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:htm)
Using the escape char doesn’t work
=> it seems as some XPath interpretation is done on the content of Literal

Bug with typeNames=“gmd:MD_Metadata”
The same request with :

<csw:Query typeNames=“csw:Record”> works : we received a response with the requested fields
<csw:Query typeNames=“gmd:MD_Metadata”> does not work : we receive a response with empty tags.
=> we feel there is a link with this issue : http://trac.osgeo.org/geonetwork/ticket/724

works :

<?xml version="1.0"?>

<csw:GetRecords xmlns:csw=“http://www.opengis.net/cat/csw/2.0.2”
xmlns:gmd=“http://www.isotc211.org/2005/gmd” service=“CSW”
version=“2.0.2” resultType=“results”>
<csw:Query typeNames=“csw:Record”>
csw:ElementNamedc:title</csw:ElementName>
<csw:Constraint version=“1.1.0”>

apiso:ResourceIdentifier
http:$$www.sandre.eaufrance.fr$?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:html

</csw:Constraint>
</csw:Query>
</csw:GetRecords>

does not work :

<?xml version="1.0" encoding="UTF-8"?>

<csw:GetRecords xmlns:csw=“http://www.opengis.net/cat/csw/2.0.2” xmlns:dc=“http://purl.org/dc/elements/1.1/” xmlns:ogc=“http://www.opengis.net/ogc” xmlns:gml=“http://www.opengis.net/gml” xmlns:gmd=“http://www.isotc211.org/2005/gmd” xmlns:xsi=“http://www.w3.org/2001/XMLSchema-instance” xsi:schemaLocation=“http://www.opengis.net/cat/csw/2.0.2 http://schemas.opengis.net/csw/2.0.2/CSW-discovery.xsd” service=“CSW” version=“2.0.2” resultType=“results” outputSchema=“http://www.isotc211.org/2005/gmd” startPosition=“1” maxRecords=“50”>
<csw:Query typeNames=“gmd:MD_Metadata”>
csw:ElementName/gmd:MD_Metadata/gmd:identificationInfo/gmd:MD_DataIdentification/gmd:citation/gmd:CI_Citation/gmd:title</csw:ElementName>
<csw:Constraint version=“1.1.0”>

apiso:ResourceIdentifier
http:$$www.sandre.eaufrance.fr$?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:html

</csw:Constraint>
</csw:Query>
</csw:GetRecords>

GetDomain : a simple getdomain on keywords, receives an error in the response.

Best,
Sylvain

sppigot · November 6, 2012, 8:12am

Hi Sylvain,

I fixed the GetDomain problem in commit 582bb0f371b00023a41659fc817c0e280e276580 - it was a runtime error generated by Lucene 3.6.x about an unsupported operation we were using on an IndexReader.

Your other queries:

1. Your query filters below are actually not valid because the attribute escape is not valid on PropertyIsLike - it should be called escapeChar. My own feeling is that if the filter is not valid then an exception should be returned to the user (or at least there should be a config option to force failure on invalid queries). In this case no exception is returned and things just continue on (I think the recent change to geotools 8.2 in trunk may have forced a change in this behaviour from memory as invalid queries don't return a Filter object in 8.2).

2. After fixing the trivial escapeChar problem above I got all queries that used wildcards with PropertyIsLike to work (2nd and 3rd query below) and the first which did not use a wildcard works with PropertyIsEqualTo (I suspect it should work with PropertyIsLike and no wildcards but I saw the literal being broken up around the '/' char by the query parser when I used PropertyIsLike without a wildcard).

Cheers,
Simon
________________________________________
From: Sylvain GRELLET [s.grellet@anonymised.com]
Sent: Sunday, 4 November 2012 11:48 PM
To: geonetwork-devel@lists.sourceforge.net
Subject: [GeoNetwork-devel] CSW : regression between 2.6.4 and 2.6.5 snapshot ?

Dear list,

We have deployed a 2.6.5 shapshot to test the upgrade to the new version of lucene but we ran into 3 major issues with the CSW. All with queries that ran with a 2.6.4 a seemed compatible with the OGC specs.
Our revision is : fix for 1105: postgis query too large · geonetwork/core-geonetwork@76fcaf6 · GitHub

* GetRecords: issue with literal containing "/"
example :
<?xml version="1.0"?>
<csw:GetRecords xmlns:csw="http://www.opengis.net/cat/csw/2.0.2"
xmlns:gmd="http://www.isotc211.org/2005/gmd" service="CSW"
version="2.0.2" resultType="results">
    <csw:Query typeNames="csw:Record">
        <csw:ElementName>dc:title</csw:ElementName>
        <csw:Constraint version="1.1.0">
            <Filter xmlns="http://www.opengis.net/ogc">
                <PropertyIsLike wildCard="*" escape="\" singleChar="$">
                    <PropertyName>apiso:ResourceIdentifier</PropertyName>
                    <Literal>http://www.sandre.eaufrance.fr/?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:html</Literal>
                </PropertyIsLike>
            </Filter>
        </csw:Constraint>
    </csw:Query>
</csw:GetRecords>

If we replace "/" by the joker (ex : "$") in the content of the Literal tag, search works (with : http:$$www.sandre.eaufrance.fr$?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:htm)
Using the escape char doesn't work
=> it seems as some XPath interpretation is done on the content of Literal

* Bug with typeNames="gmd:MD_Metadata"
The same request with :
- <csw:Query typeNames="csw:Record"> works : we received a response with the requested fields
- <csw:Query typeNames="gmd:MD_Metadata"> does not work : we receive a response with empty tags.
=> we feel there is a link with this issue : #724 (CSW / ElementName) – GeoNetwork opensource Developer website

works :
<?xml version="1.0"?>
<csw:GetRecords xmlns:csw="http://www.opengis.net/cat/csw/2.0.2"
xmlns:gmd="http://www.isotc211.org/2005/gmd" service="CSW"
version="2.0.2" resultType="results">
    <csw:Query typeNames="csw:Record">
        <csw:ElementName>dc:title</csw:ElementName>
        <csw:Constraint version="1.1.0">
            <Filter xmlns="http://www.opengis.net/ogc">
                <PropertyIsLike wildCard="*" escape="\" singleChar="$">
                    <PropertyName>apiso:ResourceIdentifier</PropertyName>
                    <Literal>http:$$www.sandre.eaufrance.fr$?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:html</Literal>
                </PropertyIsLike>
            </Filter>
        </csw:Constraint>
    </csw:Query>
</csw:GetRecords>

does not work :
<?xml version="1.0" encoding="UTF-8"?>
<csw:GetRecords xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" xmlns:dc="DCMI: DCMI Metadata Terms; xmlns:ogc="http://www.opengis.net/ogc" xmlns:gml="http://www.opengis.net/gml" xmlns:gmd="http://www.isotc211.org/2005/gmd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/cat/csw/2.0.2 http://schemas.opengis.net/csw/2.0.2/CSW-discovery.xsd" service="CSW" version="2.0.2" resultType="results" outputSchema="http://www.isotc211.org/2005/gmd" startPosition="1" maxRecords="50">
    <csw:Query typeNames="gmd:MD_Metadata">
        <csw:ElementName>/gmd:MD_Metadata/gmd:identificationInfo/gmd:MD_DataIdentification/gmd:citation/gmd:CI_Citation/gmd:title</csw:ElementName>
        <csw:Constraint version="1.1.0">
            <Filter xmlns="http://www.opengis.net/ogc">
                <PropertyIsLike wildCard="*" escape="\" singleChar="$">
                    <PropertyName>apiso:ResourceIdentifier</PropertyName>
                    <Literal>http:$$www.sandre.eaufrance.fr$?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:html</Literal>
                </PropertyIsLike>
            </Filter>
        </csw:Constraint>
    </csw:Query>
</csw:GetRecords>

* GetDomain : a simple getdomain on keywords, receives an error in the response.

Best,
Sylvain

Sylvain_GRELLET · November 29, 2012, 1:57pm

Hi Simon,

As I said in another e-mail, 2.6.5 snapshot including your commit is now in production. GetDomain and escapeChar aspects are fine now.

We now encounter an issue with a CSW request that was fine with 2_6_4.
It seems that, a csw query including spatial filter (version 1.1.0 or 1.0.0) makes the paging behave weirdly.

In the response, the numberOfRecordsMatched always equals the nextRecord value when the maxRecords of your query is (strictly less than the real number of matched record minus 1).

To be sure I was not wrong in my CSW request I reproduced the query that is in page 122/127 of the “OpenGIS Catalogue Services Specification 2.0.2 - ISO Metadata Application Profile (1.0.0)” - 07-045
I just changed the word searched in the title and adapted the bbox to Metropolitan France

You can try the query below, and change the maxRecords.
If you put it to 200, you discover that the actual number of matched records is 30.
But if you set it to 5, it responds : numberOfRecordsMatched=“6” numberOfRecordsReturned=“5” elementSet=“brief” nextRecord=“6”

If you just remove the Intersects part of the filter, paging restarts working properly. The numberOfRecordsMatched is of course different (no spatial filter anymore).

Our CSW endpoint : http://services.sandre.eaufrance.fr/geonetwork_CSW/srv/fr/csw

The query:
<GetRecords xmlns=“http://www.opengis.net/cat/csw/2.0.2” xmlns:csw=“http://www.opengis.net/cat/csw/2.0.2” xmlns:ogc=“http://www.opengis.net/ogc” xmlns:ows=“http://www.opengis” xmlns:xsd=“http://www.w3.org/2001/XMLSchema” xmlns:dc=“http://purl.org/dc/elements/1.1/” xmlns:dct=“http://purl.org/dc/terms/” xmlns:gml=“http://www.opengis.net/gml” xmlns:xsi=“http://www.w3.org/2001/XMLSchema-instance” service=“CSW” version=“2.0.2” resultType=“results” outputFormat=“application/xml” outputSchema=“http://www.opengis.net/cat/csw/2.0.2” startPosition=“1” maxRecords=“5”>

brief

ogc:Filter
ogc:And
<ogc:PropertyIsLike escapeChar=“" singleChar=”?"
wildCard=“*”>
ogc:PropertyNamedc:title</ogc:PropertyName>
ogc:Literaleau</ogc:Literal>
</ogc:PropertyIsLike>
ogc:PropertyIsEqualTo
ogc:PropertyNamedc:type</ogc:PropertyName>
ogc:Literaldataset</ogc:Literal>
</ogc:PropertyIsEqualTo>
ogc:Intersects
ogc:PropertyNameows:BoundingBox</ogc:PropertyName>
gml:Envelope
gml:lowerCorner-6 41</gml:lowerCorner>
gml:upperCorner10 51</gml:upperCorner>
</gml:Envelope>
</ogc:Intersects>
</ogc:And>
</ogc:Filter>

Cheers
Sylvain

···

On 06/11/2012 09:12, Simon.Pigot@anonymised.com wrote:

Hi Sylvain,

I fixed the GetDomain problem in commit 582bb0f371b00023a41659fc817c0e280e276580 - it was a runtime error generated by Lucene 3.6.x about an unsupported operation we were using on an IndexReader.

Your other queries:

1. Your query filters below are actually not valid because the attribute escape is not valid on PropertyIsLike - it should be called escapeChar. My own feeling is that if the filter is not valid then an exception should be returned to the user (or at least there should be a config option to force failure on invalid queries). In this case no exception is returned and things just continue on (I think the recent change to geotools 8.2 in trunk may have forced a change in this behaviour from memory as invalid queries don't return a Filter object in 8.2).

2. After fixing the trivial escapeChar problem above I got all queries that used wildcards with PropertyIsLike to work (2nd and 3rd query below) and the first which did not use a wildcard works with PropertyIsEqualTo (I suspect it should work with PropertyIsLike and no wildcards but I saw the literal being broken up around the '/' char by the query parser when I used PropertyIsLike without a wildcard).

Cheers,
Simon  
________________________________________
From: Sylvain GRELLET [[s.grellet@anonymised.com](mailto:s.grellet@anonymised.com)]
Sent: Sunday, 4 November 2012 11:48 PM
To: [geonetwork-devel@lists.sourceforge.net](mailto:geonetwork-devel@lists.sourceforge.net)
Subject: [GeoNetwork-devel] CSW : regression between 2.6.4 and 2.6.5    snapshot ?

Dear list,

We have deployed a 2.6.5 shapshot to test the upgrade to the new version of lucene but we ran into 3 major issues with the CSW. All with queries that ran with a 2.6.4 a seemed compatible with the OGC specs.
Our revision is : [https://github.com/geonetwork/core-geonetwork/commit/76fcaf6078022c94444f1fb68f115d2f656c39e3](https://github.com/geonetwork/core-geonetwork/commit/76fcaf6078022c94444f1fb68f115d2f656c39e3)

* GetRecords: issue with literal containing "/"
example :
<?xml version="1.0"?>
<csw:GetRecords xmlns:csw=["http://www.opengis.net/cat/csw/2.0.2"](http://www.opengis.net/cat/csw/2.0.2)
 xmlns:gmd=["http://www.isotc211.org/2005/gmd"](http://www.isotc211.org/2005/gmd) service="CSW"
 version="2.0.2" resultType="results">
    <csw:Query typeNames="csw:Record">
        <csw:ElementName>dc:title</csw:ElementName>
        <csw:Constraint version="1.1.0">
            <Filter xmlns=["http://www.opengis.net/ogc"](http://www.opengis.net/ogc)>
                <PropertyIsLike wildCard="*" escape="\" singleChar="$">
                    <PropertyName>apiso:ResourceIdentifier</PropertyName>
                    <Literal>[http://www.sandre.eaufrance.fr/?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:html](http://www.sandre.eaufrance.fr/?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:html)</Literal>
                </PropertyIsLike>
            </Filter>
        </csw:Constraint>
    </csw:Query>
</csw:GetRecords>
======================================================
If we replace "/" by the joker (ex : "$") in the content of the Literal tag, search works (with : http:$$www.sandre.eaufrance.fr$?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:htm)
Using the escape char doesn't work
=> it seems as some XPath interpretation is done on the content of Literal

* Bug with typeNames="gmd:MD_Metadata"
The same request with :
-  <csw:Query typeNames="csw:Record"> works : we received a response with the requested fields
-  <csw:Query typeNames="gmd:MD_Metadata"> does not work : we receive a response with empty tags.
=> we feel there is a link with this issue : [http://trac.osgeo.org/geonetwork/ticket/724](http://trac.osgeo.org/geonetwork/ticket/724)

works :
<?xml version="1.0"?>
<csw:GetRecords xmlns:csw=["http://www.opengis.net/cat/csw/2.0.2"](http://www.opengis.net/cat/csw/2.0.2)
 xmlns:gmd=["http://www.isotc211.org/2005/gmd"](http://www.isotc211.org/2005/gmd) service="CSW"
 version="2.0.2" resultType="results">
    <csw:Query typeNames="csw:Record">
        <csw:ElementName>dc:title</csw:ElementName>
        <csw:Constraint version="1.1.0">
            <Filter xmlns=["http://www.opengis.net/ogc"](http://www.opengis.net/ogc)>
                <PropertyIsLike wildCard="*" escape="\" singleChar="$">
                    <PropertyName>apiso:ResourceIdentifier</PropertyName>
                    <Literal>http:$$www.sandre.eaufrance.fr$?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:html</Literal>
                </PropertyIsLike>
            </Filter>
        </csw:Constraint>
    </csw:Query>
</csw:GetRecords>

does not work :
<?xml version="1.0" encoding="UTF-8"?>
<csw:GetRecords xmlns:csw=["http://www.opengis.net/cat/csw/2.0.2"](http://www.opengis.net/cat/csw/2.0.2) xmlns:dc=["http://purl.org/dc/elements/1.1/"](http://purl.org/dc/elements/1.1/) xmlns:ogc=["http://www.opengis.net/ogc"](http://www.opengis.net/ogc) xmlns:gml=["http://www.opengis.net/gml"](http://www.opengis.net/gml) xmlns:gmd=["http://www.isotc211.org/2005/gmd"](http://www.isotc211.org/2005/gmd) xmlns:xsi=["http://www.w3.org/2001/XMLSchema-instance"](http://www.w3.org/2001/XMLSchema-instance) xsi:schemaLocation=["http://www.opengis.net/cat/csw/2.0.2 http://schemas.opengis.net/csw/2.0.2/CSW-discover!
 y.xsd"](http://www.opengis.net/cat/csw/2.0.2http://schemas.opengis.net/csw/2.0.2/CSW-discovery.xsd)
 service="CSW" version="2.0.2" resultType="results" outputSchema=["http://www.isotc211.org/2005/gmd"](http://www.isotc211.org/2005/gmd) startPosition="1" maxRecords="50">
    <csw:Query typeNames="gmd:MD_Metadata">
        <csw:ElementName>/gmd:MD_Metadata/gmd:identificationInfo/gmd:MD_DataIdentification/gmd:citation/gmd:CI_Citation/gmd:title</csw:ElementName>
        <csw:Constraint version="1.1.0">
            <Filter xmlns=["http://www.opengis.net/ogc"](http://www.opengis.net/ogc)>
                <PropertyIsLike wildCard="*" escape="\" singleChar="$">
                    <PropertyName>apiso:ResourceIdentifier</PropertyName>
                    <Literal>http:$$www.sandre.eaufrance.fr$?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:html</Literal>
                </PropertyIsLike>
            </Filter>
        </csw:Constraint>
    </csw:Query>
</csw:GetRecords>

* GetDomain : a simple getdomain on keywords, receives an error in the response.

Best,
Sylvain

Jesse_Eichar2 · November 29, 2012, 2:00pm

That is my bug. I fixed it on 2.8.x and master but forgot to port it back to 2.6.5. I will do that now.

Jesse

···

On Thu, Nov 29, 2012 at 2:57 PM, Sylvain GRELLET <s.grellet@anonymised.com> wrote:

Hi Simon,

As I said in another e-mail, 2.6.5 snapshot including your commit is now in production. GetDomain and escapeChar aspects are fine now.

We now encounter an issue with a CSW request that was fine with 2_6_4.
It seems that, a csw query including spatial filter (version 1.1.0 or 1.0.0) makes the paging behave weirdly.

In the response, the numberOfRecordsMatched always equals the nextRecord value when the maxRecords of your query is (strictly less than the real number of matched record minus 1).

To be sure I was not wrong in my CSW request I reproduced the query that is in page 122/127 of the “OpenGIS Catalogue Services Specification 2.0.2 - ISO Metadata Application Profile (1.0.0)” - 07-045
I just changed the word searched in the title and adapted the bbox to Metropolitan France

You can try the query below, and change the maxRecords.
If you put it to 200, you discover that the actual number of matched records is 30.
But if you set it to 5, it responds : numberOfRecordsMatched=“6” numberOfRecordsReturned=“5” elementSet=“brief” nextRecord=“6”

If you just remove the Intersects part of the filter, paging restarts working properly. The numberOfRecordsMatched is of course different (no spatial filter anymore).

Our CSW endpoint : http://services.sandre.eaufrance.fr/geonetwork_CSW/srv/fr/csw

The query:
<GetRecords xmlns=“http://www.opengis.net/cat/csw/2.0.2” xmlns:csw=“http://www.opengis.net/cat/csw/2.0.2” xmlns:ogc=“http://www.opengis.net/ogc” xmlns:ows=“http://www.opengis” xmlns:xsd=“http://www.w3.org/2001/XMLSchema” xmlns:dc=“http://purl.org/dc/elements/1.1/” xmlns:dct=“http://purl.org/dc/terms/” xmlns:gml=“http://www.opengis.net/gml” xmlns:xsi=“http://www.w3.org/2001/XMLSchema-instance” service=“CSW” version=“2.0.2” resultType=“results” outputFormat=“application/xml” outputSchema=“http://www.opengis.net/cat/csw/2.0.2” startPosition=“1” maxRecords=“5”>

brief

ogc:Filter
ogc:And
<ogc:PropertyIsLike escapeChar=“" singleChar=”?"
wildCard=“*”>
ogc:PropertyNamedc:title</ogc:PropertyName>
ogc:Literaleau</ogc:Literal>
</ogc:PropertyIsLike>
ogc:PropertyIsEqualTo
ogc:PropertyNamedc:type</ogc:PropertyName>
ogc:Literaldataset</ogc:Literal>
</ogc:PropertyIsEqualTo>
ogc:Intersects
ogc:PropertyNameows:BoundingBox</ogc:PropertyName>
gml:Envelope
gml:lowerCorner-6 41</gml:lowerCorner>
gml:upperCorner10 51</gml:upperCorner>
</gml:Envelope>
</ogc:Intersects>
</ogc:And>
</ogc:Filter>

Cheers
Sylvain

On 06/11/2012 09:12, Simon.Pigot@anonymised.com wrote:
Hi Sylvain,

I fixed the GetDomain problem in commit 582bb0f371b00023a41659fc817c0e280e276580 - it was a runtime error generated by Lucene 3.6.x about an unsupported operation we were using on an IndexReader.

Your other queries:

1. Your query filters below are actually not valid because the attribute escape is not valid on PropertyIsLike - it should be called escapeChar. My own feeling is that if the filter is not valid then an exception should be returned to the user (or at least there should be a config option to force failure on invalid queries). In this case no exception is returned and things just continue on (I think the recent change to geotools 8.2 in trunk may have forced a change in this behaviour from memory as invalid queries don't return a Filter object in 8.2).

2. After fixing the trivial escapeChar problem above I got all queries that used wildcards with PropertyIsLike to work (2nd and 3rd query below) and the first which did not use a wildcard works with PropertyIsEqualTo (I suspect it should work with PropertyIsLike and no wildcards but I saw the literal being broken up around the '/' char by the query parser when I used PropertyIsLike without a wildcard).

Cheers,
Simon  
________________________________________
From: Sylvain GRELLET [[s.grellet@anonymised.com](mailto:s.grellet@anonymised.com)]
Sent: Sunday, 4 November 2012 11:48 PM
To: [geonetwork-devel@lists.sourceforge.net](mailto:geonetwork-devel@lists.sourceforge.net)
Subject: [GeoNetwork-devel] CSW : regression between 2.6.4 and 2.6.5    snapshot ?

Dear list,

We have deployed a 2.6.5 shapshot to test the upgrade to the new version of lucene but we ran into 3 major issues with the CSW. All with queries that ran with a 2.6.4 a seemed compatible with the OGC specs.
Our revision is : [https://github.com/geonetwork/core-geonetwork/commit/76fcaf6078022c94444f1fb68f115d2f656c39e3](https://github.com/geonetwork/core-geonetwork/commit/76fcaf6078022c94444f1fb68f115d2f656c39e3)

* GetRecords: issue with literal containing "/"
example :
<?xml version="1.0"?>
<csw:GetRecords xmlns:csw=["http://www.opengis.net/cat/csw/2.0.2"](http://www.opengis.net/cat/csw/2.0.2)
 xmlns:gmd=["http://www.isotc211.org/2005/gmd"](http://www.isotc211.org/2005/gmd) service="CSW"
 version="2.0.2" resultType="results">
    <csw:Query typeNames="csw:Record">
        <csw:ElementName>dc:title</csw:ElementName>
        <csw:Constraint version="1.1.0">
            <Filter xmlns=["http://www.opengis.net/ogc"](http://www.opengis.net/ogc)>
                <PropertyIsLike wildCard="*" escape="\" singleChar="$">
                    <PropertyName>apiso:ResourceIdentifier</PropertyName>
                    <Literal>[http://www.sandre.eaufrance.fr/?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:html](http://www.sandre.eaufrance.fr/?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:html)</Literal>
                </PropertyIsLike>
            </Filter>
        </csw:Constraint>
    </csw:Query>
</csw:GetRecords>
======================================================
If we replace "/" by the joker (ex : "$") in the content of the Literal tag, search works (with : http:$$www.sandre.eaufrance.fr$?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:htm)
Using the escape char doesn't work
=> it seems as some XPath interpretation is done on the content of Literal

* Bug with typeNames="gmd:MD_Metadata"
The same request with :
-  <csw:Query typeNames="csw:Record"> works : we received a response with the requested fields
-  <csw:Query typeNames="gmd:MD_Metadata"> does not work : we receive a response with empty tags.
=> we feel there is a link with this issue : [http://trac.osgeo.org/geonetwork/ticket/724](http://trac.osgeo.org/geonetwork/ticket/724)

works :
<?xml version="1.0"?>
<csw:GetRecords xmlns:csw=["http://www.opengis.net/cat/csw/2.0.2"](http://www.opengis.net/cat/csw/2.0.2)
 xmlns:gmd=["http://www.isotc211.org/2005/gmd"](http://www.isotc211.org/2005/gmd) service="CSW"
 version="2.0.2" resultType="results">
    <csw:Query typeNames="csw:Record">
        <csw:ElementName>dc:title</csw:ElementName>
        <csw:Constraint version="1.1.0">
            <Filter xmlns=["http://www.opengis.net/ogc"](http://www.opengis.net/ogc)>
                <PropertyIsLike wildCard="*" escape="\" singleChar="$">
                    <PropertyName>apiso:ResourceIdentifier</PropertyName>
                    <Literal>http:$$www.sandre.eaufrance.fr$?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:html</

Literal>
                </PropertyIsLike>
            </Filter>
        </csw:Constraint>
    </csw:Query>
</csw:GetRecords>

does not work :
<?xml version="1.0" encoding="UTF-8"?>
<csw:GetRecords xmlns:csw=["http://www.opengis.net/cat/csw/2.0.2"](http://www.opengis.net/cat/csw/2.0.2) xmlns:dc=["http://purl.org/dc/elements/1.1/"](http://purl.org/dc/elements/1.1/) xmlns:ogc=["http://www.opengis.net/ogc"](http://www.opengis.net/ogc) xmlns:gml=["http://www.opengis.net/gml"](http://www.opengis.net/gml) xmlns:gmd=["http://www.isotc211.org/2005/gmd"](http://www.isotc211.org/2005/gmd) xmlns:xsi=["http://www.w3.org/2001/XMLSchema-instance"](http://www.w3.org/2001/XMLSchema-instance) xsi:schemaLocation=["http://www.opengis.net/cat/csw/2.0.2 http://schemas.opengis.net/csw/2.0.2/CSW-discover!
 y.xsd"](http://www.opengis.net/cat/csw/2.0.2http://schemas.opengis.net/csw/2.0.2/CSW-discovery.xsd)

 service="CSW" version="2.0.2" resultType="results" outputSchema=["http://www.isotc211.org/2005/gmd"](http://www.isotc211.org/2005/gmd) startPosition="1" maxRecords="50">
    <csw:Query typeNames="gmd:MD_Metadata">
        <csw:ElementName>/gmd:MD_Metadata/gmd:identificationInfo/gmd:MD_DataIdentification/gmd:citation/gmd:CI_Citation/gmd:title</csw:ElementName>
        <csw:Constraint version="1.1.0">
            <Filter xmlns=["http://www.opengis.net/ogc"](http://www.opengis.net/ogc)>
                <PropertyIsLike wildCard="*" escape="\" singleChar="$">
                    <PropertyName>apiso:ResourceIdentifier</PropertyName>
                    <Literal>http:$$www.sandre.eaufrance.fr$?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:html</Literal>
                </PropertyIsLike>
            </Filter>
        </csw:Constraint>
    </csw:Query>
</csw:GetRecords>

* GetDomain : a simple getdomain on keywords, receives an error in the response.

Best,
Sylvain
Keep yourself connected to Go Parallel:
VERIFY Test and improve your parallel project with help from experts
and peers. http://goparallel.sourceforge.net

GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

Jesse_Eichar2 · November 29, 2012, 2:09pm

Try that. My commit is a quick fix. I have to take a couple days as some point to implement a more efficient solution, but now it has the same efficiency as 2.6.4.

···

On Thu, Nov 29, 2012 at 3:00 PM, Jesse Eichar <jesse.eichar@anonymised.com> wrote:

That is my bug. I fixed it on 2.8.x and master but forgot to port it back to 2.6.5. I will do that now.

Jesse

On Thu, Nov 29, 2012 at 2:57 PM, Sylvain GRELLET <s.grellet@anonymised.com…> wrote:

Hi Simon,

As I said in another e-mail, 2.6.5 snapshot including your commit is now in production. GetDomain and escapeChar aspects are fine now.

We now encounter an issue with a CSW request that was fine with 2_6_4.
It seems that, a csw query including spatial filter (version 1.1.0 or 1.0.0) makes the paging behave weirdly.

In the response, the numberOfRecordsMatched always equals the nextRecord value when the maxRecords of your query is (strictly less than the real number of matched record minus 1).

To be sure I was not wrong in my CSW request I reproduced the query that is in page 122/127 of the “OpenGIS Catalogue Services Specification 2.0.2 - ISO Metadata Application Profile (1.0.0)” - 07-045
I just changed the word searched in the title and adapted the bbox to Metropolitan France

You can try the query below, and change the maxRecords.
If you put it to 200, you discover that the actual number of matched records is 30.
But if you set it to 5, it responds : numberOfRecordsMatched=“6” numberOfRecordsReturned=“5” elementSet=“brief” nextRecord=“6”

If you just remove the Intersects part of the filter, paging restarts working properly. The numberOfRecordsMatched is of course different (no spatial filter anymore).

Our CSW endpoint : http://services.sandre.eaufrance.fr/geonetwork_CSW/srv/fr/csw

The query:
<GetRecords xmlns=“http://www.opengis.net/cat/csw/2.0.2” xmlns:csw=“http://www.opengis.net/cat/csw/2.0.2” xmlns:ogc=“http://www.opengis.net/ogc” xmlns:ows=“http://www.opengis” xmlns:xsd=“http://www.w3.org/2001/XMLSchema” xmlns:dc=“http://purl.org/dc/elements/1.1/” xmlns:dct=“http://purl.org/dc/terms/” xmlns:gml=“http://www.opengis.net/gml” xmlns:xsi=“http://www.w3.org/2001/XMLSchema-instance” service=“CSW” version=“2.0.2” resultType=“results” outputFormat=“application/xml” outputSchema=“http://www.opengis.net/cat/csw/2.0.2” startPosition=“1” maxRecords=“5”>

brief

ogc:Filter
ogc:And
<ogc:PropertyIsLike escapeChar=“" singleChar=”?"
wildCard=“*”>
ogc:PropertyNamedc:title</ogc:PropertyName>
ogc:Literaleau</ogc:Literal>
</ogc:PropertyIsLike>
ogc:PropertyIsEqualTo
ogc:PropertyNamedc:type</ogc:PropertyName>
ogc:Literaldataset</ogc:Literal>
</ogc:PropertyIsEqualTo>
ogc:Intersects
ogc:PropertyNameows:BoundingBox</ogc:PropertyName>
gml:Envelope
gml:lowerCorner-6 41</gml:lowerCorner>
gml:upperCorner10 51</gml:upperCorner>
</gml:Envelope>
</ogc:Intersects>
</ogc:And>
</ogc:Filter>

Cheers
Sylvain

On 06/11/2012 09:12, Simon.Pigot@anonymised.com wrote:
Hi Sylvain,

I fixed the GetDomain problem in commit 582bb0f371b00023a41659fc817c0e280e276580 - it was a runtime error generated by Lucene 3.6.x about an unsupported operation we were using on an IndexReader.

Your other queries:

1. Your query filters below are actually not valid because the attribute escape is not valid on PropertyIsLike - it should be called escapeChar. My own feeling is that if the filter is not valid then an exception should be returned to the user (or at least there should be a config option to force failure on invalid queries). In this case no exception is returned and things just continue on (I think the recent change to geotools 8.2 in trunk may have forced a change in this behaviour from memory as invalid queries don't return a Filter object in 8.2).

2. After fixing the trivial escapeChar problem above I got all queries that used wildcards with PropertyIsLike to work (2nd and 3rd query below) and the first which did not use a wildcard works with PropertyIsEqualTo (I suspect it should work with PropertyIsLike and no wildcards but I saw the literal being broken up around the '/' char by the query parser when I used PropertyIsLike without a wildcard).

Cheers,
Simon  
________________________________________
From: Sylvain GRELLET [[s.grellet@anonymised.com](mailto:s.grellet@anonymised.com)]
Sent: Sunday, 4 November 2012 11:48 PM
To: [geonetwork-devel@lists.sourceforge.net](mailto:geonetwork-devel@lists.sourceforge.net)
Subject: [GeoNetwork-devel] CSW : regression between 2.6.4 and 2.6.5    snapshot ?

Dear list,

We have deployed a 2.6.5 shapshot to test the upgrade to the new version of lucene but we ran into 3 major issues with the CSW. All with queries that ran with a 2.6.4 a seemed compatible with the OGC specs.
Our revision is : [https://github.com/geonetwork/core-geonetwork/commit/76fcaf6078022c94444f1fb68f115d2f656c39e3](https://github.com/geonetwork/core-geonetwork/commit/76fcaf6078022c94444f1fb68f115d2f656c39e3)

* GetRecords: issue with literal containing "/"
example :
<?xml version="1.0"?>
<csw:GetRecords xmlns:csw=["http://www.opengis.net/cat/csw/2.0.2"](http://www.opengis.net/cat/csw/2.0.2)
 xmlns:gmd=["http://www.isotc211.org/2005/gmd"](http://www.isotc211.org/2005/gmd) service="CSW"
 version="2.0.2" resultType="results">
    <csw:Query typeNames="csw:Record">
        <csw:ElementName>dc:title</csw:ElementName>
        <csw:Constraint version="1.1.0">
            <Filter xmlns=["http://www.opengis.net/ogc"](http://www.opengis.net/ogc)>
                <PropertyIsLike wildCard="*" escape="\" singleChar="$">
                    <PropertyName>apiso:ResourceIdentifier</PropertyName>
                    <Literal>[http://www.sandre.eaufrance.fr/?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:html](http://www.sandre.eaufrance.fr/?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:html)</Literal>
                </PropertyIsLike>
            </Filter>
        </csw:Constraint>
    </csw:Query>
</csw:GetRecords>
======================================================
If we replace "/" by the joker (ex : "$") in the content of the Literal tag, search works (with : http:$$www.sandre.eaufrance.fr$?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:htm)
Using the escape char doesn't work
=> it seems as some XPath interpretation is done on the content of Literal

* Bug with typeNames="gmd:MD_Metadata"
The same request with :
-  <csw:Query typeNames="csw:Record"> works : we received a response with the requested fields
-  <csw:Query typeNames="gmd:MD_Metadata"> does not work : we receive a response with empty tags.
=> we feel there is a link with this issue : [http://trac.osgeo.org/geonetwork/ticket/724](http://trac.osgeo.org/geonetwork/ticket/724)

works :
<?xml version="1.0"?>
<csw:GetRecords xmlns:csw=["http://www.opengis.net/cat/csw/2.0.2"](http://www.opengis.net/cat/csw/2.0.2)
 xmlns:gmd=["http://www.isotc211.org/2005/gmd"](http://www.isotc211.org/2005/gmd) service="CSW"
 version="2.0.2" resultType="results">
    <csw:Query typeNames="csw:Record">
        <csw:ElementName>dc:title</csw:ElementName>
        <csw:Constraint version="1.1.0">
            <Filter xmlns=["http://www.opengis.net/ogc"](http://www.opengis.net/ogc)>
                <PropertyIsLike wildCard="*" escape="\" singleChar="$">
                    <PropertyName>apiso:ResourceIdentifier</PropertyName>
                    <Literal>http:$$www.sandre.eaufrance.fr$?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:html</

Literal>
                </PropertyIsLike>
            </Filter>
        </csw:Constraint>
    </csw:Query>
</csw:GetRecords>

does not work :
<?xml version="1.0" encoding="UTF-8"?>
<csw:GetRecords xmlns:csw=["http://www.opengis.net/cat/csw/2.0.2"](http://www.opengis.net/cat/csw/2.0.2) xmlns:dc=["http://purl.org/dc/elements/1.1/"](http://purl.org/dc/elements/1.1/) xmlns:ogc=["http://www.opengis.net/ogc"](http://www.opengis.net/ogc) xmlns:gml=["http://www.opengis.net/gml"](http://www.opengis.net/gml) xmlns:gmd=["http://www.isotc211.org/2005/gmd"](http://www.isotc211.org/2005/gmd) xmlns:xsi=["http://www.w3.org/2001/XMLSchema-instance"](http://www.w3.org/2001/XMLSchema-instance) xsi:schemaLocation=["http://www.opengis.net/cat/csw/2.0.2 http://schemas.opengis.net/csw/2.0.2/CSW-discover!
 y.xsd"](http://www.opengis.net/cat/csw/2.0.2http://schemas.opengis.net/csw/2.0.2/CSW-discovery.xsd)

 service="CSW" version="2.0.2" resultType="results" outputSchema=["http://www.isotc211.org/2005/gmd"](http://www.isotc211.org/2005/gmd) startPosition="1" maxRecords="50">
    <csw:Query typeNames="gmd:MD_Metadata">
        <csw:ElementName>/gmd:MD_Metadata/gmd:identificationInfo/gmd:MD_DataIdentification/gmd:citation/gmd:CI_Citation/gmd:title</csw:ElementName>
        <csw:Constraint version="1.1.0">
            <Filter xmlns=["http://www.opengis.net/ogc"](http://www.opengis.net/ogc)>
                <PropertyIsLike wildCard="*" escape="\" singleChar="$">
                    <PropertyName>apiso:ResourceIdentifier</PropertyName>
                    <Literal>http:$$www.sandre.eaufrance.fr$?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:html</Literal>
                </PropertyIsLike>
            </Filter>
        </csw:Constraint>
    </csw:Query>
</csw:GetRecords>

* GetDomain : a simple getdomain on keywords, receives an error in the response.

Best,
Sylvain
Keep yourself connected to Go Parallel:
VERIFY Test and improve your parallel project with help from experts
and peers. http://goparallel.sourceforge.net

GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

Sylvain_GRELLET · November 29, 2012, 5:17pm

Hi Jesse,

Isn’t there other changes to be committed ?
I have tried twice compiling your commit (the last time on a brand new local folder) and I still have the same behaviour.
I confirm the modification appears in the LuceneSearcher.java I have locally before compiling.

Sylvain

···

On 29/11/2012 15:09, Jesse Eichar wrote:

Try that. My commit is a quick fix. I have to take a couple days as some point to implement a more efficient solution, but now it has the same efficiency as 2.6.4.

On Thu, Nov 29, 2012 at 3:00 PM, Jesse Eichar <jesse.eichar@anonymised.com> wrote:
That is my bug. I fixed it on 2.8.x and master but forgot to port it back to 2.6.5. I will do that now.

Jesse

On Thu, Nov 29, 2012 at 2:57 PM, Sylvain GRELLET <s.grellet@anonymised.com> wrote:
Hi Simon,

As I said in another e-mail, 2.6.5 snapshot including your commit is now in production. GetDomain and escapeChar aspects are fine now.

We now encounter an issue with a CSW request that was fine with 2_6_4.
It seems that, a csw query including spatial filter (version 1.1.0 or 1.0.0) makes the paging behave weirdly.

In the response, the numberOfRecordsMatched always equals the nextRecord value when the maxRecords of your query is (strictly less than the real number of matched record minus 1).

To be sure I was not wrong in my CSW request I reproduced the query that is in page 122/127 of the “OpenGIS Catalogue Services Specification 2.0.2 - ISO Metadata Application Profile (1.0.0)” - 07-045
I just changed the word searched in the title and adapted the bbox to Metropolitan France

You can try the query below, and change the maxRecords.
If you put it to 200, you discover that the actual number of matched records is 30.
But if you set it to 5, it responds : numberOfRecordsMatched=“6” numberOfRecordsReturned=“5” elementSet=“brief” nextRecord=“6”

If you just remove the Intersects part of the filter, paging restarts working properly. The numberOfRecordsMatched is of course different (no spatial filter anymore).

Our CSW endpoint : http://services.sandre.eaufrance.fr/geonetwork_CSW/srv/fr/csw

The query:
<GetRecords xmlns=“http://www.opengis.net/cat/csw/2.0.2” xmlns:csw=“http://www.opengis.net/cat/csw/2.0.2” xmlns:ogc=“http://www.opengis.net/ogc” xmlns:ows=“http://www.opengis” xmlns:xsd=“http://www.w3.org/2001/XMLSchema” xmlns:dc=“http://purl.org/dc/elements/1.1/” xmlns:dct=“http://purl.org/dc/terms/” xmlns:gml=“http://www.opengis.net/gml” xmlns:xsi=“http://www.w3.org/2001/XMLSchema-instance” service=“CSW” version=“2.0.2” resultType=“results” outputFormat=“application/xml” outputSchema=“http://www.opengis.net/cat/csw/2.0.2” startPosition=“1” maxRecords=“5”>

brief

ogc:Filter
ogc:And
<ogc:PropertyIsLike escapeChar=“" singleChar=”?"
wildCard=“*”>
ogc:PropertyNamedc:title</ogc:PropertyName>
ogc:Literaleau</ogc:Literal>
</ogc:PropertyIsLike>
ogc:PropertyIsEqualTo
ogc:PropertyNamedc:type</ogc:PropertyName>
ogc:Literaldataset</ogc:Literal>
</ogc:PropertyIsEqualTo>
ogc:Intersects
ogc:PropertyNameows:BoundingBox</ogc:PropertyName>
gml:Envelope
gml:lowerCorner-6 41</gml:lowerCorner>
gml:upperCorner10 51</gml:upperCorner>
</gml:Envelope>
</ogc:Intersects>
</ogc:And>
</ogc:Filter>

Cheers
Sylvain

On 06/11/2012 09:12, Simon.Pigot@anonymised.com wrote:
Hi Sylvain,

I fixed the GetDomain problem in commit 582bb0f371b00023a41659fc817c0e280e276580 - it was a runtime error generated by Lucene 3.6.x about an unsupported operation we were using on an IndexReader.

Your other queries:

1. Your query filters below are actually not valid because the attribute escape is not valid on PropertyIsLike - it should be called escapeChar. My own feeling is that if the filter is not valid then an exception should be returned to the user (or at least there should be a config option to force failure on invalid queries). In this case no exception is returned and things just continue on (I think the recent change to geotools 8.2 in trunk may have forced a change in this behaviour from memory as invalid queries don't return a Filter object in 8.2).

2. After fixing the trivial escapeChar problem above I got all queries that used wildcards with PropertyIsLike to work (2nd and 3rd query below) and the first which did not use a wildcard works with PropertyIsEqualTo (I suspect it should work with PropertyIsLike and no wildcards but I saw the literal being broken up around the '/' char by the query parser when I used PropertyIsLike without a wildcard).

Cheers,
Simon  
________________________________________
From: Sylvain GRELLET [[s.grellet@anonymised.com](mailto:s.grellet@anonymised.com)]
Sent: Sunday, 4 November 2012 11:48 PM
To: [geonetwork-devel@lists.sourceforge.net](mailto:geonetwork-devel@lists.sourceforge.net)
Subject: [GeoNetwork-devel] CSW : regression between 2.6.4 and 2.6.5    snapshot ?

Dear list,

We have deployed a 2.6.5 shapshot to test the upgrade to the new version of lucene but we ran into 3 major issues with the CSW. All with queries that ran with a 2.6.4 a seemed compatible with the OGC specs.
Our revision is : [https://github.com/geonetwork/core-geonetwork/commit/76fcaf6078022c94444f1fb68f115d2f656c39e3](https://github.com/geonetwork/core-geonetwork/commit/76fcaf6078022c94444f1fb68f115d2f656c39e3)

* GetRecords: issue with literal containing "/"
example :
<?xml version="1.0"?>
<csw:GetRecords xmlns:csw=["http://www.opengis.net/cat/csw/2.0.2"](http://www.opengis.net/cat/csw/2.0.2)
 xmlns:gmd=["http://www.isotc211.org/2005/gmd"](http://www.isotc211.org/2005/gmd) service="CSW"
 version="2.0.2" resultType="results">
    <csw:Query typeNames="csw:Record">
        <csw:ElementName>dc:title</csw:ElementName>
        <csw:Constraint version="1.1.0">
            <Filter xmlns=["http://www.opengis.net/ogc"](http://www.opengis.net/ogc)>
                <PropertyIsLike wildCard="*" escape="\" singleChar="$">
                    <PropertyName>apiso:ResourceIdentifier</PropertyName>
                    <Literal>[http://www.sandre.eaufrance.fr/?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:html](http://www.sandre.eaufrance.fr/?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:html)</Literal>
                </PropertyIsLike>
            </Filter>
        </csw:Constraint>
    </csw:Query>
</csw:GetRecords>
======================================================
If we replace "/" by the joker (ex : "$") in the content of the Literal tag, search works (with : http:$$www.sandre.eaufrance.fr$?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:htm)
Using the escape char doesn't work
=> it seems as some XPath interpretation is done on the content of Literal

* Bug with typeNames="gmd:MD_Metadata"
The same request with :
-  <csw:Query typeNames="csw:Record"> works : we received a response with the requested fields
-  <csw:Query typeNames="gmd:MD_Metadata"> does not work : we receive a response with empty tags.
=> we feel there is a link with this issue : [http://trac.osgeo.org/geonetwork/ticket/724](http://trac.osgeo.org/geonetwork/ticket/724)

works :
<?xml version="1.0"?>
<csw:GetRecords xmlns:csw=["http://www.opengis.net/cat/csw/2.0.2"](http://www.opengis.net/cat/csw/2.0.2)
 xmlns:gmd=["http://www.isotc211.org/2005/gmd"](http://www.isotc211.org/2005/gmd) service="CSW"
 version="2.0.2" resultType="results">
    <csw:Query typeNames="csw:Record">
        <csw:ElementName>dc:title</csw:ElementName>
        <csw:Constraint version="1.1.0">
            <Filter xmlns=["http://www.opengis.net/ogc"](http://www.opengis.net/ogc)>
                <PropertyIsLike wildCard="*" escape="\" singleChar="$">
                    <PropertyName>apiso:ResourceIdentifier</PropertyName>
                    <Literal>http:$$www.sandre.eaufrance.fr$?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:html</

Literal>
                </PropertyIsLike>
            </Filter>
        </csw:Constraint>
    </csw:Query>
</csw:GetRecords>

does not work :
<?xml version="1.0" encoding="UTF-8"?>
<csw:GetRecords xmlns:csw=["http://www.opengis.net/cat/csw/2.0.2"](http://www.opengis.net/cat/csw/2.0.2) xmlns:dc=["http://purl.org/dc/elements/1.1/"](http://purl.org/dc/elements/1.1/) xmlns:ogc=["http://www.opengis.net/ogc"](http://www.opengis.net/ogc) xmlns:gml=["http://www.opengis.net/gml"](http://www.opengis.net/gml) xmlns:gmd=["http://www.isotc211.org/2005/gmd"](http://www.isotc211.org/2005/gmd) xmlns:xsi=["http://www.w3.org/2001/XMLSchema-instance"](http://www.w3.org/2001/XMLSchema-instance) xsi:schemaLocation=["http://www.opengis.net/!
 cat/csw/2.
0.2 http://schemas.opengis.net/csw/2.0.2/CSW-discover!
 y.xsd"](http://www.opengis.net/cat/csw/2.0.2http://schemas.opengis.net/csw/2.0.2/CSW-discovery.xsd)

 service="CSW" version="2.0.2" resultType="results" outputSchema=["http://www.isotc211.org/2005/gmd"](http://www.isotc211.org/2005/gmd) startPosition="1" maxRecords="50">
    <csw:Query typeNames="gmd:MD_Metadata">
        <csw:ElementName>/gmd:MD_Metadata/gmd:identificationInfo/gmd:MD_DataIdentification/gmd:citation/gmd:CI_Citation/gmd:title</csw:ElementName>
        <csw:Constraint version="1.1.0">
            <Filter xmlns=["http://www.opengis.net/ogc"](http://www.opengis.net/ogc)>
                <PropertyIsLike wildCard="*" escape="\" singleChar="$">
                    <PropertyName>apiso:ResourceIdentifier</PropertyName>
                    <Literal>http:$$www.sandre.eaufrance.fr$?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:html</Literal>
                </PropertyIsLike>
            </Filter>
        </csw:Constraint>
    </csw:Query>
</csw:GetRecords>

* GetDomain : a simple getdomain on keywords, receives an error in the response.

Best,
Sylvain
Keep yourself connected to Go Parallel:
VERIFY Test and improve your parallel project with help from experts
and peers. http://goparallel.sourceforge.net

GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

Jesse_Eichar2 · November 29, 2012, 7:14pm

odd, that fixed a similar problem for trunk… oh. On trunk I fixed the XmlSearch. CSW probably needs a different fix. 1 sec…

···

On Thu, Nov 29, 2012 at 6:17 PM, Sylvain GRELLET <s.grellet@anonymised.com> wrote:

Hi Jesse,

Isn’t there other changes to be committed ?
I have tried twice compiling your commit (the last time on a brand new local folder) and I still have the same behaviour.
I confirm the modification appears in the LuceneSearcher.java I have locally before compiling.

Sylvain

On 29/11/2012 15:09, Jesse Eichar wrote:

Try that. My commit is a quick fix. I have to take a couple days as some point to implement a more efficient solution, but now it has the same efficiency as 2.6.4.

On Thu, Nov 29, 2012 at 3:00 PM, Jesse Eichar <jesse.eichar@anonymised.com> wrote:

That is my bug. I fixed it on 2.8.x and master but forgot to port it back to 2.6.5. I will do that now.

Jesse

On Thu, Nov 29, 2012 at 2:57 PM, Sylvain GRELLET <s.grellet@anonymised.com> wrote:

Hi Simon,

As I said in another e-mail, 2.6.5 snapshot including your commit is now in production. GetDomain and escapeChar aspects are fine now.

We now encounter an issue with a CSW request that was fine with 2_6_4.
It seems that, a csw query including spatial filter (version 1.1.0 or 1.0.0) makes the paging behave weirdly.

In the response, the numberOfRecordsMatched always equals the nextRecord value when the maxRecords of your query is (strictly less than the real number of matched record minus 1).

To be sure I was not wrong in my CSW request I reproduced the query that is in page 122/127 of the “OpenGIS Catalogue Services Specification 2.0.2 - ISO Metadata Application Profile (1.0.0)” - 07-045
I just changed the word searched in the title and adapted the bbox to Metropolitan France

You can try the query below, and change the maxRecords.
If you put it to 200, you discover that the actual number of matched records is 30.
But if you set it to 5, it responds : numberOfRecordsMatched=“6” numberOfRecordsReturned=“5” elementSet=“brief” nextRecord=“6”

If you just remove the Intersects part of the filter, paging restarts working properly. The numberOfRecordsMatched is of course different (no spatial filter anymore).

Our CSW endpoint : http://services.sandre.eaufrance.fr/geonetwork_CSW/srv/fr/csw

The query:
<GetRecords xmlns=“http://www.opengis.net/cat/csw/2.0.2” xmlns:csw=“http://www.opengis.net/cat/csw/2.0.2” xmlns:ogc=“http://www.opengis.net/ogc” xmlns:ows=“http://www.opengis” xmlns:xsd=“http://www.w3.org/2001/XMLSchema” xmlns:dc=“http://purl.org/dc/elements/1.1/” xmlns:dct=“http://purl.org/dc/terms/” xmlns:gml=“http://www.opengis.net/gml” xmlns:xsi=“http://www.w3.org/2001/XMLSchema-instance” service=“CSW” version=“2.0.2” resultType=“results” outputFormat=“application/xml” outputSchema=“http://www.opengis.net/cat/csw/2.0.2” startPosition=“1” maxRecords=“5”>

brief

ogc:Filter
ogc:And
<ogc:PropertyIsLike escapeChar=“" singleChar=”?"
wildCard=“*”>
ogc:PropertyNamedc:title</ogc:PropertyName>
ogc:Literaleau</ogc:Literal>
</ogc:PropertyIsLike>
ogc:PropertyIsEqualTo
ogc:PropertyNamedc:type</ogc:PropertyName>
ogc:Literaldataset</ogc:Literal>
</ogc:PropertyIsEqualTo>
ogc:Intersects
ogc:PropertyNameows:BoundingBox</ogc:PropertyName>
gml:Envelope
gml:lowerCorner-6 41</gml:lowerCorner>
gml:upperCorner10 51</gml:upperCorner>
</gml:Envelope>
</ogc:Intersects>
</ogc:And>
</ogc:Filter>

Cheers
Sylvain

On 06/11/2012 09:12, Simon.Pigot@anonymised.com wrote:
Hi Sylvain,

I fixed the GetDomain problem in commit 582bb0f371b00023a41659fc817c0e280e276580 - it was a runtime error generated by Lucene 3.6.x about an unsupported operation we were using on an IndexReader.

Your other queries:

1. Your query filters below are actually not valid because the attribute escape is not valid on PropertyIsLike - it should be called escapeChar. My own feeling is that if the filter is not valid then an exception should be returned to the user (or at least there should be a config option to force failure on invalid queries). In this case no exception is returned and things just continue on (I think the recent change to geotools 8.2 in trunk may have forced a change in this behaviour from memory as invalid queries don't return a Filter object in 8.2).

2. After fixing the trivial escapeChar problem above I got all queries that used wildcards with PropertyIsLike to work (2nd and 3rd query below) and the first which did not use a wildcard works with PropertyIsEqualTo (I suspect it should work with PropertyIsLike and no wildcards but I saw the literal being broken up around the '/' char by the query parser when I used PropertyIsLike without a wildcard).

Cheers,
Simon  
________________________________________
From: Sylvain GRELLET [[s.grellet@anonymised.com](mailto:s.grellet@anonymised.com)]
Sent: Sunday, 4 November 2012 11:48 PM
To: [geonetwork-devel@lists.sourceforge.net](mailto:geonetwork-devel@lists.sourceforge.net)
Subject: [GeoNetwork-devel] CSW : regression between 2.6.4 and 2.6.5    snapshot ?

Dear list,

We have deployed a 2.6.5 shapshot to test the upgrade to the new version of lucene but we ran into 3 major issues with the CSW. All with queries that ran with a 2.6.4 a seemed compatible with the OGC specs.
Our revision is : [https://github.com/geonetwork/core-geonetwork/commit/76fcaf6078022c94444f1fb68f115d2f656c39e3](https://github.com/geonetwork/core-geonetwork/commit/76fcaf6078022c94444f1fb68f115d2f656c39e3)

* GetRecords: issue with literal containing "/"
example :
<?xml version="1.0"?>
<csw:GetRecords xmlns:csw=["http://www.opengis.net/cat/csw/2.0.2"](http://www.opengis.net/cat/csw/2.0.2)
 xmlns:gmd=["http://www.isotc211.org/2005/gmd"](http://www.isotc211.org/2005/gmd) service="CSW"
 version="2.0.2" resultType="results">
    <csw:Query typeNames="csw:Record">
        <csw:ElementName>dc:title</csw:ElementName>
        <csw:Constraint version="1.1.0">
            <Filter xmlns=["http://www.opengis.net/ogc"](http://www.opengis.net/ogc)>
                <PropertyIsLike wildCard="*" escape="\" singleChar="$">
                    <PropertyName>apiso:ResourceIdentifier</PropertyName>
                    <Literal>[http://www.sandre.eaufrance.fr/?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:html](http://www.sandre.eaufrance.fr/?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:html)</Literal>
                </PropertyIsLike>
            </Filter>
        </csw:Constraint>
    </csw:Query>
</csw:GetRecords>
======================================================
If we replace "/" by the joker (ex : "$") in the content of the Literal tag, search works (with : http:$$www.sandre.eaufrance.fr$?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:htm)
Using the escape char doesn't work
=> it seems as some XPath interpretation is done on the content of Literal

* Bug with typeNames="gmd:MD_Metadata"
The same request with :
-  <csw:Query typeNames="csw:Record"> works : we received a response with the requested fields
-  <csw:Query typeNames="gmd:MD_Metadata"> does not work : we receive a response with empty tags.
=> we feel there is a link with this issue : [http://trac.osgeo.org/geonetwork/ticket/724](http://trac.osgeo.org/geonetwork/ticket/724)

works :
<?xml version="1.0"?>
<csw:GetRecords xmlns:csw=["http://www.opengis.net/cat/csw/2.0.2"](http://www.opengis.net/cat/csw/2.0.2)
 xmlns:gmd=["http://www.isotc211.org/2005/gmd"](http://www.isotc211.org/2005/gmd) service="CSW"
 version="2.0.2" resultType="results">
    <csw:Query typeNames="csw:Record">
        <csw:ElementName>dc:title</csw:ElementName>
        <csw:Constraint version="1.1.0">
            <Filter xmlns=["http://www.opengis.net/ogc"](http://www.opengis.net/ogc)>
                <PropertyIsLike wildCard="*" escape="\" singleChar="$">
                    <PropertyName>apiso:ResourceIdentifier</PropertyName>
                    <Literal>http:$$www.sandre.eaufrance.fr$?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:html</

Literal>
                </PropertyIsLike>
            </Filter>
        </csw:Constraint>
    </csw:Query>
</csw:GetRecords>

does not work :
<?xml version="1.0" encoding="UTF-8"?>
<csw:GetRecords xmlns:csw=["http://www.opengis.net/cat/csw/2.0.2"](http://www.opengis.net/cat/csw/2.0.2) xmlns:dc=["http://purl.org/dc/elements/1.1/"](http://purl.org/dc/elements/1.1/) xmlns:ogc=["http://www.opengis.net/ogc"](http://www.opengis.net/ogc) xmlns:gml=["http://www.opengis.net/gml"](http://www.opengis.net/gml) xmlns:gmd=["http://www.isotc211.org/2005/gmd"](http://www.isotc211.org/2005/gmd) xmlns:xsi=["http://www.w3.org/2001/XMLSchema-instance"](http://www.w3.org/2001/XMLSchema-instance) xsi:schemaLocation=["http://www.opengis.net/!
 cat/csw/2.
0.2 http://schemas.opengis.net/csw/2.0.2/CSW-discover!
 y.xsd"](http://www.opengis.net/cat/csw/2.0.2http://schemas.opengis.net/csw/2.0.2/CSW-discovery.xsd)

 service="CSW" version="2.0.2" resultType="results" outputSchema=["http://www.isotc211.org/2005/gmd"](http://www.isotc211.org/2005/gmd) startPosition="1" maxRecords="50">
    <csw:Query typeNames="gmd:MD_Metadata">
        <csw:ElementName>/gmd:MD_Metadata/gmd:identificationInfo/gmd:MD_DataIdentification/gmd:citation/gmd:CI_Citation/gmd:title</csw:ElementName>
        <csw:Constraint version="1.1.0">
            <Filter xmlns=["http://www.opengis.net/ogc"](http://www.opengis.net/ogc)>
                <PropertyIsLike wildCard="*" escape="\" singleChar="$">
                    <PropertyName>apiso:ResourceIdentifier</PropertyName>
                    <Literal>http:$$www.sandre.eaufrance.fr$?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:html</Literal>
                </PropertyIsLike>
            </Filter>
        </csw:Constraint>
    </csw:Query>
</csw:GetRecords>

* GetDomain : a simple getdomain on keywords, receives an error in the response.

Best,
Sylvain
Keep yourself connected to Go Parallel:
VERIFY Test and improve your parallel project with help from experts
and peers. http://goparallel.sourceforge.net

GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

Jesse_Eichar2 · November 29, 2012, 8:03pm

Sorry about that. Fixed the issue on master, 2.8.x and 2.6.x

···

On Thu, Nov 29, 2012 at 8:14 PM, Jesse Eichar <jesse.eichar@anonymised.com> wrote:

odd, that fixed a similar problem for trunk… oh. On trunk I fixed the XmlSearch. CSW probably needs a different fix. 1 sec…

On Thu, Nov 29, 2012 at 6:17 PM, Sylvain GRELLET <s.grellet@anonymised.com> wrote:

Hi Jesse,

Isn’t there other changes to be committed ?
I have tried twice compiling your commit (the last time on a brand new local folder) and I still have the same behaviour.
I confirm the modification appears in the LuceneSearcher.java I have locally before compiling.

Sylvain

On 29/11/2012 15:09, Jesse Eichar wrote:

Try that. My commit is a quick fix. I have to take a couple days as some point to implement a more efficient solution, but now it has the same efficiency as 2.6.4.

On Thu, Nov 29, 2012 at 3:00 PM, Jesse Eichar <jesse.eichar@anonymised.com> wrote:

That is my bug. I fixed it on 2.8.x and master but forgot to port it back to 2.6.5. I will do that now.

Jesse

On Thu, Nov 29, 2012 at 2:57 PM, Sylvain GRELLET <s.grellet@anonymised.com> wrote:

Hi Simon,

As I said in another e-mail, 2.6.5 snapshot including your commit is now in production. GetDomain and escapeChar aspects are fine now.

We now encounter an issue with a CSW request that was fine with 2_6_4.
It seems that, a csw query including spatial filter (version 1.1.0 or 1.0.0) makes the paging behave weirdly.

In the response, the numberOfRecordsMatched always equals the nextRecord value when the maxRecords of your query is (strictly less than the real number of matched record minus 1).

To be sure I was not wrong in my CSW request I reproduced the query that is in page 122/127 of the “OpenGIS Catalogue Services Specification 2.0.2 - ISO Metadata Application Profile (1.0.0)” - 07-045
I just changed the word searched in the title and adapted the bbox to Metropolitan France

You can try the query below, and change the maxRecords.
If you put it to 200, you discover that the actual number of matched records is 30.
But if you set it to 5, it responds : numberOfRecordsMatched=“6” numberOfRecordsReturned=“5” elementSet=“brief” nextRecord=“6”

If you just remove the Intersects part of the filter, paging restarts working properly. The numberOfRecordsMatched is of course different (no spatial filter anymore).

Our CSW endpoint : http://services.sandre.eaufrance.fr/geonetwork_CSW/srv/fr/csw

The query:
<GetRecords xmlns=“http://www.opengis.net/cat/csw/2.0.2” xmlns:csw=“http://www.opengis.net/cat/csw/2.0.2” xmlns:ogc=“http://www.opengis.net/ogc” xmlns:ows=“http://www.opengis” xmlns:xsd=“http://www.w3.org/2001/XMLSchema” xmlns:dc=“http://purl.org/dc/elements/1.1/” xmlns:dct=“http://purl.org/dc/terms/” xmlns:gml=“http://www.opengis.net/gml” xmlns:xsi=“http://www.w3.org/2001/XMLSchema-instance” service=“CSW” version=“2.0.2” resultType=“results” outputFormat=“application/xml” outputSchema=“http://www.opengis.net/cat/csw/2.0.2” startPosition=“1” maxRecords=“5”>

brief

ogc:Filter
ogc:And
<ogc:PropertyIsLike escapeChar=“" singleChar=”?"
wildCard=“*”>
ogc:PropertyNamedc:title</ogc:PropertyName>
ogc:Literaleau</ogc:Literal>
</ogc:PropertyIsLike>
ogc:PropertyIsEqualTo
ogc:PropertyNamedc:type</ogc:PropertyName>
ogc:Literaldataset</ogc:Literal>
</ogc:PropertyIsEqualTo>
ogc:Intersects
ogc:PropertyNameows:BoundingBox</ogc:PropertyName>
gml:Envelope
gml:lowerCorner-6 41</gml:lowerCorner>
gml:upperCorner10 51</gml:upperCorner>
</gml:Envelope>
</ogc:Intersects>
</ogc:And>
</ogc:Filter>

Cheers
Sylvain

On 06/11/2012 09:12, Simon.Pigot@anonymised.com wrote:
Hi Sylvain,

I fixed the GetDomain problem in commit 582bb0f371b00023a41659fc817c0e280e276580 - it was a runtime error generated by Lucene 3.6.x about an unsupported operation we were using on an IndexReader.

Your other queries:

1. Your query filters below are actually not valid because the attribute escape is not valid on PropertyIsLike - it should be called escapeChar. My own feeling is that if the filter is not valid then an exception should be returned to the user (or at least there should be a config option to force failure on invalid queries). In this case no exception is returned and things just continue on (I think the recent change to geotools 8.2 in trunk may have forced a change in this behaviour from memory as invalid queries don't return a Filter object in 8.2).

2. After fixing the trivial escapeChar problem above I got all queries that used wildcards with PropertyIsLike to work (2nd and 3rd query below) and the first which did not use a wildcard works with PropertyIsEqualTo (I suspect it should work with PropertyIsLike and no wildcards but I saw the literal being broken up around the '/' char by the query parser when I used PropertyIsLike without a wildcard).

Cheers,
Simon  
________________________________________
From: Sylvain GRELLET [[s.grellet@anonymised.com](mailto:s.grellet@anonymised.com)]
Sent: Sunday, 4 November 2012 11:48 PM
To: [geonetwork-devel@lists.sourceforge.net](mailto:geonetwork-devel@lists.sourceforge.net)
Subject: [GeoNetwork-devel] CSW : regression between 2.6.4 and 2.6.5    snapshot ?

Dear list,

We have deployed a 2.6.5 shapshot to test the upgrade to the new version of lucene but we ran into 3 major issues with the CSW. All with queries that ran with a 2.6.4 a seemed compatible with the OGC specs.
Our revision is : [https://github.com/geonetwork/core-geonetwork/commit/76fcaf6078022c94444f1fb68f115d2f656c39e3](https://github.com/geonetwork/core-geonetwork/commit/76fcaf6078022c94444f1fb68f115d2f656c39e3)

* GetRecords: issue with literal containing "/"
example :
<?xml version="1.0"?>
<csw:GetRecords xmlns:csw=["http://www.opengis.net/cat/csw/2.0.2"](http://www.opengis.net/cat/csw/2.0.2)
 xmlns:gmd=["http://www.isotc211.org/2005/gmd"](http://www.isotc211.org/2005/gmd) service="CSW"
 version="2.0.2" resultType="results">
    <csw:Query typeNames="csw:Record">
        <csw:ElementName>dc:title</csw:ElementName>
        <csw:Constraint version="1.1.0">
            <Filter xmlns=["http://www.opengis.net/ogc"](http://www.opengis.net/ogc)>
                <PropertyIsLike wildCard="*" escape="\" singleChar="$">
                    <PropertyName>apiso:ResourceIdentifier</PropertyName>
                    <Literal>[http://www.sandre.eaufrance.fr/?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:html](http://www.sandre.eaufrance.fr/?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:html)</Literal>
                </PropertyIsLike>
            </Filter>
        </csw:Constraint>
    </csw:Query>
</csw:GetRecords>
======================================================
If we replace "/" by the joker (ex : "$") in the content of the Literal tag, search works (with : http:$$www.sandre.eaufrance.fr$?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:htm)
Using the escape char doesn't work
=> it seems as some XPath interpretation is done on the content of Literal

* Bug with typeNames="gmd:MD_Metadata"
The same request with :
-  <csw:Query typeNames="csw:Record"> works : we received a response with the requested fields
-  <csw:Query typeNames="gmd:MD_Metadata"> does not work : we receive a response with empty tags.
=> we feel there is a link with this issue : [http://trac.osgeo.org/geonetwork/ticket/724](http://trac.osgeo.org/geonetwork/ticket/724)

works :
<?xml version="1.0"?>
<csw:GetRecords xmlns:csw=["http://www.opengis.net/cat/csw/2.0.2"](http://www.opengis.net/cat/csw/2.0.2)
 xmlns:gmd=["http://www.isotc211.org/2005/gmd"](http://www.isotc211.org/2005/gmd) service="CSW"
 version="2.0.2" resultType="results">
    <csw:Query typeNames="csw:Record">
        <csw:ElementName>dc:title</csw:ElementName>
        <csw:Constraint version="1.1.0">
            <Filter xmlns=["http://www.opengis.net/ogc"](http://www.opengis.net/ogc)>
                <PropertyIsLike wildCard="*" escape="\" singleChar="$">
                    <PropertyName>apiso:ResourceIdentifier</PropertyName>
                    <Literal>http:$$www.sandre.eaufrance.fr$?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:html</

Literal>
                </PropertyIsLike>
            </Filter>
        </csw:Constraint>
    </csw:Query>
</csw:GetRecords>

does not work :
<?xml version="1.0" encoding="UTF-8"?>
<csw:GetRecords xmlns:csw=["http://www.opengis.net/cat/csw/2.0.2"](http://www.opengis.net/cat/csw/2.0.2) xmlns:dc=["http://purl.org/dc/elements/1.1/"](http://purl.org/dc/elements/1.1/) xmlns:ogc=["http://www.opengis.net/ogc"](http://www.opengis.net/ogc) xmlns:gml=["http://www.opengis.net/gml"](http://www.opengis.net/gml) xmlns:gmd=["http://www.isotc211.org/2005/gmd"](http://www.isotc211.org/2005/gmd) xmlns:xsi=["http://www.w3.org/2001/XMLSchema-instance"](http://www.w3.org/2001/XMLSchema-instance) xsi:schemaLocation=["http://www.opengis.net/!
 cat/csw/2.
0.2 http://schemas.opengis.net/csw/2.0.2/CSW-discover!
 y.xsd"](http://www.opengis.net/cat/csw/2.0.2http://schemas.opengis.net/csw/2.0.2/CSW-discovery.xsd)

 service="CSW" version="2.0.2" resultType="results" outputSchema=["http://www.isotc211.org/2005/gmd"](http://www.isotc211.org/2005/gmd) startPosition="1" maxRecords="50">
    <csw:Query typeNames="gmd:MD_Metadata">
        <csw:ElementName>/gmd:MD_Metadata/gmd:identificationInfo/gmd:MD_DataIdentification/gmd:citation/gmd:CI_Citation/gmd:title</csw:ElementName>
        <csw:Constraint version="1.1.0">
            <Filter xmlns=["http://www.opengis.net/ogc"](http://www.opengis.net/ogc)>
                <PropertyIsLike wildCard="*" escape="\" singleChar="$">
                    <PropertyName>apiso:ResourceIdentifier</PropertyName>
                    <Literal>http:$$www.sandre.eaufrance.fr$?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:html</Literal>
                </PropertyIsLike>
            </Filter>
        </csw:Constraint>
    </csw:Query>
</csw:GetRecords>

* GetDomain : a simple getdomain on keywords, receives an error in the response.

Best,
Sylvain
Keep yourself connected to Go Parallel:
VERIFY Test and improve your parallel project with help from experts
and peers. http://goparallel.sourceforge.net

GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

Sylvain_GRELLET · November 30, 2012, 8:52am

Sorry this is still not it with your latest commit : https://github.com/geonetwork/core-geonetwork/commit/93128576608d888fd36b3c18581596ec1560a6d1
I confirm, I have the modification in CatalogSearcher.java.

Maybe another update missing.
No need to rush as I’ll be travelling most of next week.

Best
Sylvain

···

On 29/11/2012 21:03, Jesse Eichar wrote:

Sorry about that. Fixed the issue on master, 2.8.x and 2.6.x

On Thu, Nov 29, 2012 at 8:14 PM, Jesse Eichar <jesse.eichar@anonymised.com> wrote:
odd, that fixed a similar problem for trunk… oh. On trunk I fixed the XmlSearch. CSW probably needs a different fix. 1 sec…

On Thu, Nov 29, 2012 at 6:17 PM, Sylvain GRELLET <s.grellet@anonymised.com> wrote:
Hi Jesse,

Isn’t there other changes to be committed ?
I have tried twice compiling your commit (the last time on a brand new local folder) and I still have the same behaviour.
I confirm the modification appears in the LuceneSearcher.java I have locally before compiling.

Sylvain

On 29/11/2012 15:09, Jesse Eichar wrote:
Try that. My commit is a quick fix. I have to take a couple days as some point to implement a more efficient solution, but now it has the same efficiency as 2.6.4.

On Thu, Nov 29, 2012 at 3:00 PM, Jesse Eichar <jesse.eichar@anonymised.com> wrote:
That is my bug. I fixed it on 2.8.x and master but forgot to port it back to 2.6.5. I will do that now.

Jesse

On Thu, Nov 29, 2012 at 2:57 PM, Sylvain GRELLET <s.grellet@anonymised.com> wrote:
Hi Simon,

As I said in another e-mail, 2.6.5 snapshot including your commit is now in production. GetDomain and escapeChar aspects are fine now.

We now encounter an issue with a CSW request that was fine with 2_6_4.
It seems that, a csw query including spatial filter (version 1.1.0 or 1.0.0) makes the paging behave weirdly.

In the response, the numberOfRecordsMatched always equals the nextRecord value when the maxRecords of your query is (strictly less than the real number of matched record minus 1).

To be sure I was not wrong in my CSW request I reproduced the query that is in page 122/127 of the “OpenGIS Catalogue Services Specification 2.0.2 - ISO Metadata Application Profile (1.0.0)” - 07-045
I just changed the word searched in the title and adapted the bbox to Metropolitan France

You can try the query below, and change the maxRecords.
If you put it to 200, you discover that the actual number of matched records is 30.
But if you set it to 5, it responds : numberOfRecordsMatched=“6” numberOfRecordsReturned=“5” elementSet=“brief” nextRecord=“6”

If you just remove the Intersects part of the filter, paging restarts working properly. The numberOfRecordsMatched is of course different (no spatial filter anymore).

Our CSW endpoint : http://services.sandre.eaufrance.fr/geonetwork_CSW/srv/fr/csw

The query:
<GetRecords xmlns=“http://www.opengis.net/cat/csw/2.0.2” xmlns:csw=“http://www.opengis.net/cat/csw/2.0.2” xmlns:ogc=“http://www.opengis.net/ogc” xmlns:ows=“http://www.opengis” xmlns:xsd=“http://www.w3.org/2001/XMLSchema” xmlns:dc=“http://purl.org/dc/elements/1.1/” xmlns:dct=“http://purl.org/dc/terms/” xmlns:gml=“http://www.opengis.net/gml” xmlns:xsi=“http://www.w3.org/2001/XMLSchema-instance” service=“CSW” version=“2.0.2” resultType=“results” outputFormat=“application/xml” outputSchema=“http://www.opengis.net/cat/csw/2.0.2” startPosition=“1” maxRecords=“5”>

brief

ogc:Filter
ogc:And
<ogc:PropertyIsLike escapeChar=“" singleChar=”?"
wildCard=“*”>
ogc:PropertyNamedc:title</ogc:PropertyName>
ogc:Literaleau</ogc:Literal>
</ogc:PropertyIsLike>
ogc:PropertyIsEqualTo
ogc:PropertyNamedc:type</ogc:PropertyName>
ogc:Literaldataset</ogc:Literal>
</ogc:PropertyIsEqualTo>
ogc:Intersects
ogc:PropertyNameows:BoundingBox</ogc:PropertyName>
gml:Envelope
gml:lowerCorner-6 41</gml:lowerCorner>
gml:upperCorner10 51</gml:upperCorner>
</gml:Envelope>
</ogc:Intersects>
</ogc:And>
</ogc:Filter>

Cheers
Sylvain

On 06/11/2012 09:12, Simon.Pigot@anonymised.com wrote:
Hi Sylvain,

I fixed the GetDomain problem in commit 582bb0f371b00023a41659fc817c0e280e276580 - it was a runtime error generated by Lucene 3.6.x about an unsupported operation we were using on an IndexReader.

Your other queries:

1. Your query filters below are actually not valid because the attribute escape is not valid on PropertyIsLike - it should be called escapeChar. My own feeling is that if the filter is not valid then an exception should be returned to the user (or at least there should be a config option to force failure on invalid queries). In this case no exception is returned and things just continue on (I think the recent change to geotools 8.2 in trunk may have forced a change in this behaviour from memory as invalid queries don't return a Filter object in 8.2).

2. After fixing the trivial escapeChar problem above I got all queries that used wildcards with PropertyIsLike to work (2nd and 3rd query below) and the first which did not use a wildcard works with PropertyIsEqualTo (I suspect it should work with PropertyIsLike and no wildcards but I saw the literal being broken up around the '/' char by the query parser when I used PropertyIsLike without a wildcard).

Cheers,
Simon  
________________________________________
From: Sylvain GRELLET [[s.grellet@anonymised.com](mailto:s.grellet@anonymised.com)]
Sent: Sunday, 4 November 2012 11:48 PM
To: [geonetwork-devel@lists.sourceforge.net](mailto:geonetwork-devel@lists.sourceforge.net)
Subject: [GeoNetwork-devel] CSW : regression between 2.6.4 and 2.6.5    snapshot ?

Dear list,

We have deployed a 2.6.5 shapshot to test the upgrade to the new version of lucene but we ran into 3 major issues with the CSW. All with queries that ran with a 2.6.4 a seemed compatible with the OGC specs.
Our revision is : [https://github.com/geonetwork/core-geonetwork/commit/76fcaf6078022c94444f1fb68f115d2f656c39e3](https://github.com/geonetwork/core-geonetwork/commit/76fcaf6078022c94444f1fb68f115d2f656c39e3)

* GetRecords: issue with literal containing "/"
example :
<?xml version="1.0"?>
<csw:GetRecords xmlns:csw=["http://www.opengis.net/cat/csw/2.0.2"](http://www.opengis.net/cat/csw/2.0.2)
 xmlns:gmd=["http://www.isotc211.org/2005/gmd"](http://www.isotc211.org/2005/gmd) service="CSW"
 version="2.0.2" resultType="results">
    <csw:Query typeNames="csw:Record">
        <csw:ElementName>dc:title</csw:ElementName>
        <csw:Constraint version="1.1.0">
            <Filter xmlns=["http://www.opengis.net/ogc"](http://www.opengis.net/ogc)>
                <PropertyIsLike wildCard="*" escape="\" singleChar="$">
                    <PropertyName>apiso:ResourceIdentifier</PropertyName>
                    <Literal>[http://www.sandre.eaufrance.fr/?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:html](http://www.sandre.eaufrance.fr/?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:html)</Literal>
                </PropertyIsLike>
            </Filter>
        </csw:Constraint>
    </csw:Query>
</csw:GetRecords>
======================================================
If we replace "/" by the joker (ex : "$") in the content of the Literal tag, search works (with : http:$$www.sandre.eaufrance.fr$?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:htm)
Using the escape char doesn't work
=> it seems as some XPath interpretation is done on the content of Literal

* Bug with typeNames="gmd:MD_Metadata"
The same request with :
-  <csw:Query typeNames="csw:Record"> works : we received a response with the requested fields
-  <csw:Query typeNames="gmd:MD_Metadata"> does not work : we receive a response with empty tags.
=> we feel there is a link with this issue : [http://trac.osgeo.org/geonetwork/ticket/724](http://trac.osgeo.org/geonetwork/ticket/724)

works :
<?xml version="1.0"?>
<csw:GetRecords xmlns:csw=["http://www.opengis.net/cat/csw/2.0.2"](http://www.opengis.net/cat/csw/2.0.2)
 xmlns:gmd=["http://www.isotc211.org/2005/gmd"](http://www.isotc211.org/2005/gmd) service="CSW"
 version="2.0.2" resultType="results">
    <csw:Query typeNames="csw:Record">
        <csw:ElementName>dc:title</csw:ElementName>
        <csw:Constraint version="1.1.0">
            <Filter xmlns=["http://www.opengis.net/ogc"](http://www.opengis.net/ogc)>
                <PropertyIsLike wildCard="*" escape="\" singleChar="$">
                    <PropertyName>apiso:ResourceIdentifier</PropertyName>
                    <Literal>http:$$www.sandre.eaufrance.fr$?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:html</

Literal>
                </PropertyIsLike>
            </Filter>
        </csw:Constraint>
    </csw:Query>
</csw:GetRecords>

does not work :
<?xml version="1.0" encoding="UTF-8"?>
<csw:GetRecords xmlns:csw=["http://www.opengis.net/cat/csw/2.0.2"](http://www.opengis.net/cat/csw/2.0.2) xmlns:dc=["http://purl.org/dc/elements/1.1/"](http://purl.org/dc/elements/1.1/) xmlns:ogc=["http://www.opengis.net/ogc"](http://www.opengis.net/ogc) xmlns:gml=["http://www.opengis.net/gml"](http://www.opengis.net/gml) xmlns:gmd=["http://www.isotc211.org/2005/gmd"](http://www.isotc211.org/2005/gmd) xmlns:xsi=["http://www.w3.org/2001/XMLSchema-instance"](http://www.w3.org/2001/XMLSchema-instance) xsi:schemaLocation=["http://www.opengis.net/!
 cat/csw/2.
0.2 http://schemas.opengis.net/csw/2.0.2/CSW-discover!
 y.xsd"](http://www.opengis.net/cat/csw/2.0.2http://schemas.opengis.net/csw/2.0.2/CSW-discovery.xsd)

 service="CSW" version="2.0.2" resultType="results" outputSchema=["http://www.isotc211.org/2005/gmd"](http://www.isotc211.org/2005/gmd) startPosition="1" maxRecords="50">
    <csw:Query typeNames="gmd:MD_Metadata">
        <csw:ElementName>/gmd:MD_Metadata/gmd:identificationInfo/gmd:MD_DataIdentification/gmd:citation/gmd:CI_Citation/gmd:title</csw:ElementName>
        <csw:Constraint version="1.1.0">
            <Filter xmlns=["http://www.opengis.net/ogc"](http://www.opengis.net/ogc)>
                <PropertyIsLike wildCard="*" escape="\" singleChar="$">
                    <PropertyName>apiso:ResourceIdentifier</PropertyName>
                    <Literal>http:$$www.sandre.eaufrance.fr$?urn=urn:sandre:ensembledonnees:BDLISA:FRA:::ressource:2012::Beta:html</Literal>
                </PropertyIsLike>
            </Filter>
        </csw:Constraint>
    </csw:Query>
</csw:GetRecords>

* GetDomain : a simple getdomain on keywords, receives an error in the response.

Best,
Sylvain
Keep yourself connected to Go Parallel:
VERIFY Test and improve your parallel project with help from experts
and peers. http://goparallel.sourceforge.net

GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork