[GeoNetwork-devel] harvester fails with deegree CSW 2.0.2 service

Wolfgang_Grunberg · September 30, 2009, 1:20am

GeoNetwork 2.41’s CSW harvester fails with our deegree 2.3pre CSW service (http://75.101.143.247:8080/deegree-csw/services?REQUEST=GetCapabilities&version=2.0.2&service=CSW).

GeoNetwork’s GetRecords transaction request returns what appears to be a valid CSW and Dublin Core GetRecordsResponse response. I have attached the truncated response. This is a great improvement because deegree didn’t accept the GeoNetwork response in the past.

However, GeoNetwork’s log gives me a “[geonetwork.harvester] - Skipped record not in supported format : metadata” error and I am having difficulties figuring out what the problem is.

What kind of response does GeoNetwork require?
Also, how do I enable more verbose logging in GeoNetwork?

Thank you for your time!

POST request to http://75.101.143.247:8080/deegree-csw/services :

<?xml version="1.0" encoding="UTF-8"?>

<csw:GetRecords xmlns:csw=“http://www.opengis.net/cat/csw/2.0.2” service=“CSW” version=“2.0.2” resultType=“results” startPosition=“1301” maxRecords=“20”>
<csw:Query typeNames=“csw:Record”>
csw:ElementSetNamesummary</csw:ElementSetName>
</csw:Query>
</csw:GetRecords>

Response:
See attachment with a truncated response.

(attachments)

GetRecordsResponse.zip (14.3 KB)

Francois_Prunayre · September 30, 2009, 10:18am

Hello Wolfgang,

2009/9/30 Wolfgang Grunberg <wgrunberg@anonymised.com>:

GeoNetwork 2.41's CSW harvester fails with our deegree 2.3pre CSW service
(http://75.101.143.247:8080/deegree-csw/services?REQUEST=GetCapabilities&version=2.0.2&service=CSW).

GeoNetwork's GetRecords transaction request returns what appears to be a
valid CSW and Dublin Core GetRecordsResponse response. I have attached the
truncated response. This is a great improvement because deegree didn't
accept the GeoNetwork response in the past.

However, GeoNetwork's log gives me a "[geonetwork.harvester] - Skipped
record not in supported format : metadata" error and I am having
difficulties figuring out what the problem is.

It looks like dc:metadata elements are returned in deegree CSW
response. The harvester ask for a summary response with default schema
"Default value is http://www.opengis.net/cat/csw/2.0.2" CSW 2.0.2 spec.
I think we should get csw:SummaryRecord instead. Not sure if the
dc:metadata elements are 2.0.1 spec elements ?

Also if you set outputSchema="http://www.opengis.net/cat/csw/2.0.2"
in the http://testing.deegree.org/deegree-csw/client/client.html you
get ISO records.
If you did not set outputSchema, you get an exception.

We discussed that point with Heikki, and the harvester is doing the following:
* GetRecords in summary format first to retrieve ids and date
* GetRecordById in ISO or OGC format to load the records.

More test needed.

Francois

What kind of response does GeoNetwork require?
Also, how do I enable more verbose logging in GeoNetwork?

Thank you for your time!

POST request to http://75.101.143.247:8080/deegree-csw/services :
<?xml version="1.0" encoding="UTF-8"?>
<csw:GetRecords xmlns:csw="http://www.opengis.net/cat/csw/2.0.2"
service="CSW" version="2.0.2" resultType="results" startPosition="1301"
maxRecords="20">
<csw:Query typeNames="csw:Record">
<csw:ElementSetName>summary</csw:ElementSetName>
</csw:Query>
</csw:GetRecords>

Response:
See attachment with a truncated response.

--
_______________________________
Wolfgang Grunberg
Arizona Geological Survey
wgrunberg@anonymised.com
520-770-3500

------------------------------------------------------------------------------
Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9-12, 2009. Register now!
http://p.sf.net/sfu/devconf
_______________________________________________
GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at
http://sourceforge.net/projects/geonetwork

Heikki_Doeleman · September 30, 2009, 11:14am

hello Wolfgang, François, and list :

I’ve looked at it a bit more with Jose Garcia, and we found :

(1) GeoNetwork did not send a outputSchema in its GetRecords request. This means that the default per the CSW202 spec should be used by the deegree server, which I think means we should receive results of type csw:SummaryRecord, as says François;

(2) GeoNetwork should actually put a “more preferred” ISO outputSchema in the request, if the Capabilities advertizes it (as does the deegree server’s Capabilities). Jose is committing a fix for this to GeoNetwork today;

(3) GeoNetwork can only handle GetRecordsResponse that contains SearchResults of type csw:SummaryRecord (CSW) or gmd:MD_Metadata (ISO). Other types, like the dc:metadata receivede from deegree, leads to the message “Skipped record not in supported format :” you got

(4) sending GetRecords requests to the deegree server with various outputSchema I get :

outputSchema=“http://www.isotc211.org/2005/gmd” → response: ISO records, correct

outputSchema=“http://www.opengis.net/cat/csw/2.0.2” → response: dc:metada, expected response: csw:SummaryRecord

→ response: dc:metada, expected response: csw:SummaryRecord

Hopefully things are more clear ?

Kind regards
Heikki Doeleman

On Wed, Sep 30, 2009 at 12:18 PM, Francois Prunayre <fx.prunayre@anonymised.com> wrote:

Hello Wolfgang,

2009/9/30 Wolfgang Grunberg <wgrunberg@anonymised.com>:

GeoNetwork 2.41’s CSW harvester fails with our deegree 2.3pre CSW service
(http://75.101.143.247:8080/deegree-csw/services?REQUEST=GetCapabilities&version=2.0.2&service=CSW).

GeoNetwork’s GetRecords transaction request returns what appears to be a
valid CSW and Dublin Core GetRecordsResponse response. I have attached the
truncated response. This is a great improvement because deegree didn’t
accept the GeoNetwork response in the past.

However, GeoNetwork’s log gives me a “[geonetwork.harvester] - Skipped
record not in supported format : metadata” error and I am having
difficulties figuring out what the problem is.

It looks like dc:metadata elements are returned in deegree CSW
response. The harvester ask for a summary response with default schema
“Default value is http://www.opengis.net/cat/csw/2.0.2” CSW 2.0.2 spec.
I think we should get csw:SummaryRecord instead. Not sure if the
dc:metadata elements are 2.0.1 spec elements ?

Also if you set outputSchema=“http://www.opengis.net/cat/csw/2.0.2”
in the http://testing.deegree.org/deegree-csw/client/client.html you
get ISO records.
If you did not set outputSchema, you get an exception.

We discussed that point with Heikki, and the harvester is doing the following:

GetRecords in summary format first to retrieve ids and date

GetRecordById in ISO or OGC format to load the records.

More test needed.

Francois

What kind of response does GeoNetwork require?
Also, how do I enable more verbose logging in GeoNetwork?

Thank you for your time!

POST request to http://75.101.143.247:8080/deegree-csw/services :
<?xml version="1.0" encoding="UTF-8"?>
<csw:GetRecords xmlns:csw=“http://www.opengis.net/cat/csw/2.0.2”
service=“CSW” version=“2.0.2” resultType=“results” startPosition=“1301”
maxRecords=“20”>
<csw:Query typeNames=“csw:Record”>
csw:ElementSetNamesummary</csw:ElementSetName>
</csw:Query>
</csw:GetRecords>

Response:
See attachment with a truncated response.

–

Wolfgang Grunberg
Arizona Geological Survey
wgrunberg@anonymised.com
520-770-3500

Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9-12, 2009. Register now!
http://p.sf.net/sfu/devconf

GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at
http://sourceforge.net/projects/geonetwork

Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9-12, 2009. Register now!
http://p.sf.net/sfu/devconf

GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

Wolfgang_Grunberg · September 30, 2009, 5:40pm

Hi Heikki, Francois, and list,

Thank you for your help!

The demo deegree-csw service on the deegree site is outdated (v 2.2) and has known bugs.
Please, use our v2.3pre build at http://75.101.143.247:8080/deegree-csw/client/client.html which I compiled from SVN in August.

Yes, according to the OGC document 07-006r1 on CSW 2.0.2 service specifications (page 143, Table 65), the default for the optional outputSchema value in a GetRecords request is http://www.opengis.net/cat/csw/2.0.2.

Please correct me but from what I understand, both csw:SummaryRecord and csw:GetRecordsResponse responses should have DC formated metadata if no ouputSchema is defined.
To paraphrase our discussion: Should a GetRecords request with a "summary " ElementSetName return a csw:SummaryRecord response and not a csw:GetRecordsResponse response?

I have asked the deegree folks to comment on the csw:SummaryRecord vs. csw:GetRecordsResponse issue.

Ciao, Wolfgang

heikki wrote:

Heikki_Doeleman · September 30, 2009, 6:49pm

hi Wolfgang,

the tests I did were already using your URL at http://75.101.143.247:8080/deegree-csw/services. For good measure I tried them again, both with your online CSW client that you mentioned and with another HTTP client tool (Poster), yielding the same results :

outputSchema=“http://www.isotc211.org/2005/gmd” → response: ISO records, correct

outputSchema=“http://www.opengis.net/cat/csw/2.0.2” → response: dc:metada, expected response: csw:SummaryRecord

→ response: dc:metada, expected response: csw:SummaryRecord

So if I’m not mistaken, a bug in the deegree CSW server is that outputSchema=“http://www.opengis.net/cat/csw/2.0.2” is not handled correctly. Maybe you do handle the case of a missing outputSchema correctly in the sense that you treat it as “http://www.opengis.net/cat/csw/2.0.2” which would explain that we get the same results ?

As for the contents that should be in the output when outputSchema is absent or is “http://www.opengis.net/cat/csw/2.0.2”, it definitely still should be a csw:GetRecordsResponse, but one that inside its descendant csw:SearchResults contains a list of csw:SummaryRecord elements (and not as it does now, a list of dc:metadata elements).

Inside those csw:SummaryRecord elements I would expect dc: formatted data, yes, starting with a dc:identifier or a dct:bibliographicCitation element. For the full spec see the CSW202 schema at http://schemas.opengis.net/csw/2.0.2/CSW-discovery.xsd.

Does this make sense ?

Kind regards
Heikki Doeleman

On Wed, Sep 30, 2009 at 7:40 PM, Wolfgang Grunberg <wgrunberg@anonymised.com> wrote:

Hi Heikki, Francois, and list,

Thank you for your help!

The demo deegree-csw service on the deegree site is outdated (v 2.2) and has known bugs.
Please, use our v2.3pre build at http://75.101.143.247:8080/deegree-csw/client/client.html which I compiled from SVN in August.

Yes, according to the OGC document 07-006r1 on CSW 2.0.2 service specifications (page 143, Table 65), the default for the optional outputSchema value in a GetRecords request is http://www.opengis.net/cat/csw/2.0.2.

Please correct me but from what I understand, both csw:SummaryRecord and csw:GetRecordsResponse responses should have DC formated metadata if no ouputSchema is defined.
To paraphrase our discussion: Should a GetRecords request with a "summary " ElementSetName return a csw:SummaryRecord response and not a csw:GetRecordsResponse response?

I have asked the deegree folks to comment on the csw:SummaryRecord vs. csw:GetRecordsResponse issue.

Ciao, Wolfgang

heikki wrote:

hello Wolfgang, François, and list :

I’ve looked at it a bit more with Jose Garcia, and we found :

(1) GeoNetwork did not send a outputSchema in its GetRecords request. This means that the default per the CSW202 spec should be used by the deegree server, which I think means we should receive results of type csw:SummaryRecord, as says François

(2) GeoNetwork should actually put a “more preferred” ISO outputSchema in the request, if the Capabilities advertizes it (as does the deegree server’s Capabilities). Jose is committing a fix for this to GeoNetwork today;

(3) GeoNetwork can only handle GetRecordsResponse that contains SearchResults of type csw:SummaryRecord (CSW) or gmd:MD_Metadata (ISO). Other types, like the dc:metadata receivede from deegree, leads to the message “Skipped record not in supported format :” you got

(4) sending GetRecords requests to the deegree server with various outputSchema I get :

outputSchema=“http://www.isotc211.org/2005/gmd” → response: ISO records, correct

outputSchema=“http://www.opengis.net/cat/csw/2.0.2” → response: dc:metada, expected response: csw:SummaryRecord

→ response: dc:metada, expected response: csw:SummaryRecord

Hopefully things are more clear ?

Kind regards
Heikki Doeleman

On Wed, Sep 30, 2009 at 12:18 PM, Francois Prunayre <fx.prunayre@anonymised.com> wrote:

Hello Wolfgang,

2009/9/30 Wolfgang Grunberg <wgrunberg@anonymised.com>:

GeoNetwork 2.41’s CSW harvester fails with our deegree 2.3pre CSW service
(http://75.101.143.247:8080/deegree-csw/services?REQUEST=GetCapabilities&version=2.0.2&service=CSW).

GeoNetwork’s GetRecords transaction request returns what appears to be a
valid CSW and Dublin Core GetRecordsResponse response. I have attached the
truncated response. This is a great improvement because deegree didn’t
accept the GeoNetwork response in the past.

However, GeoNetwork’s log gives me a “[geonetwork.harvester] - Skipped
record not in supported format : metadata” error and I am having
difficulties figuring out what the problem is.

It looks like dc:metadata elements are returned in deegree CSW
response. The harvester ask for a summary response with default schema
“Default value is http://www.opengis.net/cat/csw/2.0.2” CSW 2.0.2 spec.
I think we should get csw:SummaryRecord instead. Not sure if the
dc:metadata elements are 2.0.1 spec elements ?

Also if you set outputSchema=“http://www.opengis.net/cat/csw/2.0.2”
in the http://testing.deegree.org/deegree-csw/client/client.html you
get ISO records.
If you did not set outputSchema, you get an exception.

We discussed that point with Heikki, and the harvester is doing the following:

GetRecords in summary format first to retrieve ids and date

GetRecordById in ISO or OGC format to load the records.

More test needed.

Francois

What kind of response does GeoNetwork require?
Also, how do I enable more verbose logging in GeoNetwork?

Thank you for your time!

POST request to http://75.101.143.247:8080/deegree-csw/services :
<?xml version="1.0" encoding="UTF-8"?>
<csw:GetRecords xmlns:csw=“http://www.opengis.net/cat/csw/2.0.2”
service=“CSW” version=“2.0.2” resultType=“results” startPosition=“1301”
maxRecords=“20”>
<csw:Query typeNames=“csw:Record”>
csw:ElementSetNamesummary</csw:ElementSetName>
</csw:Query>
</csw:GetRecords>

Response:
See attachment with a truncated response.

–

Wolfgang Grunberg
Arizona Geological Survey
wgrunberg@anonymised.com601…
520-770-3500

Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9-12, 2009. Register now!
http://p.sf.net/sfu/devconf

GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at
http://sourceforge.net/projects/geonetwork

Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9-12, 2009. Register now!
http://p.sf.net/sfu/devconf

GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork