[GeoNetwork-users] CSW Metadata Harvesting (terracatalog)

Dear Mailinglist users,

I have an issue pretty similar to this one: http://sourceforge.net/p/geonetwork/mailman/geonetwork-users/thread/1377002018793-5073365.post%40n6.nabble.com/#msg31307007

I want to harvest a terraCatalog CSW with GN (running on an Ubuntu 14 server). From the above thread I concluded that the problem could be that the server isn't able to send a request, but that isn't the case here because firstly I talked to our admin and I tried the following cURL command via the shell of the server (rszh.txt attached):
curl -X POST -d @rszh.txt https://www.geoseaportal.de/soapServices/CSWStartup? --header "Content-Type:text/xml"

The response are all the records of the CSW. If I request the CSW using RESTClient (Firefox Addon) I get the same response. Here are the stats:
URL: https://www.geoseaportal.de/soapServices/CSWStartup?

“POST” request:
<GetRecords
xmlns="http://www.opengis.net/cat/csw/2.0.2&quot;
                            service="CSW" version="2.0.2"
                            outputFormat="application/xml"
outputSchema="http://www.isotc211.org/2005/gmd&quot;
xmlns:csw="http://www.opengis.net/cat/csw/2.0.2&quot;
                            xmlns:ogc="http://www.opengis.net/ogc&quot;
xmlns:gmd="http://www.isotc211.org/2005/gmd&quot;
xmlns:apiso="http://www.opengis.net/cat/csw/apiso/1.0&quot;
                            xmlns:ows="http://www.opengis.net/ows&quot;
xmlns:dc="http://purl.org/dc/elements/1.1/&quot;
                            xmlns:dct="http://purl.org/dc/terms/&quot;
                            xmlns:gml="http://www.opengis.net/gml&quot;
resultType="results"
startPosition="1"
maxRecords="5">
<Query typeNames="gmd:MD_Metadata">
<ElementSetName typeNames="">brief</ElementSetName>
<Constraint version="1.1.0">
<ogc:Filter>
<ogc:PropertyIsLike escapeChar="\" singleChar="?" wildCard="*">
<ogc:PropertyName>apiso:anytext</ogc:PropertyName>
<ogc:Literal>Strömung</ogc:Literal>
</ogc:PropertyIsLike>
</ogc:Filter>
</Constraint>
</Query>
</GetRecords>

And now to the error messages of the two different GN versions installed on the server:

      GN 2.10.3

/with "Catalogue Services for the Web ISO profile 2.0" Harvesting/
"Error: Raised exception when searching: Error on line 1: White spaces are required between publicId and systemId.
Class: OperationAbortedEx
Stack:
at: org.fao.geonet.kernel.harvest.harvester.csw.Harvester file: Harvester.java line: 508 method: doSearch
at: org.fao.geonet.kernel.harvest.harvester.csw.Harvester file: Harvester.java line: 231 method: search
at: org.fao.geonet.kernel.harvest.harvester.csw.Harvester file: Harvester.java line: 103 method: harvest
at: org.fao.geonet.kernel.harvest.harvester.csw.CswHarvester file: CswHarvester.java line: 244 method: doHarvest
at: org.fao.geonet.kernel.harvest.harvester.AbstractHarvester$HarvestWithIndexProcessor file: AbstractHarvester.java line: 400 method: process
at: org.fao.geonet.kernel.harvest.harvester.AbstractHarvester file: AbstractHarvester.java line: 470 method: harvest
at: org.fao.geonet.kernel.harvest.harvester.HarvesterJob file: HarvesterJob.java line: 29 method: execute
at: org.quartz.core.JobRunShell file: JobRunShell.java line: 213 method: run
at: org.quartz.simpl.SimpleThreadPool$WorkerThread file: SimpleThreadPool.java line: 557 method: run"

/with "OGC Web services (ie WMS, WFS, etc.)" Harvesting/
no error message, just one records returned (metadata about the the CSW itself)

      GN 3.0.0RC0

"2015-04-16 09:50:31,682 WARN [GSP_1] - Raised exception when searching : org.fao.geonet.csw.common.exceptions.InvalidParameterValueEx: code=InvalidParameterValue, locator=null, message=bad service type: null
2015-04-16 09:50:31,757 WARN [GSP_1] - Raised exception when searching : org.fao.geonet.csw.common.exceptions.InvalidParameterValueEx: code=InvalidParameterValue, locator=null, message=bad service type: null
2015-04-16 09:50:31,757 ERROR [GSP_1] - Unknown error trying to harvest
2015-04-16 09:50:31,757 ERROR [GSP_1] - Raised exception when searching: bad service type: null
2015-04-16 09:50:31,758 ERROR [GSP_1] - Raised exception when searching: bad service type: null
OperationAbortedEx : Raised exception when searching: bad service type: null
     at org.fao.geonet.kernel.harvest.harvester.csw.Harvester.doSearch(Harvester.java:553)
     at org.fao.geonet.kernel.harvest.harvester.csw.Harvester.search(Harvester.java:251)
     at org.fao.geonet.kernel.harvest.harvester.csw.Harvester.harvest(Harvester.java:110)
     at org.fao.geonet.kernel.harvest.harvester.csw.CswHarvester.doHarvest(CswHarvester.java:174)
     at org.fao.geonet.kernel.harvest.harvester.AbstractHarvester$HarvestWithIndexProcessor.process(AbstractHarvester.java:494)
     at org.fao.geonet.kernel.harvest.harvester.AbstractHarvester.harvest(AbstractHarvester.java:563)
     at org.fao.geonet.kernel.harvest.harvester.HarvesterJob.execute(HarvesterJob.java:31)
     at org.quartz.core.JobRunShell.run(JobRunShell.java:213)
     at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557)
Caused by: org.fao.geonet.csw.common.exceptions.InvalidParameterValueEx: code=InvalidParameterValue, locator=null, message=bad service type: null
     at org.fao.geonet.csw.common.exceptions.CatalogException.createException(CatalogException.java:173)
     at org.fao.geonet.csw.common.exceptions.CatalogException.unmarshal(CatalogException.java:146)
     at org.fao.geonet.csw.common.requests.CatalogRequest.execute(CatalogRequest.java:151)
     at org.fao.geonet.kernel.harvest.harvester.csw.Harvester.doSearch(Harvester.java:541)
     ... 8 more
2015-04-16 09:50:31,778 WARN [GSP_1] - Raised exception while harvesting from : GSP_1 (CswHarvester)
2015-04-16 09:50:31,778 WARN [GSP_1] - (C) Class : ArrayIndexOutOfBoundsException
2015-04-16 09:50:31,778 WARN [GSP_1] - (C) Message : 1"

I'm totally lost on this issue so I'd be so glad about any help. Thanks in advance :slight_smile:

Cheers,
Christian

(attachments)

rszh.txt (966 Bytes)

Dear Mailinglist users ... again,

so I'm replying to my own question because I've digged deeper into this, after reading this thread <http://osgeo-org.1560.x6.nabble.com/CSW-Metadata-Harvesting-td5072288.html&gt;\.
I found another conterra CSW there (this one <http://www.geomis.sachsen.de/soapServices/CSWStartup&gt;\) and tested this with GN and what can I say it works perfectly fine. The only real difference to the (conterra) CSW I want to harvest <https://www.geoseaportal.de/soapServices/CSWStartup&gt; I see is that it uses HTTPS instead of just HTTP. Could it be that GN has problems harvesting HTTPS CSW's?

Cheers and thanks,
Christian

P.S.: I saw that there once was a problem harvesting GN instances using HTTPS: http://trac.osgeo.org/geonetwork/ticket/803

Am 16.04.2015 um 13:38 schrieb Christian Seip:

Dear Mailinglist users,

I have an issue pretty similar to this one: http://sourceforge.net/p/geonetwork/mailman/geonetwork-users/thread/1377002018793-5073365.post%40n6.nabble.com/#msg31307007

I want to harvest a terraCatalog CSW with GN (running on an Ubuntu 14 server). From the above thread I concluded that the problem could be that the server isn't able to send a request, but that isn't the case here because firstly I talked to our admin and I tried the following cURL command via the shell of the server (rszh.txt attached):
curl -X POST -d @rszh.txt https://www.geoseaportal.de/soapServices/CSWStartup? --header "Content-Type:text/xml"

The response are all the records of the CSW. If I request the CSW using RESTClient (Firefox Addon) I get the same response. Here are the stats:
URL: https://www.geoseaportal.de/soapServices/CSWStartup?

“POST” request:
<GetRecords
xmlns="http://www.opengis.net/cat/csw/2.0.2&quot;
                           service="CSW" version="2.0.2"
                           outputFormat="application/xml"
outputSchema="http://www.isotc211.org/2005/gmd&quot;
xmlns:csw="http://www.opengis.net/cat/csw/2.0.2&quot;
                           xmlns:ogc="http://www.opengis.net/ogc&quot;
xmlns:gmd="http://www.isotc211.org/2005/gmd&quot;
xmlns:apiso="http://www.opengis.net/cat/csw/apiso/1.0&quot;
                           xmlns:ows="http://www.opengis.net/ows&quot;
xmlns:dc="http://purl.org/dc/elements/1.1/&quot;
                           xmlns:dct="http://purl.org/dc/terms/&quot;
                           xmlns:gml="http://www.opengis.net/gml&quot;
resultType="results"
startPosition="1"
maxRecords="5">
<Query typeNames="gmd:MD_Metadata">
<ElementSetName typeNames="">brief</ElementSetName>
<Constraint version="1.1.0">
<ogc:Filter>
<ogc:PropertyIsLike escapeChar="\" singleChar="?" wildCard="*">
<ogc:PropertyName>apiso:anytext</ogc:PropertyName>
<ogc:Literal>Strömung</ogc:Literal>
</ogc:PropertyIsLike>
</ogc:Filter>
</Constraint>
</Query>
</GetRecords>

And now to the error messages of the two different GN versions installed on the server:

     GN 2.10.3

/with "Catalogue Services for the Web ISO profile 2.0" Harvesting/
"Error: Raised exception when searching: Error on line 1: White spaces are required between publicId and systemId.
Class: OperationAbortedEx
Stack:
at: org.fao.geonet.kernel.harvest.harvester.csw.Harvester file: Harvester.java line: 508 method: doSearch
at: org.fao.geonet.kernel.harvest.harvester.csw.Harvester file: Harvester.java line: 231 method: search
at: org.fao.geonet.kernel.harvest.harvester.csw.Harvester file: Harvester.java line: 103 method: harvest
at: org.fao.geonet.kernel.harvest.harvester.csw.CswHarvester file: CswHarvester.java line: 244 method: doHarvest
at: org.fao.geonet.kernel.harvest.harvester.AbstractHarvester$HarvestWithIndexProcessor file: AbstractHarvester.java line: 400 method: process
at: org.fao.geonet.kernel.harvest.harvester.AbstractHarvester file: AbstractHarvester.java line: 470 method: harvest
at: org.fao.geonet.kernel.harvest.harvester.HarvesterJob file: HarvesterJob.java line: 29 method: execute
at: org.quartz.core.JobRunShell file: JobRunShell.java line: 213 method: run
at: org.quartz.simpl.SimpleThreadPool$WorkerThread file: SimpleThreadPool.java line: 557 method: run"

/with "OGC Web services (ie WMS, WFS, etc.)" Harvesting/
no error message, just one records returned (metadata about the the CSW itself)

     GN 3.0.0RC0

"2015-04-16 09:50:31,682 WARN [GSP_1] - Raised exception when searching : org.fao.geonet.csw.common.exceptions.InvalidParameterValueEx: code=InvalidParameterValue, locator=null, message=bad service type: null
2015-04-16 09:50:31,757 WARN [GSP_1] - Raised exception when searching : org.fao.geonet.csw.common.exceptions.InvalidParameterValueEx: code=InvalidParameterValue, locator=null, message=bad service type: null
2015-04-16 09:50:31,757 ERROR [GSP_1] - Unknown error trying to harvest
2015-04-16 09:50:31,757 ERROR [GSP_1] - Raised exception when searching: bad service type: null
2015-04-16 09:50:31,758 ERROR [GSP_1] - Raised exception when searching: bad service type: null
OperationAbortedEx : Raised exception when searching: bad service type: null
    at org.fao.geonet.kernel.harvest.harvester.csw.Harvester.doSearch(Harvester.java:553)
    at org.fao.geonet.kernel.harvest.harvester.csw.Harvester.search(Harvester.java:251)
    at org.fao.geonet.kernel.harvest.harvester.csw.Harvester.harvest(Harvester.java:110)
    at org.fao.geonet.kernel.harvest.harvester.csw.CswHarvester.doHarvest(CswHarvester.java:174)
    at org.fao.geonet.kernel.harvest.harvester.AbstractHarvester$HarvestWithIndexProcessor.process(AbstractHarvester.java:494)
    at org.fao.geonet.kernel.harvest.harvester.AbstractHarvester.harvest(AbstractHarvester.java:563)
    at org.fao.geonet.kernel.harvest.harvester.HarvesterJob.execute(HarvesterJob.java:31)
    at org.quartz.core.JobRunShell.run(JobRunShell.java:213)
    at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557)
Caused by: org.fao.geonet.csw.common.exceptions.InvalidParameterValueEx: code=InvalidParameterValue, locator=null, message=bad service type: null
    at org.fao.geonet.csw.common.exceptions.CatalogException.createException(CatalogException.java:173)
    at org.fao.geonet.csw.common.exceptions.CatalogException.unmarshal(CatalogException.java:146)
    at org.fao.geonet.csw.common.requests.CatalogRequest.execute(CatalogRequest.java:151)
    at org.fao.geonet.kernel.harvest.harvester.csw.Harvester.doSearch(Harvester.java:541)
    ... 8 more
2015-04-16 09:50:31,778 WARN [GSP_1] - Raised exception while harvesting from : GSP_1 (CswHarvester)
2015-04-16 09:50:31,778 WARN [GSP_1] - (C) Class : ArrayIndexOutOfBoundsException
2015-04-16 09:50:31,778 WARN [GSP_1] - (C) Message : 1"

I'm totally lost on this issue so I'd be so glad about any help. Thanks in advance :slight_smile:

Cheers,
Christian

------------------------------------------------------------------------------
BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
Develop your own process in accordance with the BPMN 2 standard
Learn Process modeling best practices with Bonita BPM through live exercises
http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF

_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork