[GeoNetwork-users] Geoportal issues

GeoNetwork GeoNetwork User
1

Hi

Has anyone had any problems trying to run a CSW harvest against ESRI Geoportal. Geonetwork seems to catch a 'NoApplicableCodeException' from Geoportal when trying to do a GetRecordsRequest. I've only noticed this with requests to Geoportal.

cheers,

Brian.

--
Geospatial Software Developer (Geo Services)
EDINA
Causewayside House
160 Causewayside
Edinburgh EH9 1PR

p: +44 (0)131 5601184
e: brian.ohare@anonymised.com
w: http://edina.ac.uk

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

2

Brian,

I understand the Geonetwork catalogue behind http://www.geodata.govt.nz is harvesting records from the ESRI geoportal catalogue at http://geoportal.doc.govt.nz/geoportal/catalog/main/home.page

The person looking after the geodata catalogue is rainer (at) silverstripe.com who may be able to advise you. I believe Simon Pigot was also involved in making this work successfully.

Cheers,

Brent Wood

--- On Tue, 4/16/13, Brian O'Hare <brian.ohare@anonymised.com> wrote:

From: Brian O'Hare <brian.ohare@anonymised.com>
Subject: [GeoNetwork-users] Geoportal issues
To: geonetwork-users@lists.sourceforge.net
Date: Tuesday, April 16, 2013, 11:21 PM

Hi

Has anyone had any problems trying to run a CSW harvest against ESRI
Geoportal. Geonetwork seems to catch a 'NoApplicableCodeException' from
Geoportal when trying to do a GetRecordsRequest. I've only noticed this
with requests to Geoportal.

cheers,

Brian.

--
Geospatial Software Developer (Geo Services)
EDINA
Causewayside House
160 Causewayside
Edinburgh EH9 1PR

p: +44 (0)131 5601184
e: brian.ohare@anonymised.com
w: http://edina.ac.uk

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

3

Can you share the CSW requests that Geonetwork sends? Note that Geoportal Server expects OGC Filter, not CQL.

Marten Hogeweg
Esri

-----Original Message-----
From: pcreso@anonymised.com [mailto:pcreso@anonymised.com]
Sent: Tuesday, April 16, 2013 7:48 PM
To: geonetwork-users@lists.sourceforge.net; Brian O'Hare
Subject: Re: [GeoNetwork-users] Geoportal issues

Brian,

I understand the Geonetwork catalogue behind http://www.geodata.govt.nz is harvesting records from the ESRI geoportal catalogue at http://geoportal.doc.govt.nz/geoportal/catalog/main/home.page

The person looking after the geodata catalogue is rainer (at) silverstripe.com who may be able to advise you. I believe Simon Pigot was also involved in making this work successfully.

Cheers,

Brent Wood

--- On Tue, 4/16/13, Brian O'Hare <brian.ohare@anonymised.com> wrote:

From: Brian O'Hare <brian.ohare@anonymised.com>
Subject: [GeoNetwork-users] Geoportal issues
To: geonetwork-users@lists.sourceforge.net
Date: Tuesday, April 16, 2013, 11:21 PM

Hi

Has anyone had any problems trying to run a CSW harvest against ESRI Geoportal. Geonetwork seems to catch a 'NoApplicableCodeException' from Geoportal when trying to do a GetRecordsRequest. I've only noticed this with requests to Geoportal.

cheers,

Brian.

--
Geospatial Software Developer (Geo Services) EDINA Causewayside House
160 Causewayside
Edinburgh EH9 1PR

p: +44 (0)131 5601184
e: brian.ohare@anonymised.com
w: http://edina.ac.uk

The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork
------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

4

I implemented a harvester that uses the geoportal REST api instead of CSW (would prefer to use the standard yes but it could be an alternative if you need it to work and don't have time for sorting this out) so you could try this if you can't nail down the problems with CSW. The harvester is documented in the GeoNetwork 2.8.x manual at GeoPortal REST Harvesting - GeoNetwork Opensource - a lot depends on the version of the geoportal you are harvesting from (the open source geoportal should work with CSW, but for other versions such as 9.3.x, it might be better to use the geoportal REST api harvester in 2.8.x).

Cheers,
Simon
________________________________________
From: Marten Hogeweg [mhogeweg@anonymised.com]
Sent: Wednesday, 17 April 2013 4:10 AM
To: pcreso@anonymised.com; geonetwork-users@lists.sourceforge.net; Brian O'Hare
Subject: Re: [GeoNetwork-users] Geoportal issues

Can you share the CSW requests that Geonetwork sends? Note that Geoportal Server expects OGC Filter, not CQL.

Marten Hogeweg
Esri

-----Original Message-----
From: pcreso@anonymised.com [mailto:pcreso@anonymised.com]
Sent: Tuesday, April 16, 2013 7:48 PM
To: geonetwork-users@lists.sourceforge.net; Brian O'Hare
Subject: Re: [GeoNetwork-users] Geoportal issues

Brian,

I understand the Geonetwork catalogue behind http://www.geodata.govt.nz is harvesting records from the ESRI geoportal catalogue at http://geoportal.doc.govt.nz/geoportal/catalog/main/home.page

The person looking after the geodata catalogue is rainer (at) silverstripe.com who may be able to advise you. I believe Simon Pigot was also involved in making this work successfully.

Cheers,

   Brent Wood

--- On Tue, 4/16/13, Brian O'Hare <brian.ohare@anonymised.com> wrote:

From: Brian O'Hare <brian.ohare@anonymised.com>
Subject: [GeoNetwork-users] Geoportal issues
To: geonetwork-users@lists.sourceforge.net
Date: Tuesday, April 16, 2013, 11:21 PM

Hi

Has anyone had any problems trying to run a CSW harvest against ESRI Geoportal. Geonetwork seems to catch a 'NoApplicableCodeException' from Geoportal when trying to do a GetRecordsRequest. I've only noticed this with requests to Geoportal.

cheers,

Brian.

--
Geospatial Software Developer (Geo Services) EDINA Causewayside House
160 Causewayside
Edinburgh EH9 1PR

p: +44 (0)131 5601184
e: brian.ohare@anonymised.com
w: http://edina.ac.uk

The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net

GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork
------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

5

Hi, GeoNetwork checks the capability in order to send CQL or FE request.
I've been harvesting ESRI GeoPortal with CSW with no issue.
Could you share the URL to be harvested ?

Cheers.

Francois

2013/4/17 <Simon.Pigot@anonymised.com>

I implemented a harvester that uses the geoportal REST api instead of CSW
(would prefer to use the standard yes but it could be an alternative if you
need it to work and don't have time for sorting this out) so you could try
this if you can't nail down the problems with CSW. The harvester is
documented in the GeoNetwork 2.8.x manual at
http://geonetwork-opensource.org/manuals/2.8.0/eng/users/managing_metadata/harvesting/geoPREST/index.html#geoportal-rest-harvester - a lot depends on the version of the geoportal you are harvesting from
(the open source geoportal should work with CSW, but for other versions
such as 9.3.x, it might be better to use the geoportal REST api harvester
in 2.8.x).

Cheers,
Simon
________________________________________
From: Marten Hogeweg [mhogeweg@anonymised.com]
Sent: Wednesday, 17 April 2013 4:10 AM
To: pcreso@anonymised.com; geonetwork-users@lists.sourceforge.net; Brian
O'Hare
Subject: Re: [GeoNetwork-users] Geoportal issues

Can you share the CSW requests that Geonetwork sends? Note that Geoportal
Server expects OGC Filter, not CQL.

Marten Hogeweg
Esri

-----Original Message-----
From: pcreso@anonymised.com [mailto:pcreso@anonymised.com]
Sent: Tuesday, April 16, 2013 7:48 PM
To: geonetwork-users@lists.sourceforge.net; Brian O'Hare
Subject: Re: [GeoNetwork-users] Geoportal issues

Brian,

I understand the Geonetwork catalogue behind http://www.geodata.govt.nzis harvesting records from the ESRI geoportal catalogue at
http://geoportal.doc.govt.nz/geoportal/catalog/main/home.page

The person looking after the geodata catalogue is rainer (at)
silverstripe.com who may be able to advise you. I believe Simon Pigot was
also involved in making this work successfully.

Cheers,

   Brent Wood

--- On Tue, 4/16/13, Brian O'Hare <brian.ohare@anonymised.com> wrote:

From: Brian O'Hare <brian.ohare@anonymised.com>
Subject: [GeoNetwork-users] Geoportal issues
To: geonetwork-users@lists.sourceforge.net
Date: Tuesday, April 16, 2013, 11:21 PM

Hi

Has anyone had any problems trying to run a CSW harvest against ESRI
Geoportal. Geonetwork seems to catch a 'NoApplicableCodeException' from
Geoportal when trying to do a GetRecordsRequest. I've only noticed this
with requests to Geoportal.

cheers,

Brian.

--
Geospatial Software Developer (Geo Services) EDINA Causewayside House
160 Causewayside
Edinburgh EH9 1PR

p: +44 (0)131 5601184
e: brian.ohare@anonymised.com
w: http://edina.ac.uk

The University of Edinburgh is a charitable body, registered in Scotland,
with registration number SC005336.

------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use our
toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at
http://sourceforge.net/projects/geonetwork

------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use our
toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at
http://sourceforge.net/projects/geonetwork

------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at
http://sourceforge.net/projects/geonetwork

6

Many thanks everyone for your help.

I've enabled HttpClient debug logging in Geonetwork but can't see the request going to geoportal. However I have used a debugger on the running tomcat and can see that the request is using Filter not CQL. It should just be a straightforward GetRecords request. The capabilities file is parsed without problems. Here's some logging output - you can see the Geoportal url here also:

2013-04-17 09:32:58,384 [schema: kb] DEBUG [org.apache.commons.httpclient.HttpConnection] - Open connection to www.ngdc.noaa.gov:80
2013-04-17 09:32:58,529 [schema: kb] DEBUG [org.apache.commons.httpclient.HttpMethodBase] - Adding Host request header
2013-04-17 09:32:58,529 [schema: kb] WARN [org.apache.commons.httpclient.cookie.CookieSpec] - Invalid cookie state: domain not specified
2013-04-17 09:32:58,529 [schema: kb] DEBUG [org.apache.commons.httpclient.methods.EntityEnclosingMethod] - Request body sent
2013-04-17 09:32:58,760 [schema: kb] WARN [org.apache.commons.httpclient.HttpMethodBase] - Cookie rejected: "JSESSIONID=70FF886352F32B7A0FB5996515DFB29E". Illegal path attribute "/geoportal". Path of origin: "http://www.ngdc.noaa.gov/geoportal/csw&quot;
2013-04-17 09:32:58,763 [schema: kb] DEBUG [org.apache.commons.httpclient.HttpMethodBase] - Should close connection in response to directive: close
2013-04-17 09:32:58,763 [schema: kb] DEBUG [org.apache.commons.httpclient.HttpConnection] - Releasing connection back to connection manager.
2013-04-17 09:32:58,763 [schema: kb] WARN [geonetwork.harvester] - Raised exception when searching : org.fao.geonet.csw.common.exceptions.NoApplicableCodeEx: code=NoApplicableCode, locator=null, message=
An exception occurred with no applicable code

Looks like its to do with cookies. I've rebuilt geonetwork and set cookie policy to 'ignoreCookies' with no success - it still fails. Geoportal version is reported as.

NGDC Geoportal Instance using ArcGIS Server Geoportal Extension 10 - OGC CSW 2.0.2 ISO AP

I'll try Simon's restful harvester and see if I have more success.

thanks again all,

Brian.

On 17/04/13 07:24, Francois Prunayre wrote:

Hi, GeoNetwork checks the capability in order to send CQL or FE request. I've been harvesting ESRI GeoPortal with CSW with no issue.
Could you share the URL to be harvested ?

Cheers.

Francois

2013/4/17 <Simon.Pigot@anonymised.com <mailto:Simon.Pigot@anonymised.com>>

    I implemented a harvester that uses the geoportal REST api instead
    of CSW (would prefer to use the standard yes but it could be an
    alternative if you need it to work and don't have time for sorting
    this out) so you could try this if you can't nail down the
    problems with CSW. The harvester is documented in the GeoNetwork
    2.8.x manual at
    http://geonetwork-opensource.org/manuals/2.8.0/eng/users/managing_metadata/harvesting/geoPREST/index.html#geoportal-rest-harvester
     - a lot depends on the version of the geoportal you are
    harvesting from (the open source geoportal should work with CSW,
    but for other versions such as 9.3.x, it might be better to use
    the geoportal REST api harvester in 2.8.x).

    Cheers,
    Simon
    ________________________________________
    From: Marten Hogeweg [mhogeweg@anonymised.com <mailto:mhogeweg@anonymised.com>]
    Sent: Wednesday, 17 April 2013 4:10 AM
    To: pcreso@anonymised.com <mailto:pcreso@anonymised.com>;
    geonetwork-users@lists.sourceforge.net
    <mailto:geonetwork-users@lists.sourceforge.net>; Brian O'Hare
    Subject: Re: [GeoNetwork-users] Geoportal issues

    Can you share the CSW requests that Geonetwork sends? Note that
    Geoportal Server expects OGC Filter, not CQL.

    Marten Hogeweg
    Esri

    -----Original Message-----
    From: pcreso@anonymised.com <mailto:pcreso@anonymised.com>
    [mailto:pcreso@anonymised.com]
    Sent: Tuesday, April 16, 2013 7:48 PM
    To: geonetwork-users@lists.sourceforge.net
    <mailto:geonetwork-users@lists.sourceforge.net>; Brian O'Hare
    Subject: Re: [GeoNetwork-users] Geoportal issues

    Brian,

    I understand the Geonetwork catalogue behind
    http://www.geodata.govt.nz is harvesting records from the ESRI
    geoportal catalogue at
    http://geoportal.doc.govt.nz/geoportal/catalog/main/home.page

    The person looking after the geodata catalogue is rainer (at)
    silverstripe.com <http://silverstripe.com> who may be able to
    advise you. I believe Simon Pigot was also involved in making this
    work successfully.

    Cheers,

       Brent Wood

    --- On Tue, 4/16/13, Brian O'Hare <brian.ohare@anonymised.com
    <mailto:brian.ohare@anonymised.com>> wrote:

    From: Brian O'Hare <brian.ohare@anonymised.com
    <mailto:brian.ohare@anonymised.com>>
    Subject: [GeoNetwork-users] Geoportal issues
    To: geonetwork-users@lists.sourceforge.net
    <mailto:geonetwork-users@lists.sourceforge.net>
    Date: Tuesday, April 16, 2013, 11:21 PM

    Hi

    Has anyone had any problems trying to run a CSW harvest against
    ESRI Geoportal. Geonetwork seems to catch a
    'NoApplicableCodeException' from Geoportal when trying to do a
    GetRecordsRequest. I've only noticed this with requests to Geoportal.

    cheers,

    Brian.

    --
    Geospatial Software Developer (Geo Services) EDINA Causewayside House
    160 Causewayside
    Edinburgh EH9 1PR

    p: +44 (0)131 5601184 <tel:%2B44%20%280%29131%205601184>
    e: brian.ohare@anonymised.com <mailto:brian.ohare@anonymised.com>
    w: http://edina.ac.uk

    The University of Edinburgh is a charitable body, registered in
    Scotland, with registration number SC005336.

    ------------------------------------------------------------------------------
    Precog is a next-generation analytics platform capable of advanced
    analytics on semi-structured data. The platform includes APIs for
    building apps and a phenomenal toolset for data science.
    Developers can use our toolset for easy data analysis &
    visualization. Get a free account!
    http://www2.precog.com/precogplatform/slashdotnewsletter
    _______________________________________________
    GeoNetwork-users mailing list
    GeoNetwork-users@lists.sourceforge.net
    <mailto:GeoNetwork-users@lists.sourceforge.net>
    https://lists.sourceforge.net/lists/listinfo/geonetwork-users
    GeoNetwork OpenSource is maintained at
    http://sourceforge.net/projects/geonetwork
    ------------------------------------------------------------------------------
    Precog is a next-generation analytics platform capable of advanced
    analytics on semi-structured data. The platform includes APIs for
    building apps and a phenomenal toolset for data science.
    Developers can use our toolset for easy data analysis &
    visualization. Get a free account!
    http://www2.precog.com/precogplatform/slashdotnewsletter
    _______________________________________________
    GeoNetwork-users mailing list
    GeoNetwork-users@lists.sourceforge.net
    <mailto:GeoNetwork-users@lists.sourceforge.net>
    https://lists.sourceforge.net/lists/listinfo/geonetwork-users
    GeoNetwork OpenSource is maintained at
    http://sourceforge.net/projects/geonetwork

    ------------------------------------------------------------------------------
    Precog is a next-generation analytics platform capable of advanced
    analytics on semi-structured data. The platform includes APIs for
    building
    apps and a phenomenal toolset for data science. Developers can use
    our toolset for easy data analysis & visualization. Get a free
    account!
    http://www2.precog.com/precogplatform/slashdotnewsletter
    _______________________________________________
    GeoNetwork-users mailing list
    GeoNetwork-users@lists.sourceforge.net
    <mailto:GeoNetwork-users@lists.sourceforge.net>
    https://lists.sourceforge.net/lists/listinfo/geonetwork-users
    GeoNetwork OpenSource is maintained at
    http://sourceforge.net/projects/geonetwork

--
Geospatial Software Developer (Geo Services)
EDINA
Causewayside House
160 Causewayside
Edinburgh EH9 1PR

p: +44 (0)131 5601184
e: brian.ohare@anonymised.com
w: http://edina.ac.uk

7

Brian,

GeoPortal REST harvester in GeoNetwork does seem to work ok on the NOAA site. I set up a harvester as shown in the image attached. I did have to create a gmiTogmd.xsl (attached), install it in web/geonetwork/xsl/conversion/import as the NOAA site is using ISO19115-2 (this xslt converts the top element and loses the rest of the gmi content) and select it in the harvester as an XSLT to apply to the content harvested from the geoportal site.

Cheers,
Simon
________________________________________
From: Brian O'Hare [brian.ohare@anonymised.com]
Sent: Wednesday, 17 April 2013 6:51 PM
To: Francois Prunayre
Cc: Pigot, Simon (CMAR, Hobart); Geonetwork-Users@anonymised.com; mhogeweg@anonymised.com3...; pcreso@anonymised.com
Subject: Re: [GeoNetwork-users] Geoportal issues

Many thanks everyone for your help.

I've enabled HttpClient debug logging in Geonetwork but can't see the request going to geoportal. However I have used a debugger on the running tomcat and can see that the request is using Filter not CQL. It should just be a straightforward GetRecords request. The capabilities file is parsed without problems. Here's some logging output - you can see the Geoportal url here also:

2013-04-17 09:32:58,384 [schema: kb] DEBUG [org.apache.commons.httpclient.HttpConnection] - Open connection to www.ngdc.noaa.gov:80<http://www.ngdc.noaa.gov:80>
2013-04-17 09:32:58,529 [schema: kb] DEBUG [org.apache.commons.httpclient.HttpMethodBase] - Adding Host request header
2013-04-17 09:32:58,529 [schema: kb] WARN [org.apache.commons.httpclient.cookie.CookieSpec] - Invalid cookie state: domain not specified
2013-04-17 09:32:58,529 [schema: kb] DEBUG [org.apache.commons.httpclient.methods.EntityEnclosingMethod] - Request body sent
2013-04-17 09:32:58,760 [schema: kb] WARN [org.apache.commons.httpclient.HttpMethodBase] - Cookie rejected: "JSESSIONID=70FF886352F32B7A0FB5996515DFB29E". Illegal path attribute "/geoportal". Path of origin: "http://www.ngdc.noaa.gov/geoportal/csw&quot;&lt;http://www.ngdc.noaa.gov/geoportal/csw&gt;
2013-04-17 09:32:58,763 [schema: kb] DEBUG [org.apache.commons.httpclient.HttpMethodBase] - Should close connection in response to directive: close
2013-04-17 09:32:58,763 [schema: kb] DEBUG [org.apache.commons.httpclient.HttpConnection] - Releasing connection back to connection manager.
2013-04-17 09:32:58,763 [schema: kb] WARN [geonetwork.harvester] - Raised exception when searching : org.fao.geonet.csw.common.exceptions.NoApplicableCodeEx: code=NoApplicableCode, locator=null, message=
An exception occurred with no applicable code

Looks like its to do with cookies. I've rebuilt geonetwork and set cookie policy to 'ignoreCookies' with no success - it still fails. Geoportal version is reported as.

NGDC Geoportal Instance using ArcGIS Server Geoportal Extension 10 - OGC CSW 2.0.2 ISO AP

I'll try Simon's restful harvester and see if I have more success.

thanks again all,

Brian.

On 17/04/13 07:24, Francois Prunayre wrote:
Hi, GeoNetwork checks the capability in order to send CQL or FE request. I've been harvesting ESRI GeoPortal with CSW with no issue.
Could you share the URL to be harvested ?

Cheers.

Francois

2013/4/17 <Simon.Pigot@anonymised.com<mailto:Simon.Pigot@anonymised.com>>
I implemented a harvester that uses the geoportal REST api instead of CSW (would prefer to use the standard yes but it could be an alternative if you need it to work and don't have time for sorting this out) so you could try this if you can't nail down the problems with CSW. The harvester is documented in the GeoNetwork 2.8.x manual at GeoPortal REST Harvesting - GeoNetwork Opensource - a lot depends on the version of the geoportal you are harvesting from (the open source geoportal should work with CSW, but for other versions such as 9.3.x, it might be better to use the geoportal REST api harvester in 2.8.x).

Cheers,
Simon
________________________________________
From: Marten Hogeweg [mhogeweg@anonymised.com<mailto:mhogeweg@anonymised.com>]
Sent: Wednesday, 17 April 2013 4:10 AM
To: pcreso@anonymised.com<mailto:pcreso@anonymised.com>; geonetwork-users@anonymised.comeforge.net<mailto:geonetwork-users@lists.sourceforge.net>; Brian O'Hare
Subject: Re: [GeoNetwork-users] Geoportal issues

Can you share the CSW requests that Geonetwork sends? Note that Geoportal Server expects OGC Filter, not CQL.

Marten Hogeweg
Esri

-----Original Message-----
From: pcreso@anonymised.com<mailto:pcreso@anonymised.com> [mailto:pcreso@anonymised.com]
Sent: Tuesday, April 16, 2013 7:48 PM
To: geonetwork-users@lists.sourceforge.net<mailto:geonetwork-users@anonymised.comurceforge.net>; Brian O'Hare
Subject: Re: [GeoNetwork-users] Geoportal issues

Brian,

I understand the Geonetwork catalogue behind http://www.geodata.govt.nz is harvesting records from the ESRI geoportal catalogue at http://geoportal.doc.govt.nz/geoportal/catalog/main/home.page

The person looking after the geodata catalogue is rainer (at) silverstripe.com<http://silverstripe.com> who may be able to advise you. I believe Simon Pigot was also involved in making this work successfully.

Cheers,

   Brent Wood

--- On Tue, 4/16/13, Brian O'Hare <brian.ohare@anonymised.com<mailto:brian.ohare@anonymised.com>> wrote:

From: Brian O'Hare <brian.ohare@anonymised.com<mailto:brian.ohare@anonymised.com>>
Subject: [GeoNetwork-users] Geoportal issues
To: geonetwork-users@lists.sourceforge.net<mailto:geonetwork-users@anonymised.comurceforge.net>
Date: Tuesday, April 16, 2013, 11:21 PM

Hi

Has anyone had any problems trying to run a CSW harvest against ESRI Geoportal. Geonetwork seems to catch a 'NoApplicableCodeException' from Geoportal when trying to do a GetRecordsRequest. I've only noticed this with requests to Geoportal.

cheers,

Brian.

--
Geospatial Software Developer (Geo Services) EDINA Causewayside House
160 Causewayside
Edinburgh EH9 1PR

p: +44 (0)131 5601184<tel:%2B44%20%280%29131%205601184>
e: brian.ohare@anonymised.com<mailto:brian.ohare@anonymised.com>
w: http://edina.ac.uk

The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net<mailto:GeoNetwork-users@anonymised.comforge.net>

GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork
------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net<mailto:GeoNetwork-users@anonymised.comforge.net>
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net<mailto:GeoNetwork-users@anonymised.comforge.net>

GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

--
Geospatial Software Developer (Geo Services)
EDINA
Causewayside House
160 Causewayside
Edinburgh EH9 1PR

p: +44 (0)131 5601184
e: brian.ohare@anonymised.com<mailto:brian.ohare@anonymised.com>
w: http://edina.ac.uk

(attachments)

geoportal-rest-harvester.png
geoportal-rest-harvester.png767×724 60.2 KB

1_Warning.txt (299 Bytes)

8

Just in case the gmiTogmd.xsl converter got blocked by some zealous email server, I've committed it to 2.8.x and trunk (see web/src/main/webapp/xsl/conversion/import/gmiTogmd.xsl).

Cheers,
Simon
________________________________________
From: Pigot, Simon (CMAR, Hobart)
Sent: Thursday, 18 April 2013 9:29 AM
To: Brian O'Hare; Francois Prunayre
Cc: Geonetwork-Users@anonymised.com
Subject: RE: [GeoNetwork-users] Geoportal issues

Brian,

GeoPortal REST harvester in GeoNetwork does seem to work ok on the NOAA site. I set up a harvester as shown in the image attached. I did have to create a gmiTogmd.xsl (attached), install it in web/geonetwork/xsl/conversion/import as the NOAA site is using ISO19115-2 (this xslt converts the top element and loses the rest of the gmi content) and select it in the harvester as an XSLT to apply to the content harvested from the geoportal site.

Cheers,
Simon
________________________________________
From: Brian O'Hare [brian.ohare@anonymised.com]
Sent: Wednesday, 17 April 2013 6:51 PM
To: Francois Prunayre
Cc: Pigot, Simon (CMAR, Hobart); Geonetwork-Users@anonymised.com; mhogeweg@anonymised.com3...; pcreso@anonymised.com
Subject: Re: [GeoNetwork-users] Geoportal issues

Many thanks everyone for your help.

I've enabled HttpClient debug logging in Geonetwork but can't see the request going to geoportal. However I have used a debugger on the running tomcat and can see that the request is using Filter not CQL. It should just be a straightforward GetRecords request. The capabilities file is parsed without problems. Here's some logging output - you can see the Geoportal url here also:

2013-04-17 09:32:58,384 [schema: kb] DEBUG [org.apache.commons.httpclient.HttpConnection] - Open connection to www.ngdc.noaa.gov:80<http://www.ngdc.noaa.gov:80>
2013-04-17 09:32:58,529 [schema: kb] DEBUG [org.apache.commons.httpclient.HttpMethodBase] - Adding Host request header
2013-04-17 09:32:58,529 [schema: kb] WARN [org.apache.commons.httpclient.cookie.CookieSpec] - Invalid cookie state: domain not specified
2013-04-17 09:32:58,529 [schema: kb] DEBUG [org.apache.commons.httpclient.methods.EntityEnclosingMethod] - Request body sent
2013-04-17 09:32:58,760 [schema: kb] WARN [org.apache.commons.httpclient.HttpMethodBase] - Cookie rejected: "JSESSIONID=70FF886352F32B7A0FB5996515DFB29E". Illegal path attribute "/geoportal". Path of origin: "http://www.ngdc.noaa.gov/geoportal/csw&quot;&lt;http://www.ngdc.noaa.gov/geoportal/csw&gt;
2013-04-17 09:32:58,763 [schema: kb] DEBUG [org.apache.commons.httpclient.HttpMethodBase] - Should close connection in response to directive: close
2013-04-17 09:32:58,763 [schema: kb] DEBUG [org.apache.commons.httpclient.HttpConnection] - Releasing connection back to connection manager.
2013-04-17 09:32:58,763 [schema: kb] WARN [geonetwork.harvester] - Raised exception when searching : org.fao.geonet.csw.common.exceptions.NoApplicableCodeEx: code=NoApplicableCode, locator=null, message=
An exception occurred with no applicable code

Looks like its to do with cookies. I've rebuilt geonetwork and set cookie policy to 'ignoreCookies' with no success - it still fails. Geoportal version is reported as.

NGDC Geoportal Instance using ArcGIS Server Geoportal Extension 10 - OGC CSW 2.0.2 ISO AP

I'll try Simon's restful harvester and see if I have more success.

thanks again all,

Brian.

On 17/04/13 07:24, Francois Prunayre wrote:
Hi, GeoNetwork checks the capability in order to send CQL or FE request. I've been harvesting ESRI GeoPortal with CSW with no issue.
Could you share the URL to be harvested ?

Cheers.

Francois

2013/4/17 <Simon.Pigot@anonymised.com<mailto:Simon.Pigot@anonymised.com>>
I implemented a harvester that uses the geoportal REST api instead of CSW (would prefer to use the standard yes but it could be an alternative if you need it to work and don't have time for sorting this out) so you could try this if you can't nail down the problems with CSW. The harvester is documented in the GeoNetwork 2.8.x manual at GeoPortal REST Harvesting - GeoNetwork Opensource - a lot depends on the version of the geoportal you are harvesting from (the open source geoportal should work with CSW, but for other versions such as 9.3.x, it might be better to use the geoportal REST api harvester in 2.8.x).

Cheers,
Simon
________________________________________
From: Marten Hogeweg [mhogeweg@anonymised.com<mailto:mhogeweg@anonymised.com>]
Sent: Wednesday, 17 April 2013 4:10 AM
To: pcreso@anonymised.com<mailto:pcreso@anonymised.com>; geonetwork-users@anonymised.comeforge.net<mailto:geonetwork-users@lists.sourceforge.net>; Brian O'Hare
Subject: Re: [GeoNetwork-users] Geoportal issues

Can you share the CSW requests that Geonetwork sends? Note that Geoportal Server expects OGC Filter, not CQL.

Marten Hogeweg
Esri

-----Original Message-----
From: pcreso@anonymised.com<mailto:pcreso@anonymised.com> [mailto:pcreso@anonymised.com]
Sent: Tuesday, April 16, 2013 7:48 PM
To: geonetwork-users@lists.sourceforge.net<mailto:geonetwork-users@anonymised.comurceforge.net>; Brian O'Hare
Subject: Re: [GeoNetwork-users] Geoportal issues

Brian,

I understand the Geonetwork catalogue behind http://www.geodata.govt.nz is harvesting records from the ESRI geoportal catalogue at http://geoportal.doc.govt.nz/geoportal/catalog/main/home.page

The person looking after the geodata catalogue is rainer (at) silverstripe.com<http://silverstripe.com> who may be able to advise you. I believe Simon Pigot was also involved in making this work successfully.

Cheers,

   Brent Wood

--- On Tue, 4/16/13, Brian O'Hare <brian.ohare@anonymised.com<mailto:brian.ohare@anonymised.com>> wrote:

From: Brian O'Hare <brian.ohare@anonymised.com<mailto:brian.ohare@anonymised.com>>
Subject: [GeoNetwork-users] Geoportal issues
To: geonetwork-users@lists.sourceforge.net<mailto:geonetwork-users@anonymised.comurceforge.net>
Date: Tuesday, April 16, 2013, 11:21 PM

Hi

Has anyone had any problems trying to run a CSW harvest against ESRI Geoportal. Geonetwork seems to catch a 'NoApplicableCodeException' from Geoportal when trying to do a GetRecordsRequest. I've only noticed this with requests to Geoportal.

cheers,

Brian.

--
Geospatial Software Developer (Geo Services) EDINA Causewayside House
160 Causewayside
Edinburgh EH9 1PR

p: +44 (0)131 5601184<tel:%2B44%20%280%29131%205601184>
e: brian.ohare@anonymised.com<mailto:brian.ohare@anonymised.com>
w: http://edina.ac.uk

The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net<mailto:GeoNetwork-users@anonymised.comforge.net>

GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork
------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net<mailto:GeoNetwork-users@anonymised.comforge.net>
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net<mailto:GeoNetwork-users@anonymised.comforge.net>

GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

--
Geospatial Software Developer (Geo Services)
EDINA
Causewayside House
160 Causewayside
Edinburgh EH9 1PR

p: +44 (0)131 5601184
e: brian.ohare@anonymised.com<mailto:brian.ohare@anonymised.com>
w: http://edina.ac.uk

9

Many thanks Simon, might give this a go.

cheers,

Brian.

On 18/04/13 00:37, Simon.Pigot@anonymised.com wrote:

Just in case the gmiTogmd.xsl converter got blocked by some zealous email server, I've committed it to 2.8.x and trunk (see web/src/main/webapp/xsl/conversion/import/gmiTogmd.xsl).

Cheers,
Simon
________________________________________
From: Pigot, Simon (CMAR, Hobart)
Sent: Thursday, 18 April 2013 9:29 AM
To: Brian O'Hare; Francois Prunayre
Cc: Geonetwork-Users@anonymised.com
Subject: RE: [GeoNetwork-users] Geoportal issues

Brian,

GeoPortal REST harvester in GeoNetwork does seem to work ok on the NOAA site. I set up a harvester as shown in the image attached. I did have to create a gmiTogmd.xsl (attached), install it in web/geonetwork/xsl/conversion/import as the NOAA site is using ISO19115-2 (this xslt converts the top element and loses the rest of the gmi content) and select it in the harvester as an XSLT to apply to the content harvested from the geoportal site.

Cheers,
Simon
________________________________________
From: Brian O'Hare [brian.ohare@anonymised.com]
Sent: Wednesday, 17 April 2013 6:51 PM
To: Francois Prunayre
Cc: Pigot, Simon (CMAR, Hobart); Geonetwork-Users@anonymised.com; mhogeweg@anonymised.com; pcreso@anonymised.com
Subject: Re: [GeoNetwork-users] Geoportal issues

Many thanks everyone for your help.

I've enabled HttpClient debug logging in Geonetwork but can't see the request going to geoportal. However I have used a debugger on the running tomcat and can see that the request is using Filter not CQL. It should just be a straightforward GetRecords request. The capabilities file is parsed without problems. Here's some logging output - you can see the Geoportal url here also:

2013-04-17 09:32:58,384 [schema: kb] DEBUG [org.apache.commons.httpclient.HttpConnection] - Open connection to www.ngdc.noaa.gov:80<http://www.ngdc.noaa.gov:80>
2013-04-17 09:32:58,529 [schema: kb] DEBUG [org.apache.commons.httpclient.HttpMethodBase] - Adding Host request header
2013-04-17 09:32:58,529 [schema: kb] WARN [org.apache.commons.httpclient.cookie.CookieSpec] - Invalid cookie state: domain not specified
2013-04-17 09:32:58,529 [schema: kb] DEBUG [org.apache.commons.httpclient.methods.EntityEnclosingMethod] - Request body sent
2013-04-17 09:32:58,760 [schema: kb] WARN [org.apache.commons.httpclient.HttpMethodBase] - Cookie rejected: "JSESSIONID=70FF886352F32B7A0FB5996515DFB29E". Illegal path attribute "/geoportal". Path of origin: "http://www.ngdc.noaa.gov/geoportal/csw&quot;&lt;http://www.ngdc.noaa.gov/geoportal/csw&gt;
2013-04-17 09:32:58,763 [schema: kb] DEBUG [org.apache.commons.httpclient.HttpMethodBase] - Should close connection in response to directive: close
2013-04-17 09:32:58,763 [schema: kb] DEBUG [org.apache.commons.httpclient.HttpConnection] - Releasing connection back to connection manager.
2013-04-17 09:32:58,763 [schema: kb] WARN [geonetwork.harvester] - Raised exception when searching : org.fao.geonet.csw.common.exceptions.NoApplicableCodeEx: code=NoApplicableCode, locator=null, message=
An exception occurred with no applicable code

Looks like its to do with cookies. I've rebuilt geonetwork and set cookie policy to 'ignoreCookies' with no success - it still fails. Geoportal version is reported as.

NGDC Geoportal Instance using ArcGIS Server Geoportal Extension 10 - OGC CSW 2.0.2 ISO AP

I'll try Simon's restful harvester and see if I have more success.

thanks again all,

Brian.

On 17/04/13 07:24, Francois Prunayre wrote:
Hi, GeoNetwork checks the capability in order to send CQL or FE request. I've been harvesting ESRI GeoPortal with CSW with no issue.
Could you share the URL to be harvested ?

Cheers.

Francois

2013/4/17 <Simon.Pigot@anonymised.com<mailto:Simon.Pigot@anonymised.com>>
I implemented a harvester that uses the geoportal REST api instead of CSW (would prefer to use the standard yes but it could be an alternative if you need it to work and don't have time for sorting this out) so you could try this if you can't nail down the problems with CSW. The harvester is documented in the GeoNetwork 2.8.x manual at http://geonetwork-opensource.org/manuals/2.8.0/eng/users/managing_metadata/harvesting/geoPREST/index.html#geoportal-rest-harvester - a lot depends on the version of the geoportal you are harvesting from (the open source geoportal should work with CSW, but for other versions such as 9.3.x, it might be better to use the geoportal REST api harvester in 2.8.x).

Cheers,
Simon
________________________________________
From: Marten Hogeweg [mhogeweg@anonymised.com<mailto:mhogeweg@anonymised.com>]
Sent: Wednesday, 17 April 2013 4:10 AM
To: pcreso@anonymised.com<mailto:pcreso@anonymised.com>; geonetwork-users@lists.sourceforge.net<mailto:geonetwork-users@lists.sourceforge.net>; Brian O'Hare
Subject: Re: [GeoNetwork-users] Geoportal issues

Can you share the CSW requests that Geonetwork sends? Note that Geoportal Server expects OGC Filter, not CQL.

Marten Hogeweg
Esri

-----Original Message-----
From: pcreso@anonymised.com<mailto:pcreso@anonymised.com> [mailto:pcreso@anonymised.com]
Sent: Tuesday, April 16, 2013 7:48 PM
To: geonetwork-users@lists.sourceforge.net<mailto:geonetwork-users@lists.sourceforge.net>; Brian O'Hare
Subject: Re: [GeoNetwork-users] Geoportal issues

Brian,

I understand the Geonetwork catalogue behind http://www.geodata.govt.nz is harvesting records from the ESRI geoportal catalogue at http://geoportal.doc.govt.nz/geoportal/catalog/main/home.page

The person looking after the geodata catalogue is rainer (at) silverstripe.com<http://silverstripe.com> who may be able to advise you. I believe Simon Pigot was also involved in making this work successfully.

Cheers,

    Brent Wood

--- On Tue, 4/16/13, Brian O'Hare <brian.ohare@anonymised.com<mailto:brian.ohare@anonymised.com>> wrote:

From: Brian O'Hare <brian.ohare@anonymised.com<mailto:brian.ohare@anonymised.com>>
Subject: [GeoNetwork-users] Geoportal issues
To: geonetwork-users@lists.sourceforge.net<mailto:geonetwork-users@lists.sourceforge.net>
Date: Tuesday, April 16, 2013, 11:21 PM

Hi

Has anyone had any problems trying to run a CSW harvest against ESRI Geoportal. Geonetwork seems to catch a 'NoApplicableCodeException' from Geoportal when trying to do a GetRecordsRequest. I've only noticed this with requests to Geoportal.

cheers,

Brian.

--
Geospatial Software Developer (Geo Services) EDINA Causewayside House
160 Causewayside
Edinburgh EH9 1PR

p: +44 (0)131 5601184<tel:%2B44%20%280%29131%205601184>
e: brian.ohare@anonymised.com<mailto:brian.ohare@anonymised.com>
w: http://edina.ac.uk

The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net<mailto:GeoNetwork-users@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork
------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net<mailto:GeoNetwork-users@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net<mailto:GeoNetwork-users@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

--
Geospatial Software Developer (Geo Services)
EDINA
Causewayside House
160 Causewayside
Edinburgh EH9 1PR

p: +44 (0)131 5601184
e: brian.ohare@anonymised.com<mailto:brian.ohare@anonymised.com>
w: http://edina.ac.uk

--
Geospatial Software Developer (Geo Services)
EDINA
Causewayside House
160 Causewayside
Edinburgh EH9 1PR

p: +44 (0)131 5601184
e: brian.ohare@anonymised.com
w: http://edina.ac.uk

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

10

Simon,

Your Geoportal REST harvester works very nicely thanks! I had to add 'sys.schema.key:iso19115-2' as a search criteria in the harvester config to just harvest records with this schema as I was getting errors in schema auto-detection when it came across RDF records in the NOAA catalogue. See https://github.com/Esri/geoportal-server/wiki/How-to-find-all-documents-of-a-particular-metadata-standard.

thanks again,

Brian

On 18/04/13 00:37, Simon.Pigot@anonymised.com wrote:

Just in case the gmiTogmd.xsl converter got blocked by some zealous email server, I've committed it to 2.8.x and trunk (see web/src/main/webapp/xsl/conversion/import/gmiTogmd.xsl).

Cheers,
Simon
________________________________________
From: Pigot, Simon (CMAR, Hobart)
Sent: Thursday, 18 April 2013 9:29 AM
To: Brian O'Hare; Francois Prunayre
Cc:Geonetwork-Users@anonymised.com
Subject: RE: [GeoNetwork-users] Geoportal issues

Brian,

GeoPortal REST harvester in GeoNetwork does seem to work ok on the NOAA site. I set up a harvester as shown in the image attached. I did have to create a gmiTogmd.xsl (attached), install it in web/geonetwork/xsl/conversion/import as the NOAA site is using ISO19115-2 (this xslt converts the top element and loses the rest of the gmi content) and select it in the harvester as an XSLT to apply to the content harvested from the geoportal site.

Cheers,
Simon
________________________________________
From: Brian O'Hare [brian.ohare@anonymised.com]
Sent: Wednesday, 17 April 2013 6:51 PM
To: Francois Prunayre
Cc: Pigot, Simon (CMAR, Hobart);Geonetwork-Users@anonymised.com;mhogeweg@anonymised.com;pcreso@anonymised.com
Subject: Re: [GeoNetwork-users] Geoportal issues

Many thanks everyone for your help.

I've enabled HttpClient debug logging in Geonetwork but can't see the request going to geoportal. However I have used a debugger on the running tomcat and can see that the request is using Filter not CQL. It should just be a straightforward GetRecords request. The capabilities file is parsed without problems. Here's some logging output - you can see the Geoportal url here also:

2013-04-17 09:32:58,384 [schema: kb] DEBUG [org.apache.commons.httpclient.HttpConnection] - Open connection towww.ngdc.noaa.gov:80<http://www.ngdc.noaa.gov:80>
2013-04-17 09:32:58,529 [schema: kb] DEBUG [org.apache.commons.httpclient.HttpMethodBase] - Adding Host request header
2013-04-17 09:32:58,529 [schema: kb] WARN [org.apache.commons.httpclient.cookie.CookieSpec] - Invalid cookie state: domain not specified
2013-04-17 09:32:58,529 [schema: kb] DEBUG [org.apache.commons.httpclient.methods.EntityEnclosingMethod] - Request body sent
2013-04-17 09:32:58,760 [schema: kb] WARN [org.apache.commons.httpclient.HttpMethodBase] - Cookie rejected: "JSESSIONID=70FF886352F32B7A0FB5996515DFB29E". Illegal path attribute "/geoportal". Path of origin:"http://www.ngdc.noaa.gov/geoportal/csw&quot;&lt;http://www.ngdc.noaa.gov/geoportal/csw&gt;
2013-04-17 09:32:58,763 [schema: kb] DEBUG [org.apache.commons.httpclient.HttpMethodBase] - Should close connection in response to directive: close
2013-04-17 09:32:58,763 [schema: kb] DEBUG [org.apache.commons.httpclient.HttpConnection] - Releasing connection back to connection manager.
2013-04-17 09:32:58,763 [schema: kb] WARN [geonetwork.harvester] - Raised exception when searching : org.fao.geonet.csw.common.exceptions.NoApplicableCodeEx: code=NoApplicableCode, locator=null, message=
An exception occurred with no applicable code

Looks like its to do with cookies. I've rebuilt geonetwork and set cookie policy to 'ignoreCookies' with no success - it still fails. Geoportal version is reported as.

NGDC Geoportal Instance using ArcGIS Server Geoportal Extension 10 - OGC CSW 2.0.2 ISO AP

I'll try Simon's restful harvester and see if I have more success.

thanks again all,

Brian.

On 17/04/13 07:24, Francois Prunayre wrote:
Hi, GeoNetwork checks the capability in order to send CQL or FE request. I've been harvesting ESRI GeoPortal with CSW with no issue.
Could you share the URL to be harvested ?

Cheers.

Francois

2013/4/17 <Simon.Pigot@anonymised.com<mailto:Simon.Pigot@anonymised.com>>
I implemented a harvester that uses the geoportal REST api instead of CSW (would prefer to use the standard yes but it could be an alternative if you need it to work and don't have time for sorting this out) so you could try this if you can't nail down the problems with CSW. The harvester is documented in the GeoNetwork 2.8.x manual athttp://geonetwork-opensource.org/manuals/2.8.0/eng/users/managing_metadata/harvesting/geoPREST/index.html#geoportal-rest-harvester - a lot depends on the version of the geoportal you are harvesting from (the open source geoportal should work with CSW, but for other versions such as 9.3.x, it might be better to use the geoportal REST api harvester in 2.8.x).

Cheers,
Simon
________________________________________
From: Marten Hogeweg [mhogeweg@anonymised.com<mailto:mhogeweg@anonymised.com>]
Sent: Wednesday, 17 April 2013 4:10 AM
To:pcreso@anonymised.com<mailto:pcreso@anonymised.com>;geonetwork-users@lists.sourceforge.net<mailto:geonetwork-users@lists.sourceforge.net>; Brian O'Hare
Subject: Re: [GeoNetwork-users] Geoportal issues

Can you share the CSW requests that Geonetwork sends? Note that Geoportal Server expects OGC Filter, not CQL.

Marten Hogeweg
Esri

-----Original Message-----
From:pcreso@anonymised.com<mailto:pcreso@anonymised.com> [mailto:pcreso@anonymised.com]
Sent: Tuesday, April 16, 2013 7:48 PM
To:geonetwork-users@lists.sourceforge.net<mailto:geonetwork-users@lists.sourceforge.net>; Brian O'Hare
Subject: Re: [GeoNetwork-users] Geoportal issues

Brian,

I understand the Geonetwork catalogue behindhttp://www.geodata.govt.nz is harvesting records from the ESRI geoportal catalogue athttp://geoportal.doc.govt.nz/geoportal/catalog/main/home.page

The person looking after the geodata catalogue is rainer (at) silverstripe.com<http://silverstripe.com> who may be able to advise you. I believe Simon Pigot was also involved in making this work successfully.

Cheers,

    Brent Wood

--- On Tue, 4/16/13, Brian O'Hare <brian.ohare@anonymised.com<mailto:brian.ohare@anonymised.com>> wrote:

From: Brian O'Hare <brian.ohare@anonymised.com<mailto:brian.ohare@anonymised.com>>
Subject: [GeoNetwork-users] Geoportal issues
To:geonetwork-users@lists.sourceforge.net<mailto:geonetwork-users@lists.sourceforge.net>
Date: Tuesday, April 16, 2013, 11:21 PM

Hi

Has anyone had any problems trying to run a CSW harvest against ESRI Geoportal. Geonetwork seems to catch a 'NoApplicableCodeException' from Geoportal when trying to do a GetRecordsRequest. I've only noticed this with requests to Geoportal.

cheers,

Brian.

--
Geospatial Software Developer (Geo Services) EDINA Causewayside House
160 Causewayside
Edinburgh EH9 1PR

p: +44 (0)131 5601184<tel:%2B44%20%280%29131%205601184>
e:brian.ohare@anonymised.com<mailto:brian.ohare@anonymised.com>
w:http://edina.ac.uk

The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net<mailto:GeoNetwork-users@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained athttp://sourceforge.net/projects/geonetwork
------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net<mailto:GeoNetwork-users@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained athttp://sourceforge.net/projects/geonetwork

------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net<mailto:GeoNetwork-users@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained athttp://sourceforge.net/projects/geonetwork

--
Geospatial Software Developer (Geo Services)
EDINA
Causewayside House
160 Causewayside
Edinburgh EH9 1PR

p: +44 (0)131 5601184
e:brian.ohare@anonymised.com<mailto:brian.ohare@anonymised.com>
w:http://edina.ac.uk

--
Geospatial Software Developer (Geo Services)
EDINA
Causewayside House
160 Causewayside
Edinburgh EH9 1PR

p: +44 (0)131 5601184
e:brian.ohare@anonymised.com
w:http://edina.ac.uk

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

11

Hi everyone,

**GeoNetwork Version:2.10.1 ** http://geocatalog.webservice-energy.org

I went on this interesting thread as I have trouble setting up a CSW harvest
from my GeoNetwork to an ESRI Geoportal. At harvest set-up stage, I try to
add "Search Criteria" to filter the harvest. We I do so, I'm not able to
save this new entry. The save button does not work. If I do not add such
"Search criteria" I can save the harvest target.

I need these "Search Criteria" as I don't want to harvest the whole huge
content of the remote Geoportal.
For the same reason I've not yet try the harvest of the whole Geoportal.

Does anyone has the same pb. ?

Has someone successfully harvest a remote CSW ESRI Geoportal yet ?

Thanks in advance for your help.

Lionel

--
View this message in context: http://osgeo-org.1560.x6.nabble.com/Geoportal-issues-tp5047261p5090611.html
Sent from the GeoNetwork users mailing list archive at Nabble.com.

12

Hi Lionel,

We have had problems connecting to Geoportal.

We are developng a free Open Source tool for ESRI Arc Server to fix the lack of effective CSW support for ESRI geodabases.

It is a python based tool that harvests metadata from the geodabase and provides a CSW service (tested against Geometwork).

Hopefully this will allow ESRI users to have a valid CSW service with their ESRI system, and introduce them to Open Source as well :slight_smile:

On a related note - we have upgraded our Geonetwork catalogue at NIWA - which may have addressed some of the issues you had with it?

Cheers,

Brent Wood

________________________________
From: Lionel Menard <lionel.menard@anonymised.com>
To: geonetwork-users@lists.sourceforge.net
Sent: Friday, November 22, 2013 11:18 PM
Subject: Re: [GeoNetwork-users] Geoportal issues

Hi everyone,

**GeoNetwork Version:2.10.1 ** http://geocatalog.webservice-energy.org

I went on this interesting thread as I have trouble setting up a CSW harvest
from my GeoNetwork to an ESRI Geoportal. At harvest set-up stage, I try to
add "Search Criteria" to filter the harvest. We I do so, I'm not able to
save this new entry. The save button does not work. If I do not add such
"Search criteria" I can save the harvest target.

I need these "Search Criteria" as I don't want to harvest the whole huge
content of the remote Geoportal.
For the same reason I've not yet try the harvest of the whole Geoportal.

Does anyone has the same pb. ?

Has someone successfully harvest a remote CSW ESRI Geoportal yet ?

Thanks in advance for your help.

Lionel

--
View this message in context: http://osgeo-org.1560.x6.nabble.com/Geoportal-issues-tp5047261p5090611.html
Sent from the GeoNetwork users mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Shape the Mobile Experience: Free Subscription
Software experts and developers: Be at the forefront of tech innovation.
Intel(R) Software Adrenaline delivers strategic insight and game-changing
conversations that shape the rapidly evolving mobile landscape. Sign up now.
http://pubads.g.doubleclick.net/gampad/clk?id=63431311&iu=/4140/ostg.clktrk
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

13

Hi all,

Esri Geoportal Server IS open source and provide OGC compliant CSW services. Geoportal Server does NOT require any Esri software license and works with services from Esri or with OGC services from other software.

We have a open source tool already that allows for registering metadata for files in file systems and and feature classes in geodatabases (personal, file, enterprise) with the Geoportal Server for exposing through CSW. Users of ArcGIS Server may also publish geodatabase services that Geoportal Server can index (along with all other ArcGIS Server service types including KML, W*S, and REST services).

We also have an open source CSW client for ArcGIS Desktop users that can connect to may flavors, versions, and brands of CSW (OGCCORE, APISO, ebRIM) and can be extended with custom profiles for specific catalogs.

You can fork, customize, and contribute to Geoportal Server at https://github.com/Esri/geoportal-server

And while you're at github, check out the 160 public repos (https://github.com/Esri) of other Esri open source tools, components, API, templates, SDK, and what have you.

Cheers,

@martenhogeweg

-----Original Message-----
From: Brent Wood [mailto:pcreso@anonymised.com]
Sent: Friday, November 22, 2013 3:36 PM
To: Lionel Menard; geonetwork-users@lists.sourceforge.net
Subject: Re: [GeoNetwork-users] Geoportal issues

Hi Lionel,

We have had problems connecting to Geoportal.

We are developng a free Open Source tool for ESRI Arc Server to fix the lack of effective CSW support for ESRI geodabases.

It is a python based tool that harvests metadata from the geodabase and provides a CSW service (tested against Geometwork).

Hopefully this will allow ESRI users to have a valid CSW service with their ESRI system, and introduce them to Open Source as well :slight_smile:

On a related note - we have upgraded our Geonetwork catalogue at NIWA - which may have addressed some of the issues you had with it?

Cheers,

Brent Wood

________________________________
From: Lionel Menard <lionel.menard@anonymised.com>
To: geonetwork-users@lists.sourceforge.net
Sent: Friday, November 22, 2013 11:18 PM
Subject: Re: [GeoNetwork-users] Geoportal issues

Hi everyone,

**GeoNetwork Version:2.10.1 ** http://geocatalog.webservice-energy.org

I went on this interesting thread as I have trouble setting up a CSW harvest
from my GeoNetwork to an ESRI Geoportal. At harvest set-up stage, I try to
add "Search Criteria" to filter the harvest. We I do so, I'm not able to
save this new entry. The save button does not work. If I do not add such
"Search criteria" I can save the harvest target.

I need these "Search Criteria" as I don't want to harvest the whole huge
content of the remote Geoportal.
For the same reason I've not yet try the harvest of the whole Geoportal.

Does anyone has the same pb. ?

Has someone successfully harvest a remote CSW ESRI Geoportal yet ?

Thanks in advance for your help.

Lionel

--
View this message in context: http://osgeo-org.1560.x6.nabble.com/Geoportal-issues-tp5047261p5090611.html
Sent from the GeoNetwork users mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Shape the Mobile Experience: Free Subscription
Software experts and developers: Be at the forefront of tech innovation.
Intel(R) Software Adrenaline delivers strategic insight and game-changing
conversations that shape the rapidly evolving mobile landscape. Sign up now.
http://pubads.g.doubleclick.net/gampad/clk?id=63431311&iu=/4140/ostg.clktrk
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork
------------------------------------------------------------------------------
Shape the Mobile Experience: Free Subscription
Software experts and developers: Be at the forefront of tech innovation.
Intel(R) Software Adrenaline delivers strategic insight and game-changing
conversations that shape the rapidly evolving mobile landscape. Sign up now.
http://pubads.g.doubleclick.net/gampad/clk?id=63431311&iu=/4140/ostg.clktrk
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

14

Without getting into a debate about the standards support (or lack) of various implementations of OGC services in various implementations - there are plenty of examples of poor interoperability - and not just from proprietary implementations... but I know of no (yes ZERO) institutions who have implemented a working ESRI geoportal without some painful issues to work through. I'm not saying there are none - just none I know of.

Even the ESRI developers we are working with wanted the new tool to be open source licenced so that other ESRI users could benefit, so I assume others I don't know of would also benefit..

I would hope the ESRI community will welcome having a choice of open source tools in this area to enable them to reliably participate in a federated, open standards based metadata network.

Brent Wood

________________________________
From: Marten Hogeweg <mhogeweg@anonymised.com>
To: Brent Wood <pcreso@anonymised.com>; Lionel Menard <lionel.menard@anonymised.com>; "geonetwork-users@lists.sourceforge.net" <geonetwork-users@anonymised.comrge.net>
Sent: Saturday, November 23, 2013 12:46 PM
Subject: RE: [GeoNetwork-users] Geoportal issues

Hi all,

Esri Geoportal Server IS open source and provide OGC compliant CSW services. Geoportal Server does NOT require any Esri software license and works with services from Esri or with OGC services from other software.

We have a open source tool already that allows for registering metadata for files in file systems and and feature classes in geodatabases (personal, file, enterprise) with the Geoportal Server for exposing through CSW. Users of ArcGIS Server may also publish geodatabase services that Geoportal Server can index (along with all other ArcGIS Server service types including KML, W*S, and REST services).

We also have an open source CSW client for ArcGIS Desktop users that can connect to may flavors, versions, and brands of CSW (OGCCORE, APISO, ebRIM) and can be extended with custom profiles for specific catalogs.

You can fork, customize, and contribute to Geoportal Server at https://github.com/Esri/geoportal-server

And while you're at github, check out the 160 public repos (https://github.com/Esri) of other Esri open source tools, components, API, templates, SDK, and what have you.

Cheers,

@martenhogeweg

-----Original Message-----
From: Brent Wood [mailto:pcreso@anonymised.com]
Sent: Friday, November 22, 2013 3:36 PM
To: Lionel Menard; geonetwork-users@lists.sourceforge.net
Subject: Re: [GeoNetwork-users] Geoportal issues

Hi Lionel,

We have had problems connecting to Geoportal.

We are developng a free Open Source tool for ESRI Arc Server to fix the lack of effective CSW support for ESRI geodabases.

It is a python based tool that harvests metadata from the geodabase and provides a CSW service (tested against Geometwork).

Hopefully this will allow ESRI users to have a valid CSW service with their ESRI system, and introduce them to Open Source as well :slight_smile:

On a related note - we have upgraded our Geonetwork catalogue at NIWA - which may have addressed some of the issues you had with it?

Cheers,

Brent Wood

________________________________
From: Lionel Menard <lionel.menard@anonymised.com>
To: geonetwork-users@lists.sourceforge.net
Sent: Friday, November 22, 2013 11:18 PM
Subject: Re: [GeoNetwork-users] Geoportal issues

Hi everyone,

**GeoNetwork Version:2.10.1 ** http://geocatalog.webservice-energy.org

I went on this interesting thread as I have trouble setting up a CSW harvest
from my GeoNetwork to an ESRI Geoportal. At harvest set-up stage, I try to
add "Search Criteria" to filter the harvest. We I do so, I'm not able to
save this new entry. The save button does not work. If I do not add such
"Search criteria" I can save the harvest target.

I need these "Search Criteria" as I don't want to harvest the whole huge
content of the remote Geoportal.
For the same reason I've not yet try the harvest of the whole Geoportal.

Does anyone has the same pb. ?

Has someone successfully harvest a remote CSW ESRI Geoportal yet ?

Thanks in advance for your help.

Lionel

--
View this message in context: http://osgeo-org.1560.x6.nabble.com/Geoportal-issues-tp5047261p5090611.html
Sent from the GeoNetwork users mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Shape the Mobile Experience: Free Subscription
Software experts and developers: Be at the forefront of tech innovation.
Intel(R) Software Adrenaline delivers strategic insight and game-changing
conversations that shape the rapidly evolving mobile landscape. Sign up now.
http://pubads.g.doubleclick.net/gampad/clk?id=63431311&iu=/4140/ostg.clktrk
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

------------------------------------------------------------------------------
Shape the Mobile Experience: Free Subscription
Software experts and developers: Be at the forefront of tech innovation.
Intel(R) Software Adrenaline delivers strategic insight and game-changing
conversations that shape the rapidly evolving mobile landscape. Sign up now.
http://pubads.g.doubleclick.net/gampad/clk?id=63431311&iu=/4140/ostg.clktrk
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

15

For sure an OGC CSW service would be best, but I guess you tried the GeoPortal havester in GeoNetwork 2.10.x?

Cheers,
Simon
________________________________________
From: Brent Wood [pcreso@anonymised.com]
Sent: Saturday, 23 November 2013 10:36 AM
To: Lionel Menard; geonetwork-users@lists.sourceforge.net
Subject: Re: [GeoNetwork-users] Geoportal issues

Hi Lionel,

We have had problems connecting to Geoportal.

We are developng a free Open Source tool for ESRI Arc Server to fix the lack of effective CSW support for ESRI geodabases.

It is a python based tool that harvests metadata from the geodabase and provides a CSW service (tested against Geometwork).

Hopefully this will allow ESRI users to have a valid CSW service with their ESRI system, and introduce them to Open Source as well :slight_smile:

On a related note - we have upgraded our Geonetwork catalogue at NIWA - which may have addressed some of the issues you had with it?

Cheers,

  Brent Wood

________________________________
From: Lionel Menard <lionel.menard@anonymised.com>
To: geonetwork-users@lists.sourceforge.net
Sent: Friday, November 22, 2013 11:18 PM
Subject: Re: [GeoNetwork-users] Geoportal issues

Hi everyone,

**GeoNetwork Version:2.10.1 ** http://geocatalog.webservice-energy.org

I went on this interesting thread as I have trouble setting up a CSW harvest
from my GeoNetwork to an ESRI Geoportal. At harvest set-up stage, I try to
add "Search Criteria" to filter the harvest. We I do so, I'm not able to
save this new entry. The save button does not work. If I do not add such
"Search criteria" I can save the harvest target.

I need these "Search Criteria" as I don't want to harvest the whole huge
content of the remote Geoportal.
For the same reason I've not yet try the harvest of the whole Geoportal.

Does anyone has the same pb. ?

Has someone successfully harvest a remote CSW ESRI Geoportal yet ?

Thanks in advance for your help.

Lionel

--
View this message in context: http://osgeo-org.1560.x6.nabble.com/Geoportal-issues-tp5047261p5090611.html
Sent from the GeoNetwork users mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Shape the Mobile Experience: Free Subscription
Software experts and developers: Be at the forefront of tech innovation.
Intel(R) Software Adrenaline delivers strategic insight and game-changing
conversations that shape the rapidly evolving mobile landscape. Sign up now.
http://pubads.g.doubleclick.net/gampad/clk?id=63431311&iu=/4140/ostg.clktrk
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net

GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

16

Gidday Simon,

We did consider Geonetwork based solutions, but we are working with other institutions looking at non-Geonetwork catalogues as well as QGIS usres using CSW plugins, so rather than having Geonetwork able to interoperate with Geoportal, we believed it was a better way forward to enable ESRI (Arc Server) users to work robustly within a CSW federation.

Cheers,

Brent

________________________________
From: "Simon.Pigot@anonymised.com" <Simon.Pigot@anonymised.com>
To: pcreso@anonymised.com; lionel.menard@anonymised.com; geonetwork-users@anonymised.comceforge.net
Sent: Saturday, November 23, 2013 3:12 PM
Subject: RE: [GeoNetwork-users] Geoportal issues

For sure an OGC CSW service would be best, but I guess you tried the GeoPortal havester in GeoNetwork 2.10.x?

http://geonetwork-opensource.org/manuals/2.10.2/eng/users/managing_metadata/harvesting/geoPREST/index.html#geoportal-rest-harvester

Cheers,
Simon
________________________________________

From: Brent Wood [pcreso@anonymised.com]
Sent: Saturday, 23 November 2013 10:36 AM
To: Lionel Menard; geonetwork-users@lists.sourceforge.net
Subject: Re: [GeoNetwork-users] Geoportal issues

Hi Lionel,

We have had problems connecting to Geoportal.

We are developng a free Open Source tool for ESRI Arc Server to fix the lack of effective CSW support for ESRI geodabases.

It is a python based tool that harvests metadata from the geodabase and provides a CSW service (tested against Geometwork).

Hopefully this will allow ESRI users to have a valid CSW service with their ESRI system, and introduce them to Open Source as well :slight_smile:

On a related note - we have upgraded our Geonetwork catalogue at NIWA - which may have addressed some of the issues you had with it?

Cheers,

Brent Wood

________________________________
From: Lionel Menard <lionel.menard@anonymised.com>
To: geonetwork-users@lists.sourceforge.net
Sent: Friday, November 22, 2013 11:18 PM
Subject: Re: [GeoNetwork-users] Geoportal issues

Hi everyone,

**GeoNetwork Version:2.10.1 ** http://geocatalog.webservice-energy.org

I went on this interesting thread as I have trouble setting up a CSW harvest
from my GeoNetwork to an ESRI Geoportal. At harvest set-up stage, I try to
add "Search Criteria" to filter the harvest. We I do so, I'm not able to
save this new entry. The save button does not work. If I do not add such
"Search criteria" I can save the harvest target.

I need these "Search Criteria" as I don't want to harvest the whole huge
content of the remote Geoportal.
For the same reason I've not yet try the harvest of the whole Geoportal.

Does anyone has the same pb. ?

Has someone successfully harvest a remote CSW ESRI Geoportal yet ?

Thanks in advance for your help.

Lionel

--
View this message in context: http://osgeo-org.1560.x6.nabble.com/Geoportal-issues-tp5047261p5090611.html
Sent from the GeoNetwork users mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Shape the Mobile Experience: Free Subscription
Software experts and developers: Be at the forefront of tech innovation.
Intel(R) Software Adrenaline delivers strategic insight and game-changing
conversations that shape the rapidly evolving mobile landscape. Sign up now.
http://pubads.g.doubleclick.net/gampad/clk?id=63431311&iu=/4140/ostg.clktrk
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

17

Dear Marten, Brent and Simon, dear list,

thanks a lot for the comments, solutions, (and related note ;-)).

@Simon: No I did not test this feature yet. I will for sure and provide
feedback.

@Marten and Brent. Thanks for your experiences feedback and clarification. I
think this need to be a win win solution here. The more platform available
the more dynamic the domain might be (No "Yoda" talk here :slight_smile: ). Of course
we need to manage this cross-platform interoperability but pb. also occurs
between the various versions of a given platform. So lets try to smooth that
out.

I'm gonna get my hand back in the Geoportal Harvester option right now and
hopefully provide you with successful feedback.

Thanks again to you all for your commitment.

Cheers,

Lionel

--
View this message in context: http://osgeo-org.1560.x6.nabble.com/Geoportal-issues-tp5047261p5091066.html
Sent from the GeoNetwork users mailing list archive at Nabble.com.

18

I have had great success with our geoportal instances. I currently have
three implementations. Marten and team have been extremely helpful anytime
I hit a small snag. I'd be happy to demo any of our geoportals to you.
Check this one out
http://www.match.globalchange.gov/geoportal/catalog/main/home.page

On Mon, Nov 25, 2013 at 10:48 AM, Lionel Menard <
lionel.menard@anonymised.com> wrote:

Dear Marten, Brent and Simon, dear list,

thanks a lot for the comments, solutions, (and related note ;-)).

@Simon: No I did not test this feature yet. I will for sure and provide
feedback.

@Marten and Brent. Thanks for your experiences feedback and clarification.
I
think this need to be a win win solution here. The more platform available
the more dynamic the domain might be (No "Yoda" talk here :slight_smile: ). Of course
we need to manage this cross-platform interoperability but pb. also occurs
between the various versions of a given platform. So lets try to smooth
that
out.

I'm gonna get my hand back in the Geoportal Harvester option right now and
hopefully provide you with successful feedback.

Thanks again to you all for your commitment.

Cheers,

Lionel

--
View this message in context:
http://osgeo-org.1560.x6.nabble.com/Geoportal-issues-tp5047261p5091066.html
Sent from the GeoNetwork users mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Shape the Mobile Experience: Free Subscription
Software experts and developers: Be at the forefront of tech innovation.
Intel(R) Software Adrenaline delivers strategic insight and game-changing
conversations that shape the rapidly evolving mobile landscape. Sign up
now.
http://pubads.g.doubleclick.net/gampad/clk?id=63431311&iu=/4140/ostg.clktrk
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at
http://sourceforge.net/projects/geonetwork

--
Jacqueline Mize
Metadata Specialist
Radiance Technologies, Inc.
DOC/NOAA/NESDIS/NODC/NCDDC
1021 Balch Blvd, Suite 1003
Stennis Space Center, MS 39529
email: Jacqueline.Mize@anonymised.com
ph: 228.688.2174

19

Hi everyone,
thanks Jacqueline for your testimony. Unfortunately implementing is one
thing and harvesting between implementations is an other thing...Have you
had any successful harvest from GeoNetwork to Geoportal you could report ?
@Simon: I tested the Geoportal Harvester with the help of the link you
provide. The target version is:
ArcGIS Server Geoportal Extension 10 - OGC CSW 2.0.2 ISO AP
Unfortunately this still does not work.
I set one keyword for Search Criteria: (solar)
I tested with both XSTL options set to: Add-gmd-prefix and gmiTogmd. Same
results.

From the "History" link in the "Harvest Manager" section I get the following

message: "Harvested Nothing".
I also get this message in the log:
2013-11-25 18:36:34,862 WARN [geonetwork.harvester] - Skipped record not in
supported format

I also did the test with the example Geoportal of the doc you provide:
http://www.ngdc.noaa.gov/geoportal

Same behavior...

Any thoughts about what's going wrong ?

Lionel

--
View this message in context: http://osgeo-org.1560.x6.nabble.com/Geoportal-issues-tp5047261p5091082.html
Sent from the GeoNetwork users mailing list archive at Nabble.com.

20

Our sandbox site at http://gptogc.esri.com regularly harvests from a variety of catalogs including several GeoNetwork implementations. Here is a list: http://gptogc.esri.com/geoportal/rest/repositories

One of these is the web services energy site:

http://geocatalog.webservice-energy.org/geonetwork/srv/eng/csw?SERVICE=CSW&VERSION=2.0.2&REQUEST=GetCapabilities

below is a screenshot of the harvesting results for this catalog. What you see here is that out of the 907 records the last harvest validated and published 777. This illustrates one possible challenge: different catalogs may implement different validation rules for metadata (see second screenshot for a piece of a detailed harvesting report).

Another issue I have seen many times is in the registration itself where we require the CSW to have a title in the capabilities statement. Many GeoNetwork CSW don't have this and without that you won't get through the registration process. This even holds for significant CSW like the GEOSS clearinghouse:

http://ec2-50-19-223-225.compute-1.amazonaws.com/geonetwork/srv/en/csw?request=GetCapabilities&service=CSW&version=2.0.2

what I then do is rehost the capabilities file as a straight XML file and register that with geoportal as a proxy for the actual CSW. The above is available here:

https://dl.dropboxusercontent.com/u/14831068/cswproxy/geoss_clearinghouse.xml

Furthermore: different CSW respond differently. For example the NIWA (New Zealand) catalog responds with ISO metadata in a different namespace

http://dc.niwa.co.nz/niwa_dc/srv/eng/main.home

http://dc.niwa.co.nz/niwa_dc/srv/eng/csw?service=CSW&request=GetRecordById&version=2.0.2&ElementSetName=full&OutputSchema=csw:IsoRecord&ID=c583f085-f2aa-42fd-a1ac-076703d09dde
mcp:MD_Metadata xmlns:mcp="http://bluenet3.antcrc.utas.edu.au/mcp&quot;

that means we have to update our profiles of GeoNetwork CSW interaction when we see these things.

[cid:image002.jpg@anonymised.com]

[cid:image004.jpg@anonymised.com]

-----Original Message-----
From: Lionel Menard [mailto:lionel.menard@anonymised.com]
Sent: Monday, November 25, 2013 10:17 AM
To: geonetwork-users@lists.sourceforge.net
Subject: Re: [GeoNetwork-users] Geoportal issues

Hi everyone,

thanks Jacqueline for your testimony. Unfortunately implementing is one thing and harvesting between implementations is an other thing...Have you had any successful harvest from GeoNetwork to Geoportal you could report ?

@Simon: I tested the Geoportal Harvester with the help of the link you provide. The target version is:

ArcGIS Server Geoportal Extension 10 - OGC CSW 2.0.2 ISO AP Unfortunately this still does not work.

I set one keyword for Search Criteria: (solar) I tested with both XSTL options set to: Add-gmd-prefix and gmiTogmd. Same results.

From the "History" link in the "Harvest Manager" section I get the

following

message: "Harvested Nothing".

I also get this message in the log:

2013-11-25 18:36:34,862 WARN [geonetwork.harvester] - Skipped record not in supported format

I also did the test with the example Geoportal of the doc you provide:

http://www.ngdc.noaa.gov/geoportal

Same behavior...

Any thoughts about what's going wrong ?

Lionel

--

View this message in context: http://osgeo-org.1560.x6.nabble.com/Geoportal-issues-tp5047261p5091082.html

Sent from the GeoNetwork users mailing list archive at Nabble.com.

------------------------------------------------------------------------------

Shape the Mobile Experience: Free Subscription Software experts and developers: Be at the forefront of tech innovation.

Intel(R) Software Adrenaline delivers strategic insight and game-changing conversations that shape the rapidly evolving mobile landscape. Sign up now.

http://pubads.g.doubleclick.net/gampad/clk?id=63431311&iu=/4140/ostg.clktrk

_______________________________________________

GeoNetwork-users mailing list

GeoNetwork-users@lists.sourceforge.net<mailto:GeoNetwork-users@anonymised.comforge.net>

https://lists.sourceforge.net/lists/listinfo/geonetwork-users

GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

(attachments)

image002.jpg
image004.jpg