[GeoNetwork-users] Thredds harvest error

I've looked through past messages on the list and while I see similar questions, did not see any replies with solutions so am hoping that someone out there has run into this and/or can help me find an answer.

I am attempting to set up a harvest to a Thredds catalog. I am making sure I put in the .xml extension. I've tried a variety of options based on the user guide and tried single datasets as well as a folder and the full catalog. However, regardless of what I set in the form I get the following error on harvest (*I took the full path and filename out just so it's not published). I'm wondering if there is some other setting I need to do at the server end or if it is something wrong with our .xml? >

Error: Invalid catalog <filename*>.xml ----Catalog Validation version 1.0.01 **Fatal: InvCatalogFactory.readXML failed Exception= java.io.FileNotFoundException <filename*>.xml fatalMessages= errMessages= warnMessages=
Class: BadXmlResponseEx
Stack:
at: org.fao.geonet.kernel.harvest.harvester.thredds.Harvester file: Harvester.java line: 318 method: harvestCatalog
at: org.fao.geonet.kernel.harvest.harvester.thredds.Harvester file: Harvester.java line: 251 method: harvest
at: org.fao.geonet.kernel.harvest.harvester.thredds.ThreddsHarvester file: ThreddsHarvester.java line: 245 method: doHarvest
at: org.fao.geonet.kernel.harvest.harvester.AbstractHarvester$HarvestWithIndexProcessor file: AbstractHarvester.java line: 399 method: process
at: org.fao.geonet.kernel.harvest.harvester.AbstractHarvester file: AbstractHarvester.java line: 429 method: harvest
at: org.fao.geonet.kernel.harvest.harvester.HarvesterJob file: HarvesterJob.java line: 29 method: execute
at: org.quartz.core.JobRunShell file: JobRunShell.java line: 213 method: run
at: org.quartz.simpl.SimpleThreadPool$WorkerThread file: SimpleThreadPool.java line: 557 method: run

TIA!

Kathy K.

Hi Kathy,

The error message below is returned because the harvester is unable to find the catalog using the provided url (you are getting a FileNotFoundException!).

Check the URL you are entering in the harvester returns an xml catalog on the server (e.g. using wget <filename*>.xml on the server).

For example, this url to a thredds.ucar.edu catalog entry works for me:

http://thredds.ucar.edu/thredds/catalog/grib/FNMOC/WW3/Global_1p0deg/FNMOC_WW3_Global_1p0deg_20140921_1200.grib1/catalog.xml

Regards,
Craig Jones
Integrated Marine observing System

On 23/09/14 03:33, Kathy Koch wrote:

I've looked through past messages on the list and while I see similar questions, did not see any replies with solutions so am hoping that someone out there has run into this and/or can help me find an answer.

I am attempting to set up a harvest to a Thredds catalog. I am making sure I put in the .xml extension. I've tried a variety of options based on the user guide and tried single datasets as well as a folder and the full catalog. However, regardless of what I set in the form I get the following error on harvest (*I took the full path and filename out just so it's not published). I'm wondering if there is some other setting I need to do at the server end or if it is something wrong with our .xml? >

Error: Invalid catalog <filename*>.xml ----Catalog Validation version 1.0.01 **Fatal: InvCatalogFactory.readXML failed Exception= java.io.FileNotFoundException <filename*>.xml fatalMessages= errMessages= warnMessages=
Class: BadXmlResponseEx
Stack:
at: org.fao.geonet.kernel.harvest.harvester.thredds.Harvester file: Harvester.java line: 318 method: harvestCatalog
at: org.fao.geonet.kernel.harvest.harvester.thredds.Harvester file: Harvester.java line: 251 method: harvest
at: org.fao.geonet.kernel.harvest.harvester.thredds.ThreddsHarvester file: ThreddsHarvester.java line: 245 method: doHarvest
at: org.fao.geonet.kernel.harvest.harvester.AbstractHarvester$HarvestWithIndexProcessor file: AbstractHarvester.java line: 399 method: process
at: org.fao.geonet.kernel.harvest.harvester.AbstractHarvester file: AbstractHarvester.java line: 429 method: harvest
at: org.fao.geonet.kernel.harvest.harvester.HarvesterJob file: HarvesterJob.java line: 29 method: execute
at: org.quartz.core.JobRunShell file: JobRunShell.java line: 213 method: run
at: org.quartz.simpl.SimpleThreadPool$WorkerThread file: SimpleThreadPool.java line: 557 method: run

  TIA!

Kathy K.

------------------------------------------------------------------------------
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

I was able to do a wget on the server, listed out (cat) the xml just fine, but still doesn't work as a harvest from GN. Same file not found error - settings exactly like you have. Since you had success wondering if there is an issue at the geoserver level? I am using 2.8 right now (have to start testing 2.10 and see if that makes a diff I guess).

Kathy

-----Original Message-----
From: Craig Jones [mailto:Craig.Jones@anonymised.com]
Sent: Tuesday, September 23, 2014 6:23 PM
To: Kathy Koch
Subject: Re: [GeoNetwork-users] Thredds harvest error

Hi Kathy,

I'm able to harvest directly from the first catalog url below using a default GeoNetwork 2.10.3 install (I've included a screen dump of the settings I used). The second didn't return any metadata and I think this is because the harvester can't handle the direct reference to the catalog entry for a specific dataset.

Have a look at the settings I used for the harvester, perhaps you can spot the difference? Or perhaps you are using an older version of GeoNetwork? If so let me know the version and I can check the result there, but as far as I'm aware this should work in 2.8 as well.

Also, it would be good to keep this on the mailing list if possible for future reference by others.

Regards,

Craig Jones

PS. When I said check to make sure you can access the URL on the server I meant check that you can access the URL from the server where GeoNetwork is installed. Perhaps there is a proxy or network issue preventing this server from accessing your thredds server.

On 24/09/14 04:04, Kathy Koch wrote:

Thanks for the reply. Here are the links I've been testing and get the error every time.

What else am I missing?

http://tds.glos.us/thredds/mtri/aoc.xml

http://tds.glos.us/thredds/mtri/aoc.xml?dataset=LakeErieSST-Agg

(Eventually I will want to harvest the entire catalog at

http://tds.glos.us/thredds/catalog.xml but am trying one of the above

two as a test for right now. I did try the entire catalog as a last

resort but got the same error.)

Kathy

_____________________________________

Kathy Koch

Data Curator

Contractor (LimnoTech / GLOS)

501 Avis Drive, Ann Arbor, MI 48108

Office: 734-332-1200 email:kkoch@anonymised.com

LimnoTech www.limno.com<http://www.limno.com>

________________________________________

From: Craig Jones [Craig.Jones@anonymised.com]

Sent: Monday, September 22, 2014 7:30 PM

To: geonetwork-users@lists.sourceforge.net<mailto:geonetwork-users@anonymised.comsourceforge.net>

Subject: Re: [GeoNetwork-users] Thredds harvest error

Hi Kathy,

The error message below is returned because the harvester is unable to

find the catalog using the provided url (you are getting a

FileNotFoundException!).

Check the URL you are entering in the harvester returns an xml catalog

on the server (e.g. using wget <filename*>.xml on the server).

For example, this url to a thredds.ucar.edu catalog entry works for me:

http://thredds.ucar.edu/thredds/catalog/grib/FNMOC/WW3/Global_1p0deg/F

NMOC_WW3_Global_1p0deg_20140921_1200.grib1/catalog.xml

Regards,

Craig Jones

Integrated Marine observing System

On 23/09/14 03:33, Kathy Koch wrote:

I've looked through past messages on the list and while I see similar questions, did not see any replies with solutions so am hoping that someone out there has run into this and/or can help me find an answer.

I am attempting to set up a harvest to a Thredds catalog. I am

making sure I put in the .xml extension. I've tried a variety of

options based on the user guide and tried single datasets as well as

a folder and the full catalog. However, regardless of what I set in

the form I get the following error on harvest (*I took the full path

and filename out just so it's not published). I'm wondering if there

is some other setting I need to do at the server end or if it is

something wrong with our .xml? >

Error: Invalid catalog <filename*>.xml ----Catalog Validation version

1.0.01 **Fatal: InvCatalogFactory.readXML failed Exception=

java.io.FileNotFoundException <filename*>.xml fatalMessages=

errMessages= warnMessages=

Class: BadXmlResponseEx

Stack:

at: org.fao.geonet.kernel.harvest.harvester.thredds.Harvester file:

Harvester.java line: 318 method: harvestCatalog

at: org.fao.geonet.kernel.harvest.harvester.thredds.Harvester file:

Harvester.java line: 251 method: harvest

at: org.fao.geonet.kernel.harvest.harvester.thredds.ThreddsHarvester

file: ThreddsHarvester.java line: 245 method: doHarvest

at:

org.fao.geonet.kernel.harvest.harvester.AbstractHarvester$HarvestWith

IndexProcessor file: AbstractHarvester.java line: 399 method: process

at: org.fao.geonet.kernel.harvest.harvester.AbstractHarvester file:

AbstractHarvester.java line: 429 method: harvest

at: org.fao.geonet.kernel.harvest.harvester.HarvesterJob file:

HarvesterJob.java line: 29 method: execute

at: org.quartz.core.JobRunShell file: JobRunShell.java line: 213

method: run

at: org.quartz.simpl.SimpleThreadPool$WorkerThread file:

SimpleThreadPool.java line: 557 method: run

   TIA!

Kathy K.

---------------------------------------------------------------------

--------- Meet PCI DSS 3.0 Compliance Requirements with EventLog

Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI

DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download

White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with

EventLog Analyzer

http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg

.clktrk _______________________________________________

GeoNetwork-users mailing list

GeoNetwork-users@lists.sourceforge.net<mailto:GeoNetwork-users@anonymised.comrceforge.net>

https://lists.sourceforge.net/lists/listinfo/geonetwork-users

GeoNetwork OpenSource is maintained at

http://sourceforge.net/projects/geonetwork

----------------------------------------------------------------------

-------- Meet PCI DSS 3.0 Compliance Requirements with EventLog

Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI

DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download

White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with

EventLog Analyzer

http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.

clktrk _______________________________________________

GeoNetwork-users mailing list

GeoNetwork-users@lists.sourceforge.net<mailto:GeoNetwork-users@anonymised.comceforge.net>

https://lists.sourceforge.net/lists/listinfo/geonetwork-users

GeoNetwork OpenSource is maintained at

http://sourceforge.net/projects/geonetwork

Okay. Looks like it's a 2.8 issue. I brought up 2.10 and the harvest worked just fine. Guess I'm off to do a full migration and make 2.10 live. Thanks for your help! (although if you had an easy fix to make it work on 2.8 that would be nice, not necessary to investigate much further)

Kathy
_____________________________________
Kathy Koch
Data Curator
Contractor (LimnoTech / GLOS)
501 Avis Drive, Ann Arbor, MI 48108
Office: 734-332-1200 email:kkoch@anonymised.com<mailto:kkoch@anonymised.com>
LimnoTech www.limno.com<https://mail.limno.com/owa/UrlBlockedError.aspx&gt;
<../../../../tdekker/AppData/Local/Microsoft/Windows/Temporary%20Internet%20Files/Content.Outlook/8B0OUOWO/www.limno.com>
________________________________
From: Kathy Koch
Sent: Wednesday, September 24, 2014 10:47 AM
To: Craig Jones
Cc: geonetwork-users@lists.sourceforge.net
Subject: RE: [GeoNetwork-users] Thredds harvest error

I was able to do a wget on the server, listed out (cat) the xml just fine, but still doesn't work as a harvest from GN. Same file not found error - settings exactly like you have. Since you had success wondering if there is an issue at the geoserver level? I am using 2.8 right now (have to start testing 2.10 and see if that makes a diff I guess).

Kathy

-----Original Message-----
From: Craig Jones [mailto:Craig.Jones@anonymised.com]
Sent: Tuesday, September 23, 2014 6:23 PM
To: Kathy Koch
Subject: Re: [GeoNetwork-users] Thredds harvest error

Hi Kathy,

I'm able to harvest directly from the first catalog url below using a default GeoNetwork 2.10.3 install (I've included a screen dump of the settings I used). The second didn't return any metadata and I think this is because the harvester can't handle the direct reference to the catalog entry for a specific dataset.

Have a look at the settings I used for the harvester, perhaps you can spot the difference? Or perhaps you are using an older version of GeoNetwork? If so let me know the version and I can check the result there, but as far as I'm aware this should work in 2.8 as well.

Also, it would be good to keep this on the mailing list if possible for future reference by others.

Regards,

Craig Jones

PS. When I said check to make sure you can access the URL on the server I meant check that you can access the URL from the server where GeoNetwork is installed. Perhaps there is a proxy or network issue preventing this server from accessing your thredds server.

On 24/09/14 04:04, Kathy Koch wrote:

Thanks for the reply. Here are the links I've been testing and get the error every time.

What else am I missing?

http://tds.glos.us/thredds/mtri/aoc.xml

http://tds.glos.us/thredds/mtri/aoc.xml?dataset=LakeErieSST-Agg

(Eventually I will want to harvest the entire catalog at

http://tds.glos.us/thredds/catalog.xml but am trying one of the above

two as a test for right now. I did try the entire catalog as a last

resort but got the same error.)

Kathy

_____________________________________

Kathy Koch

Data Curator

Contractor (LimnoTech / GLOS)

501 Avis Drive, Ann Arbor, MI 48108

Office: 734-332-1200 email:kkoch@anonymised.com

LimnoTech www.limno.com<http://www.limno.com>

________________________________________

From: Craig Jones [Craig.Jones@anonymised.com]

Sent: Monday, September 22, 2014 7:30 PM

To: geonetwork-users@lists.sourceforge.net<mailto:geonetwork-users@anonymised.comsourceforge.net>

Subject: Re: [GeoNetwork-users] Thredds harvest error

Hi Kathy,

The error message below is returned because the harvester is unable to

find the catalog using the provided url (you are getting a

FileNotFoundException!).

Check the URL you are entering in the harvester returns an xml catalog

on the server (e.g. using wget <filename*>.xml on the server).

For example, this url to a thredds.ucar.edu catalog entry works for me:

http://thredds.ucar.edu/thredds/catalog/grib/FNMOC/WW3/Global_1p0deg/F

NMOC_WW3_Global_1p0deg_20140921_1200.grib1/catalog.xml

Regards,

Craig Jones

Integrated Marine observing System

On 23/09/14 03:33, Kathy Koch wrote:

I've looked through past messages on the list and while I see similar questions, did not see any replies with solutions so am hoping that someone out there has run into this and/or can help me find an answer.

I am attempting to set up a harvest to a Thredds catalog. I am

making sure I put in the .xml extension. I've tried a variety of

options based on the user guide and tried single datasets as well as

a folder and the full catalog. However, regardless of what I set in

the form I get the following error on harvest (*I took the full path

and filename out just so it's not published). I'm wondering if there

is some other setting I need to do at the server end or if it is

something wrong with our .xml? >

Error: Invalid catalog <filename*>.xml ----Catalog Validation version

1.0.01 **Fatal: InvCatalogFactory.readXML failed Exception=

java.io.FileNotFoundException <filename*>.xml fatalMessages=

errMessages= warnMessages=

Class: BadXmlResponseEx

Stack:

at: org.fao.geonet.kernel.harvest.harvester.thredds.Harvester file:

Harvester.java line: 318 method: harvestCatalog

at: org.fao.geonet.kernel.harvest.harvester.thredds.Harvester file:

Harvester.java line: 251 method: harvest

at: org.fao.geonet.kernel.harvest.harvester.thredds.ThreddsHarvester

file: ThreddsHarvester.java line: 245 method: doHarvest

at:

org.fao.geonet.kernel.harvest.harvester.AbstractHarvester$HarvestWith

IndexProcessor file: AbstractHarvester.java line: 399 method: process

at: org.fao.geonet.kernel.harvest.harvester.AbstractHarvester file:

AbstractHarvester.java line: 429 method: harvest

at: org.fao.geonet.kernel.harvest.harvester.HarvesterJob file:

HarvesterJob.java line: 29 method: execute

at: org.quartz.core.JobRunShell file: JobRunShell.java line: 213

method: run

at: org.quartz.simpl.SimpleThreadPool$WorkerThread file:

SimpleThreadPool.java line: 557 method: run

   TIA!

Kathy K.

---------------------------------------------------------------------

--------- Meet PCI DSS 3.0 Compliance Requirements with EventLog

Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI

DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download

White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with

EventLog Analyzer

http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg

.clktrk _______________________________________________

GeoNetwork-users mailing list

GeoNetwork-users@lists.sourceforge.net<mailto:GeoNetwork-users@anonymised.comrceforge.net>

https://lists.sourceforge.net/lists/listinfo/geonetwork-users

GeoNetwork OpenSource is maintained at

http://sourceforge.net/projects/geonetwork

----------------------------------------------------------------------

-------- Meet PCI DSS 3.0 Compliance Requirements with EventLog

Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI

DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download

White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with

EventLog Analyzer

http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.

clktrk _______________________________________________

GeoNetwork-users mailing list

GeoNetwork-users@lists.sourceforge.net<mailto:GeoNetwork-users@anonymised.comceforge.net>

https://lists.sourceforge.net/lists/listinfo/geonetwork-users

GeoNetwork OpenSource is maintained at

http://sourceforge.net/projects/geonetwork