Ahh - the xlink resolver. Checked, restarted, and my records are 'coming
together' nicer now.
However, The WFS harvester gives me endpoints with bonus content like a
protocol string relating to the endpoint - ie
'OGC:WFS-1.1.0-http-get-feature' and similar for WMS
OGC:WMS-1.1.1-http-get-map.
Is there a way to educate the Thredds harvester in doing so? Currently I get
OnLine resource:
Linkage: http://cgsrv3.arrc.csiro.au/thredds/wms/c3dmm/bhdemsub_latlong.nc
Protocol: WWW:LINK-1.0-http--downloaddata
It would be great to get the same protocol for WMS in both types of
harvested WMS's
Terry
-----Original Message-----
From: Pigot, Simon (CMAR, Hobart)
Sent: Monday, 13 June 2011 4:58 PM
To: Rankine, Terry (CESRE, Kensington);
geonetwork-users@lists.sourceforge.net
Cc: Craig.Jones@anonymised.com
Subject: RE: [GeoNetwork-users] Thredds harvester
Hi Terry,
Did you check the 'Create subtemplates for fragments and XLink them into
template' option? If so, then you need to enable the xlink resolver in
system configuration and either repeat the harvest or rebuild the index,
otherwise the created metadata records will not include the fragments of
harvested metadata.
Using the 'Create subtemplates....' option is the recommended path for
harvesting if you want to harvest fragments of metadata from the THREDDS
catalog in order to create metadata records that are a mix of harvested and
manually entered content.
Trunk compiles fine without your patch for me curiously!?
Cheers,
Simon
________________________________________
From: Rankine, Terry (CESRE, Kensington)
Sent: Monday, 13 June 2011 4:29 PM
To: Pigot, Simon (CMAR, Hobart); geonetwork-users@lists.sourceforge.net
Cc: Craig.Jones@anonymised.com
Subject: RE: [GeoNetwork-users] Thredds harvester
Hi Simon
I got the trunk compiling (with that patch sent to the devel list), but when
I setup the harvester, I don't get what I was expecting.
I still can't seem to get a record which says "BOM data, and transfer
options of DAP, WCS, WMS" or something like that.
Also all the 'names' of the harvested content are "HARVESTING TEMPLATE -
THREDDS - UNIDATA DISCOVERY" which makes it really hard to find anything.
You should have access to the
http://cgsrv3.arrc.csiro.au/thredds/catalog.xml catalogue, can you replicate
this using trunk? svn r7841.
Regards,
Terry Rankine
-----Original Message-----
From: Pigot, Simon (CMAR, Hobart)
Sent: Wednesday, 8 June 2011 9:56 PM
To: Pigot, Simon (CMAR, Hobart); Rankine, Terry (CESRE, Kensington);
geonetwork-users@lists.sourceforge.net
Cc: Craig.Jones@anonymised.com
Subject: RE: [GeoNetwork-users] Thredds harvester
Ignore the reference to the trunk 2.7 manual I've given below because it is
actually available on http://geonetwork-opensource.org. The THREDDS section
is at:
http://geonetwork-opensource.org/manuals/trunk/users/admin/harvesting/index.
html#adding-a-thredds-catalog
Cheers,
Simon
________________________________________
From: Pigot, Simon (CMAR, Hobart)
Sent: Wednesday, 8 June 2011 11:14 PM
To: Rankine, Terry (CESRE, Kensington);
geonetwork-users@lists.sourceforge.net
Cc: Craig.Jones@anonymised.com
Subject: RE: [GeoNetwork-users] Thredds harvester
Hi Terry,
Unfortunately the options I'm referring to are in 2.7 (and documented in the
2.7 manual) - not 2.6.3 - sorry about that.
However there is another option which will work with 2.6.x in the interim -
if you replace <install_dir>/web/geonetwork/WEB-INF/lib/netcdf-4.0.patch.jar
with http://geonetwork.globaldial.com/testdownloads/netcdf-4.0.patch.jar -
you will find additional services added to your metadata record that match
the THREDDS services that deliver them. (As a side issue, you will also find
that each THREDDS service actually appears twice - this appears to be due to
the DIFToISO converter).
Note that the 'Create default metadata' in 2.6.x is actually using the
enhanced DIFWriter package in the unidata netcdf-java library to write out
metadata in DIF format which is then converted to ISO. In 2.7 this option
has been relabeled as DIF (hence my earlier references to DIF).
The DIF path was the first approach we used for harvesting metadata from
THREDDS catalogs. My own thoughts are that in 2.7, the Unidata Data
Discovery option for metadata in THREDDS catalogs (see
http://www.unidata.ucar.edu/software/netcdf-java/formats/DataDiscoveryAttCon
vention.html) should be preferred over the DIF path as it is more direct
(the DIFToISO converter is not required) and the mapping from THREDDS to ISO
metadata is controlled by an XSLT in GeoNetwork (as opposed to making
changes to the DIFWriter class in the netcdf jar).
There is an early/draft/snapshot copy of the 2.7 manual (with all THREDDS
options for 2.7 fully documented) at
http://geonetwork.globaldial.com/testdownloads/GeoNetworkUserManual.pdf
Cheers,
Simon
________________________________________
From: Rankine, Terry (CESRE, Kensington)
Sent: Wednesday, 8 June 2011 4:04 PM
To: Pigot, Simon (CMAR, Hobart); geonetwork-users@lists.sourceforge.net
Cc: Craig.Jones@anonymised.com
Subject: RE: [GeoNetwork-users] Thredds harvester
Hi Simon
I have had a look at the harvester in 2.6.3 and cannot see any of the
options your mentioning in the email.
Is it possible to have a few screen shots or a quick skype with screen share
to see what you're saying?
Regards
Terry Rankine
-----Original Message-----
From: Pigot, Simon (CMAR, Hobart)
Sent: Saturday, 4 June 2011 8:18 PM
To: Rankine, Terry (CESRE, Kensington);
geonetwork-users@lists.sourceforge.net
Cc: Craig.Jones@anonymised.com
Subject: RE: [GeoNetwork-users] Thredds harvester
Hi Terry,
The short answer is if you use 'Extract Unidata dataset discovery metadata'
(instead of 'Extract DIF .....'), output schema iso19139, select the
'thredds-metadata' stylesheet for creating fragments and the 'HARVESTING
TEMPLATE - THREDDS - UNIDATA DISCOVERY' template in the harvester
definition, then you'll get what you want.
However, with regard to the DIF functionality, looking at the mappings from
THREDDS catalog datasets fields to DIF fields on
http://www.unidata.ucar.edu/projects/THREDDS/tech/catalog/DL.html, it seems
to me that all the service elements (not just the dods/opendap URK)
inherited by your datasets should be mapped to DIF 'Related_URL' fields and
thus should appear in your metadata records.
I'll do a bit more digging into the DIFWriter module in the netcdf java
lib......
Cheers,
Simon
________________________________________,
From: Terry.Rankine@anonymised.com [Terry.Rankine@anonymised.com]
Sent: Thursday, 2 June 2011 11:48 AM
To: geonetwork-users@lists.sourceforge.net
Subject: [GeoNetwork-users] Thredds harvester
Hi guys
The Thredds harvester doesn't operate like I thought it would.
Thredds treats URL's (and workflows) differently to Geoserver.
Geoserver's Model:
Service has many layers -> the one service endpoint serves multiple data
sets
Thredds Model
Data has many services -> the data set(collection/aggregation) has multiple
services (wcs,opendap,wms)
Is there a way of making the Thredds harvester behave like this?
I would like a record of my DATASET with all its associated services inside
it. Ie -
http://auscope-portal-2.arrc.csiro.au/geonetwork/srv/en/metadata.show?uuid=u
nidata.ucar.edu--ga-onshore_only_Bouguer_geodetic_reprojected_fixed.nc would
have an online resources for all the services which Thredds supported on
that dataset.
Terry
