[GeoNetwork-users] Harvesting your own data

I am looking for some advice on what people have done regarding the initial
load of their spatial data / metadata into GeoNetwork and to maintain it
when datasets are updated?

The reason I ask is that we have over 400 datasets from a variety of
sources that are currently being stored in PostGIS and manipulated in a
range of software including QGIS. I want to try and find a way to quickly
upload the metadata into GeoNetwork and harvest and updates on a regular
basis from our own databases to reflect the current status of the data.

We are not currently set on the technology solution, although we would like
to try and stick with an open source solution, so want to try and find out
what others have done and the processes they have used to keep the metadata
current.

Thanks

Dean Howell
deanhowell2009@anonymised.com

Hi, An option would be to use Talend Spatial with this set of jobs
https://github.com/talend-spatial/workspace-metadata-crawler
which are scanning folders, databases or webservices, generates metadata
records and populates a catalog using CSW.

Cheers.

Francois

2015-05-18 7:16 GMT+02:00 Dean Howell <deanhowell2009@anonymised.com>:

I am looking for some advice on what people have done regarding the initial
load of their spatial data / metadata into GeoNetwork and to maintain it
when datasets are updated?

The reason I ask is that we have over 400 datasets from a variety of
sources that are currently being stored in PostGIS and manipulated in a
range of software including QGIS. I want to try and find a way to quickly
upload the metadata into GeoNetwork and harvest and updates on a regular
basis from our own databases to reflect the current status of the data.

We are not currently set on the technology solution, although we would like
to try and stick with an open source solution, so want to try and find out
what others have done and the processes they have used to keep the metadata
current.

Thanks

Dean Howell
deanhowell2009@anonymised.com

------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at
http://sourceforge.net/projects/geonetwork

Thanks I will give that a go :slight_smile:

On Mon, May 18, 2015 at 4:13 PM Francois Prunayre <fx.prunayre@anonymised.com>
wrote:

Hi, An option would be to use Talend Spatial with this set of jobs
https://github.com/talend-spatial/workspace-metadata-crawler
which are scanning folders, databases or webservices, generates metadata
records and populates a catalog using CSW.

Cheers.

Francois

2015-05-18 7:16 GMT+02:00 Dean Howell <deanhowell2009@anonymised.com>:

I am looking for some advice on what people have done regarding the
initial
load of their spatial data / metadata into GeoNetwork and to maintain it
when datasets are updated?

The reason I ask is that we have over 400 datasets from a variety of
sources that are currently being stored in PostGIS and manipulated in a
range of software including QGIS. I want to try and find a way to quickly
upload the metadata into GeoNetwork and harvest and updates on a regular
basis from our own databases to reflect the current status of the data.

We are not currently set on the technology solution, although we would
like
to try and stick with an open source solution, so want to try and find out
what others have done and the processes they have used to keep the
metadata
current.

Thanks

Dean Howell
deanhowell2009@anonymised.com

------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at
http://sourceforge.net/projects/geonetwork

Hello again Francois, this looks like a great tool and perfect fit to what
I am trying to achieve. I have just one question, do we need the main
Talend software installed first or will the metadata crawler work
independently of it.

Thanks

Dean

On Mon, May 18, 2015 at 4:38 PM Dean Howell <deanhowell2009@anonymised.com>
wrote:

Thanks I will give that a go :slight_smile:

On Mon, May 18, 2015 at 4:13 PM Francois Prunayre <fx.prunayre@anonymised.com>
wrote:

Hi, An option would be to use Talend Spatial with this set of jobs
https://github.com/talend-spatial/workspace-metadata-crawler
which are scanning folders, databases or webservices, generates metadata
records and populates a catalog using CSW.

Cheers.

Francois

2015-05-18 7:16 GMT+02:00 Dean Howell <deanhowell2009@anonymised.com>:

I am looking for some advice on what people have done regarding the
initial
load of their spatial data / metadata into GeoNetwork and to maintain it
when datasets are updated?

The reason I ask is that we have over 400 datasets from a variety of
sources that are currently being stored in PostGIS and manipulated in a
range of software including QGIS. I want to try and find a way to quickly
upload the metadata into GeoNetwork and harvest and updates on a regular
basis from our own databases to reflect the current status of the data.

We are not currently set on the technology solution, although we would
like
to try and stick with an open source solution, so want to try and find
out
what others have done and the processes they have used to keep the
metadata
current.

Thanks

Dean Howell
deanhowell2009@anonymised.com

------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at
http://sourceforge.net/projects/geonetwork

2015-05-18 9:52 GMT+02:00 Dean Howell <deanhowell2009@anonymised.com>:

Hello again Francois, this looks like a great tool and perfect fit to what
I am trying to achieve. I have just one question, do we need the main
Talend software installed first or will the metadata crawler work
independently of it.

You should have a look to the doc
https://docs.google.com/document/d/1Z-lJDGMX1tO3377Ytgoos0nGxqqN2rF-GyqiOB4iAVY/edit

Usually, you first test it from Talend Studio and then you can export the
job as an autonomous Java app or webservice and run it without the Studio.

Cheers.

Francois

Thanks

Dean

On Mon, May 18, 2015 at 4:38 PM Dean Howell <deanhowell2009@anonymised.com>
wrote:

Thanks I will give that a go :slight_smile:

On Mon, May 18, 2015 at 4:13 PM Francois Prunayre <fx.prunayre@anonymised.com>
wrote:

Hi, An option would be to use Talend Spatial with this set of jobs
https://github.com/talend-spatial/workspace-metadata-crawler
which are scanning folders, databases or webservices, generates metadata
records and populates a catalog using CSW.

Cheers.

Francois

2015-05-18 7:16 GMT+02:00 Dean Howell <deanhowell2009@anonymised.com>:

I am looking for some advice on what people have done regarding the
initial
load of their spatial data / metadata into GeoNetwork and to maintain it
when datasets are updated?

The reason I ask is that we have over 400 datasets from a variety of
sources that are currently being stored in PostGIS and manipulated in a
range of software including QGIS. I want to try and find a way to
quickly
upload the metadata into GeoNetwork and harvest and updates on a regular
basis from our own databases to reflect the current status of the data.

We are not currently set on the technology solution, although we would
like
to try and stick with an open source solution, so want to try and find
out
what others have done and the processes they have used to keep the
metadata
current.

Thanks

Dean Howell
deanhowell2009@anonymised.com

------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at
http://sourceforge.net/projects/geonetwork

Thanks Francois, terrific :slight_smile: thanks for all your help. It is very much
appreciated - Merci beaucoup :slight_smile:

On Mon, May 18, 2015 at 5:54 PM Francois Prunayre <fx.prunayre@anonymised.com>
wrote:

2015-05-18 9:52 GMT+02:00 Dean Howell <deanhowell2009@anonymised.com>:

Hello again Francois, this looks like a great tool and perfect fit to
what I am trying to achieve. I have just one question, do we need the main
Talend software installed first or will the metadata crawler work
independently of it.

You should have a look to the doc

https://docs.google.com/document/d/1Z-lJDGMX1tO3377Ytgoos0nGxqqN2rF-GyqiOB4iAVY/edit

Usually, you first test it from Talend Studio and then you can export the
job as an autonomous Java app or webservice and run it without the Studio.

Cheers.

Francois

Thanks

Dean

On Mon, May 18, 2015 at 4:38 PM Dean Howell <deanhowell2009@anonymised.com>
wrote:

Thanks I will give that a go :slight_smile:

On Mon, May 18, 2015 at 4:13 PM Francois Prunayre <fx.prunayre@anonymised.com>
wrote:

Hi, An option would be to use Talend Spatial with this set of jobs
https://github.com/talend-spatial/workspace-metadata-crawler
which are scanning folders, databases or webservices, generates
metadata records and populates a catalog using CSW.

Cheers.

Francois

2015-05-18 7:16 GMT+02:00 Dean Howell <deanhowell2009@anonymised.com>:

I am looking for some advice on what people have done regarding the
initial
load of their spatial data / metadata into GeoNetwork and to maintain
it
when datasets are updated?

The reason I ask is that we have over 400 datasets from a variety of
sources that are currently being stored in PostGIS and manipulated in a
range of software including QGIS. I want to try and find a way to
quickly
upload the metadata into GeoNetwork and harvest and updates on a
regular
basis from our own databases to reflect the current status of the data.

We are not currently set on the technology solution, although we would
like
to try and stick with an open source solution, so want to try and find
out
what others have done and the processes they have used to keep the
metadata
current.

Thanks

Dean Howell
deanhowell2009@anonymised.com

------------------------------------------------------------------------------
One dashboard for servers and applications across
Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable
Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at
http://sourceforge.net/projects/geonetwork