[GeoNetwork-devel] Proposal for vote: Metadata Notifications to Remote Targets

Heikki_Doeleman · October 16, 2010, 4:02pm

hello developers, and PSC members,

Jose and me have put up a proposal for GN2.7.x about notifications to remote destinations.

See http://trac.osgeo.org/geonetwork/wiki/MetadataNotifications.

Please vote !

Kind regards
Hikki Doeleman

Francois_Prunayre · October 17, 2010, 5:40pm

Hi Heikki, +1 from me and some questions

When you say "all the metadata created previously is notified to the
new destination" when we add a new node, do you send all the metadata
records in one call ? This could be huge if you have a catalogue with
thousands of records ?

This will expose all metadata records to the remote node ? Public and
private records ? No link to the groups & privileges ?

Do you keep an history of all changes (at least to be able to notify
after failure) ? If yes, we could also create a RSS or atom feed, so a
remote node could listen to changes and pull.

Any patch to have a look at ?

Ciao.

Francois

2010/10/16 heikki <tropicano@anonymised.com>:

hello developers, and PSC members,

Jose and me have put up a proposal for GN2.7.x about notifications to remote
destinations.

See http://trac.osgeo.org/geonetwork/wiki/MetadataNotifications.

Please vote !

Kind regards
Hikki Doeleman

------------------------------------------------------------------------------
Download new Adobe(R) Flash(R) Builder(TM) 4
The new Adobe(R) Flex(R) 4 and Flash(R) Builder(TM) 4 (formerly
Flex(R) Builder(TM)) enable the development of rich applications that run
across multiple browsers and platforms. Download your free trials today!
http://p.sf.net/sfu/adobe-dev2dev
_______________________________________________
GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at
http://sourceforge.net/projects/geonetwork

Jose_Garcia · October 18, 2010, 7:04am

Hi Francois

I respond below.

Regards,
Jose García

On Sun, Oct 17, 2010 at 7:40 PM, Francois Prunayre <fx.prunayre@anonymised.com> wrote:

Hi Heikki, +1 from me and some questions

When you say “all the metadata created previously is notified to the
new destination” when we add a new node, do you send all the metadata
records in one call ? This could be huge if you have a catalogue with
thousands of records ?

This process is done by a periodical thread, sending each metadata in an indiviudal call. Sure that for large catalogues will take quite time.

This will expose all metadata records to the remote node ? Public and
private records ? No link to the groups & privileges ?

For now nothing about permissions. For the project that has been initially implemented the synch database seem an internal one and no need to manage about permissions. But we can think about this to improve the proposal for trunk.

Do you keep an history of all changes (at least to be able to notify
after failure) ? If yes, we could also create a RSS or atom feed, so a
remote node could listen to changes and pull.

If a notitication fails, the request is stored in error state so later can be retried. I like the idea of publish as RSS or atom feed. We can discuss in next IRC meeting.

Any patch to have a look at ?

Not now, but we can provide it later this week i think.

Ciao.

Francois

2010/10/16 heikki <tropicano@anonymised.com…>:

hello developers, and PSC members,

Jose and me have put up a proposal for GN2.7.x about notifications to remote
destinations.

See http://trac.osgeo.org/geonetwork/wiki/MetadataNotifications.

Please vote !

Kind regards
Hikki Doeleman

Download new Adobe(R) Flash(R) Builder™ 4
The new Adobe(R) Flex(R) 4 and Flash(R) Builder™ 4 (formerly
Flex(R) Builder™) enable the development of rich applications that run
across multiple browsers and platforms. Download your free trials today!
http://p.sf.net/sfu/adobe-dev2dev

GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at
http://sourceforge.net/projects/geonetwork

Download new Adobe(R) Flash(R) Builder™ 4
The new Adobe(R) Flex(R) 4 and Flash(R) Builder™ 4 (formerly
Flex(R) Builder™) enable the development of rich applications that run
across multiple browsers and platforms. Download your free trials today!
http://p.sf.net/sfu/adobe-dev2dev

GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

sppigot · October 18, 2010, 10:56am

+1 from me - What about GeoNetwork itself being a destination for these notifications? I have a need for something generic like this because I'm working on auto configuring things like the list of Z3950 servers from a category of metadata service records and also automatically updating metadata schemas that are plugged into GeoNetwork when the record they are attached to changes. Presumably harvested records arriving into GeoNetwork will also trigger these notifications.

Cheers,
Simon

PS: Still have to catch up on some earlier votes a little later.
________________________________________
From: jose garcia [josegar74@anonymised.com]
Sent: Monday, 18 October 2010 6:04 PM
To: Francois Prunayre
Cc: Devel geonetwork-devel@lists.sourceforge.net
Subject: Re: [GeoNetwork-devel] Proposal for vote: Metadata Notifications to Remote Targets

Hi Francois

I respond below.

Regards,
Jose García

On Sun, Oct 17, 2010 at 7:40 PM, Francois Prunayre <fx.prunayre@anonymised.com<mailto:fx.prunayre@anonymised.com>> wrote:
Hi Heikki, +1 from me and some questions

When you say "all the metadata created previously is notified to the
new destination" when we add a new node, do you send all the metadata
records in one call ? This could be huge if you have a catalogue with
thousands of records ?

This process is done by a periodical thread, sending each metadata in an indiviudal call. Sure that for large catalogues will take quite time.

This will expose all metadata records to the remote node ? Public and
private records ? No link to the groups & privileges ?

For now nothing about permissions. For the project that has been initially implemented the synch database seem an internal one and no need to manage about permissions. But we can think about this to improve the proposal for trunk.

Do you keep an history of all changes (at least to be able to notify
after failure) ? If yes, we could also create a RSS or atom feed, so a
remote node could listen to changes and pull.

If a notitication fails, the request is stored in error state so later can be retried. I like the idea of publish as RSS or atom feed. We can discuss in next IRC meeting.

Any patch to have a look at ?

Not now, but we can provide it later this week i think.

Ciao.

Francois

2010/10/16 heikki <tropicano@anonymised.com<mailto:tropicano@anonymised.com>>:

hello developers, and PSC members,

Jose and me have put up a proposal for GN2.7.x about notifications to remote
destinations.

See MetadataNotifications – GeoNetwork opensource Developer website.

Please vote !

Kind regards
Hikki Doeleman

------------------------------------------------------------------------------
Download new Adobe(R) Flash(R) Builder(TM) 4
The new Adobe(R) Flex(R) 4 and Flash(R) Builder(TM) 4 (formerly
Flex(R) Builder(TM)) enable the development of rich applications that run
across multiple browsers and platforms. Download your free trials today!
http://p.sf.net/sfu/adobe-dev2dev
_______________________________________________
GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net<mailto:GeoNetwork-devel@anonymised.comceforge.net>
geonetwork-devel List Signup and Options
GeoNetwork OpenSource is maintained at
http://sourceforge.net/projects/geonetwork

------------------------------------------------------------------------------
Download new Adobe(R) Flash(R) Builder(TM) 4
The new Adobe(R) Flex(R) 4 and Flash(R) Builder(TM) 4 (formerly
Flex(R) Builder(TM)) enable the development of rich applications that run
across multiple browsers and platforms. Download your free trials today!

_______________________________________________
GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net<mailto:GeoNetwork-devel@anonymised.comforge.net>

GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

Doug_Nebert · October 18, 2010, 1:26pm

On 10/16/10 12:02 PM, heikki wrote:

hello developers, and PSC members,

Jose and me have put up a proposal for GN2.7.x about notifications to remote destinations.

-1. I do not see the value or appropriateness of a push notification service when an Atom or RSS feed of changes would serve the same purpose and is much more in-line with industry practice. In the geospatial world, a synchronization capability has been achieved on both catalogs and feature repositories that advertises all changes (add/update/delete) so that replicating or peer servers can be aware and adopt the change streams. I think this is a much more reasonable approach - using Atom to carry the ID, synopsis, change type, and date/time stamp of the changes. Coupled with OpenSearch, a remote catalog can query based on date-time since last harvest and retrieve only those records - a very polite way to synchronize content.

Doug.

--
Douglas D. Nebert
Senior Advisor for Geospatial Technology, System-of-Systems Architect
FGDC Secretariat T:703 648 4151 F:703 648-5755 C:703 459-5860

Monteduro_Patrizia_C · October 19, 2010, 8:56am

+1 from me.

Patrizia

-----Original Message-----
From: heikki [mailto:tropicano@anonymised.com]
Sent: 16 October 2010 18:03
To: Devel geonetwork-devel@lists.sourceforge.net
Subject: [GeoNetwork-devel] Proposal for vote: Metadata Notifications to Remote Targets

hello developers, and PSC members,

Jose and me have put up a proposal for GN2.7.x about notifications to remote destinations.

See http://trac.osgeo.org/geonetwork/wiki/MetadataNotifications.

Please vote !

Kind regards
Hikki Doeleman

ticheler · October 22, 2010, 5:16pm

Hi Jose and Heikki,
This is indeed a useful extra function to have.+1 from me.
Cheers,
Jeroen

On 16 okt 2010, at 18:02, heikki wrote:

hello developers, and PSC members,

Jose and me have put up a proposal for GN2.7.x about notifications to remote destinations.

See http://trac.osgeo.org/geonetwork/wiki/MetadataNotifications.

Please vote !

Kind regards
Hikki Doeleman

Download new Adobe(R) Flash(R) Builder™ 4
The new Adobe(R) Flex(R) 4 and Flash(R) Builder™ 4 (formerly
Flex(R) Builder™) enable the development of rich applications that run
across multiple browsers and platforms. Download your free trials today!
http://p.sf.net/sfu/adobe-dev2dev_______________________________________________
GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

Francois_Prunayre · October 27, 2010, 5:02pm

Hello Doug,

2010/10/18 Douglas Nebert <ddnebert@anonymised.com>:

On 10/16/10 12:02 PM, heikki wrote:

hello developers, and PSC members,

Jose and me have put up a proposal for GN2.7.x about notifications to
remote destinations.

-1.

This is the first veto on this list, so probably a good idea to have a
bit of discussion !
... even if I'm not involved in that proposal.

I do not see the value or appropriateness of a push notification
service

At least from my point of view, it's one requirement for the CSW ebRIM
module we have in a sandbox for quite some times now.
So it's one step further to have it as an optional module for main
GeoNetwork and I'm sure that some users would like to test and use it.

Heikki & Jose, will it be lot of work to add the ebrim module once
that mechanism will be there ?

when an Atom or RSS feed of changes would serve the same purpose
and is much more in-line with industry practice. In the geospatial
world, a synchronization capability has been achieved on both catalogs
and feature repositories that advertises all changes (add/update/delete)
so that replicating or peer servers can be aware and adopt the change
streams. I think this is a much more reasonable approach - using Atom to
carry the ID, synopsis, change type, and date/time stamp of the changes.
Coupled with OpenSearch, a remote catalog can query based on date-time
since last harvest and retrieve only those records - a very polite way
to synchronize content.

I'm also in favor of that approach, and it could be nice to have a
"all changes" feed available.

What we're missing now is:
* Atom output format (only RSS for now)
* History storage
* Add "all changes" services
* maybe some OpenSearch protocol improvements (for date criteria)

Do not hesitate to create a ticket for enhancement and to support
people to work on it !

Francois

Doug.

--
Douglas D. Nebert
Senior Advisor for Geospatial Technology, System-of-Systems Architect
FGDC Secretariat T:703 648 4151 F:703 648-5755 C:703 459-5860

------------------------------------------------------------------------------
Download new Adobe(R) Flash(R) Builder(TM) 4
The new Adobe(R) Flex(R) 4 and Flash(R) Builder(TM) 4 (formerly
Flex(R) Builder(TM)) enable the development of rich applications that run
across multiple browsers and platforms. Download your free trials today!
http://p.sf.net/sfu/adobe-dev2dev
_______________________________________________
GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

Jose_Garcia · October 27, 2010, 5:46pm

Hi

About this new functionality I commented Francois to discuss in IRC this week, but unfortunately i could not be available. We can discuss next week in IRC, hopefully Heikki and me will we available in the meeting.

I think the functionality interesting to have, maybe not as a push service (that’s the discussion ).

Regards,
Jose García

On Wed, Oct 27, 2010 at 7:02 PM, Francois Prunayre <fx.prunayre@anonymised.com> wrote:

Hello Doug,

2010/10/18 Douglas Nebert <ddnebert@anonymised.com>:

On 10/16/10 12:02 PM, heikki wrote:

hello developers, and PSC members,

Jose and me have put up a proposal for GN2.7.x about notifications to
remote destinations.

-1.

This is the first veto on this list, so probably a good idea to have a
bit of discussion !
… even if I’m not involved in that proposal.

I do not see the value or appropriateness of a push notification
service

At least from my point of view, it’s one requirement for the CSW ebRIM
module we have in a sandbox for quite some times now.
So it’s one step further to have it as an optional module for main
GeoNetwork and I’m sure that some users would like to test and use it.

Heikki & Jose, will it be lot of work to add the ebrim module once
that mechanism will be there ?

when an Atom or RSS feed of changes would serve the same purpose
and is much more in-line with industry practice. In the geospatial
world, a synchronization capability has been achieved on both catalogs
and feature repositories that advertises all changes (add/update/delete)
so that replicating or peer servers can be aware and adopt the change
streams. I think this is a much more reasonable approach - using Atom to
carry the ID, synopsis, change type, and date/time stamp of the changes.
Coupled with OpenSearch, a remote catalog can query based on date-time
since last harvest and retrieve only those records - a very polite way
to synchronize content.

I’m also in favor of that approach, and it could be nice to have a
“all changes” feed available.

What we’re missing now is:

Atom output format (only RSS for now)

History storage

Add “all changes” services

maybe some OpenSearch protocol improvements (for date criteria)

Do not hesitate to create a ticket for enhancement and to support
people to work on it !

Francois

Doug.

–
Douglas D. Nebert
Senior Advisor for Geospatial Technology, System-of-Systems Architect
FGDC Secretariat T:703 648 4151 F:703 648-5755 C:703 459-5860

Download new Adobe(R) Flash(R) Builder™ 4
The new Adobe(R) Flex(R) 4 and Flash(R) Builder™ 4 (formerly
Flex(R) Builder™) enable the development of rich applications that run
across multiple browsers and platforms. Download your free trials today!
http://p.sf.net/sfu/adobe-dev2dev

GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

Nokia and AT&T present the 2010 Calling All Innovators-North America contest
Create new apps & games for the Nokia N8 for consumers in U.S. and Canada
$10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing
Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store
http://p.sf.net/sfu/nokia-dev2dev

GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

Emanuele_Tajariol · October 28, 2010, 12:19am

Hi Heikki,

it's late, but I see in the trac that this proposal is still in a "Proposed
for vote" status.

+1 from me.

Ciao,
Emanuele

Alle 18:02:52 di sabato 16 ottobre 2010, heikki ha scritto:

hello developers, and PSC members,

Jose and me have put up a proposal for GN2.7.x about notifications to
remote destinations.

See http://trac.osgeo.org/geonetwork/wiki/MetadataNotifications.

Please vote !

Kind regards
Hikki Doeleman