[GeoNetwork-devel] Drafts for published metadata

GeoNetwork GeoNetwork Developer
1

Hi,

We are working on a proposal to have a basic versioning on published
metadata. We are working over some design we made two years ago in
Bolsena with Jesse's colaboration, but as we know that not everyone
participated, we thought it might be useful to share it here.

The idea is simple:

When a published metadata is edited, a new copy is saved on both the
database and the index marked with the draft flag. This way a user
with enough privileges can view, search and edit published metadata
without modifying what the rest of the users see about this metadata.
This is specially useful for long editings (that last more than one
session) or edits that need some kind of review from several users
before getting published.

This draft is blocked while a user is editing it to prevent other
concurrent editing with other users. This block is released either
when the editor is closed or after a timeout of inactivity.

Once the draft copy is published, the original published metadata gets
replaced with the draft copy and the draft is removed from the system.

We will probably extend the blocking system to the normal metadata
editing, as right now it can be very confusing if more than one user
edit the same metadata at the same time.

Drafts will be saved on a different table on the database, so we can
keep a unique identifier by uuid on the metadata table. The lucene
index will be the same, just adding the "draft" flag.

All this draft thing will be enabled by configuration, so you can keep
the current implementation if you want. Unless there is consense that
it should be defaulted because it doesn't make sense what we have now.
Should we make the locking over editing also configurable?

So this is the time for brainstorm. Any ideas? Any parallel
implementation? Paul remembers Simon saying something about using svn
for this, but I am not sure what was the idea and he didn't remember
it completely, so if you remember something about it, this is the time
and place.

Kind regards,

María Arias de Reyna Domínguez

Veenderweg 13
6721 WD Bennekom
The Netherlands
T: +31 (0)318 416664

Please consider the environment before printing this email.

2

Hi Maria,

···

2015-12-01 12:44 GMT+01:00 Maria Arias de Reyna <maria.arias@anonymised.com>:

Hi,

We are working on a proposal to have a basic versioning on published
metadata. We are working over some design we made two years ago in
Bolsena with Jesse’s colaboration, but as we know that not everyone
participated, we thought it might be useful to share it here.

The idea is simple:

When a published metadata is edited, a new copy is saved on both the
database and the index marked with the draft flag. This way a user
with enough privileges can view, search and edit published metadata
without modifying what the rest of the users see about this metadata.
This is specially useful for long editings (that last more than one
session) or edits that need some kind of review from several users
before getting published.

This draft is blocked while a user is editing it to prevent other
concurrent editing with other users. This block is released either
when the editor is closed or after a timeout of inactivity.

Once the draft copy is published, the original published metadata gets
replaced with the draft copy and the draft is removed from the system.

We will probably extend the blocking system to the normal metadata
editing, as right now it can be very confusing if more than one user
edit the same metadata at the same time.

Drafts will be saved on a different table on the database, so we can
keep a unique identifier by uuid on the metadata table. The lucene
index will be the same, just adding the “draft” flag.

All this draft thing will be enabled by configuration, so you can keep
the current implementation if you want.

+1

Unless there is consense that
it should be defaulted because it doesn’t make sense what we have now.
Should we make the locking over editing also configurable?

Maybe not, because currently users get an exception in that case so the lock will avoid that case.

So this is the time for brainstorm. Any ideas? Any parallel
implementation?

Not on my side.

Cheers.

Francois

Paul remembers Simon saying something about using svn
for this, but I am not sure what was the idea and he didn’t remember
it completely, so if you remember something about it, this is the time
and place.

Kind regards,

María Arias de Reyna Domínguez

Veenderweg 13
6721 WD Bennekom
The Netherlands
T: +31 (0)318 416664

Please consider the environment before printing this email.

3

+1

Will this get into conflicts with metadata status management ?

···

On Tue, Dec 1, 2015 at 1:34 PM, Francois Prunayre <fx.prunayre@anonymised.com> wrote:

Hi Maria,

Go from Idea to Many App Stores Faster with Intel(R) XDK
Give your users amazing mobile app experiences with Intel(R) XDK.
Use one codebase in this all-in-one HTML5 development environment.
Design, debug & build mobile apps & 2D/3D high-impact games for multiple OSs.
http://pubads.g.doubleclick.net/gampad/clk?id=254741911&iu=/4140

GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

–

2015-12-01 12:44 GMT+01:00 Maria Arias de Reyna <maria.arias@anonymised.com>:

Hi,

We are working on a proposal to have a basic versioning on published
metadata. We are working over some design we made two years ago in
Bolsena with Jesse’s colaboration, but as we know that not everyone
participated, we thought it might be useful to share it here.

The idea is simple:

When a published metadata is edited, a new copy is saved on both the
database and the index marked with the draft flag. This way a user
with enough privileges can view, search and edit published metadata
without modifying what the rest of the users see about this metadata.
This is specially useful for long editings (that last more than one
session) or edits that need some kind of review from several users
before getting published.

This draft is blocked while a user is editing it to prevent other
concurrent editing with other users. This block is released either
when the editor is closed or after a timeout of inactivity.

Once the draft copy is published, the original published metadata gets
replaced with the draft copy and the draft is removed from the system.

We will probably extend the blocking system to the normal metadata
editing, as right now it can be very confusing if more than one user
edit the same metadata at the same time.

Drafts will be saved on a different table on the database, so we can
keep a unique identifier by uuid on the metadata table. The lucene
index will be the same, just adding the “draft” flag.

All this draft thing will be enabled by configuration, so you can keep
the current implementation if you want.

+1

Unless there is consense that
it should be defaulted because it doesn’t make sense what we have now.
Should we make the locking over editing also configurable?

Maybe not, because currently users get an exception in that case so the lock will avoid that case.

So this is the time for brainstorm. Any ideas? Any parallel
implementation?

Not on my side.

Cheers.

Francois

Paul remembers Simon saying something about using svn
for this, but I am not sure what was the idea and he didn’t remember
it completely, so if you remember something about it, this is the time
and place.

Kind regards,

María Arias de Reyna Domínguez

Veenderweg 13
6721 WD Bennekom
The Netherlands
T: +31 (0)318 416664

Please consider the environment before printing this email.

camptocamp
INNOVATIVE SOLUTIONS
BY OPEN SOURCE EXPERTS

Florent Gravin
0479444492

4

Hi,

The idea is that it doesn't conflict. The draft metadata will have a
status draft and the published as status published. We can either
ignore the status of the draft or just add it so we know that there is
simultaneously a drafted and a published metadata with the same uuid.

On Tue, Dec 1, 2015 at 3:12 PM, Florent Gravin
<florent.gravin@anonymised.com> wrote:

+1

Will this get into conflicts with metadata status management ?

On Tue, Dec 1, 2015 at 1:34 PM, Francois Prunayre <fx.prunayre@anonymised.com>
wrote:

Hi Maria,

2015-12-01 12:44 GMT+01:00 Maria Arias de Reyna <maria.arias@anonymised.com>:

Hi,

We are working on a proposal to have a basic versioning on published
metadata. We are working over some design we made two years ago in
Bolsena with Jesse's colaboration, but as we know that not everyone
participated, we thought it might be useful to share it here.

The idea is simple:

When a published metadata is edited, a new copy is saved on both the
database and the index marked with the draft flag. This way a user
with enough privileges can view, search and edit published metadata
without modifying what the rest of the users see about this metadata.
This is specially useful for long editings (that last more than one
session) or edits that need some kind of review from several users
before getting published.

This draft is blocked while a user is editing it to prevent other
concurrent editing with other users. This block is released either
when the editor is closed or after a timeout of inactivity.

Once the draft copy is published, the original published metadata gets
replaced with the draft copy and the draft is removed from the system.

We will probably extend the blocking system to the normal metadata
editing, as right now it can be very confusing if more than one user
edit the same metadata at the same time.

Drafts will be saved on a different table on the database, so we can
keep a unique identifier by uuid on the metadata table. The lucene
index will be the same, just adding the "draft" flag.

All this draft thing will be enabled by configuration, so you can keep
the current implementation if you want.

+1

Unless there is consense that
it should be defaulted because it doesn't make sense what we have now.
Should we make the locking over editing also configurable?

Maybe not, because currently users get an exception in that case so the
lock will avoid that case.

So this is the time for brainstorm. Any ideas? Any parallel
implementation?

Not on my side.

Cheers.

Francois

Paul remembers Simon saying something about using svn
for this, but I am not sure what was the idea and he didn't remember
it completely, so if you remember something about it, this is the time
and place.

Kind regards,

María Arias de Reyna Domínguez

Veenderweg 13
6721 WD Bennekom
The Netherlands
T: +31 (0)318 416664

Please consider the environment before printing this email.

------------------------------------------------------------------------------
Go from Idea to Many App Stores Faster with Intel(R) XDK
Give your users amazing mobile app experiences with Intel(R) XDK.
Use one codebase in this all-in-one HTML5 development environment.
Design, debug & build mobile apps & 2D/3D high-impact games for multiple
OSs.
http://pubads.g.doubleclick.net/gampad/clk?id=254741911&iu=/4140
_______________________________________________
GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at
http://sourceforge.net/projects/geonetwork

--
camptocamp
INNOVATIVE SOLUTIONS
BY OPEN SOURCE EXPERTS

Florent Gravin
0479444492

------------------------------------------------------------------------------
Go from Idea to Many App Stores Faster with Intel(R) XDK
Give your users amazing mobile app experiences with Intel(R) XDK.
Use one codebase in this all-in-one HTML5 development environment.
Design, debug & build mobile apps & 2D/3D high-impact games for multiple
OSs.
http://pubads.g.doubleclick.net/gampad/clk?id=254741911&iu=/4140
_______________________________________________
GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at
http://sourceforge.net/projects/geonetwork

5

Hi,

This is what we have right now:
https://docs.google.com/document/d/1fp1qnwlNOadOynonb3Y6BsMUb-jsYoPzfpLYKHaEzjg/edit?usp=sharing

It's not very detailed, but works for a proposal. We have omitted the
svn repository part because we will not have funding enough for it,
but we will add events so it will be easy to develop a repository
addon based on the changes on the metadata.

On Tue, Dec 1, 2015 at 12:44 PM, Maria Arias de Reyna
<maria.arias@anonymised.com> wrote:

Hi,

We are working on a proposal to have a basic versioning on published
metadata. We are working over some design we made two years ago in
Bolsena with Jesse's colaboration, but as we know that not everyone
participated, we thought it might be useful to share it here.

The idea is simple:

When a published metadata is edited, a new copy is saved on both the
database and the index marked with the draft flag. This way a user
with enough privileges can view, search and edit published metadata
without modifying what the rest of the users see about this metadata.
This is specially useful for long editings (that last more than one
session) or edits that need some kind of review from several users
before getting published.

This draft is blocked while a user is editing it to prevent other
concurrent editing with other users. This block is released either
when the editor is closed or after a timeout of inactivity.

Once the draft copy is published, the original published metadata gets
replaced with the draft copy and the draft is removed from the system.

We will probably extend the blocking system to the normal metadata
editing, as right now it can be very confusing if more than one user
edit the same metadata at the same time.

Drafts will be saved on a different table on the database, so we can
keep a unique identifier by uuid on the metadata table. The lucene
index will be the same, just adding the "draft" flag.

All this draft thing will be enabled by configuration, so you can keep
the current implementation if you want. Unless there is consense that
it should be defaulted because it doesn't make sense what we have now.
Should we make the locking over editing also configurable?

So this is the time for brainstorm. Any ideas? Any parallel
implementation? Paul remembers Simon saying something about using svn
for this, but I am not sure what was the idea and he didn't remember
it completely, so if you remember something about it, this is the time
and place.

Kind regards,

María Arias de Reyna Domínguez

Veenderweg 13
6721 WD Bennekom
The Netherlands
T: +31 (0)318 416664

Please consider the environment before printing this email.

--
Kind regards,

María Arias de Reyna Domínguez

Veenderweg 13
6721 WD Bennekom
The Netherlands
T: +31 (0)318 416664

Please consider the environment before printing this email.

6

Hi,

As suggested yesterday, I moved the developments to the develop branch:
https://github.com/geonetwork/core-geonetwork/tree/feature_workspace

Feel free to test it.

There are two new implementations:

  • Lock metadata editing: There is a new setting on the configuration that defines the timeout (minutes) of the lock (so it won’t stay forever if the editor forgets to close the session). By default this is disabled (set as -1, which means: the lock will always be timed out). The lock is released when the edit session is cancelled or closed too.

If you are using an already deployed version, don’t forget to add the new setting to the database:
INSERT INTO Settings (name, value, datatype, position, internal)
VALUES (‘metadata/lock’, ‘-1’, 1, 100003, ‘n’);

  • Draft metadata: When you edit a published metadata, instead of modify the published version, a ghost/draft metadata will be created where you can edit whatever you want for as long as you need. Once you want to publish the changes, just publish the drafted version and it will replace the already published metadata with your edited one.

I prefer not to disclose the exact steps to do things yet because if you are going to test, I want you to do blind tests. The steps I always do are fine, I want to know if other order or other steps cause failures.

Let me know if you find unexpected things.

Once I make sure it has no evident bugs, I will fix the tests. Last time I checked, it failed. So don’t worry about that.

Regards,
María.