[GeoNetwork-devel] Harvesting question / issue / idea

Carlos_Figueiredo · May 2, 2018, 4:54pm

Hello all,

We are using the Geonetwork software package in a European project (h2020 initiative) in order to catalogue the project resources.

We are brainstorming some ideas to use the catalogue and its CSW protocol to make a sys2sys queries in order to use it on the project geoportal (using the GeoNetwork GUI as back office to maintain and manage metadata and harvesting procedures).

Our idea is to use some keywording to characterize and categorize de metadata records. This is of course ok for the metadata records we will create and manage under the project.

But for the harvesting metadata records we cannot do this as it will be overwritten the next harvesting cycle. It is understood that the metadata should (must) be maintained at the source and other should not change it.

Nevertheless, we would like to create just some keywords under one xml iso element on the harvested records. These will not change the metadata record meaning but it is just for our project better handle those records.

How do you see this as an option to the harvesting procedure ? i.e. the harvested metadata records would be updated and not deleted and inserted again, on all elements (if changes are available) thus maintaining the changes we made ourselves (the keywords on the element) ?

Have you came across with similar situations ?

Please let me know.

Thank you and best regards,

Carlos Figueiredo

Senior System Analyst
Defence & Security Systems

carlos.figueiredo@anonymised.com

Tel: +351.212945900

Mobile: +351. 937 673 523

Fax: +351.212945999

Rua Calvet Magalhães, 245

2770-153 Paço de Arcos · Portugal

www.edisoft.pt

……………………………………………………………………………

A informação contida neste e-mail e quaisquer documentos anexos são propriedade da EDISOFT e poderão ser confidenciais. Se não for o destinatário pretendido, por favor, comunique-nos de imediato, enviando-nos esta mensagem de volta e destruindo-a em seguida. Fica desde já advertido de que é estritamente proibida qualquer revisão, divulgação, distribuição, cópia ou qualquer outra utilização deste e-mail.

The information contained in this e-mail and any attachments are the property of EDISOFT and may be confidential. If you are not the intended recipient, please notify us immediately, send this message back to us and destroy it. You are hereby notified that any review, dissemination, distribution, copying or otherwise use of this e-mail is strictly prohibited

Jose_Garcia1 · May 3, 2018, 6:20am

Hi Carlos

In the CSW harvester there’s an option XSL transformation to apply that allows to process the harvested records, I think you can use that mechanism creating a new xslt process that adds to the harvested records the required information.

You need to create the new xslt file and add it to TOMCAT_DIR/webapps/geonetwork/xsl/conversion/import folder, then should be displayed in the CSW harvester in the XSL transformation to apply.

Regards,
Jose García

(attachments)

···

On Wed, May 2, 2018 at 6:54 PM, Carlos Figueiredo <carlos.figueiredo@anonymised.com> wrote:

Hello all,

We are using the Geonetwork software package in a European project (h2020 initiative) in order to catalogue the project resources.

We are brainstorming some ideas to use the catalogue and its CSW protocol to make a sys2sys queries in order to use it on the project geoportal (using the GeoNetwork GUI as back office to maintain and manage metadata and harvesting procedures).

Our idea is to use some keywording to characterize and categorize de metadata records. This is of course ok for the metadata records we will create and manage under the project.

But for the harvesting metadata records we cannot do this as it will be overwritten the next harvesting cycle. It is understood that the metadata should (must) be maintained at the source and other should not change it.

Nevertheless, we would like to create just some keywords under one xml iso element on the harvested records. These will not change the metadata record meaning but it is just for our project better handle those records.

How do you see this as an option to the harvesting procedure ? i.e. the harvested metadata records would be updated and not deleted and inserted again, on all elements (if changes are available) thus maintaining the changes we made ourselves (the keywords on the element) ?

Have you came across with similar situations ?

Please let me know.

Thank you and best regards,

Carlos Figueiredo

Senior System Analyst
Defence & Security Systems

carlos.figueiredo@anonymised.com

Tel: +351.212945900

Mobile: +351. 937 673 523

Fax: +351.212945999

Rua Calvet Magalhães, 245

2770-153 Paço de Arcos · Portugal

www.edisoft.pt

……………………………………………………………………………

A informação contida neste e-mail e quaisquer documentos anexos são propriedade da EDISOFT e poderão ser confidenciais. Se não for o destinatário pretendido, por favor, comunique-nos de imediato, enviando-nos esta mensagem de volta e destruindo-a em seguida. Fica desde já advertido de que é estritamente proibida qualquer revisão, divulgação, distribuição, cópia ou qualquer outra utilização deste e-mail.

The information contained in this e-mail and any attachments are the property of EDISOFT and may be confidential. If you are not the intended recipient, please notify us immediately, send this message back to us and destroy it. You are hereby notified that any review, dissemination, distribution, copying or otherwise use of this e-mail is strictly prohibited

Check out the vibrant tech community on one of the world’s most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

GeoNetwork-devel mailing list
GeoNetwork-devel@…537…sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

–

Vriendelijke groeten / Kind regards,

Jose García

Veenderweg 13
6721 WD Bennekom
The Netherlands
T: +31 (0)318 416664

Please consider the environment before printing this email.

Carlos_Figueiredo · May 3, 2018, 8:23am

Hi José,

May thanks for the quick response on this subject.

I will try that approach.

Do you think I can start with the existent XSL files (the one used to do the regular CSW 2.0.2 and the one for GeoNetwork Node) ?

Can you please be so kind to name those files and location ?

Thank you and best regards,

Carlos Figueiredo

Senior System Analyst
Defence & Security Systems

carlos.figueiredo@…1177…

Tel: +351.212945900

Mobile: +351. 937 673 523

Fax: +351.212945999

Rua Calvet Magalhães, 245

2770-153 Paço de Arcos · Portugal

www.edisoft.pt

……………………………………………………………………………

A informação contida neste e-mail e quaisquer documentos anexos são propriedade da EDISOFT e poderão ser confidenciais. Se não for o destinatário pretendido, por favor, comunique-nos de imediato, enviando-nos esta mensagem de volta e destruindo-a em seguida. Fica desde já advertido de que é estritamente proibida qualquer revisão, divulgação, distribuição, cópia ou qualquer outra utilização deste e-mail.

The information contained in this e-mail and any attachments are the property of EDISOFT and may be confidential. If you are not the intended recipient, please notify us immediately, send this message back to us and destroy it. You are hereby notified that any review, dissemination, distribution, copying or otherwise use of this e-mail is strictly prohibited

From: Jose Garcia [mailto:jose.garcia@…437…]
Sent: quinta-feira, 3 de maio de 2018 07:21
To: Carlos Figueiredo
Cc: geonetwork-devel@lists.sourceforge.net
Subject: Re: [GeoNetwork-devel] Harvesting question / issue / idea

Hi Carlos

In the CSW harvester there’s an option XSL transformation to apply that allows to process the harvested records, I think you can use that mechanism creating a new xslt process that adds to the harvested records the required information.

You need to create the new xslt file and add it to TOMCAT_DIR/webapps/geonetwork/xsl/conversion/import folder, then should be displayed in the CSW harvester in the XSL transformation to apply.

Regards,

Jose García

On Wed, May 2, 2018 at 6:54 PM, Carlos Figueiredo <carlos.figueiredo@…1177…> wrote:

Hello all,

We are using the Geonetwork software package in a European project (h2020 initiative) in order to catalogue the project resources.

We are brainstorming some ideas to use the catalogue and its CSW protocol to make a sys2sys queries in order to use it on the project geoportal (using the GeoNetwork GUI as back office to maintain and manage metadata and harvesting procedures).

Our idea is to use some keywording to characterize and categorize de metadata records. This is of course ok for the metadata records we will create and manage under the project.

But for the harvesting metadata records we cannot do this as it will be overwritten the next harvesting cycle. It is understood that the metadata should (must) be maintained at the source and other should not change it.

Nevertheless, we would like to create just some keywords under one xml iso element on the harvested records. These will not change the metadata record meaning but it is just for our project better handle those records.

How do you see this as an option to the harvesting procedure ? i.e. the harvested metadata records would be updated and not deleted and inserted again, on all elements (if changes are available) thus maintaining the changes we made ourselves (the keywords on the element) ?

Have you came across with similar situations ?

Please let me know.

Thank you and best regards,

Carlos Figueiredo

Senior System Analyst
Defence & Security Systems

carlos.figueiredo@…1177…

Tel: +351.212945900

Mobile: +351. 937 673 523

Fax: +351.212945999

Rua Calvet Magalhães, 245

2770-153 Paço de Arcos · Portugal

www.edisoft.pt

……………………………………………………………………………

A informação contida neste e-mail e quaisquer documentos anexos são propriedade da EDISOFT e poderão ser confidenciais. Se não for o destinatário pretendido, por favor, comunique-nos de imediato, enviando-nos esta mensagem de volta e destruindo-a em seguida. Fica desde já advertido de que é estritamente proibida qualquer revisão, divulgação, distribuição, cópia ou qualquer outra utilização deste e-mail.

The information contained in this e-mail and any attachments are the property of EDISOFT and may be confidential. If you are not the intended recipient, please notify us immediately, send this message back to us and destroy it. You are hereby notified that any review, dissemination, distribution, copying or otherwise use of this e-mail is strictly prohibited

Check out the vibrant tech community on one of the world’s most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

–

Vriendelijke groeten / Kind regards,

Jose García

Veenderweg 13
6721 WD Bennekom
The Netherlands
T: +31 (0)318 416664

Please consider the environment before printing this email.

(attachments)

Maria_Arias_de_Reyn2 · May 3, 2018, 8:48am

Dear Carlos,

The current examples are here:

https://github.com/geonetwork/core-geonetwork/tree/develop/web/src/main/webapp/xsl/conversion/import

(attachments)

···

On Thu, May 3, 2018 at 10:23 AM, Carlos Figueiredo <carlos.figueiredo@anonymised.com> wrote:

Hi José,

May thanks for the quick response on this subject.

I will try that approach.

Do you think I can start with the existent XSL files (the one used to do the regular CSW 2.0.2 and the one for GeoNetwork Node) ?

Can you please be so kind to name those files and location ?

Thank you and best regards,

Carlos Figueiredo

Senior System Analyst
Defence & Security Systems

carlos.figueiredo@anonymised.com

Tel: +351.212945900

Mobile: +351. 937 673 523

Fax: +351.212945999

Rua Calvet Magalhães, 245

2770-153 Paço de Arcos · Portugal

www.edisoft.pt

……………………………………………………………………………

A informação contida neste e-mail e quaisquer documentos anexos são propriedade da EDISOFT e poderão ser confidenciais. Se não for o destinatário pretendido, por favor, comunique-nos de imediato, enviando-nos esta mensagem de volta e destruindo-a em seguida. Fica desde já advertido de que é estritamente proibida qualquer revisão, divulgação, distribuição, cópia ou qualquer outra utilização deste e-mail.

The information contained in this e-mail and any attachments are the property of EDISOFT and may be confidential. If you are not the intended recipient, please notify us immediately, send this message back to us and destroy it. You are hereby notified that any review, dissemination, distribution, copying or otherwise use of this e-mail is strictly prohibited

From: Jose Garcia [mailto:jose.garcia@anonymised.com]
Sent: quinta-feira, 3 de maio de 2018 07:21
To: Carlos Figueiredo
Cc: geonetwork-devel@anonymised.comsourceforge.net
Subject: Re: [GeoNetwork-devel] Harvesting question / issue / idea

Hi Carlos

In the CSW harvester there’s an option XSL transformation to apply that allows to process the harvested records, I think you can use that mechanism creating a new xslt process that adds to the harvested records the required information.

You need to create the new xslt file and add it to TOMCAT_DIR/webapps/geonetwork/xsl/conversion/import folder, then should be displayed in the CSW harvester in the XSL transformation to apply.

Regards,

Jose García

On Wed, May 2, 2018 at 6:54 PM, Carlos Figueiredo <carlos.figueiredo@anonymised.com> wrote:

Hello all,

We are using the Geonetwork software package in a European project (h2020 initiative) in order to catalogue the project resources.

We are brainstorming some ideas to use the catalogue and its CSW protocol to make a sys2sys queries in order to use it on the project geoportal (using the GeoNetwork GUI as back office to maintain and manage metadata and harvesting procedures).

Our idea is to use some keywording to characterize and categorize de metadata records. This is of course ok for the metadata records we will create and manage under the project.

But for the harvesting metadata records we cannot do this as it will be overwritten the next harvesting cycle. It is understood that the metadata should (must) be maintained at the source and other should not change it.

Nevertheless, we would like to create just some keywords under one xml iso element on the harvested records. These will not change the metadata record meaning but it is just for our project better handle those records.

How do you see this as an option to the harvesting procedure ? i.e. the harvested metadata records would be updated and not deleted and inserted again, on all elements (if changes are available) thus maintaining the changes we made ourselves (the keywords on the element) ?

Have you came across with similar situations ?

Please let me know.

Thank you and best regards,

Carlos Figueiredo

Senior System Analyst
Defence & Security Systems

carlos.figueiredo@anonymised.com

Tel: +351.212945900

Mobile: +351. 937 673 523

Fax: +351.212945999

Rua Calvet Magalhães, 245

2770-153 Paço de Arcos · Portugal

www.edisoft.pt

……………………………………………………………………………

A informação contida neste e-mail e quaisquer documentos anexos são propriedade da EDISOFT e poderão ser confidenciais. Se não for o destinatário pretendido, por favor, comunique-nos de imediato, enviando-nos esta mensagem de volta e destruindo-a em seguida. Fica desde já advertido de que é estritamente proibida qualquer revisão, divulgação, distribuição, cópia ou qualquer outra utilização deste e-mail.

The information contained in this e-mail and any attachments are the property of EDISOFT and may be confidential. If you are not the intended recipient, please notify us immediately, send this message back to us and destroy it. You are hereby notified that any review, dissemination, distribution, copying or otherwise use of this e-mail is strictly prohibited

Check out the vibrant tech community on one of the world’s most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

GeoNetwork-devel mailing list
GeoNetwork-devel@anonymised.comsourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

–

Vriendelijke groeten / Kind regards,

Jose García

Veenderweg 13
6721 WD Bennekom
The Netherlands
T: +31 (0)318 416664

Please consider the environment before printing this email.

Check out the vibrant tech community on one of the world’s most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

GeoNetwork-devel mailing list
GeoNetwork-devel@…537…sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

Carlos_Figueiredo · May 3, 2018, 9:10am

Hi Maria,

Thanks for your email.

Just a question, following José’s suggestion.

The “default” harvesting to a CSW or GeoNetwork node will just collected the metadata records and according to the update date overwrite or not the existing ones.

This procedure doesn’t use a XSL doesn’t it ?

Considering that I want to use the ISO profile, which are the xsl files I should use to achive that transformation I want, i.e. not to overwrite but to keep some elements of the existent metadata records ?

On the folder you mentioned there are 19 xsl files but which ones you suggest to achive the harvesting of CSW and Geonetwork ?

Thanks and BR,

Carlos Figueiredo

Senior System Analyst
Defence & Security Systems

carlos.figueiredo@…1177…

Tel: +351.212945900

Mobile: +351. 937 673 523

Fax: +351.212945999

Rua Calvet Magalhães, 245

2770-153 Paço de Arcos · Portugal

www.edisoft.pt

……………………………………………………………………………

A informação contida neste e-mail e quaisquer documentos anexos são propriedade da EDISOFT e poderão ser confidenciais. Se não for o destinatário pretendido, por favor, comunique-nos de imediato, enviando-nos esta mensagem de volta e destruindo-a em seguida. Fica desde já advertido de que é estritamente proibida qualquer revisão, divulgação, distribuição, cópia ou qualquer outra utilização deste e-mail.

The information contained in this e-mail and any attachments are the property of EDISOFT and may be confidential. If you are not the intended recipient, please notify us immediately, send this message back to us and destroy it. You are hereby notified that any review, dissemination, distribution, copying or otherwise use of this e-mail is strictly prohibited

From: María Arias de Reyna [mailto:delawen@…31…]
Sent: quinta-feira, 3 de maio de 2018 09:48
To: Carlos Figueiredo
Cc: Jose Garcia; geonetwork-devel@lists.sourceforge.net
Subject: Re: [GeoNetwork-devel] Harvesting question / issue / idea

Dear Carlos,

The current examples are here:

https://github.com/geonetwork/core-geonetwork/tree/develop/web/src/main/webapp/xsl/conversion/import

On Thu, May 3, 2018 at 10:23 AM, Carlos Figueiredo <carlos.figueiredo@…1177…> wrote:

Hi José,

May thanks for the quick response on this subject.

I will try that approach.

Do you think I can start with the existent XSL files (the one used to do the regular CSW 2.0.2 and the one for GeoNetwork Node) ?

Can you please be so kind to name those files and location ?

Thank you and best regards,

Carlos Figueiredo

Senior System Analyst
Defence & Security Systems

carlos.figueiredo@…1177…

Tel: +351.212945900

Mobile: +351. 937 673 523

Fax: +351.212945999

Rua Calvet Magalhães, 245

2770-153 Paço de Arcos · Portugal

www.edisoft.pt

……………………………………………………………………………

A informação contida neste e-mail e quaisquer documentos anexos são propriedade da EDISOFT e poderão ser confidenciais. Se não for o destinatário pretendido, por favor, comunique-nos de imediato, enviando-nos esta mensagem de volta e destruindo-a em seguida. Fica desde já advertido de que é estritamente proibida qualquer revisão, divulgação, distribuição, cópia ou qualquer outra utilização deste e-mail.

The information contained in this e-mail and any attachments are the property of EDISOFT and may be confidential. If you are not the intended recipient, please notify us immediately, send this message back to us and destroy it. You are hereby notified that any review, dissemination, distribution, copying or otherwise use of this e-mail is strictly prohibited

From: Jose Garcia [mailto:jose.garcia@…437…]
Sent: quinta-feira, 3 de maio de 2018 07:21
To: Carlos Figueiredo
Cc: geonetwork-devel@lists.sourceforge.net
Subject: Re: [GeoNetwork-devel] Harvesting question / issue / idea

Hi Carlos

In the CSW harvester there’s an option XSL transformation to apply that allows to process the harvested records, I think you can use that mechanism creating a new xslt process that adds to the harvested records the required information.

You need to create the new xslt file and add it to TOMCAT_DIR/webapps/geonetwork/xsl/conversion/import folder, then should be displayed in the CSW harvester in the XSL transformation to apply.

Regards,

Jose García

On Wed, May 2, 2018 at 6:54 PM, Carlos Figueiredo <carlos.figueiredo@…1177…> wrote:

Hello all,

We are using the Geonetwork software package in a European project (h2020 initiative) in order to catalogue the project resources.

We are brainstorming some ideas to use the catalogue and its CSW protocol to make a sys2sys queries in order to use it on the project geoportal (using the GeoNetwork GUI as back office to maintain and manage metadata and harvesting procedures).

Our idea is to use some keywording to characterize and categorize de metadata records. This is of course ok for the metadata records we will create and manage under the project.

But for the harvesting metadata records we cannot do this as it will be overwritten the next harvesting cycle. It is understood that the metadata should (must) be maintained at the source and other should not change it.

Nevertheless, we would like to create just some keywords under one xml iso element on the harvested records. These will not change the metadata record meaning but it is just for our project better handle those records.

How do you see this as an option to the harvesting procedure ? i.e. the harvested metadata records would be updated and not deleted and inserted again, on all elements (if changes are available) thus maintaining the changes we made ourselves (the keywords on the element) ?

Have you came across with similar situations ?

Please let me know.

Thank you and best regards,

Carlos Figueiredo

Senior System Analyst
Defence & Security Systems

carlos.figueiredo@…1177…

Tel: +351.212945900

Mobile: +351. 937 673 523

Fax: +351.212945999

Rua Calvet Magalhães, 245

2770-153 Paço de Arcos · Portugal

www.edisoft.pt

……………………………………………………………………………

A informação contida neste e-mail e quaisquer documentos anexos são propriedade da EDISOFT e poderão ser confidenciais. Se não for o destinatário pretendido, por favor, comunique-nos de imediato, enviando-nos esta mensagem de volta e destruindo-a em seguida. Fica desde já advertido de que é estritamente proibida qualquer revisão, divulgação, distribuição, cópia ou qualquer outra utilização deste e-mail.

The information contained in this e-mail and any attachments are the property of EDISOFT and may be confidential. If you are not the intended recipient, please notify us immediately, send this message back to us and destroy it. You are hereby notified that any review, dissemination, distribution, copying or otherwise use of this e-mail is strictly prohibited

Check out the vibrant tech community on one of the world’s most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

–

Vriendelijke groeten / Kind regards,

Jose García

Veenderweg 13
6721 WD Bennekom
The Netherlands
T: +31 (0)318 416664

Please consider the environment before printing this email.

Check out the vibrant tech community on one of the world’s most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

(attachments)

Jose_Garcia1 · May 3, 2018, 9:20am

Hi Carlos

To clarify about the xslt, GeoNetwork has some processes available that are displayed in the CSW harvester field XSL transformation to apply , if you select a value the metadata will be harvested and before added/updated to GeoNetwork will be applied the process, not matter if it’s a new metadata or one to update.

The processes are:

In the deployed GeoNetwork: TOMCAT_DIR/webapps/geonetwork/xsl/conversion/import
In the source code: https://github.com/geonetwork/core-geonetwork/tree/develop/web/src/main/webapp/xsl/conversion/import

You can check the available processes to see how should be developed your process, but you need to develop your own xslt process that does the work you require to add keywords or whatever are your requirements and add it to the previous folder so appears in the Harvester list for the field XSL transformation to apply

Regards,
Jose García

(attachments)

···

On Thu, May 3, 2018 at 11:10 AM, Carlos Figueiredo <carlos.figueiredo@anonymised.com> wrote:

Hi Maria,

Thanks for your email.

Just a question, following José’s suggestion.

The “default” harvesting to a CSW or GeoNetwork node will just collected the metadata records and according to the update date overwrite or not the existing ones.

This procedure doesn’t use a XSL doesn’t it ?

Considering that I want to use the ISO profile, which are the xsl files I should use to achive that transformation I want, i.e. not to overwrite but to keep some elements of the existent metadata records ?

On the folder you mentioned there are 19 xsl files but which ones you suggest to achive the harvesting of CSW and Geonetwork ?

Thanks and BR,

Carlos Figueiredo

Senior System Analyst
Defence & Security Systems

carlos.figueiredo@anonymised.com

Tel: +351.212945900

Mobile: +351. 937 673 523

Fax: +351.212945999

Rua Calvet Magalhães, 245

2770-153 Paço de Arcos · Portugal

www.edisoft.pt

……………………………………………………………………………

A informação contida neste e-mail e quaisquer documentos anexos são propriedade da EDISOFT e poderão ser confidenciais. Se não for o destinatário pretendido, por favor, comunique-nos de imediato, enviando-nos esta mensagem de volta e destruindo-a em seguida. Fica desde já advertido de que é estritamente proibida qualquer revisão, divulgação, distribuição, cópia ou qualquer outra utilização deste e-mail.

The information contained in this e-mail and any attachments are the property of EDISOFT and may be confidential. If you are not the intended recipient, please notify us immediately, send this message back to us and destroy it. You are hereby notified that any review, dissemination, distribution, copying or otherwise use of this e-mail is strictly prohibited

From: María Arias de Reyna [mailto:delawen@anonymised.com]
Sent: quinta-feira, 3 de maio de 2018 09:48
To: Carlos Figueiredo
Cc: Jose Garcia; geonetwork-devel@anonymised.comsourceforge.net

Subject: Re: [GeoNetwork-devel] Harvesting question / issue / idea

Dear Carlos,

The current examples are here:

https://github.com/geonetwork/core-geonetwork/tree/develop/web/src/main/webapp/xsl/conversion/import

On Thu, May 3, 2018 at 10:23 AM, Carlos Figueiredo <carlos.figueiredo@anonymised.com> wrote:

Hi José,

May thanks for the quick response on this subject.

I will try that approach.

Do you think I can start with the existent XSL files (the one used to do the regular CSW 2.0.2 and the one for GeoNetwork Node) ?

Can you please be so kind to name those files and location ?

Thank you and best regards,

Carlos Figueiredo

Senior System Analyst
Defence & Security Systems

carlos.figueiredo@anonymised.com

Tel: +351.212945900

Mobile: +351. 937 673 523

Fax: +351.212945999

Rua Calvet Magalhães, 245

2770-153 Paço de Arcos · Portugal

www.edisoft.pt

……………………………………………………………………………

A informação contida neste e-mail e quaisquer documentos anexos são propriedade da EDISOFT e poderão ser confidenciais. Se não for o destinatário pretendido, por favor, comunique-nos de imediato, enviando-nos esta mensagem de volta e destruindo-a em seguida. Fica desde já advertido de que é estritamente proibida qualquer revisão, divulgação, distribuição, cópia ou qualquer outra utilização deste e-mail.

The information contained in this e-mail and any attachments are the property of EDISOFT and may be confidential. If you are not the intended recipient, please notify us immediately, send this message back to us and destroy it. You are hereby notified that any review, dissemination, distribution, copying or otherwise use of this e-mail is strictly prohibited

From: Jose Garcia [mailto:jose.garcia@anonymised.com]
Sent: quinta-feira, 3 de maio de 2018 07:21
To: Carlos Figueiredo
Cc: geonetwork-devel@anonymised.comsourceforge.net
Subject: Re: [GeoNetwork-devel] Harvesting question / issue / idea

Hi Carlos

In the CSW harvester there’s an option XSL transformation to apply that allows to process the harvested records, I think you can use that mechanism creating a new xslt process that adds to the harvested records the required information.

You need to create the new xslt file and add it to TOMCAT_DIR/webapps/geonetwork/xsl/conversion/import folder, then should be displayed in the CSW harvester in the XSL transformation to apply.

Regards,

Jose García

On Wed, May 2, 2018 at 6:54 PM, Carlos Figueiredo <carlos.figueiredo@anonymised.com> wrote:

Hello all,

We are using the Geonetwork software package in a European project (h2020 initiative) in order to catalogue the project resources.

We are brainstorming some ideas to use the catalogue and its CSW protocol to make a sys2sys queries in order to use it on the project geoportal (using the GeoNetwork GUI as back office to maintain and manage metadata and harvesting procedures).

Our idea is to use some keywording to characterize and categorize de metadata records. This is of course ok for the metadata records we will create and manage under the project.

But for the harvesting metadata records we cannot do this as it will be overwritten the next harvesting cycle. It is understood that the metadata should (must) be maintained at the source and other should not change it.

Nevertheless, we would like to create just some keywords under one xml iso element on the harvested records. These will not change the metadata record meaning but it is just for our project better handle those records.

How do you see this as an option to the harvesting procedure ? i.e. the harvested metadata records would be updated and not deleted and inserted again, on all elements (if changes are available) thus maintaining the changes we made ourselves (the keywords on the element) ?

Have you came across with similar situations ?

Please let me know.

Thank you and best regards,

Carlos Figueiredo

Senior System Analyst
Defence & Security Systems

carlos.figueiredo@anonymised.com

Tel: +351.212945900

Mobile: +351. 937 673 523

Fax: +351.212945999

Rua Calvet Magalhães, 245

2770-153 Paço de Arcos · Portugal

www.edisoft.pt

……………………………………………………………………………

A informação contida neste e-mail e quaisquer documentos anexos são propriedade da EDISOFT e poderão ser confidenciais. Se não for o destinatário pretendido, por favor, comunique-nos de imediato, enviando-nos esta mensagem de volta e destruindo-a em seguida. Fica desde já advertido de que é estritamente proibida qualquer revisão, divulgação, distribuição, cópia ou qualquer outra utilização deste e-mail.

The information contained in this e-mail and any attachments are the property of EDISOFT and may be confidential. If you are not the intended recipient, please notify us immediately, send this message back to us and destroy it. You are hereby notified that any review, dissemination, distribution, copying or otherwise use of this e-mail is strictly prohibited

Check out the vibrant tech community on one of the world’s most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

GeoNetwork-devel mailing list
GeoNetwork-devel@anonymised.comsourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

–

Vriendelijke groeten / Kind regards,

Jose García

Veenderweg 13
6721 WD Bennekom
The Netherlands
T: +31 (0)318 416664

Please consider the environment before printing this email.

Check out the vibrant tech community on one of the world’s most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

GeoNetwork-devel mailing list
GeoNetwork-devel@anonymised.comsourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

–

Vriendelijke groeten / Kind regards,

Jose García

Veenderweg 13
6721 WD Bennekom
The Netherlands
T: +31 (0)318 416664

Please consider the environment before printing this email.

Carlos_Figueiredo · May 7, 2018, 1:35pm

Hi José and Maria,

I was trying to make a use case with the pointers you gave me.

But I still have some doubts about it and your interpretation of my request, which, I think invalidates your solution:

I will try to enumerate the use case:

1/ My Geonetwork is set to harvest system A (CSW or GeoNetword node)

2/ My Geonetwork is configured to edit the harvested metadata records

3/ My Geonetwork finishes the Havesting procedure, several records are acquired.

4/ My GeoNetwork is configured to stop the harvesting procedure.

5/ My Geonetwork administrator (or other owner) modifies the harvested metadata records by adding some “keyword” elements on the ISO metadata (ex: gmd:MD_Metadata\gmd:identificationInfo\gmd:MD_DataIdentification\gmd:descriptiveKeywords\gmd:MD_Keywords\gmd:keyword<gco:CharacterString> ). Or something similar.

6/ The harvested records are changed in the My Geonetwork database.

7/ if I kept the My Geoentwork Harvesting stopped I will not get the metadata updates of system A.

8/ If I start the My Geoenetwork Harvesting of system A, I will lose the “keywords” created.

Your suggestion: Add a XSLT template to intermediate the My Geonetwork procedure.

Usually the XSLT transformations (forgetting now GeoNetwork) work by using a XSLT XML files with rules, applied to an input XML file … that outputs a XML according to those rules.

Now applying it with GeoNetwork Harvesting procedure:

a/ the source/input file is the harvested metadata record. OK

b/ the output xml file shall be written to the Geonetwork database. OK

c/ a XSLT file with rules must be created to supply the transformation rules. NOT SO OK.

i/ please remember that the changes are made to the harvested metadata records that are stored in My Geonetwork DB

ii/ the XSLT rules are applied to the metadata records harvested from system A

iii/ how do you see that the XSLT file knows that the “keyword” elements exist in My GeoNetwork database if the only input is the harvested metadata records from system A ?

iv/ where in the flow the XSLT will consult the existing metadata record in My Geonetwork along with the input metadata records harvested from system A ?

v/ is sum, how in the geonetwork harvesting configuration (or in a regular XSLT script) you can use two input files, one is the updated record, the other the existing record and with that you create an output record with the merge of both. Ie. The ouput will have all the updates of the input harvested source and some of existing elements of the current metadata record.

I hope that the explanation is better than the previous ones.

Please let me know.

Thank you and best regards,

Carlos Figueiredo

Senior System Analyst
Defence & Security Systems

carlos.figueiredo@…1177…

Tel: +351.212945900

Mobile: +351. 937 673 523

Fax: +351.212945999

Rua Calvet Magalhães, 245

2770-153 Paço de Arcos · Portugal

www.edisoft.pt

……………………………………………………………………………

A informação contida neste e-mail e quaisquer documentos anexos são propriedade da EDISOFT e poderão ser confidenciais. Se não for o destinatário pretendido, por favor, comunique-nos de imediato, enviando-nos esta mensagem de volta e destruindo-a em seguida. Fica desde já advertido de que é estritamente proibida qualquer revisão, divulgação, distribuição, cópia ou qualquer outra utilização deste e-mail.

The information contained in this e-mail and any attachments are the property of EDISOFT and may be confidential. If you are not the intended recipient, please notify us immediately, send this message back to us and destroy it. You are hereby notified that any review, dissemination, distribution, copying or otherwise use of this e-mail is strictly prohibited

From: Carlos Figueiredo
Sent: quinta-feira, 3 de maio de 2018 10:38
To: ‘Jose Garcia’
Cc: María Arias de Reyna; geonetwork-devel@lists.sourceforge.net
Subject: RE: [GeoNetwork-devel] Harvesting question / issue / idea

Hi José.

Understood.

Thank you.

BR.

Carlos Figueiredo

Senior System Analyst
Defence & Security Systems

carlos.figueiredo@…1177…

Tel: +351.212945900

Mobile: +351. 937 673 523

Fax: +351.212945999

Rua Calvet Magalhães, 245

2770-153 Paço de Arcos · Portugal

www.edisoft.pt

(attachments)

Jose_Garcia1 · May 8, 2018, 6:25am

Hi Carlos

See feedback inline.

Regards,
Jose García

(attachments)

···

On Mon, May 7, 2018 at 3:35 PM, Carlos Figueiredo <carlos.figueiredo@anonymised.com> wrote:

Hi José and Maria,

I was trying to make a use case with the pointers you gave me.

But I still have some doubts about it and your interpretation of my request, which, I think invalidates your solution:

I will try to enumerate the use case:

1/ My Geonetwork is set to harvest system A (CSW or GeoNetword node)

2/ My Geonetwork is configured to edit the harvested metadata records

3/ My Geonetwork finishes the Havesting procedure, several records are acquired.

4/ My GeoNetwork is configured to stop the harvesting procedure.

5/ My Geonetwork administrator (or other owner) modifies the harvested metadata records by adding some “keyword” elements on the ISO metadata (ex: gmd:MD_Metadata\gmd:identificationInfo\gmd:MD_DataIdentification\gmd:descriptiveKeywords\gmd:MD_Keywords\gmd:keyword<gco:CharacterString> ). Or something similar.

6/ The harvested records are changed in the My Geonetwork database.

7/ if I kept the My Geoentwork Harvesting stopped I will not get the metadata updates of system A.

8/ If I start the My Geoenetwork Harvesting of system A, I will lose the “keywords” created.

About items 7 / 8, the option to allow editing harvested records is intended for cases where is required a 1 time harvesting, otherwise if you edit the harvested metadata and re-harvest again, you loose your edits as you indicate in item 8.

Your suggestion: Add a XSLT template to intermediate the My Geonetwork procedure.

Usually the XSLT transformations (forgetting now GeoNetwork) work by using a XSLT XML files with rules, applied to an input XML file … that outputs a XML according to those rules.

Now applying it with GeoNetwork Harvesting procedure:

a/ the source/input file is the harvested metadata record. OK

b/ the output xml file shall be written to the Geonetwork database. OK

c/ a XSLT file with rules must be created to supply the transformation rules. NOT SO OK.

i/ please remember that the changes are made to the harvested metadata records that are stored in My Geonetwork DB

ii/ the XSLT rules are applied to the metadata records harvested from system A

iii/ how do you see that the XSLT file knows that the “keyword” elements exist in My GeoNetwork database if the only input is the harvested metadata records from system A ?

iv/ where in the flow the XSLT will consult the existing metadata record in My Geonetwork along with the input metadata records harvested from system A ?

v/ is sum, how in the geonetwork harvesting configuration (or in a regular XSLT script) you can use two input files, one is the updated record, the other the existing record and with that you create an output record with the merge of both. Ie. The ouput will have all the updates of the input harvested source and some of existing elements of the current metadata record.

The case to use an xslt is to update the harvested metadata adding/removing/updating content, based on the current metadata that you’re harvesting. It only uses as a source the metadata that is harvested.

But the case you describe requires also to use the metadata that is already in the system, that is not supported by Geonetwork currently.

Geonetwork can be extended to support your use case doing the required developments. But I don’t fully get how this keyword will end in initially and how using the local metadata makes any difference. I mean the first harvesting will only use the remote metadata to add keywords, so whats the advantage to use this local version later for other re-harvests, instead of the remote version only, but I don’t have also the full picture.

In any case, as indicated your use case requires development in Geonetwork. In case you can’t manage these developments, there’re several companies providing development/consultancy services in GeoNetwork, you can check some of them in https://geonetwork-opensource.org/ at the bottom in the Acknowledgements section.

I hope that the explanation is better than the previous ones.

Please let me know.

Thank you and best regards,

Carlos Figueiredo

Senior System Analyst
Defence & Security Systems

carlos.figueiredo@anonymised.com

Tel: +351.212945900

Mobile: +351. 937 673 523

Fax: +351.212945999

Rua Calvet Magalhães, 245

2770-153 Paço de Arcos · Portugal

www.edisoft.pt

……………………………………………………………………………

A informação contida neste e-mail e quaisquer documentos anexos são propriedade da EDISOFT e poderão ser confidenciais. Se não for o destinatário pretendido, por favor, comunique-nos de imediato, enviando-nos esta mensagem de volta e destruindo-a em seguida. Fica desde já advertido de que é estritamente proibida qualquer revisão, divulgação, distribuição, cópia ou qualquer outra utilização deste e-mail.

The information contained in this e-mail and any attachments are the property of EDISOFT and may be confidential. If you are not the intended recipient, please notify us immediately, send this message back to us and destroy it. You are hereby notified that any review, dissemination, distribution, copying or otherwise use of this e-mail is strictly prohibited

From: Carlos Figueiredo
Sent: quinta-feira, 3 de maio de 2018 10:38
To: ‘Jose Garcia’
Cc: María Arias de Reyna; geonetwork-devel@anonymised.com…537…sourceforge.net
Subject: RE: [GeoNetwork-devel] Harvesting question / issue / idea

Hi José.

Understood.

Thank you.

BR.

Carlos Figueiredo

Senior System Analyst
Defence & Security Systems

carlos.figueiredo@anonymised.com

Tel: +351.212945900

Mobile: +351. 937 673 523

Fax: +351.212945999

Rua Calvet Magalhães, 245

2770-153 Paço de Arcos · Portugal

www.edisoft.pt

–

Vriendelijke groeten / Kind regards,

Jose García

Veenderweg 13
6721 WD Bennekom
The Netherlands
T: +31 (0)318 416664

Please consider the environment before printing this email.

Carlos_Figueiredo · May 8, 2018, 9:09am

Hi José,

Thank you for your feedback.

BR,

Carlos Figueiredo

Senior System Analyst
Defence & Security Systems

carlos.figueiredo@…1177…

Tel: +351.212945900

Mobile: +351. 937 673 523

Fax: +351.212945999

Rua Calvet Magalhães, 245

2770-153 Paço de Arcos · Portugal

www.edisoft.pt

……………………………………………………………………………

A informação contida neste e-mail e quaisquer documentos anexos são propriedade da EDISOFT e poderão ser confidenciais. Se não for o destinatário pretendido, por favor, comunique-nos de imediato, enviando-nos esta mensagem de volta e destruindo-a em seguida. Fica desde já advertido de que é estritamente proibida qualquer revisão, divulgação, distribuição, cópia ou qualquer outra utilização deste e-mail.

The information contained in this e-mail and any attachments are the property of EDISOFT and may be confidential. If you are not the intended recipient, please notify us immediately, send this message back to us and destroy it. You are hereby notified that any review, dissemination, distribution, copying or otherwise use of this e-mail is strictly prohibited

From: Jose Garcia [mailto:jose.garcia@…437…]
Sent: terça-feira, 8 de maio de 2018 07:25
To: Carlos Figueiredo
Cc: María Arias de Reyna; geonetwork-devel@lists.sourceforge.net
Subject: Re: [GeoNetwork-devel] Harvesting question / issue / idea

Hi Carlos

See feedback inline.

Regards,

Jose García

On Mon, May 7, 2018 at 3:35 PM, Carlos Figueiredo <carlos.figueiredo@…1177…> wrote: