We have developed a new module for geoserver that we'd like to contribute to the community, and I would like to ask for approval.
I remember once already suggesting the idea behind the module to the list a while ago and getting positive feedback / interest. The main thing it does is adding an extra tab to the layer page with custom metadata fields, based on a YAML configuration file in the data directory. Each field as a type: text fields, numbers, dates, checkboxes, UUID's, dropdowns, etcetera. You have multi-dimensional fields and unlimited nested complex fields. These fields are stored in the metadata map of the layer and therefore accessible via CSW mappings.
What is the problem we wanted to solve? Having complex metadata information in geonetwork, but wanting a straight-forward way to sync this with geoserver (for example title, keywords, etcetera). The chosen solution is to keep all the metadata close to your data, and let geonetwork automatically harvest everything from geoserver using the CSW service, therefore removing the need to add/edit metadata in geonetwork itself and avoiding differences between the two.
For this reason I also had to fix a number of bugs and add new features in CSW with respect to the ISO metadata profile. There are already a number of PR's related to this.
Here is a non-exclusive list of additional features included:
- creating templates and linking layers to (multiple) templates (for example, with info about owner)
- import the metadata from geonetwork (in case you do already have a geonetwork entry)
- automatically generating feature catalogue data from the feature type of your vector layer
That is quite the experiment. I know James was using keywords to host
some external config in a standards compatible way.
+1 on the community module.
Are you sure you should not build out the CSW functionality more? XML
vs YAML etc..
On Wed, Feb 6, 2019 at 2:45 AM Niels Charlier <niels@anonymised.com
<mailto:niels@anonymised.com>> wrote:
Hello,
We have developed a new module for geoserver that we'd like to
contribute to the community, and I would like to ask for approval.
I remember once already suggesting the idea behind the module to the
list a while ago and getting positive feedback / interest. The main
thing it does is adding an extra tab to the layer page with custom
metadata fields, based on a YAML configuration file in the data
directory. Each field as a type: text fields, numbers, dates,
checkboxes, UUID's, dropdowns, etcetera. You have multi-dimensional
fields and unlimited nested complex fields. These fields are
stored in
the metadata map of the layer and therefore accessible via CSW
mappings.
What is the problem we wanted to solve? Having complex metadata
information in geonetwork, but wanting a straight-forward way to sync
this with geoserver (for example title, keywords, etcetera). The
chosen
solution is to keep all the metadata close to your data, and let
geonetwork automatically harvest everything from geoserver using
the CSW
service, therefore removing the need to add/edit metadata in
geonetwork
itself and avoiding differences between the two.
For this reason I also had to fix a number of bugs and add new
features
in CSW with respect to the ISO metadata profile. There are already a
number of PR's related to this.
Here is a non-exclusive list of additional features included:
- creating templates and linking layers to (multiple) templates (for
example, with info about owner)
- import the metadata from geonetwork (in case you do already have a
geonetwork entry)
- automatically generating feature catalogue data from the feature
type
of your vector layer
Are you sure you should not build out the CSW functionality more? XML vs YAML etc..
Well, they do two very different things -> csw = output (service), metadata = input (GUI). The YAML is for config, while CSW currently uses .properties files for its config which could rather easily be extended to support yaml as well. The .properties mapping files seemed like a good idea at first to keep things very simple, but they get really out of hand when configuring a complex ISO scheme and become hard to maintain. An idea for the future.
This work has already lead me to a bunch of CSW improvements and bug fixes. Apparently the ISO metadata profile is really quite faulty atm. It would be great if I can find one or more people to help review all that stuff (a lot of the PR's currently have failed builds but that is because they depend on others that need to be review/merged first.)
Regards
Niels
On Wed, Feb 6, 2019 at 2:45 AM Niels Charlier <niels@anonymised.com <mailto:niels@anonymised.com>> wrote:
Hello,
We have developed a new module for geoserver that we'd like to
contribute to the community, and I would like to ask for approval.
I remember once already suggesting the idea behind the module to the
list a while ago and getting positive feedback / interest. The main
thing it does is adding an extra tab to the layer page with custom
metadata fields, based on a YAML configuration file in the data
directory. Each field as a type: text fields, numbers, dates,
checkboxes, UUID's, dropdowns, etcetera. You have multi-dimensional
fields and unlimited nested complex fields. These fields are
stored in
the metadata map of the layer and therefore accessible via CSW
mappings.
What is the problem we wanted to solve? Having complex metadata
information in geonetwork, but wanting a straight-forward way to sync
this with geoserver (for example title, keywords, etcetera). The
chosen
solution is to keep all the metadata close to your data, and let
geonetwork automatically harvest everything from geoserver using
the CSW
service, therefore removing the need to add/edit metadata in
geonetwork
itself and avoiding differences between the two.
For this reason I also had to fix a number of bugs and add new
features
in CSW with respect to the ISO metadata profile. There are already a
number of PR's related to this.
Here is a non-exclusive list of additional features included:
- creating templates and linking layers to (multiple) templates (for
example, with info about owner)
- import the metadata from geonetwork (in case you do already have a
geonetwork entry)
- automatically generating feature catalogue data from the feature
type
of your vector layer
The first version of the metadata module is ready, and it is as expected a very powerful tool. I do need approval for two small changes I made to the web-core module (hooks).
What is even more wonderful, is that we managed to create a configuration that allows us to configure and store all of the INSPIRE compliant metadata in geoserver, expose it via the CSW service using the ISO metadata profile, and harvest it with geonetwork. We have added all of our configuration files in a tutorial inside the metadata module PR.
However, to make all of this work we had to fix a number of bugs in the CSW module (in particular related to the ISO metadata profile) and add some improvements as well. The ISO metadata profile is required for INSPIRE, but support for it is currently still limited and buggy in the geoserver CSW extension. We have resolved the bugs and limitations.
Several Pull Requests have been made for this over the past few months, they all contain unit tests and documentation. Merging this would vastly improve the CSW extension of geoserver and in particular in combination with the new metadata profile would vastly improve geoserver’s metadata capabilities. There is already another government department that wants to roll out the same system, but they would like it to be merged into the official geoserver.
Are there people out there who are willing to review some of these PR’s? I am of course ready to follow up and make improvements to the code and documentation as requested.
Kind Regards
Niels
···
On 14/02/2019 17:03, Gabriel Roldan wrote:
Hey Niels,
that sounds really cool.
Have a community +1 vote here.
Cheers!
Gabriel.
El mar., 12 de feb. de 2019 a la(s) 10:03, Niels Charlier (niels@anonymised.com) escribió:
On 12/02/2019 01:21, Jody Garnett wrote:
Are you sure you should not build out the CSW functionality more? XML vs YAML etc…
Well, they do two very different things → csw = output (service), metadata = input (GUI). The YAML is for config, while CSW currently uses .properties files for its config which could rather easily be extended to support yaml as well. The .properties mapping files seemed like a good idea at first to keep things very simple, but they get really out of hand when configuring a complex ISO scheme and become hard to maintain. An idea for the future.
This work has already lead me to a bunch of CSW improvements and bug fixes. Apparently the ISO metadata profile is really quite faulty atm. It would be great if I can find one or more people to help review all that stuff (a lot of the PR’s currently have failed builds but that is because they depend on others that need to be review/merged first.)
We have developed a new module for geoserver that we’d like to
contribute to the community, and I would like to ask for approval.
I remember once already suggesting the idea behind the module to the
list a while ago and getting positive feedback / interest. The main
thing it does is adding an extra tab to the layer page with custom
metadata fields, based on a YAML configuration file in the data
directory. Each field as a type: text fields, numbers, dates,
checkboxes, UUID’s, dropdowns, etcetera. You have multi-dimensional
fields and unlimited nested complex fields. These fields are stored in
the metadata map of the layer and therefore accessible via CSW mappings.
What is the problem we wanted to solve? Having complex metadata
information in geonetwork, but wanting a straight-forward way to sync
this with geoserver (for example title, keywords, etcetera). The chosen
solution is to keep all the metadata close to your data, and let
geonetwork automatically harvest everything from geoserver using the CSW
service, therefore removing the need to add/edit metadata in geonetwork
itself and avoiding differences between the two.
For this reason I also had to fix a number of bugs and add new features
in CSW with respect to the ISO metadata profile. There are already a
number of PR’s related to this.
Here is a non-exclusive list of additional features included:
creating templates and linking layers to (multiple) templates (for
example, with info about owner)
import the metadata from geonetwork (in case you do already have a
geonetwork entry)
automatically generating feature catalogue data from the feature type
of your vector layer