[GeoNetwork-devel] Filtering based on availability

GeoNetwork GeoNetwork Developer
1

Hi All,

We’re looking at how we might filter out or disable collections which are not accessible in our portal, which uses Geonetwork as the backend search engine.

Our use case is as follows:

  • Our metadata records relate one to one to feature types served by wms/wfs services

  • If the wms/wfs service is unavailable/slow or just unreliable we’d like to be able to flag it as such or not show it at all in our portal (in which case facet counts would need to reflect only what’s visible),

  • We intend to harvest records of this type from a number of providers including our own so records in our GeoNetwork instance will be harvested records

  • We want unavailable services to be detected and records filtered out or marked as unavailable automatically.

We’ve considered a number of options including a filtering process separate to Geonetwork, but have some issues with this approach with respect to accuracy of facet counts etc.

We’ve also looking at unpublishing records, but this conflicts with normal publishing workflow and is problematic for harvested records.

We are now looking at how we may include filtering based on accessibility as a plugin to GeoNetwork. I’m thinking of some type of custom filter functionality where the filter value is not sourced from the metadata record, but from another service and is regularly updated or to allow custom filters to be applied at search time.

What I’m wondering is a) whether anyone else has looked at this problem/if there are solutions already implemented or in progess, or, if not, b) whether the ability to plugin in this functionality may be a desirable enhancement for GeoNetwork when we’ve finished, in which case we can invest some time in ensuring that the functionality will be suitable for this purpose.

Thanks,
Craig Jones
Integrated Marine Observing System

2

Hi All,

I’ll take the lack of response as confirmation that there is no work going on in this space.

I’d be interested in getting comments on our proposed approach prior to starting on the work (so that we can take this into account if possible before we start rather than after)

Please refer to:

https://github.com/geonetwork/core-geonetwork/wiki/201411PluggableFilters

If you could make the time to review and comment on approach/suitability for inclusion in core that would be great.

Thanks,
Craig Jones
Integrated Marine Observing System

···

-------- Forwarded Message --------

Subject:

Filtering based on availability

Date:

Fri, 14 Nov 2014 17:42:19 +1100

From:

Craig Jones <Craig.Jones@anonymised.com>

To:

Devel geonetwork-devel@lists.sourceforge.net <geonetwork-devel@lists.sourceforge.net>

CC:

Peter Blain <Peter.Blain@anonymised.com>

Hi All,

We’re looking at how we might filter out or disable collections which are not accessible in our portal, which uses Geonetwork as the backend search engine.

Our use case is as follows:

  • Our metadata records relate one to one to feature types served by wms/wfs services

  • If the wms/wfs service is unavailable/slow or just unreliable we’d like to be able to flag it as such or not show it at all in our portal (in which case facet counts would need to reflect only what’s visible),

  • We intend to harvest records of this type from a number of providers including our own so records in our GeoNetwork instance will be harvested records

  • We want unavailable services to be detected and records filtered out or marked as unavailable automatically.

We’ve considered a number of options including a filtering process separate to Geonetwork, but have some issues with this approach with respect to accuracy of facet counts etc.

We’ve also looking at unpublishing records, but this conflicts with normal publishing workflow and is problematic for harvested records.

We are now looking at how we may include filtering based on accessibility as a plugin to GeoNetwork. I’m thinking of some type of custom filter functionality where the filter value is not sourced from the metadata record, but from another service and is regularly updated or to allow custom filters to be applied at search time.

What I’m wondering is a) whether anyone else has looked at this problem/if there are solutions already implemented or in progess, or, if not, b) whether the ability to plugin in this functionality may be a desirable enhancement for GeoNetwork when we’ve finished, in which case we can invest some time in ensuring that the functionality will be suitable for this purpose.

Thanks,
Craig Jones
Integrated Marine Observing System

3

Hi Craig,

I love the proposal. The one thing comment I have is about the isFilterRequired(Element) method. We are migrating away from Jeeves to Spring MVC and the search parameters will probably be created from Map<String,String> instead of Element in the future. That said it is just something to be aware of. That change will probably take some time yet.

Jesse

···

On Thu, Nov 20, 2014 at 8:20 AM, Craig Jones <Craig.Jones@anonymised.com> wrote:

Hi All,

I’ll take the lack of response as confirmation that there is no work going on in this space.

I’d be interested in getting comments on our proposed approach prior to starting on the work (so that we can take this into account if possible before we start rather than after)

Please refer to:

https://github.com/geonetwork/core-geonetwork/wiki/201411PluggableFilters

If you could make the time to review and comment on approach/suitability for inclusion in core that would be great.

Thanks,
Craig Jones
Integrated Marine Observing System

-------- Forwarded Message --------

Subject:

Filtering based on availability

Date:

Fri, 14 Nov 2014 17:42:19 +1100

From:

Craig Jones <Craig.Jones@anonymised.com>

To:

Devel geonetwork-devel@lists.sourceforge.net <geonetwork-devel@lists.sourceforge.net>

CC:

Peter Blain <Peter.Blain@anonymised.com>

Hi All,

We’re looking at how we might filter out or disable collections which are not accessible in our portal, which uses Geonetwork as the backend search engine.

Our use case is as follows:

  • Our metadata records relate one to one to feature types served by wms/wfs services

  • If the wms/wfs service is unavailable/slow or just unreliable we’d like to be able to flag it as such or not show it at all in our portal (in which case facet counts would need to reflect only what’s visible),

  • We intend to harvest records of this type from a number of providers including our own so records in our GeoNetwork instance will be harvested records

  • We want unavailable services to be detected and records filtered out or marked as unavailable automatically.

We’ve considered a number of options including a filtering process separate to Geonetwork, but have some issues with this approach with respect to accuracy of facet counts etc.

We’ve also looking at unpublishing records, but this conflicts with normal publishing workflow and is problematic for harvested records.

We are now looking at how we may include filtering based on accessibility as a plugin to GeoNetwork. I’m thinking of some type of custom filter functionality where the filter value is not sourced from the metadata record, but from another service and is regularly updated or to allow custom filters to be applied at search time.

What I’m wondering is a) whether anyone else has looked at this problem/if there are solutions already implemented or in progess, or, if not, b) whether the ability to plugin in this functionality may be a desirable enhancement for GeoNetwork when we’ve finished, in which case we can invest some time in ensuring that the functionality will be suitable for this purpose.

Thanks,
Craig Jones
Integrated Marine Observing System

Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk

GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

4

Thanks Jesse. No problems incorporating that when the time comes.

On 20/11/14 18:30, Jesse Eichar wrote:

Hi Craig,

I love the proposal. The one thing comment I have is about the isFilterRequired(Element) method. We are migrating away from Jeeves to Spring MVC and the search parameters will probably be created from Map<String,String> instead of Element in the future. That said it is just something to be aware of. That change will probably take some time yet.

Jesse

On Thu, Nov 20, 2014 at 8:20 AM, Craig Jones <Craig.Jones@anonymised.com <mailto:Craig.Jones@anonymised.com>> wrote:

    Hi All,

    I'll take the lack of response as confirmation that there is no
    work going on in this space.

    I'd be interested in getting comments on our proposed approach
    prior to starting on the work (so that we can take this into
    account if possible before we start rather than after)

    Please refer to:

    https://github.com/geonetwork/core-geonetwork/wiki/201411PluggableFilters

    If you could make the time to review and comment on
    approach/suitability for inclusion in core that would be great.

    Thanks,
    Craig Jones
    Integrated Marine Observing System

    -------- Forwarded Message --------
    Subject: Filtering based on availability
    Date: Fri, 14 Nov 2014 17:42:19 +1100
    From: Craig Jones <Craig.Jones@anonymised.com>
    <mailto:Craig.Jones@anonymised.com>
    To: Devel geonetwork-devel@lists.sourceforge.net
    <mailto:geonetwork-devel@lists.sourceforge.net>
    <geonetwork-devel@lists.sourceforge.net>
    <mailto:geonetwork-devel@lists.sourceforge.net>
    CC: Peter Blain <Peter.Blain@anonymised.com>
    <mailto:Peter.Blain@anonymised.com>

    Hi All,

    We're looking at how we might filter out or disable collections
    which are not accessible in our portal, which uses Geonetwork as
    the backend search engine.

    Our use case is as follows:

      * Our metadata records relate one to one to feature types served
        by wms/wfs services
      * If the wms/wfs service is unavailable/slow or just unreliable
        we'd like to be able to flag it as such or not show it at all
        in our portal (in which case facet counts would need to
        reflect only what's visible),
      * We intend to harvest records of this type from a number of
        providers including our own so records in our GeoNetwork
        instance will be harvested records
      * We want unavailable services to be detected and records
        filtered out or marked as unavailable automatically.

    We've considered a number of options including a filtering process
    separate to Geonetwork, but have some issues with this approach
    with respect to accuracy of facet counts etc.

    We've also looking at unpublishing records, but this conflicts
    with normal publishing workflow and is problematic for harvested
    records.

    We are now looking at how we may include filtering based on
    accessibility as a plugin to GeoNetwork. I'm thinking of some type
    of custom filter functionality where the filter value is not
    sourced from the metadata record, but from another service and is
    regularly updated or to allow custom filters to be applied at
    search time.

    What I'm wondering is a) whether anyone else has looked at this
    problem/if there are solutions already implemented or in progess,
    or, if not, b) whether the ability to plugin in this functionality
    may be a desirable enhancement for GeoNetwork when we've finished,
    in which case we can invest some time in ensuring that the
    functionality will be suitable for this purpose.

    Thanks,
    Craig Jones
    Integrated Marine Observing System

    ------------------------------------------------------------------------------
    Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
    from Actuate! Instantly Supercharge Your Business Reports and
    Dashboards
    with Interactivity, Sharing, Native Excel Exports, App Integration
    & more
    Get technology previously reserved for billion-dollar
    corporations, FREE
    http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
    _______________________________________________
    GeoNetwork-devel mailing list
    GeoNetwork-devel@lists.sourceforge.net
    <mailto:GeoNetwork-devel@lists.sourceforge.net>
    https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
    GeoNetwork OpenSource is maintained at
    http://sourceforge.net/projects/geonetwork

5

Hi Craig,

There are definitely times (as in your use case) when this is desirable and so a filter plugin architecture would be good for supporting not just your use case but others as well. I'm not working on anything like this but I am working on other projects that rely on the catalog for information about services delivering data so any mechanism that can improve the reliability of those results would be very useful. I like the approach as stated in your proposal and I believe it will be suitable for inclusion into trunk when complete.

+1 from me.

Cheers and thanks,
Simon
________________________________________
From: Craig Jones [Craig.Jones@anonymised.com]
Sent: Thursday, 20 November 2014 6:20 PM
To: Devel geonetwork-devel@lists.sourceforge.net
Subject: [GeoNetwork-devel] Filtering based on availability

Hi All,

I'll take the lack of response as confirmation that there is no work going on in this space.

I'd be interested in getting comments on our proposed approach prior to starting on the work (so that we can take this into account if possible before we start rather than after)

Please refer to:

If you could make the time to review and comment on approach/suitability for inclusion in core that would be great.

Thanks,
Craig Jones
Integrated Marine Observing System

-------- Forwarded Message --------
Subject: Filtering based on availability
Date: Fri, 14 Nov 2014 17:42:19 +1100
From: Craig Jones <Craig.Jones@anonymised.com><mailto:Craig.Jones@anonymised.com>
To: Devel geonetwork-devel@lists.sourceforge.net<mailto:geonetwork-devel@lists.sourceforge.net> <geonetwork-devel@lists.sourceforge.net><mailto:geonetwork-devel@lists.sourceforge.net>
CC: Peter Blain <Peter.Blain@anonymised.com><mailto:Peter.Blain@anonymised.com>

Hi All,

We're looking at how we might filter out or disable collections which are not accessible in our portal, which uses Geonetwork as the backend search engine.

Our use case is as follows:

  * Our metadata records relate one to one to feature types served by wms/wfs services
  * If the wms/wfs service is unavailable/slow or just unreliable we'd like to be able to flag it as such or not show it at all in our portal (in which case facet counts would need to reflect only what's visible),
  * We intend to harvest records of this type from a number of providers including our own so records in our GeoNetwork instance will be harvested records
  * We want unavailable services to be detected and records filtered out or marked as unavailable automatically.

We've considered a number of options including a filtering process separate to Geonetwork, but have some issues with this approach with respect to accuracy of facet counts etc.

We've also looking at unpublishing records, but this conflicts with normal publishing workflow and is problematic for harvested records.

We are now looking at how we may include filtering based on accessibility as a plugin to GeoNetwork. I'm thinking of some type of custom filter functionality where the filter value is not sourced from the metadata record, but from another service and is regularly updated or to allow custom filters to be applied at search time.

What I'm wondering is a) whether anyone else has looked at this problem/if there are solutions already implemented or in progess, or, if not, b) whether the ability to plugin in this functionality may be a desirable enhancement for GeoNetwork when we've finished, in which case we can invest some time in ensuring that the functionality will be suitable for this purpose.

Thanks,
Craig Jones
Integrated Marine Observing System