[GeoNetwork-devel] Vote for Health monitoring proposal

GeoNetwork GeoNetwork Developer
1

Hi,

I have the health monitoring work done and ready for commit. Could we vote on it? Make improvement suggestions?

http://trac.osgeo.org/geonetwork/wiki/HealthMonitoring

Thanks Jesse

2

Hi Jesse,
I suggest you put this proposal forward for voting. It is a very welcome utility indeed!
Cheers,
Jeroen

**GeoCat** introduces Bridge© An extension to ArcGIS© to instantly publish data and metadata on GeoServer and GeoNetwork. See [http://geocat.net](http://geocat.net/) for more details.


Jeroen Ticheler

GeoCat bv
Veenderweg 13
6721 WD Bennekom
The Netherlands
Tel: +31 (0)6 81286572
http://geocat.net

Please consider the environment before printing this email.

On 2 apr 2012, at 10:39, Jesse Eichar wrote:

Hi,

I have the health monitoring work done and ready for commit. Could we vote on it? Make improvement suggestions?

http://trac.osgeo.org/geonetwork/wiki/HealthMonitoring

Thanks Jesse

This SF email is sponsosred by:
Try Windows Azure free for 90 days Click Here
http://p.sf.net/sfu/sfd2d-msazure_______________________________________________
GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork
3

I have changed the status to “Proposed for vote” and added an associated ticket for the new feature.

Jesse

On Mon, Apr 2, 2012 at 11:35 AM, Jeroen Ticheler <jeroen.ticheler@anonymised.com> wrote:

Hi Jesse,
I suggest you put this proposal forward for voting. It is a very welcome utility indeed!
Cheers,
Jeroen

**GeoCat** introduces Bridge© An extension to ArcGIS© to instantly publish data and metadata on GeoServer and GeoNetwork. See [http://geocat.net](http://geocat.net/) for more details.


Jeroen Ticheler

GeoCat bv
Veenderweg 13
6721 WD Bennekom
The Netherlands
Tel: +31 (0)6 81286572
http://geocat.net

Please consider the environment before printing this email.

On 2 apr 2012, at 10:39, Jesse Eichar wrote:

Hi,

I have the health monitoring work done and ready for commit. Could we vote on it? Make improvement suggestions?

http://trac.osgeo.org/geonetwork/wiki/HealthMonitoring

Thanks Jesse

This SF email is sponsosred by:
Try Windows Azure free for 90 days Click Here
http://p.sf.net/sfu/sfd2d-msazure_______________________________________________
GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

4

The github link is on http://trac.osgeo.org/geonetwork/wiki/HealthMonitoring

But for your convenience I have attached a patch to http://trac.osgeo.org/geonetwork/ticket/846 with the current diff.

Jesse

On Mon, Apr 2, 2012 at 2:44 PM, Francois Prunayre <fx.prunayre@anonymised.com> wrote:

Hello Jesse, do you have any patch to have a look to the changes ? or any git magic command ?

Thanks.

Francois

Le 2 avril 2012 12:38, Jesse Eichar <jesse.eichar@anonymised.com> a écrit :

I have changed the status to “Proposed for vote” and added an associated ticket for the new feature.

Jesse

On Mon, Apr 2, 2012 at 11:35 AM, Jeroen Ticheler <jeroen.ticheler@…437…> wrote:

Hi Jesse,
I suggest you put this proposal forward for voting. It is a very welcome utility indeed!
Cheers,
Jeroen

**GeoCat** introduces Bridge© An extension to ArcGIS© to instantly publish data and metadata on GeoServer and GeoNetwork. See [http://geocat.net](http://geocat.net/) for more details.


Jeroen Ticheler

GeoCat bv
Veenderweg 13
6721 WD Bennekom
The Netherlands
Tel: +31 (0)6 81286572
http://geocat.net

Please consider the environment before printing this email.

On 2 apr 2012, at 10:39, Jesse Eichar wrote:

Hi,

I have the health monitoring work done and ready for commit. Could we vote on it? Make improvement suggestions?

http://trac.osgeo.org/geonetwork/wiki/HealthMonitoring

Thanks Jesse

This SF email is sponsosred by:
Try Windows Azure free for 90 days Click Here
http://p.sf.net/sfu/sfd2d-msazure_______________________________________________
GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

This SF email is sponsosred by:
Try Windows Azure free for 90 days Click Here
http://p.sf.net/sfu/sfd2d-msazure

GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

5

Hi Jesse,

May I suggest that you add some monitoring of the harvesting process?
For example, if there are errors during the harvesting process those
errors are sent to the administrator's email address.

Also if there are connection errors to the remote site after a certain
number of attempts then an email is sent to the GN administrator as well
as the remote site's administrator.

Thanks.

John Hockaday

-----Original Message-----
From: Jesse Eichar [mailto:jesse.eichar@anonymised.com]
Sent: Monday, April 02, 2012 6:39 PM
To: Devel geonetwork-devel@lists.sourceforge.net; Heikki
Doeleman; Francois Prunayre; Jose Garcia
Subject: [GeoNetwork-devel] Vote for Health monitoring proposal

Hi,

I have the health monitoring work done and ready for commit.
Could we vote on it? Make improvement suggestions?

http://trac.osgeo.org/geonetwork/wiki/HealthMonitoring

Thanks Jesse

*************************************************************************
The information contained in this e-mail, and any attachments to it,
is intended for the use of the addressee and is confidential. If you
are not the intended recipient you must not use, disclose, read,
forward, copy or retain any of the information. If you received this
e-mail in error, please delete it and notify the sender by return
e-mail or telephone.

The Commonwealth does not warrant that any attachments are free
from viruses or any other defects. You assume all liability for any
loss, damage or other consequences which may arise from opening
or using the attachments.

The security of emails transmitted in an unencrypted environment
cannot be guaranteed. By forwarding or replying to this email, you
acknowledge and accept these risks.
*************************************************************************

6

A good suggestion. I will look into adding these new monitors.

Jesse

On Tue, Apr 3, 2012 at 2:45 AM, Hockaday, John <John.Hockaday@anonymised.com> wrote:

Hi Jesse,

May I suggest that you add some monitoring of the harvesting process?
For example, if there are errors during the harvesting process those
errors are sent to the administrator’s email address.

Also if there are connection errors to the remote site after a certain
number of attempts then an email is sent to the GN administrator as well
as the remote site’s administrator.

Thanks.

John Hockaday

-----Original Message-----
From: Jesse Eichar [mailto:jesse.eichar@anonymised.com]
Sent: Monday, April 02, 2012 6:39 PM
To: Devel geonetwork-devel@lists.sourceforge.net; Heikki
Doeleman; Francois Prunayre; Jose Garcia
Subject: [GeoNetwork-devel] Vote for Health monitoring proposal

Hi,

I have the health monitoring work done and ready for commit.
Could we vote on it? Make improvement suggestions?

http://trac.osgeo.org/geonetwork/wiki/HealthMonitoring

Thanks Jesse

The information contained in this e-mail, and any attachments to it,
is intended for the use of the addressee and is confidential. If you
are not the intended recipient you must not use, disclose, read,
forward, copy or retain any of the information. If you received this
e-mail in error, please delete it and notify the sender by return
e-mail or telephone.

The Commonwealth does not warrant that any attachments are free
from viruses or any other defects. You assume all liability for any
loss, damage or other consequences which may arise from opening
or using the attachments.

The security of emails transmitted in an unencrypted environment
cannot be guaranteed. By forwarding or replying to this email, you
acknowledge and accept these risks.

7

+1 for me. Thanks Jesse.

Francois

Le 2 avril 2012 14:55, Jesse Eichar <jesse.eichar@anonymised.com> a écrit :

The github link is on http://trac.osgeo.org/geonetwork/wiki/HealthMonitoring

But for your convenience I have attached a patch to http://trac.osgeo.org/geonetwork/ticket/846 with the current diff.

Jesse

On Mon, Apr 2, 2012 at 2:44 PM, Francois Prunayre <fx.prunayre@…31…> wrote:

Hello Jesse, do you have any patch to have a look to the changes ? or any git magic command ?

Thanks.

Francois

Le 2 avril 2012 12:38, Jesse Eichar <jesse.eichar@anonymised.com> a écrit :

I have changed the status to “Proposed for vote” and added an associated ticket for the new feature.

Jesse

On Mon, Apr 2, 2012 at 11:35 AM, Jeroen Ticheler <jeroen.ticheler@…437…> wrote:

Hi Jesse,
I suggest you put this proposal forward for voting. It is a very welcome utility indeed!
Cheers,
Jeroen

**GeoCat** introduces Bridge© An extension to ArcGIS© to instantly publish data and metadata on GeoServer and GeoNetwork. See [http://geocat.net](http://geocat.net/) for more details.


Jeroen Ticheler

GeoCat bv
Veenderweg 13
6721 WD Bennekom
The Netherlands
Tel: +31 (0)6 81286572
http://geocat.net

Please consider the environment before printing this email.

On 2 apr 2012, at 10:39, Jesse Eichar wrote:

Hi,

I have the health monitoring work done and ready for commit. Could we vote on it? Make improvement suggestions?

http://trac.osgeo.org/geonetwork/wiki/HealthMonitoring

Thanks Jesse

This SF email is sponsosred by:
Try Windows Azure free for 90 days Click Here
http://p.sf.net/sfu/sfd2d-msazure_______________________________________________
GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

This SF email is sponsosred by:
Try Windows Azure free for 90 days Click Here
http://p.sf.net/sfu/sfd2d-msazure

GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

8

+1 from me.
Cheers,
Jeroen

On 3 apr. 2012, at 08:37, Francois Prunayre wrote:

+1 for me. Thanks Jesse.

Francois

Le 2 avril 2012 14:55, Jesse Eichar <jesse.eichar@anonymised.com> a écrit :

The github link is on http://trac.osgeo.org/geonetwork/wiki/HealthMonitoring

But for your convenience I have attached a patch to http://trac.osgeo.org/geonetwork/ticket/846 with the current diff.

Jesse

On Mon, Apr 2, 2012 at 2:44 PM, Francois Prunayre <fx.prunayre@…31…> wrote:

Hello Jesse, do you have any patch to have a look to the changes ? or any git magic command ?

Thanks.

Francois

Le 2 avril 2012 12:38, Jesse Eichar <jesse.eichar@anonymised.com> a écrit :

I have changed the status to “Proposed for vote” and added an associated ticket for the new feature.

Jesse

On Mon, Apr 2, 2012 at 11:35 AM, Jeroen Ticheler <jeroen.ticheler@…437…> wrote:

Hi Jesse,
I suggest you put this proposal forward for voting. It is a very welcome utility indeed!
Cheers,
Jeroen

**GeoCat** introduces Bridge© An extension to ArcGIS© to instantly publish data and metadata on GeoServer and GeoNetwork. See [http://geocat.net](http://geocat.net/) for more details.


Jeroen Ticheler

GeoCat bv
Veenderweg 13
6721 WD Bennekom
The Netherlands
Tel: +31 (0)6 81286572
http://geocat.net

Please consider the environment before printing this email.

On 2 apr 2012, at 10:39, Jesse Eichar wrote:

Hi,

I have the health monitoring work done and ready for commit. Could we vote on it? Make improvement suggestions?

http://trac.osgeo.org/geonetwork/wiki/HealthMonitoring

Thanks Jesse

This SF email is sponsosred by:
Try Windows Azure free for 90 days Click Here
http://p.sf.net/sfu/sfd2d-msazure_______________________________________________
GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

This SF email is sponsosred by:
Try Windows Azure free for 90 days Click Here
http://p.sf.net/sfu/sfd2d-msazure

GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

Better than sec? Nothing is better than sec when it comes to
monitoring Big Data applications. Try Boundary one-second
resolution app monitoring today. Free.
http://p.sf.net/sfu/Boundary-dev2dev_______________________________________________
GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

9

+1 from me too Jesse - looking forward to using this.

Cheers,
Simon
________________________________________
From: Jesse Eichar [jesse.eichar@anonymised.com]
Sent: Monday, 2 April 2012 8:38 PM
To: Jeroen Ticheler
Cc: Devel geonetwork-devel@lists.sourceforge.net
Subject: Re: [GeoNetwork-devel] Vote for Health monitoring proposal

I have changed the status to "Proposed for vote" and added an associated ticket for the new feature.

Jesse

On Mon, Apr 2, 2012 at 11:35 AM, Jeroen Ticheler <jeroen.ticheler@anonymised.com<mailto:jeroen.ticheler@anonymised.com>> wrote:
Hi Jesse,
I suggest you put this proposal forward for voting. It is a very welcome utility indeed!
Cheers,
Jeroen

____________________________________________________
GeoCat
introduces Bridge©
An extension to ArcGIS© to instantly publish data and metadata on GeoServer and GeoNetwork.
See http://geocat.net/&gt; for more details.

Jeroen Ticheler
GeoCat bv
Veenderweg 13
6721 WD Bennekom
The Netherlands
Tel: +31 (0)6 81286572<tel:%2B31%20%280%296%2081286572>

Please consider the environment before printing this email.

On 2 apr 2012, at 10:39, Jesse Eichar wrote:

Hi,

I have the health monitoring work done and ready for commit. Could we vote on it? Make improvement suggestions?

http://trac.osgeo.org/geonetwork/wiki/HealthMonitoring

Thanks Jesse

------------------------------------------------------------------------------
This SF email is sponsosred by:
Try Windows Azure free for 90 days Click Here

GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net<mailto:GeoNetwork-devel@anonymised.comforge.net>

GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

10

On Tue, Apr 3, 2012 at 2:45 AM, Hockaday, John <John.Hockaday@anonymised.com> wrote:

Hi Jesse,

May I suggest that you add some monitoring of the harvesting process?
For example, if there are errors during the harvesting process those
errors are sent to the administrator’s email address.

I have added a counter that tracks tracks the errors raised by the harvesters. This can be used for this purpose.

The monitoring doesn’t quite work like you explained above. It monitors various aspects of the system and provides an API for a dedicated monitor software to access. For example here we have collectd with the json module collecting the statistics of geonetwork. We have it connected to nagios which pings both collectd to see if the values have crossed an acceptable threshhold and it also calls Geonetwork’s health checks url to ensure that the system is functioning correctly. If either a threshhold or a health check fails then administrators are notified.

If you look at the metrics there are some very good metrics. For example you can see the rate of occurance of a particular even in the last minute, last 5 minutes, last 15 minutes. You can see the average and median values of the rate. With collectd, ganglia, graphite (for example) one can easily visualize the history of the various values over time allowing one to determine when a particular behavior (like rate of Error level logs to log4j) started to be a problem and hopefully then consult the logs and see what requests or changes to the system occurred then.

Also if there are connection errors to the remote site after a certain
number of attempts then an email is sent to the GN administrator as well
as the remote site’s administrator.

Right now if an error occurs during harvesting it is propagated up and the harvester error counter is incremented. If that is not enough, then can you give me a bit more of a concrete example of the code monitored and how you would use it?

Thanks.

John Hockaday

-----Original Message-----
From: Jesse Eichar [mailto:jesse.eichar@anonymised.com]
Sent: Monday, April 02, 2012 6:39 PM
To: Devel geonetwork-devel@lists.sourceforge.net; Heikki
Doeleman; Francois Prunayre; Jose Garcia
Subject: [GeoNetwork-devel] Vote for Health monitoring proposal

Hi,

I have the health monitoring work done and ready for commit.
Could we vote on it? Make improvement suggestions?

http://trac.osgeo.org/geonetwork/wiki/HealthMonitoring

Thanks Jesse

The information contained in this e-mail, and any attachments to it,
is intended for the use of the addressee and is confidential. If you
are not the intended recipient you must not use, disclose, read,
forward, copy or retain any of the information. If you received this
e-mail in error, please delete it and notify the sender by return
e-mail or telephone.

The Commonwealth does not warrant that any attachments are free
from viruses or any other defects. You assume all liability for any
loss, damage or other consequences which may arise from opening
or using the attachments.

The security of emails transmitted in an unencrypted environment
cannot be guaranteed. By forwarding or replying to this email, you
acknowledge and accept these risks.

11

+1 from me too!

Cheers,
Patrizia
________________________________________
From: Jesse Eichar [jesse.eichar@anonymised.com]
Sent: Monday, 2 April 2012 8:38 PM
To: Jeroen Ticheler
Cc: Devel geonetwork-devel@lists.sourceforge.net
Subject: Re: [GeoNetwork-devel] Vote for Health monitoring proposal

I have changed the status to "Proposed for vote" and added an associated ticket for the new feature.

Jesse

On Mon, Apr 2, 2012 at 11:35 AM, Jeroen Ticheler <jeroen.ticheler@anonymised.com<mailto:jeroen.ticheler@anonymised.com>> wrote: Hi Jesse, I suggest you put this proposal forward for voting. It is a very welcome utility indeed! Cheers, Jeroen

____________________________________________________
GeoCat
introduces Bridge(c)
An extension to ArcGIS(c) to instantly publish data and metadata on GeoServer and GeoNetwork. See http://geocat.net/&gt; for more details.

Jeroen Ticheler
GeoCat bv
Veenderweg 13
6721 WD Bennekom
The Netherlands
Tel: +31 (0)6 81286572<tel:%2B31%20%280%296%2081286572>
http://geocat.net

Please consider the environment before printing this email.

On 2 apr 2012, at 10:39, Jesse Eichar wrote:

Hi,

I have the health monitoring work done and ready for commit. Could we vote on it? Make improvement suggestions?

http://trac.osgeo.org/geonetwork/wiki/HealthMonitoring

Thanks Jesse

------------------------------------------------------------------------------
This SF email is sponsosred by:
Try Windows Azure free for 90 days Click Here http://p.sf.net/sfu/sfd2d-msazure_______________________________________________
GeoNetwork-devel mailing list GeoNetwork-devel@lists.sourceforge.net<mailto:GeoNetwork-devel@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

------------------------------------------------------------------------------
Better than sec? Nothing is better than sec when it comes to monitoring Big Data applications. Try Boundary one-second
resolution app monitoring today. Free. http://p.sf.net/sfu/Boundary-dev2dev
_______________________________________________
GeoNetwork-devel mailing list GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

12

Cool stuff, +1 for me too.

  Ciao,
  Emanuele

Alle 14:55:41 di Monday 2 April 2012, Jesse Eichar ha scritto:

The github link is on
http://trac.osgeo.org/geonetwork/wiki/HealthMonitoring

But for your convenience I have attached a patch to
http://trac.osgeo.org/geonetwork/ticket/846 with the current diff.

Jesse

On Mon, Apr 2, 2012 at 2:44 PM, Francois Prunayre

<fx.prunayre@anonymised.com>wrote:

> Hello Jesse, do you have any patch to have a look to the changes ? or any
> git magic command ?
>
> Thanks.
>
> Francois
>
> Le 2 avril 2012 12:38, Jesse Eichar <jesse.eichar@anonymised.com> a écrit
>
>
> I have changed the status to "Proposed for vote" and added an associated
>
>> ticket for the new feature.
>>
>> Jesse
>>