[GeoNetwork-users] Geonetwork 2.4 re-indexing every time on start-up

Heikki_Doeleman · August 14, 2009, 7:15pm

hello lists,

I'm having the following behaviour : everytime I re-start a GeoNetwork
installation, it starts rebuilding the Lucene index, even though there
already exists a Lucene index that's not empty and up-to-date.

I've this same issue using GeoNetwork 2.4 (both when built from SVN or when
installed from the installer distro) and also using a modified version of
Geonetwork 2.2.

I'm using Windows XP and my Java version is

java version "1.6.0_15"
Java(TM) SE Runtime Environment (build 1.6.0_15-b03)
Java HotSpot(TM) Client VM (build 14.1-b02, mixed mode, sharing)

The indexes that already exist when GN starts, are valid Luke indexes, which
I can successfully view using Luke.

I've tried running a GN version with some extra debug logging, and
SearchManager.java, when opening a Lucene IndexReader reports that the index
exists, that the index reader is current, and (somehow) that the number of
docs in the index is 0 (which is not true and causes the re-indexing).

I don't remember seeing this before. Did anyone else see this ? Could it be
because of some Java or Windows XP update, that all of a sudden I'm getting
this behaviour ?

Your comments welcome

kind regards
Heikki Doeleman

Heikki_Doeleman · August 14, 2009, 8:06pm

OK -- testing again, I only have it using GN2.4, not GN2.2.

But it's consistently reproduceable for GN2.4. So I guess I can rule out
that it could have anything to do with Java or XP, instead it must be
something about the index reading / writing introduced with 2.4. I tried it
with both a MySQL and a Postgres DB, and that doesn't make a difference (as
it shouldn't).

Still I have no clue as to what that cause might be, though.. everything
seems fine, the index does actually work perfectly fine for GN while the app
is running, and before and after killing GN it's also just fine to view the
index using Luke. When GN restarts, everything seems dandy, except that the
IndexReader thinks that there are 0 docs in the index.

I'd be happy to hear from other people whether or not you have this issue in
GN2.4.

Kind regards
Heikki Doeleman

On Fri, Aug 14, 2009 at 9:15 PM, heikki <tropicano@anonymised.com> wrote:

hello lists,

I'm having the following behaviour : everytime I re-start a GeoNetwork
installation, it starts rebuilding the Lucene index, even though there
already exists a Lucene index that's not empty and up-to-date.

I've this same issue using GeoNetwork 2.4 (both when built from SVN or when
installed from the installer distro) and also using a modified version of
Geonetwork 2.2.

I'm using Windows XP and my Java version is

java version "1.6.0_15"
Java(TM) SE Runtime Environment (build 1.6.0_15-b03)
Java HotSpot(TM) Client VM (build 14.1-b02, mixed mode, sharing)

The indexes that already exist when GN starts, are valid Luke indexes,
which I can successfully view using Luke.

I've tried running a GN version with some extra debug logging, and
SearchManager.java, when opening a Lucene IndexReader reports that the index
exists, that the index reader is current, and (somehow) that the number of
docs in the index is 0 (which is not true and causes the re-indexing).

I don't remember seeing this before. Did anyone else see this ? Could it be
because of some Java or Windows XP update, that all of a sudden I'm getting
this behaviour ?

Your comments welcome

kind regards
Heikki Doeleman

andrew_walsh · August 16, 2009, 11:26pm

Heikki,

I get the same thing, on restarting the system my geonetwork.log says:

2009-08-17 09:13:25,453 ERROR [geonetwork.search] - Rebuilding lucene index

I am running GN2.4 built from the installer using Java 1.5.0_14 and Win XP SP3.

Andrew Walsh
  ----- Original Message -----
  From: heikki
  To: Devel geonetwork-devel@lists.sourceforge.net ; Geonetwork Users
  Sent: Saturday, August 15, 2009 6:06 AM
  Subject: Re: [GeoNetwork-devel] Geonetwork 2.4 re-indexing every time onstart-up

OK -- testing again, I only have it using GN2.4, not GN2.2.

But it's consistently reproduceable for GN2.4. So I guess I can rule out that it could have anything to do with Java or XP, instead it must be something about the index reading / writing introduced with 2.4. I tried it with both a MySQL and a Postgres DB, and that doesn't make a difference (as it shouldn't).

Still I have no clue as to what that cause might be, though.. everything seems fine, the index does actually work perfectly fine for GN while the app is running, and before and after killing GN it's also just fine to view the index using Luke. When GN restarts, everything seems dandy, except that the IndexReader thinks that there are 0 docs in the index.

I'd be happy to hear from other people whether or not you have this issue in GN2.4.

Kind regards
Heikki Doeleman

On Fri, Aug 14, 2009 at 9:15 PM, heikki <tropicano@anonymised.com> wrote:

hello lists,

I'm having the following behaviour : everytime I re-start a GeoNetwork installation, it starts rebuilding the Lucene index, even though there already exists a Lucene index that's not empty and up-to-date.

I've this same issue using GeoNetwork 2.4 (both when built from SVN or when installed from the installer distro) and also using a modified version of Geonetwork 2.2.

I'm using Windows XP and my Java version is

    java version "1.6.0_15"
    Java(TM) SE Runtime Environment (build 1.6.0_15-b03)
    Java HotSpot(TM) Client VM (build 14.1-b02, mixed mode, sharing)

The indexes that already exist when GN starts, are valid Luke indexes, which I can successfully view using Luke.

I've tried running a GN version with some extra debug logging, and SearchManager.java, when opening a Lucene IndexReader reports that the index exists, that the index reader is current, and (somehow) that the number of docs in the index is 0 (which is not true and causes the re-indexing).

I don't remember seeing this before. Did anyone else see this ? Could it be because of some Java or Windows XP update, that all of a sudden I'm getting this behaviour ?

Your comments welcome

kind regards
Heikki Doeleman

------------------------------------------------------------------------------

  ------------------------------------------------------------------------------
  Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
  trial. Simplify your report design, integration and deployment - and focus on
  what you do best, core application coding. Discover what's new with
  Crystal Reports now. http://p.sf.net/sfu/bobj-july

------------------------------------------------------------------------------

  _______________________________________________
  GeoNetwork-devel mailing list
  GeoNetwork-devel@lists.sourceforge.net
  https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
  GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

Francois_Prunayre · August 17, 2009, 9:03am

Hello Heikki, fixed in trunk and 2.4 branch, this bug was
unfortunately introduced with the CSW patch.

Francois

2009/8/17 awalsh <awalsh@anonymised.com>:

Heikki,

I get the same thing, on restarting the system my geonetwork.log says:

2009-08-17 09:13:25,453 ERROR [geonetwork.search] - Rebuilding lucene index

I am running GN2.4 built from the installer using Java 1.5.0_14 and Win XP
SP3.

Andrew Walsh

----- Original Message -----
From: heikki
To: Devel geonetwork-devel@lists.sourceforge.net ; Geonetwork Users
Sent: Saturday, August 15, 2009 6:06 AM
Subject: Re: [GeoNetwork-devel] Geonetwork 2.4 re-indexing every time
onstart-up
OK -- testing again, I only have it using GN2.4, not GN2.2.

But it's consistently reproduceable for GN2.4. So I guess I can rule out
that it could have anything to do with Java or XP, instead it must be
something about the index reading / writing introduced with 2.4. I tried it
with both a MySQL and a Postgres DB, and that doesn't make a difference (as
it shouldn't).

Still I have no clue as to what that cause might be, though.. everything
seems fine, the index does actually work perfectly fine for GN while the app
is running, and before and after killing GN it's also just fine to view the
index using Luke. When GN restarts, everything seems dandy, except that the
IndexReader thinks that there are 0 docs in the index.

I'd be happy to hear from other people whether or not you have this issue in
GN2.4.

Kind regards
Heikki Doeleman

On Fri, Aug 14, 2009 at 9:15 PM, heikki <tropicano@anonymised.com> wrote:

hello lists,

I'm having the following behaviour : everytime I re-start a GeoNetwork
installation, it starts rebuilding the Lucene index, even though there
already exists a Lucene index that's not empty and up-to-date.

I've this same issue using GeoNetwork 2.4 (both when built from SVN or
when installed from the installer distro) and also using a modified version
of Geonetwork 2.2.

I'm using Windows XP and my Java version is

java version "1.6.0_15"
Java(TM) SE Runtime Environment (build 1.6.0_15-b03)
Java HotSpot(TM) Client VM (build 14.1-b02, mixed mode, sharing)

The indexes that already exist when GN starts, are valid Luke indexes,
which I can successfully view using Luke.

I've tried running a GN version with some extra debug logging, and
SearchManager.java, when opening a Lucene IndexReader reports that the index
exists, that the index reader is current, and (somehow) that the number of
docs in the index is 0 (which is not true and causes the re-indexing).

I don't remember seeing this before. Did anyone else see this ? Could it
be because of some Java or Windows XP update, that all of a sudden I'm
getting this behaviour ?

Your comments welcome

kind regards
Heikki Doeleman

________________________________

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus
on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july

________________________________

_______________________________________________
GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at
http://sourceforge.net/projects/geonetwork

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus
on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at
http://sourceforge.net/projects/geonetwork

Mael_IG · August 20, 2009, 8:45am

I have a similar problem on my test machine (Windows XP SP3, java 1.5, GN /
GeoSource (french) but it always blocked on the last metadata created :

2009-08-20 10:03:50,443 DEBUG [geonetwork.datamanager] - - record (11)
2009-08-20 10:03:50,443 DEBUG [geonetwork.datamanager] - Indexing record
(11)
2009-08-20 10:03:50,505 DEBUG [geonetwork.datamanager] - record schema
(iso19139fra)
2009-08-20 10:03:50,505 DEBUG [geonetwork.datamanager] - record createDate
(2009-08-19T09:16:01)
2009-08-20 10:03:50,990 DEBUG [geonetwork.datamanager] - - record (12)
2009-08-20 10:03:50,990 DEBUG [geonetwork.datamanager] - Indexing record
(12)

The only solution consist in deleting the MD :
DELETE FROM metadata WHERE id = 12;
DELETE FROM operationallowed WHERE metadataId = 12;

Did anyone see that before ?

Maël REBOUX
--
View this message in context: http://n2.nabble.com/Geonetwork-2.4-re-indexing-every-time-on-start-up-tp3446993p3477727.html
Sent from the GeoNetwork users mailing list archive at Nabble.com.

ticheler · August 17, 2009, 10:47am

Hi Heikki,
Yes, I think we should have an IRC meeting to discuss the priority bugs we want to resolve and then make a bug fix release ASAP. Can we meet on IRC on Wednesday morning 10 AM?
Ciao,
Jeroen

On Aug 17, 2009, at 12:43 PM, heikki wrote:

BTW don't you think the fix for GN2.4 merits a new bugfix release ?

I think so; to me it's unacceptable to re-create the index at every application start-up, because it takes easy 10 minutes on a DB with 500-1000 metadata in it. Which certainly for development is very unhandy (or I should develop against a DB that doesn't have the test data I want) and als in production, imagine if the GN node has many more records, it could take many hours *every time*.

Jeroen?

Heikki_Doeleman · August 17, 2009, 10:47am

OK

On Mon, Aug 17, 2009 at 12:47 PM, Jeroen Ticheler <Jeroen.Ticheler@anonymised.com> wrote:

Hi Heikki,
Yes, I think we should have an IRC meeting to discuss the priority bugs we want to resolve and then make a bug fix release ASAP. Can we meet on IRC on Wednesday morning 10 AM?
Ciao,
Jeroen

On Aug 17, 2009, at 12:43 PM, heikki wrote:

BTW don’t you think the fix for GN2.4 merits a new bugfix release ?

I think so; to me it’s unacceptable to re-create the index at every application start-up, because it takes easy 10 minutes on a DB with 500-1000 metadata in it. Which certainly for development is very unhandy (or I should develop against a DB that doesn’t have the test data I want) and als in production, imagine if the GN node has many more records, it could take many hours every time.

Jeroen?