[GeoNetwork-devel] Indexing in 2.10

Hi

The indexing in 2.10 if I’m not confused was changed to be asych, but this causes some bizarre results depending how fast the index happens:

Sometimes changing the title and saving the metadata, the results list is displaying still the old title. Is possible to revert to synch indexing using some config property? Possibly a bit slower, but more consistent I think.

Regards,
Jose García


GeoCat Bridge for ArcGIS allows instant publishing of data and metadata on GeoServer and GeoNetwork. Visit http://geocat.net for details.


Jose García
GeoCat bv
Veenderweg 13
6721 WD Bennekom
The Netherlands
http://GeoCat.net

Hi Jose,

I did some work on develop and believe I figured out the problem. It turns out to mainly be the fault of a single method call:

https://github.com/geonetwork/core-geonetwork/blob/2.10.x/web/src/main/java/org/fao/geonet/kernel/search/index/GeonetworkNRTManager.java#L117

Is the 2.10 version.

Compare it to the develop version:

https://github.com/geonetwork/core-geonetwork/blob/develop/core/src/main/java/org/fao/geonet/kernel/search/index/GeonetworkNRTManager.java#L118

The call should be maybeRefreshBlocking() to get the behaviour you want.

Jesse

···

On Thu, Mar 6, 2014 at 11:50 AM, Jose Garcia <jose.garcia@anonymised.com> wrote:

Hi

The indexing in 2.10 if I’m not confused was changed to be asych, but this causes some bizarre results depending how fast the index happens:

Sometimes changing the title and saving the metadata, the results list is displaying still the old title. Is possible to revert to synch indexing using some config property? Possibly a bit slower, but more consistent I think.

Regards,
Jose García


GeoCat Bridge for ArcGIS allows instant publishing of data and metadata on GeoServer and GeoNetwork. Visit http://geocat.net for details.


Jose García
GeoCat bv
Veenderweg 13
6721 WD Bennekom
The Netherlands
http://GeoCat.net


Subversion Kills Productivity. Get off Subversion & Make the Move to Perforce.
With Perforce, you get hassle-free workflows. Merge that actually works.
Faster operations. Version large binaries. Built-in WAN optimization and the
freedom to use Git, Perforce or both. Make the move to Perforce.
http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk


GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

Hi Jesse

Thanks, I had to set this setting to false in config-lucene.xml:

false

I understand that if set to false no NRT features are used? Do we have any performance statistics about using NRT or not? Would be good to know if disabling NRT is going to be very bad.

The issue described is not very nice, at least in catalogues where users are going to edit metadata seem better to disable NRT (if performance is not that bad) to avoid the issue described, what do you think?

Regards,
Jose García

···

On Thu, Mar 6, 2014 at 3:20 PM, Jesse Eichar <jesse.eichar@anonymised.com> wrote:

Hi Jose,

I did some work on develop and believe I figured out the problem. It turns out to mainly be the fault of a single method call:

https://github.com/geonetwork/core-geonetwork/blob/2.10.x/web/src/main/java/org/fao/geonet/kernel/search/index/GeonetworkNRTManager.java#L117

Is the 2.10 version.

Compare it to the develop version:

https://github.com/geonetwork/core-geonetwork/blob/develop/core/src/main/java/org/fao/geonet/kernel/search/index/GeonetworkNRTManager.java#L118

The call should be maybeRefreshBlocking() to get the behaviour you want.

Jesse


GeoCat Bridge for ArcGIS allows instant publishing of data and metadata on GeoServer and GeoNetwork. Visit http://geocat.net for details.


Jose García
GeoCat bv
Veenderweg 13
6721 WD Bennekom
The Netherlands
http://GeoCat.net

On Thu, Mar 6, 2014 at 11:50 AM, Jose Garcia <jose.garcia@anonymised.com> wrote:

Hi

The indexing in 2.10 if I’m not confused was changed to be asych, but this causes some bizarre results depending how fast the index happens:

Sometimes changing the title and saving the metadata, the results list is displaying still the old title. Is possible to revert to synch indexing using some config property? Possibly a bit slower, but more consistent I think.

Regards,
Jose García


GeoCat Bridge for ArcGIS allows instant publishing of data and metadata on GeoServer and GeoNetwork. Visit http://geocat.net for details.


Jose García
GeoCat bv
Veenderweg 13
6721 WD Bennekom
The Netherlands
http://GeoCat.net


Subversion Kills Productivity. Get off Subversion & Make the Move to Perforce.
With Perforce, you get hassle-free workflows. Merge that actually works.
Faster operations. Version large binaries. Built-in WAN optimization and the
freedom to use Git, Perforce or both. Make the move to Perforce.
http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk


GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

On Fri, Mar 7, 2014 at 10:43 AM, Jose Garcia <jose.garcia@anonymised.com> wrote:

Hi Jesse

Thanks, I had to set this setting to false in config-lucene.xml:

<useNRTManagerReopenThread>false</useNRTManagerReopenThread>

That is right.

An additional point. On develope we have changed the index API to force
refreshing of the readers. for example DataManager.indexMetadata has a
forceRefresh parameter. So now many operations force the readers to be
refreshed.

I understand that if set to false no NRT features are used? Do we have any
performance statistics about using NRT or not? Would be good to know if
disabling NRT is going to be very bad.

It will make a difference when the number of concurrent users is high.
Probably not critical for many of our systems. But that is where the risk
is. Develop handles this much better.

The issue described is not very nice, at least in catalogues where users
are going to edit metadata seem better to disable NRT (if performance is
not that bad) to avoid the issue described, what do you think?

for 2.10 I would make the fix I mentioned and disable NRT. I think for
most of our systems it will perform acceptably.

Jesse