#566: Use ScheduledExecutorService instead of Timer for index metadata tasks
-------------------------+--------------------------------------------------
Reporter: jesseeichar | Owner: geonetwork-devel@…
Type: defect | Status: new
Priority: major | Milestone: v2.6.5
Component: General | Version: v2.6.3
Keywords: |
-------------------------+--------------------------------------------------
Right now when a metadata needs to be indexed asynchonously a new Timer
object and IndexMetadataTask are created. This has the potential to blow
up into hundreds of Timer objects when doing a full re-index.
The ScheduledThreadExecutor allows the number of threads to be controlled
through configuration.
I have a patch that makes the change and the number of threads are
configured in the config.xml file. The default is 4 threads. Each thread
has the name "Index Task Thread" and normal thread priority. Thread
priority can also be configured in config.xml.
--
Ticket URL: <http://trac.osgeo.org/geonetwork/ticket/566>
GeoNetwork opensource Developer website <http://sourceforge.net/projects/geonetwork/>
GeoNetwork opensource is a standards based, Free and Open Source catalog application to manage spatially referenced resources through the web. It provides powerful metadata editing and search functions as well as an embedded interactive web map viewer. This website contains information related to the development of the software.
Hi Jesse,
I don't think it's quite as potentially explosive as that! 
By my reading, one IndexMetadataTask is created for rebuilding the entire metadata index (or those records in it that have XLinks if that is enabled) from the index rebuilding functions on the Admin page/or on startup. I don't see IndexMetadataTasks being created to index individual records (check the constructor in DataManager.java and the rebuildIndex method in SearchManager.java).
That said, I also think it should use a more sophisticated and modern approach such as the one you've suggested. I've been implementing something similar to your patch for speeding up indexing of batch imports and batch/massive ops - so maybe we can combine our work? (see http://geonetwork.svn.sourceforge.net/geonetwork/?rev=8053&view=rev) Looks like we could share the same/similar config options etc.
Cheers and thanks,
Simon
________________________________________
From: GeoNetwork opensource Developer website [trac_osgeo@anonymised.com]
Sent: Tuesday, 2 August 2011 7:08 PM
Subject: [GeoNetwork-devel] [GeoNetwork opensource Developer website] #566: Use ScheduledExecutorService instead of Timer for index metadata tasks
#566: Use ScheduledExecutorService instead of Timer for index metadata tasks
-------------------------+--------------------------------------------------
Reporter: jesseeichar | Owner: geonetwork-devel@…
Type: defect | Status: new
Priority: major | Milestone: v2.6.5
Component: General | Version: v2.6.3
Keywords: |
-------------------------+--------------------------------------------------
Right now when a metadata needs to be indexed asynchonously a new Timer
object and IndexMetadataTask are created. This has the potential to blow
up into hundreds of Timer objects when doing a full re-index.
The ScheduledThreadExecutor allows the number of threads to be controlled
through configuration.
I have a patch that makes the change and the number of threads are
configured in the config.xml file. The default is 4 threads. Each thread
has the name "Index Task Thread" and normal thread priority. Thread
priority can also be configured in config.xml.
--
Ticket URL: <http://trac.osgeo.org/geonetwork/ticket/566>
GeoNetwork opensource Developer website <http://sourceforge.net/projects/geonetwork/>
GeoNetwork opensource is a standards based, Free and Open Source catalog application to manage spatially referenced resources through the web. It provides powerful metadata editing and search functions as well as an embedded interactive web map viewer. This website contains information related to the development of the software.
------------------------------------------------------------------------------
BlackBerry® DevCon Americas, Oct. 18-20, San Francisco, CA
The must-attend event for mobile developers. Connect with experts.
Get tools for creating Super Apps. See the latest technologies.
Sessions, hands-on labs, demos & much more. Register early & save!
http://p.sf.net/sfu/rim-blackberry-1
_______________________________________________
GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork
#566: Use ScheduledExecutorService instead of Timer for index metadata tasks
--------------------------+-------------------------------------------------
Reporter: jesseeichar | Owner: geonetwork-devel@…
Type: defect | Status: closed
Priority: major | Milestone: v2.6.5
Component: General | Version: v2.6.3
Resolution: fixed | Keywords:
--------------------------+-------------------------------------------------
Changes (by jesseeichar):
* status: new => closed
* resolution: => fixed
Comment:
A thread pool was introduced which fixes the issue
--
Ticket URL: <http://trac.osgeo.org/geonetwork/ticket/566#comment:1>
GeoNetwork opensource Developer website <http://sourceforge.net/projects/geonetwork/>
GeoNetwork opensource is a standards based, Free and Open Source catalog application to manage spatially referenced resources through the web. It provides powerful metadata editing and search functions as well as an embedded interactive web map viewer. This website contains information related to the development of the software.