[GeoNetwork-devel] CFV: Proposal: Performance Enhancements for supporting larger catalogs (10s or 100s of thousands of records)

Dear PSC members,

Proposal to apply performance enhancements to GeoNetwork trunk (2.5 Unstable) for supporting larger catalogs (10s or 100s of thousands of records) is available at:

http://trac.osgeo.org/geonetwork/wiki/PerformanceEnhancements2

A patch is attached to the proposal.

Effects of this proposal in brief for users of larger catalogs:

- speed up startup time
- speed up index building used by massive ops, harvesters, csw transaction, rebuild index, group delete, category delete, bulk import
- speed up returning search result to user (for large result sets) by introducing configurable limit on number of results used in search summary builder

Other effects:

- postgis can be specified as a gast database option and will then be used for spatial index (by CSW spatial searches) - adapted from work of Jessie Eichar
- system admin can configure the maximum size of 'select all' in Administration>System Configuration
- system admin can configure how often the background lucene optimizer task runs in Administration>System Configuration
- optimize lucene index now operation in Administration menu

Not related to Performance Enhancements but in the patch:

- elementName support introduced for CSW

Jar updates:

geotools 2.6.2
lucene 2.9.2
geoapi 2.3

Thanks to all contributors to this patch - especially Jose Garcia and Francois-Xavier Prunayre!

Cheers,
Simon

+1 for me Simon. Great work !

Francois

2010/3/29 <Simon.Pigot@anonymised.com>:

Dear PSC members,

Proposal to apply performance enhancements to GeoNetwork trunk (2.5 Unstable) for supporting larger catalogs (10s or 100s of thousands of records) is available at:

http://trac.osgeo.org/geonetwork/wiki/PerformanceEnhancements2

A patch is attached to the proposal.

Effects of this proposal in brief for users of larger catalogs:

- speed up startup time
- speed up index building used by massive ops, harvesters, csw transaction, rebuild index, group delete, category delete, bulk import
- speed up returning search result to user (for large result sets) by introducing configurable limit on number of results used in search summary builder

Other effects:

- postgis can be specified as a gast database option and will then be used for spatial index (by CSW spatial searches) - adapted from work of Jessie Eichar
- system admin can configure the maximum size of 'select all' in Administration>System Configuration
- system admin can configure how often the background lucene optimizer task runs in Administration>System Configuration
- optimize lucene index now operation in Administration menu

Not related to Performance Enhancements but in the patch:

- elementName support introduced for CSW

Jar updates:

geotools 2.6.2
lucene 2.9.2
geoapi 2.3

Thanks to all contributors to this patch - especially Jose Garcia and Francois-Xavier Prunayre!

Cheers,
Simon
------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

Hi Simon,

Excellent work! Congrats to all participants and to you to pull it all together!

+1 from me.
Ciao,
Jeroen

On 29 mrt 2010, at 11:54, <Simon.Pigot@anonymised.com> <Simon.Pigot@anonymised.com> wrote:

Dear PSC members,

Proposal to apply performance enhancements to GeoNetwork trunk (2.5 Unstable) for supporting larger catalogs (10s or 100s of thousands of records) is available at:

PerformanceEnhancements2 – GeoNetwork opensource Developer website

A patch is attached to the proposal.

Effects of this proposal in brief for users of larger catalogs:

- speed up startup time
- speed up index building used by massive ops, harvesters, csw transaction, rebuild index, group delete, category delete, bulk import
- speed up returning search result to user (for large result sets) by introducing configurable limit on number of results used in search summary builder

Other effects:

- postgis can be specified as a gast database option and will then be used for spatial index (by CSW spatial searches) - adapted from work of Jessie Eichar
- system admin can configure the maximum size of 'select all' in Administration>System Configuration
- system admin can configure how often the background lucene optimizer task runs in Administration>System Configuration
- optimize lucene index now operation in Administration menu

Not related to Performance Enhancements but in the patch:

- elementName support introduced for CSW

Jar updates:

geotools 2.6.2
lucene 2.9.2
geoapi 2.3

Thanks to all contributors to this patch - especially Jose Garcia and Francois-Xavier Prunayre!

Cheers,
Simon
------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork