[GeoNetwork-devel] Difference between "indexed" and "stored" field ?

GeoNetwork GeoNetwork Developer
1

Hi,

In “index-fields.xsl” file of a schema’s folder, there is “index” and “store” attributes on each field.
But what is the difference between an “indexed” field and a “stored” field ?

Best Regards

2

these terms come from Lucene, the library used to do the indexing and retrieval.

Maybe this is helpful: http://wiki.apache.org/lucene-java/LuceneFAQ#What_is_the_different_between_Stored.2C_Tokenized.2C_Indexed.2C_and_Vector.3F

kind regards
Heikki Doeleman

On Wed, May 11, 2011 at 11:35 AM, Rudy Commenge <rudywi.devel@anonymised.com> wrote:

Hi,

In “index-fields.xsl” file of a schema’s folder, there is “index” and “store” attributes on each field.
But what is the difference between an “indexed” field and a “stored” field ?

Best Regards

Achieve unprecedented app performance and reliability
What every C/C++ and Fortran developer should know.
Learn how Intel has extended the reach of its next-generation tools
to help boost performance applications - inlcuding clusters.
http://p.sf.net/sfu/intel-dev2devmay

GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

3

Ok, but I don’t understand :
“Stored = as-is value stored in the Lucene index”
“Indexed = the text (…) is made searchable”

I thought “store something in lucene index” and “index something in lucene” were synonyms.
In fact, when I index something in lucene, it is searchable. Whereas, when I store something in lucene index, it is not searchable, right ? So, if I can’t search the data, why store it in lucene index ?

2011/5/11 heikki <tropicano@anonymised.com>

these terms come from Lucene, the library used to do the indexing and retrieval.

Maybe this is helpful: http://wiki.apache.org/lucene-java/LuceneFAQ#What_is_the_different_between_Stored.2C_Tokenized.2C_Indexed.2C_and_Vector.3F

kind regards
Heikki Doeleman

On Wed, May 11, 2011 at 11:35 AM, Rudy Commenge <rudywi.devel@anonymised.com> wrote:

Hi,

In “index-fields.xsl” file of a schema’s folder, there is “index” and “store” attributes on each field.
But what is the difference between an “indexed” field and a “stored” field ?

Best Regards

Achieve unprecedented app performance and reliability
What every C/C++ and Fortran developer should know.
Learn how Intel has extended the reach of its next-generation tools
to help boost performance applications - inlcuding clusters.
http://p.sf.net/sfu/intel-dev2devmay

GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

4

2011/5/11 Rudy Commenge <rudywi.devel@anonymised.com>:

Ok, but I don't understand :
"Stored = as-is value stored in the Lucene index"
"Indexed = the text (...) is made searchable"

I thought "store something in lucene index" and "index something in lucene"
were synonyms.
In fact, when I index something in lucene, it is searchable. Whereas, when I
store something in lucene index, it is not searchable, right ? So, if I
can't search the data, why store it in lucene index ?

Because you could retrieve the field as-is (ie. not analyzed) from the
index (without the need to retrieve the original document, in our
case, from the xml doc stored in the db).

This could be used to speed up search results formatting (see
experimental search service
http://trac.osgeo.org/geonetwork/ticket/485).

Ciao.

Francois

2011/5/11 heikki <tropicano@anonymised.com>

these terms come from Lucene, the library used to do the indexing and
retrieval.

Maybe this is helpful:
http://wiki.apache.org/lucene-java/LuceneFAQ#What_is_the_different_between_Stored.2C_Tokenized.2C_Indexed.2C_and_Vector.3F

kind regards
Heikki Doeleman

On Wed, May 11, 2011 at 11:35 AM, Rudy Commenge <rudywi.devel@anonymised.com>
wrote:

Hi,

In "index-fields.xsl" file of a schema's folder, there is "index" and
"store" attributes on each field.
But what is the difference between an "indexed" field and a "stored"
field ?

Best Regards

------------------------------------------------------------------------------
Achieve unprecedented app performance and reliability
What every C/C++ and Fortran developer should know.
Learn how Intel has extended the reach of its next-generation tools
to help boost performance applications - inlcuding clusters.
http://p.sf.net/sfu/intel-dev2devmay
_______________________________________________
GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at
http://sourceforge.net/projects/geonetwork

------------------------------------------------------------------------------
Achieve unprecedented app performance and reliability
What every C/C++ and Fortran developer should know.
Learn how Intel has extended the reach of its next-generation tools
to help boost performance applications - inlcuding clusters.
http://p.sf.net/sfu/intel-dev2devmay
_______________________________________________
GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at
http://sourceforge.net/projects/geonetwork

5

Ok, I begin to understand. Thx.
Where is the Lucene index ? Can I to explore its content ?

2011/5/11 Francois Prunayre <fx.prunayre@anonymised.com>

2011/5/11 Rudy Commenge <rudywi.devel@anonymised.com>:

Ok, but I don’t understand :
“Stored = as-is value stored in the Lucene index”
“Indexed = the text (…) is made searchable”

I thought “store something in lucene index” and “index something in lucene”
were synonyms.
In fact, when I index something in lucene, it is searchable. Whereas, when I
store something in lucene index, it is not searchable, right ? So, if I
can’t search the data, why store it in lucene index ?

Because you could retrieve the field as-is (ie. not analyzed) from the
index (without the need to retrieve the original document, in our
case, from the xml doc stored in the db).

This could be used to speed up search results formatting (see
experimental search service
http://trac.osgeo.org/geonetwork/ticket/485).

Ciao.

Francois

2011/5/11 heikki <tropicano@anonymised.com…31…>

these terms come from Lucene, the library used to do the indexing and
retrieval.

Maybe this is helpful:
http://wiki.apache.org/lucene-java/LuceneFAQ#What_is_the_different_between_Stored.2C_Tokenized.2C_Indexed.2C_and_Vector.3F

kind regards
Heikki Doeleman

On Wed, May 11, 2011 at 11:35 AM, Rudy Commenge <rudywi.devel@anonymised.com>
wrote:

Hi,

In “index-fields.xsl” file of a schema’s folder, there is “index” and
“store” attributes on each field.
But what is the difference between an “indexed” field and a “stored”
field ?

Best Regards

Achieve unprecedented app performance and reliability
What every C/C++ and Fortran developer should know.
Learn how Intel has extended the reach of its next-generation tools
to help boost performance applications - inlcuding clusters.
http://p.sf.net/sfu/intel-dev2devmay

GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at
http://sourceforge.net/projects/geonetwork

Achieve unprecedented app performance and reliability
What every C/C++ and Fortran developer should know.
Learn how Intel has extended the reach of its next-generation tools
to help boost performance applications - inlcuding clusters.
http://p.sf.net/sfu/intel-dev2devmay

GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at
http://sourceforge.net/projects/geonetwork

6

Rudy,

to explore the content of the Lucene index, use Luke : http://code.google.com/p/luke.

The index is by default in /web/geonetwork/WEB-INF/lucene/nonspatial.

Kind regards
Heikki Doeleman

On Thu, May 12, 2011 at 11:16 AM, Rudy Commenge <rudywi.devel@anonymised.com> wrote:

Ok, I begin to understand. Thx.
Where is the Lucene index ? Can I to explore its content ?

2011/5/11 Francois Prunayre <fx.prunayre@anonymised.com>

2011/5/11 Rudy Commenge <rudywi.devel@anonymised.com>:

Ok, but I don’t understand :
“Stored = as-is value stored in the Lucene index”
“Indexed = the text (…) is made searchable”

I thought “store something in lucene index” and “index something in lucene”
were synonyms.
In fact, when I index something in lucene, it is searchable. Whereas, when I
store something in lucene index, it is not searchable, right ? So, if I
can’t search the data, why store it in lucene index ?

Because you could retrieve the field as-is (ie. not analyzed) from the
index (without the need to retrieve the original document, in our
case, from the xml doc stored in the db).

This could be used to speed up search results formatting (see
experimental search service
http://trac.osgeo.org/geonetwork/ticket/485).

Ciao.

Francois

2011/5/11 heikki <tropicano@anonymised.com>

these terms come from Lucene, the library used to do the indexing and
retrieval.

Maybe this is helpful:
http://wiki.apache.org/lucene-java/LuceneFAQ#What_is_the_different_between_Stored.2C_Tokenized.2C_Indexed.2C_and_Vector.3F

kind regards
Heikki Doeleman

On Wed, May 11, 2011 at 11:35 AM, Rudy Commenge <rudywi.devel@anonymised.com>
wrote:

Hi,

In “index-fields.xsl” file of a schema’s folder, there is “index” and
“store” attributes on each field.
But what is the difference between an “indexed” field and a “stored”
field ?

Best Regards

Achieve unprecedented app performance and reliability
What every C/C++ and Fortran developer should know.
Learn how Intel has extended the reach of its next-generation tools
to help boost performance applications - inlcuding clusters.
http://p.sf.net/sfu/intel-dev2devmay

GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at
http://sourceforge.net/projects/geonetwork

Achieve unprecedented app performance and reliability
What every C/C++ and Fortran developer should know.
Learn how Intel has extended the reach of its next-generation tools
to help boost performance applications - inlcuding clusters.
http://p.sf.net/sfu/intel-dev2devmay

GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at
http://sourceforge.net/projects/geonetwork

7

OK, it works very well. Thx.

I have created a new metadata schema, and I have indexed its fields using its “index-fields.xsl” file.
I have inserted some metadata with this schema.
Considering, I have a “myfield” field in this schema, how to use GeoNetwork search area or search service (xml.search) to retrieve data where value of “myfield” is equal to “myvalue” ?

2011/5/12 heikki <tropicano@anonymised.com>

Rudy,

to explore the content of the Lucene index, use Luke : http://code.google.com/p/luke.

The index is by default in /web/geonetwork/WEB-INF/lucene/nonspatial.

Kind regards
Heikki Doeleman

On Thu, May 12, 2011 at 11:16 AM, Rudy Commenge <rudywi.devel@anonymised.com> wrote:

Ok, I begin to understand. Thx.
Where is the Lucene index ? Can I to explore its content ?

2011/5/11 Francois Prunayre <fx.prunayre@anonymised.com>

2011/5/11 Rudy Commenge <rudywi.devel@anonymised.com>:

Ok, but I don’t understand :
“Stored = as-is value stored in the Lucene index”
“Indexed = the text (…) is made searchable”

I thought “store something in lucene index” and “index something in lucene”
were synonyms.
In fact, when I index something in lucene, it is searchable. Whereas, when I
store something in lucene index, it is not searchable, right ? So, if I
can’t search the data, why store it in lucene index ?

Because you could retrieve the field as-is (ie. not analyzed) from the
index (without the need to retrieve the original document, in our
case, from the xml doc stored in the db).

This could be used to speed up search results formatting (see
experimental search service
http://trac.osgeo.org/geonetwork/ticket/485).

Ciao.

Francois

2011/5/11 heikki <tropicano@anonymised.com>

these terms come from Lucene, the library used to do the indexing and
retrieval.

Maybe this is helpful:
http://wiki.apache.org/lucene-java/LuceneFAQ#What_is_the_different_between_Stored.2C_Tokenized.2C_Indexed.2C_and_Vector.3F

kind regards
Heikki Doeleman

On Wed, May 11, 2011 at 11:35 AM, Rudy Commenge <rudywi.devel@anonymised.com>
wrote:

Hi,

In “index-fields.xsl” file of a schema’s folder, there is “index” and
“store” attributes on each field.
But what is the difference between an “indexed” field and a “stored”
field ?

Best Regards

Achieve unprecedented app performance and reliability
What every C/C++ and Fortran developer should know.
Learn how Intel has extended the reach of its next-generation tools
to help boost performance applications - inlcuding clusters.
http://p.sf.net/sfu/intel-dev2devmay

GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at
http://sourceforge.net/projects/geonetwork

Achieve unprecedented app performance and reliability
What every C/C++ and Fortran developer should know.
Learn how Intel has extended the reach of its next-generation tools
to help boost performance applications - inlcuding clusters.
http://p.sf.net/sfu/intel-dev2devmay

GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at
http://sourceforge.net/projects/geonetwork