Dear all,
sorry for the cross posting on both mailing list … but well App-Schema “as a leg” on both projects
and this may be interesting for people on both mailing lists.
… and my apologies for the wall of text that follows … what can I say complex features are really
a complex subject 
Lately I have been working in integrating App-Schema with Apache Solr [1] which is a well know
solution for indexing and (faceting) searching document.
We already have an Apache Solr store in both GT and GS, configuring a layer for it requires the user
to provide some extra information, e.g. which index fields should be considered, should multivalued
fields be used, should empty fields be used and which is attribute should be considered the
default geometry.
The first step for the integration between Apache Solr and App-Schema is to allow App-Schema to
use Apache Solr has it one of its data store. When in the context of App-schema we already have
most of the information we need, we already say indirectly through the mappings which fields of
the index we are interested in, which ones are multivalued (), etc …
The Apache-Solr data store configuration doesn’t fit in the usual MAP parameters configuration
model … and we also need to automatically detect during the parsing of the data stores
configuration that we are dealing with an Apache Solr data store and do all the possible
automatic steps.
I have come up with the following syntax for the Apache Sorl data store configuration, please find
the complete mappings file attached to this mail (mappings_solr.xml):
(...)
<sourceDataStores>
<SolrDataStore>
<id>stations</id>
<url>[http://localhost:8983/solr/stations](http://localhost:8983/solr/stations)</url>
<index name="stations">
<geometry default="true">
<name>station_location</name>
<srid>4326</srid>
<type>POINT</type>
</geometry>
</index>
</SolrDataStore>
</sourceDataStores>
(...)
All the other necessary information will be automatically inferred from the App-Schema mappings
file when possible, at the exception of multivalued fields. An Apache Solr multivalued field
handles \ contains multiple values for the same attribute.
Apache Solr multivalued fields are related with App-Schema multivalued fields, i.e. simple attributes
(String, numeric, dates, etc …) that can be encoded multiple times. Currently in App-Schema
this attributes can be mapped using a denormalized data store (database) or using a specific
feature chaining syntax when client properties are involved.
Here I would like to extend the App-Schema notion of multivalued attributes and allow each data
data store to contribute a specific handler with a custom syntax has needed:
- Apache Solr could use multivalued attributes
- MongoDB could use JSON arrays
- SQL could define explicit foreign key relation
Please find full examples of mappings using custom handlers attached to this mail (multi_*). For SQL
data stores the syntax looks like this:
(...)
<AttributeMapping>
<targetAttribute>st:comment</targetAttribute>
<jdbcMultipleValue>
<sourceColumn>id</sourceColumn>
<targetTable>meteo_stations_comments</targetTable>
<targetColumn>station_id</targetColumn>
<targetValue>comment</targetValue>
</jdbcMultipleValue>
</AttributeMapping>
(...)
A station can have multiple comment associated to it, this comment are stored in meteo_stations_comments
table which has a foreign key to stations table base on columns id and station_id. The target value can be an
OCQL expression.
In the case of Apache Solr the syntax would look like this:
(...)
<AttributeMapping>
<targetAttribute>st:comment</targetAttribute>
<solrMultipleValue>station_comment</solrMultipleValue>
</AttributeMapping>
(...)
I have a prototype of all of this already working, all of the things above required creating a few extensions
points in App-Schema:
- in the mappings configuration parsing module to handle the custom syntaxes of each data store
- in the App-Schema feature iterator to give a chance to custom handlers to do their job
This changes open the doors to the usage of non relational data stores with App-Schema and IMHO this syntaxes
promote \ allow clearer mappings files, of curse this changes and new syntaxes will need to be clearly documented
in the App-Schema documentation.
All comments about this are welcome.
Kind regards,
Nuno Oliveira
(attachments)
stations_solr.xml (2.18 KB)
multi_stations_solr.xml (2.59 KB)
multi_stations_sql.xml (5.31 KB)
···
--
Regards,
Nuno Oliveira
==
GeoServer Professional Services from the experts! Visit [http://goo.gl/it488V](http://goo.gl/it488V) for more information.
==
Nuno Miguel Carvalho Oliveira
@nmcoliveira
Software Engineer
GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
[http://www.geo-solutions.it](http://www.geo-solutions.it)
[http://twitter.com/geosolutions_it](http://twitter.com/geosolutions_it)
-------------------------------------------------------
AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.
The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.
