[Geoserver-devel] build/load Imagemosaic from existing datastore

Hi all,
I’m looking into adding this (http://jira.codehaus.org/browse/GEOT-4597) feature to the geotools/geoserver.
The intention is to add to geotools the capability to create an ImageMosaicReader using hints without the need to add or modify properties files somewhere.
The idea is:
Starting from an existing schema (or Datastore), we can navigate into its structure (validating granules) to create an imagemosaic CatalogGranule.

Essentially no accessory files are needed (*.properties) but still some configuration (those located into the datastore.properties) should be passed via Hints I believe.

The Hints.MOSAIC_LOCATION_ATTRIBUTE is still needed and will be used to store generated accessory files such as the Imagemosaic.properties file and the sample_image and is also considered the default parent dir of relative paths for granules.

The entry point will be the ImageMosaicReader constructor:

//

// Check source
//
if (source instanceof ImageMosaicDescriptor) {
initReaderFromDescriptor((ImageMosaicDescriptor) source, uHints);

} else if (checkForDatastore(uHints)){
// if Hints contains all the needed params to configure a Mosaic (datastore connection, etc)
initReaderFromDescriptor(createGranuleDescriptorFromHints(uHints), uHints);
} else {

// normal filesystem scan
initReaderFromURL(source, localHints);

}


Essentially the condition to distinguish from the classic folder based scan and the new DataStore based scan is in the content of Hints.

If we can locate all the needed information to start the navigation we assume to use the DataStore scan mode (which essentially reads from existing datastore) otherwise, normal filesystem scan is performed (which writes creating a new datastore).

A better option can be adding a new Hint option to select between the modes f.e.:

  • fs
    [the ImageMosaic create the datastore scanning the passed folder as usually]
  • datastore
    [the imagemosaic scan the datastore schema for coverages (typeNames) and recursively coverages (typeNames) for granules. Empty tables creates an empty coverage see http://jira.codehaus.org/browse/GEOT-4597]
  • auto
    [ use ‘source’ and old method to create the mosaic reader. This should be the default to preserve old workflow ]

Once the ‘datastore’ mode is selected the DataStore is created and the typeNames will be queried for granules as the ImageMosaicWalker does for the folders. Note that the ImageMosaic base dir (Hints.MOSAIC_LOCATION_ATTRIBUTE) will contain a list of folder representing typenames.

Other options can be added to recalculate the bbox for each granule.

Feedback are really appreciated

Cheers,
Carlo

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Dott. Carlo Cancellieri
@cancellieric
Software Engineer

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
mobile: +39 3371094494
fax: +39 0584 1660272

http://www.geo-solutions.it
http://twitter.com/geosolutions_it


Errata corridge:
there was a mistake with the usage of Hints.MOSAIC_LOCATION_ATTRIBUTE.
It should be intended as Utils.Prop.ROOT_MOSAIC_DIR.

···

2013/10/16 carlo cancellieri <carlo.cancellieri@anonymised.com>

Hi all,
I’m looking into adding this (http://jira.codehaus.org/browse/GEOT-4597) feature to the geotools/geoserver.
The intention is to add to geotools the capability to create an ImageMosaicReader using hints without the need to add or modify properties files somewhere.
The idea is:
Starting from an existing schema (or Datastore), we can navigate into its structure (validating granules) to create an imagemosaic CatalogGranule.

Essentially no accessory files are needed (*.properties) but still some configuration (those located into the datastore.properties) should be passed via Hints I believe.

The Hints.MOSAIC_LOCATION_ATTRIBUTE is still needed and will be used to store generated accessory files such as the Imagemosaic.properties file and the sample_image and is also considered the default parent dir of relative paths for granules.

The entry point will be the ImageMosaicReader constructor:

//

// Check source
//
if (source instanceof ImageMosaicDescriptor) {
initReaderFromDescriptor((ImageMosaicDescriptor) source, uHints);

} else if (checkForDatastore(uHints)){
// if Hints contains all the needed params to configure a Mosaic (datastore connection, etc)
initReaderFromDescriptor(createGranuleDescriptorFromHints(uHints), uHints);
} else {

// normal filesystem scan
initReaderFromURL(source, localHints);

}


Essentially the condition to distinguish from the classic folder based scan and the new DataStore based scan is in the content of Hints.

If we can locate all the needed information to start the navigation we assume to use the DataStore scan mode (which essentially reads from existing datastore) otherwise, normal filesystem scan is performed (which writes creating a new datastore).

A better option can be adding a new Hint option to select between the modes f.e.:

  • fs
    [the ImageMosaic create the datastore scanning the passed folder as usually]
  • datastore
    [the imagemosaic scan the datastore schema for coverages (typeNames) and recursively coverages (typeNames) for granules. Empty tables creates an empty coverage see http://jira.codehaus.org/browse/GEOT-4597]
  • auto
    [ use ‘source’ and old method to create the mosaic reader. This should be the default to preserve old workflow ]

Once the ‘datastore’ mode is selected the DataStore is created and the typeNames will be queried for granules as the ImageMosaicWalker does for the folders. Note that the ImageMosaic base dir (Hints.MOSAIC_LOCATION_ATTRIBUTE) will contain a list of folder representing typenames.

Other options can be added to recalculate the bbox for each granule.

Feedback are really appreciated

Cheers,
Carlo

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Dott. Carlo Cancellieri
@cancellieric
Software Engineer

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
mobile: +39 3371094494
fax: +39 0584 1660272

http://www.geo-solutions.it
http://twitter.com/geosolutions_it


==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Dott. Carlo Cancellieri
@cancellieric
Software Engineer

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
mobile: +39 3371094494
fax: +39 0584 1660272

http://www.geo-solutions.it
http://twitter.com/geosolutions_it


Ciao Carlo,
please find my feedback inline below...

Regards,
Simone Giannecchini

Our support, Your Success! Visit http://opensdi.geo-solutions.it for
more information.

Ing. Simone Giannecchini
@simogeo
Founder/Director

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 333 8128928

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

On Wed, Oct 16, 2013 at 2:52 PM, carlo cancellieri
<carlo.cancellieri@anonymised.com> wrote:

Hi all,
I'm looking into adding this (http://jira.codehaus.org/browse/GEOT-4597)
feature to the geotools/geoserver.
The intention is to add to geotools the capability to create an
ImageMosaicReader using hints without the need to add or modify properties
files somewhere.
The idea is:
Starting from an existing schema (or Datastore), we can navigate into its
structure (validating granules) to create an imagemosaic CatalogGranule.

As mentioned on the JIRA, existing table or existing datastore are
very different in terms of goal.
I would clarify this and problablys tart off an existing table.

Essentially no accessory files are needed (*.properties) but still some
configuration (those located into the datastore.properties) should be passed
via Hints I believe.

Thinking, I would rather leave things as they are and I would add a
parameter to the indexer properties
in order to instructu the catalog builder to use the content of the
table rather than scanning the directories.

We would also probably want instruct the builder to relax some
constraints in case we will never use more than 1 image at a time
(e.g. same colormodel for all imagery becomes useless).

The Hints.MOSAIC_LOCATION_ATTRIBUTE is still needed and will be used to
store generated accessory files such as the Imagemosaic.properties file and
the sample_image and is also considered the default parent dir of relative
paths for granules.

This is why I would go with modifying the indexer. In the end you
still need a writable location, therefore let's use the indexer.

The entry point will be the ImageMosaicReader constructor:
-------------------------------------------------------
        //
        // Check source
        //
        if (source instanceof ImageMosaicDescriptor) {
            initReaderFromDescriptor((ImageMosaicDescriptor) source,
uHints);

        } else if (checkForDatastore(uHints)){
            // if Hints contains all the needed params to configure a Mosaic
(datastore connection, etc)

initReaderFromDescriptor(createGranuleDescriptorFromHints(uHints), uHints);
        } else {
            ...
            // normal filesystem scan
                initReaderFromURL(source, localHints);

        }
--------------------

Essentially the condition to distinguish from the classic folder based scan
and the new DataStore based scan is in the content of Hints.

If we can locate all the needed information to start the navigation we
assume to use the DataStore scan mode (which essentially reads from existing
datastore) otherwise, normal filesystem scan is performed (which writes
creating a new datastore).

A better option can be adding a new Hint option to select between the modes
f.e.:
- fs
[the ImageMosaic create the datastore scanning the passed folder as
usually]
- datastore
[the imagemosaic scan the datastore schema for coverages (typeNames) and
recursively coverages (typeNames) for granules. Empty tables creates an
empty coverage see http://jira.codehaus.org/browse/GEOT-4597\]
- auto
[ use 'source' and old method to create the mosaic reader. This should be
the default to preserve old workflow ]

Once the 'datastore' mode is selected the DataStore is created and the
typeNames will be queried for granules as the ImageMosaicWalker does for the
folders. Note that the ImageMosaic base dir
(Hints.MOSAIC_LOCATION_ATTRIBUTE) will contain a list of folder representing
typenames.

Other options can be added to recalculate the bbox for each granule.

Interesting, we might want to go over this again, but I believe this
could still fit into the current design
with reusing the indexer.

In addition we could use a hint to pass a datastore instance to the
mosaic for the place where to add/get single featuresources
for the various indexes.

Let's discuss this asap.

Feedback are really appreciated

Cheers,
Carlo

--

Our support, Your Success! Visit http://opensdi.geo-solutions.it for more
information.

Dott. Carlo Cancellieri
@cancellieric
Software Engineer

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
mobile: +39 3371094494
fax: +39 0584 1660272

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most
from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60135031&iu=/4140/ostg.clktrk
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel