[Geoserver-users] ImageMosaic plugin update

Alexander_Petkov · August 28, 2006, 3:13pm

I managed to get the ImageMosaic plugin from the WCS branch to work.
Attached is an image with satellite imagery. Thanks to the mosaicking
plugin, these images are configured as one coverage. For me, the
advantages of this are cleaner catalog and not so many subdirectories
in my data_dir.

Notice the blend between overlapping images--the alpha channel is
respected and any uderlying imagery is visible.

Also, the vector layer on top is reprojected.

Thanks for looking,
Alex

Luca_Morandini2 · September 11, 2006, 7:10pm

Alexander Petkov wrote:

I managed to get the ImageMosaic plugin from the WCS branch to work.

I have only one question: how you did it ?

Today I've downloaded and built latest GeoTools and Geoserver WCS branch, was able to add a coverage store, but not a coverage:

8127049 [INFO] org.vfny.geoserver.servlets.AbstractService - Service handled
  8194669 [CONFIG] org.geotools.factory.FactoryRegistry - Factory implementations for category GridCoverageFactory:
   org.geotools.coverage.grid.GridCoverageFactory
Sep 11, 2006 9:05:16 PM org.apache.struts.action.RequestProcessor processException
WARNING: Unhandled Exception thrown: class java.lang.NullPointerException
21:05:16.096 WARN!! Exception for /geoserver/config/data/coverageNewSubmit.do
java.lang.NullPointerException
         at org.vfny.geoserver.action.data.DataCoveragesNewAction.execute(DataCoveragesNewAction.java:139)
         at org.vfny.geoserver.action.ConfigAction.execute(ConfigAction.java:102)
         at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:431)
         at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:236)
         at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1196)
         at org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:432)
         at javax.servlet.http.HttpServlet.service(HttpServlet.java:760)

--------------------
Luca Morandini
www.lucamorandini.it
--------------------

Alexander_Petkov · September 11, 2006, 8:41pm

I suspect that it mught be a missing .properties file for your
ImageMosaic coverage. Check the sample ImageMosaic coverage
(mosaic.properties file) for example:

=============================
#

#Thu Aug 17 14:00:58 CEST 2006

Name=mosaic

Levels=0.057934032977228433,0.04039598061277999

LevelsNum=1

Envelope2D=6.34617490847439,36.4917718219401 20.8296831527815,46.5907669751351

NumFiles=25

Here "Levels" is the cell size in x and y direction, "Envelope2D" is
the bounding box for the coverage, and "NumFiles" is the number of
rasters (tiles) in your Coverage.

Alex

On 9/11/06, Luca Morandini <lmorandini@anonymised.com> wrote:

Alexander Petkov wrote:
> I managed to get the ImageMosaic plugin from the WCS branch to work.

I have only one question: how you did it ?

Today I've downloaded and built latest GeoTools and Geoserver WCS
branch, was able to add a coverage store, but not a coverage:

8127049 [INFO] org.vfny.geoserver.servlets.AbstractService - Service handled
  8194669 [CONFIG] org.geotools.factory.FactoryRegistry - Factory
implementations for category GridCoverageFactory:
   org.geotools.coverage.grid.GridCoverageFactory
Sep 11, 2006 9:05:16 PM org.apache.struts.action.RequestProcessor
processException
WARNING: Unhandled Exception thrown: class java.lang.NullPointerException
21:05:16.096 WARN!! Exception for
/geoserver/config/data/coverageNewSubmit.do
java.lang.NullPointerException
         at
org.vfny.geoserver.action.data.DataCoveragesNewAction.execute(DataCoveragesNewAction.java:139)
         at
org.vfny.geoserver.action.ConfigAction.execute(ConfigAction.java:102)
         at
org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:431)
         at
org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:236)
         at
org.apache.struts.action.ActionServlet.process(ActionServlet.java:1196)
         at
org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:432)
         at javax.servlet.http.HttpServlet.service(HttpServlet.java:760)

--------------------
    Luca Morandini
www.lucamorandini.it
--------------------

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Luca_Morandini2 · September 11, 2006, 9:12pm

Alexander Petkov wrote:

I suspect that it mught be a missing .properties file for your
ImageMosaic coverage. Check the sample ImageMosaic coverage
(mosaic.properties file) for example:

The properties file is missing indeed, but setting up one didn't work.
Actually, I've put the following in a file named "tci250.properties" (tci250 being the name of the coveragestore):

Name=tci250
Levels=70000,48000
LevelsNum=1
Envelope2D=162000,4546000 459000,4755000
NumFiles=14

Moreover, shouldn't the plugin write it by itself ?

Regards,

--------------------
Luca Morandini
www.lucamorandini.it
--------------------

Alexander_Petkov · September 11, 2006, 9:27pm

What is the error this time? The plugin is not (yet) generating the
properties file.

Alex

On 9/11/06, Luca Morandini <lmorandini@anonymised.com> wrote:

Alexander Petkov wrote:
> I suspect that it mught be a missing .properties file for your
> ImageMosaic coverage. Check the sample ImageMosaic coverage
> (mosaic.properties file) for example:

The properties file is missing indeed, but setting up one didn't work.
Actually, I've put the following in a file named "tci250.properties"
(tci250 being the name of the coveragestore):

Name=tci250
Levels=70000,48000
LevelsNum=1
Envelope2D=162000,4546000 459000,4755000
NumFiles=14

Moreover, shouldn't the plugin write it by itself ?

Regards,

--------------------
Luca Morandini
www.lucamorandini.it
--------------------

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Alexey_Leybman · September 12, 2006, 5:04am

Hi Alexander
How do you generate shp index file, using gdalindex?
What means param LevelsNum in .properties file if Levels is x,y resolutions?

Alexander Petkov wrote:

What is the error this time? The plugin is not (yet) generating the
properties file.

Alex

On 9/11/06, Luca Morandini <lmorandini@anonymised.com> wrote:

Alexander Petkov wrote:
> I suspect that it mught be a missing .properties file for your
> ImageMosaic coverage. Check the sample ImageMosaic coverage
> (mosaic.properties file) for example:

The properties file is missing indeed, but setting up one didn't work.
Actually, I've put the following in a file named "tci250.properties"
(tci250 being the name of the coveragestore):

Name=tci250
Levels=70000,48000
LevelsNum=1
Envelope2D=162000,4546000 459000,4755000
NumFiles=14

Moreover, shouldn't the plugin write it by itself ?

Regards,

--------------------
Luca Morandini
www.lucamorandini.it
--------------------

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job
easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache
Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job
easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
View this message in context: http://www.nabble.com/ImageMosaic-plugin-update-tf2177970.html#a6259659
Sent from the GeoServer - User forum at Nabble.com.

Luca_Morandini2 · September 12, 2006, 6:43am

Alexander Petkov wrote:

What is the error this time?

The same:
942815 [CONFIG] org.geotools.factory.FactoryRegistry - Factory implementations for category GridCoverageFactory:
   org.geotools.coverage.grid.GridCoverageFactory
Sep 12, 2006 8:43:11 AM org.apache.struts.action.RequestProcessor processException
WARNING: Unhandled Exception thrown: class java.lang.NullPointerException
08:43:11.891 WARN!! Exception for /geoserver/config/data/coverageNewSubmit.do
java.lang.NullPointerException
         at org.vfny.geoserver.action.data.DataCoveragesNewAction.execute(DataCoveragesNewAction.java:139)
         at org.vfny.geoserver.action.ConfigAction.execute(ConfigAction.java:102)
         at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:431)
         at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:236)
         at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1196)
         at org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:432)
         at javax.servlet.http.HttpServlet.service(HttpServlet.java:760)

--------------------
Luca Morandini
www.lucamorandini.it
--------------------

Alexander_Petkov · September 17, 2006, 3:53pm

Here is a relevant snippet from a previous post:

For example, the following command line will create an index shapefile
for all of your geotiffs:

gdaltindex -tileindex location test *.tif

Here it creates a test.shp with a field named "location" (the exact
name is needed by the Imagemosaic plugin I believe), for all images
with .tif extension in the current directory.

"Levels" in the .properties file represents the cell size in the x and
y directions (in the units of your projection). Maybe a better name
for that parameter would be "CellSize" or something... LevelsNum is
probably picking the appropriate "Levels" index, I am not too sure on
that one.

Alex

On 9/11/06, Alexey Leybman <alexey.leybman@anonymised.com> wrote:

Hi Alexander
How do you generate shp index file, using gdalindex?
What means param LevelsNum in .properties file if Levels is x,y resolutions?

Alexander Petkov wrote:
>
> What is the error this time? The plugin is not (yet) generating the
> properties file.
>
> Alex
>
> On 9/11/06, Luca Morandini <lmorandini@anonymised.com> wrote:
>> Alexander Petkov wrote:
>> > I suspect that it mught be a missing .properties file for your
>> > ImageMosaic coverage. Check the sample ImageMosaic coverage
>> > (mosaic.properties file) for example:
>>
>> The properties file is missing indeed, but setting up one didn't work.
>> Actually, I've put the following in a file named "tci250.properties"
>> (tci250 being the name of the coveragestore):
>>
>> Name=tci250
>> Levels=70000,48000
>> LevelsNum=1
>> Envelope2D=162000,4546000 459000,4755000
>> NumFiles=14
>>
>> Moreover, shouldn't the plugin write it by itself ?
>>
>> Regards,
>>
>> --------------------
>> Luca Morandini
>> www.lucamorandini.it
>> --------------------
>>
>> -------------------------------------------------------------------------
>> Using Tomcat but need to do more? Need to support web services, security?
>> Get stuff done quickly with pre-integrated technology to make your job
>> easier
>> Download IBM WebSphere Application Server v.1.0.1 based on Apache
>> Geronimo
>> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>> _______________________________________________
>> Geoserver-users mailing list
>> Geoserver-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/geoserver-users
>>
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job
> easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Geoserver-users mailing list
> Geoserver-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/geoserver-users
>

--
View this message in context: http://www.nabble.com/ImageMosaic-plugin-update-tf2177970.html#a6259659
Sent from the GeoServer - User forum at Nabble.com.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Simone3 · September 17, 2006, 10:59pm

Hi guys, hi lists,
I just committed a couple of tools for coverage preparation and
processing to the GeoTools subversion repository under
spike/coveragetools. They are command line tools, with command line
parsing capabilities built with commons-cli package.
These tools are:
1> mosaicindexbuilder tool which can be used to generate the shapefile
index and the other appropriate files for the image mosaic plugin.
2>pyramidbuilder that can be used to build a pyraimd starting from a
mosaic (the pyramid plugin will be released next week).
3>Coveragetiler that can be used to tile a very large coverage into a
mosaic of geotiffs (new formats will come soon).

Just to let you know, this week me and Alex Petkov should start with
writing some documentantion for the various coverage plugins,
including the image mosaic one.

Ciao,
Simone.

On 9/17/06, Alexander Petkov <greenkov@anonymised.com> wrote:

Here is a relevant snippet from a previous post:

> For example, the following command line will create an index shapefile
> for all of your geotiffs:
>
> gdaltindex -tileindex location test *.tif
>
> Here it creates a test.shp with a field named "location" (the exact
> name is needed by the Imagemosaic plugin I believe), for all images
> with .tif extension in the current directory.

"Levels" in the .properties file represents the cell size in the x and
y directions (in the units of your projection). Maybe a better name
for that parameter would be "CellSize" or something... LevelsNum is
probably picking the appropriate "Levels" index, I am not too sure on
that one.

Alex

On 9/11/06, Alexey Leybman <alexey.leybman@anonymised.com> wrote:
>
> Hi Alexander
> How do you generate shp index file, using gdalindex?
> What means param LevelsNum in .properties file if Levels is x,y resolutions?
>
> Alexander Petkov wrote:
> >
> > What is the error this time? The plugin is not (yet) generating the
> > properties file.
> >
> > Alex
> >
> > On 9/11/06, Luca Morandini <lmorandini@anonymised.com> wrote:
> >> Alexander Petkov wrote:
> >> > I suspect that it mught be a missing .properties file for your
> >> > ImageMosaic coverage. Check the sample ImageMosaic coverage
> >> > (mosaic.properties file) for example:
> >>
> >> The properties file is missing indeed, but setting up one didn't work.
> >> Actually, I've put the following in a file named "tci250.properties"
> >> (tci250 being the name of the coveragestore):
> >>
> >> Name=tci250
> >> Levels=70000,48000
> >> LevelsNum=1
> >> Envelope2D=162000,4546000 459000,4755000
> >> NumFiles=14
> >>
> >> Moreover, shouldn't the plugin write it by itself ?
> >>
> >> Regards,
> >>
> >> --------------------
> >> Luca Morandini
> >> www.lucamorandini.it
> >> --------------------
> >>
> >> -------------------------------------------------------------------------
> >> Using Tomcat but need to do more? Need to support web services, security?
> >> Get stuff done quickly with pre-integrated technology to make your job
> >> easier
> >> Download IBM WebSphere Application Server v.1.0.1 based on Apache
> >> Geronimo
> >> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> >> _______________________________________________
> >> Geoserver-users mailing list
> >> Geoserver-users@lists.sourceforge.net
> >> https://lists.sourceforge.net/lists/listinfo/geoserver-users
> >>
> >
> > -------------------------------------------------------------------------
> > Using Tomcat but need to do more? Need to support web services, security?
> > Get stuff done quickly with pre-integrated technology to make your job
> > easier
> > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> > _______________________________________________
> > Geoserver-users mailing list
> > Geoserver-users@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/geoserver-users
> >
>
> --
> View this message in context: http://www.nabble.com/ImageMosaic-plugin-update-tf2177970.html#a6259659
> Sent from the GeoServer - User forum at Nabble.com.
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Geoserver-users mailing list
> Geoserver-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/geoserver-users
>

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
-------------------------------------------------------
Eng. Simone Giannecchini
President /CEO GeoSolutions

http://www.geo-solutions.it

-------------------------------------------------------

Blaz · September 18, 2006, 9:13am

Great tools. Thank you!

But i have a suggestion. Sometimes my image mosaic files are not all in one
directory but in directory tree, like:

So it would be great if MosaicIndexBuilder would recurse through all the
directories.

--
View this message in context: http://www.nabble.com/ImageMosaic-plugin-update-tf2177970.html#a6360176
Sent from the GeoServer - User mailing list archive at Nabble.com.

Simone3 · September 18, 2006, 11:35am

It is a very good idea, indeed. I am going to investigate a bit to
understand how it would impact the imagemosaic and the imagepyramid
plugin.

Btw, how it going with your mosaic?

SImone.

On 9/18/06, GeoUser <blazrepnik@anonymised.com> wrote:

Great tools. Thank you!

But i have a suggestion. Sometimes my image mosaic files are not all in one
directory but in directory tree, like:

DOF1
|
|-DOF11: raster here
|-DOF12: raster here
|-DOF13: raster here
| |
| |-DOF131: raster here
| |-DOF132: raster here
| ....
|
|-DOF1n: raster here
...

So it would be great if MosaicIndexBuilder would recurse through all the
directories.

--
View this message in context: http://www.nabble.com/ImageMosaic-plugin-update-tf2177970.html#a6360176
Sent from the GeoServer - User mailing list archive at Nabble.com.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
-------------------------------------------------------
Eng. Simone Giannecchini
President /CEO GeoSolutions

http://www.geo-solutions.it

-------------------------------------------------------

Simone3 · September 18, 2006, 12:38pm

It is a very good idea, indeed.

If you feel like you can even go ahead and implement such capabilty
yourself. Afterwards we can commit it to the codebase.

Simone.

On 9/18/06, GeoUser <blazrepnik@anonymised.com> wrote:

Great tools. Thank you!

But i have a suggestion. Sometimes my image mosaic files are not all in one
directory but in directory tree, like:

DOF1
|
|-DOF11: raster here
|-DOF12: raster here
|-DOF13: raster here
| |
| |-DOF131: raster here
| |-DOF132: raster here
| ....
|
|-DOF1n: raster here
...

So it would be great if MosaicIndexBuilder would recurse through all the
directories.

--
View this message in context: http://www.nabble.com/ImageMosaic-plugin-update-tf2177970.html#a6360176
Sent from the GeoServer - User mailing list archive at Nabble.com.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
-------------------------------------------------------
Eng. Simone Giannecchini
President /CEO GeoSolutions

http://www.geo-solutions.it

-------------------------------------------------------

Blaz · September 18, 2006, 12:39pm

really well ( if i exclude the java crashes).

The mosaic operations are really fast and i have no trouble displaying large
amount of data in mapbuilder.

Example:
I have one mosaic that contains 10 TIFs, each 80MB.
I setup the bounding box so that all rasters fall in.
Then i execute the WMS query that returns png 500x250.
This usually takes under second or two.
Like i said, very fast.

I have a question. Are this image transformations (scale, clipping...)
executed only on CPU?

Anyway. I wrote a simple SDL that hides mosaic until i get close enough. i
probably won't have to use that anymore since you introduced pyramids. i'll
be testing that next week when you release the pyramid plugin.

bye.
--
View this message in context: http://www.nabble.com/ImageMosaic-plugin-update-tf2177970.html#a6362865
Sent from the GeoServer - User mailing list archive at Nabble.com.

Blaz · September 18, 2006, 12:45pm

if you say i can then i will

simone giannecchini wrote:

It is a very good idea, indeed.

If you feel like you can even go ahead and implement such capabilty
yourself. Afterwards we can commit it to the codebase.

Simone.

On 9/18/06, GeoUser <blazrepnik@anonymised.com> wrote:

Great tools. Thank you!

But i have a suggestion. Sometimes my image mosaic files are not all in
one
directory but in directory tree, like:

DOF1
|
|-DOF11: raster here
|-DOF12: raster here
|-DOF13: raster here
| |
| |-DOF131: raster here
| |-DOF132: raster here
| ....
|
|-DOF1n: raster here
...

So it would be great if MosaicIndexBuilder would recurse through all the
directories.

--
View this message in context:
http://www.nabble.com/ImageMosaic-plugin-update-tf2177970.html#a6360176
Sent from the GeoServer - User mailing list archive at Nabble.com.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job
easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache
Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
-------------------------------------------------------
Eng. Simone Giannecchini
President /CEO GeoSolutions

http://www.geo-solutions.it

-------------------------------------------------------

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job
easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
View this message in context: http://www.nabble.com/ImageMosaic-plugin-update-tf2177970.html#a6362948
Sent from the GeoServer - User mailing list archive at Nabble.com.

Simone3 · September 18, 2006, 12:47pm

Answers and advices inlined...

On 9/18/06, GeoUser <blazrepnik@anonymised.com> wrote:

really well ( if i exclude the java crashes).

The mosaic operations are really fast and i have no trouble displaying large
amount of data in mapbuilder.

Example:
I have one mosaic that contains 10 TIFs, each 80MB.
I setup the bounding box so that all rasters fall in.
Then i execute the WMS query that returns png 500x250.
This usually takes under second or two.
Like i said, very fast.

Mmmmh, let me ask a few questions in order to speed up even more:
1>Are your tiff tiled?
2>Did you embed overviews into them?

I have a question. Are this image transformations (scale, clipping...)
executed only on CPU?

I do not clearly see the point of this question (please explain ).

Anyway. I wrote a simple SDL that hides mosaic until i get close enough. i
probably won't have to use that anymore since you introduced pyramids. i'll
be testing that next week when you release the pyramid plugin.

If i can I would suggest you doing as follows:

1>using the coveragetiler, split each raster in a series of tiles
let's say of size 1 mb each
2>use the pyramid plugin once it is released (it could even be
tomorrow, anyway next week is reasable enough I hope). Use the
pyramidbuilder tool to build the pyramid for you, level by level.

Simone.

bye.
--
View this message in context: http://www.nabble.com/ImageMosaic-plugin-update-tf2177970.html#a6362865
Sent from the GeoServer - User mailing list archive at Nabble.com.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
-------------------------------------------------------
Eng. Simone Giannecchini
President /CEO GeoSolutions

http://www.geo-solutions.it

-------------------------------------------------------

Simone3 · September 18, 2006, 12:52pm

Well, I encourage you to do so. This is the essence of the open
source. I am putting there tools that are not yet completely finished
because I hope that someone else will grab them, use them and
hopefully contribute.

No one can do it all by himself but if we share the load the overall
quality and capabilities will increase quickly.

Play with the tools, modify them, let me review the changes once you
feel comfortable with them in order to commit them. You might even ask
for commit access to the geotools codebase just to enhance the tools.

Simone.
On 9/18/06, GeoUser <blazrepnik@anonymised.com> wrote:

if you say i can then i will

simone giannecchini wrote:
>
> It is a very good idea, indeed.
>
> If you feel like you can even go ahead and implement such capabilty
> yourself. Afterwards we can commit it to the codebase.
>
> Simone.
>
> On 9/18/06, GeoUser <blazrepnik@anonymised.com> wrote:
>>
>> Great tools. Thank you!
>>
>> But i have a suggestion. Sometimes my image mosaic files are not all in
>> one
>> directory but in directory tree, like:
>>
>> DOF1
>> |
>> |-DOF11: raster here
>> |-DOF12: raster here
>> |-DOF13: raster here
>> | |
>> | |-DOF131: raster here
>> | |-DOF132: raster here
>> | ....
>> |
>> |-DOF1n: raster here
>> ...
>>
>> So it would be great if MosaicIndexBuilder would recurse through all the
>> directories.
>>
>> --
>> View this message in context:
>> http://www.nabble.com/ImageMosaic-plugin-update-tf2177970.html#a6360176
>> Sent from the GeoServer - User mailing list archive at Nabble.com.
>>
>> -------------------------------------------------------------------------
>> Using Tomcat but need to do more? Need to support web services, security?
>> Get stuff done quickly with pre-integrated technology to make your job
>> easier
>> Download IBM WebSphere Application Server v.1.0.1 based on Apache
>> Geronimo
>> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>> _______________________________________________
>> Geoserver-users mailing list
>> Geoserver-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/geoserver-users
>>
>
> --
> -------------------------------------------------------
> Eng. Simone Giannecchini
> President /CEO GeoSolutions
>
> http://www.geo-solutions.it
>
> -------------------------------------------------------
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job
> easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Geoserver-users mailing list
> Geoserver-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/geoserver-users
>

--
View this message in context: http://www.nabble.com/ImageMosaic-plugin-update-tf2177970.html#a6362948
Sent from the GeoServer - User mailing list archive at Nabble.com.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
-------------------------------------------------------
Eng. Simone Giannecchini
President /CEO GeoSolutions

http://www.geo-solutions.it

-------------------------------------------------------

Blaz · September 18, 2006, 12:56pm

Mmmmh, let me ask a few questions in order to speed up even more:
1>Are your tiff tiled?
2>Did you embed overviews into them?

1. no the TIF is one big picture
2. i don't understand. How do i embed the overview?

I have a question. Are this image transformations (scale, clipping...)
executed only on CPU?

I do not clearly see the point of this question (please explain ).

Forget about it It was just something stupid.

Anyway. I wrote a simple SDL that hides mosaic until i get close enough.
i
probably won't have to use that anymore since you introduced pyramids.
i'll
be testing that next week when you release the pyramid plugin.

If i can I would suggest you doing as follows:

1>using the coveragetiler, split each raster in a series of tiles
let's say of size 1 mb each
2>use the pyramid plugin once it is released (it could even be
tomorrow, anyway next week is reasable enough I hope). Use the
pyramidbuilder tool to build the pyramid for you, level by level.

I can not tile tifs. Their original size must remain unchanged.
--
View this message in context: http://www.nabble.com/ImageMosaic-plugin-update-tf2177970.html#a6363127
Sent from the GeoServer - User mailing list archive at Nabble.com.

Simone3 · September 18, 2006, 1:05pm

On 9/18/06, GeoUser <blazrepnik@anonymised.com> wrote:

> Mmmmh, let me ask a few questions in order to speed up even more:
> 1>Are your tiff tiled?
> 2>Did you embed overviews into them?
>

1. no the TIF is one big picture
2. i don't understand. How do i embed the overview?

For the moment, until they fix an annoying bug in JAI so that i canj
commit other tools, use the gdaladdo tool from the gdal utilities. You
can download them from maptool.org. This utility is pretty
straightforward. You should write something like
gdaladdo -r average 2 4 8 16 32 64 -co "TILED=YES" sourcefile

and it will embedde din the same tiff overviews with resolution
decreasing bya power two at each level. 6 levels should be enough.

The geotools plugin I wrote for tiff will automagically recognize the
presence of the overviews and select the right one.

Once you do this you will see REAL speed, believe me.

>> I have a question. Are this image transformations (scale, clipping...)
>> executed only on CPU?
> I do not clearly see the point of this question (please explain ).
>

Forget about it It was just something stupid.

>> Anyway. I wrote a simple SDL that hides mosaic until i get close enough.
>> i
>> probably won't have to use that anymore since you introduced pyramids.
>> i'll
>> be testing that next week when you release the pyramid plugin.
>
> If i can I would suggest you doing as follows:
>
> 1>using the coveragetiler, split each raster in a series of tiles
> let's say of size 1 mb each
> 2>use the pyramid plugin once it is released (it could even be
> tomorrow, anyway next week is reasable enough I hope). Use the
> pyramidbuilder tool to build the pyramid for you, level by level.
>

I can not tile tifs. Their original size must remain unchanged.

The original size will not change. The difference will be that instead
of having a single image of 80 mb (which is not THAT big btw) you will
have let's say 80 images of 1 mb and the result will be much faster.
However if each image is only 80 mb using gdaladdo with the parameters
I gave you above is more than enough. Try that out with the same
parameters on each image, rerun the mosaicindexbuilder and then tell
me if things are faster or not, if not you made a mistake somewhere
:-).

Simone.

--
View this message in context: http://www.nabble.com/ImageMosaic-plugin-update-tf2177970.html#a6363127
Sent from the GeoServer - User mailing list archive at Nabble.com.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
-------------------------------------------------------
Eng. Simone Giannecchini
President /CEO GeoSolutions

http://www.geo-solutions.it

-------------------------------------------------------

Blaz · September 18, 2006, 1:13pm

thanks!

i will try: gdaladdo -r average 2 4 8 16 32 64 -co "TILED=YES" sourcefile

--
View this message in context: http://www.nabble.com/ImageMosaic-plugin-update-tf2177970.html#a6363406
Sent from the GeoServer - User mailing list archive at Nabble.com.

Blaz · September 18, 2006, 2:09pm

i made a little change to MosaicIndexBuilder

Tell me if the results are correct:

my mosaic files are stored in directories like this:

Dof_77: 4 rasters
|
|-dof00: 1 raster
|-dof01: 1 raster
|-dof02: 1 raster

After runing the MosaicIndexBuilder i got this result:

18.9.2006 15:57:39 it.geosolutions.utils.imagemosaic.MosaicIndexBuilder
getNotification
INFO: Progress is at 0.0
attached message is: Now indexing file C272161.tif
18.9.2006 15:57:39 it.geosolutions.utils.imagemosaic.MosaicIndexBuilder
getNotification
INFO: Progress is at 14.142857142857142
attached message is: Done with file D:\Dof_77\C272161.tif

18.9.2006 15:57:39 it.geosolutions.utils.imagemosaic.MosaicIndexBuilder
getNotification
INFO: Progress is at 14.285714285714286
attached message is: Now indexing file C272261.tif
18.9.2006 15:57:39 it.geosolutions.utils.imagemosaic.MosaicIndexBuilder
getNotification
INFO: Progress is at 28.285714285714285
attached message is: Done with file D:\Dof_77\C272261.tif

18.9.2006 15:57:39 it.geosolutions.utils.imagemosaic.MosaicIndexBuilder
getNotification
INFO: Progress is at 28.571428571428573
attached message is: Now indexing file C272361.tif
18.9.2006 15:57:39 it.geosolutions.utils.imagemosaic.MosaicIndexBuilder
getNotification
INFO: Progress is at 42.42857142857143
attached message is: Done with file D:\Dof_77\C272361.tif

18.9.2006 15:57:39 it.geosolutions.utils.imagemosaic.MosaicIndexBuilder
getNotification
INFO: Progress is at 42.857142857142854
attached message is: Now indexing file C272461.tif
18.9.2006 15:57:40 it.geosolutions.utils.imagemosaic.MosaicIndexBuilder
getNotification
INFO: Progress is at 56.57142857142857
attached message is: Done with file D:\Dof_77\C272461.tif

18.9.2006 15:57:40 it.geosolutions.utils.imagemosaic.MosaicIndexBuilder
getNotification
INFO: Progress is at 57.142857142857146
attached message is: Now indexing file dof00\eC272161.tif
18.9.2006 15:57:40 it.geosolutions.utils.imagemosaic.MosaicIndexBuilder
getNotification
INFO: Progress is at 70.71428571428571
attached message is: Done with file D:\Dof_77\dof00\eC272161.tif

18.9.2006 15:57:40 it.geosolutions.utils.imagemosaic.MosaicIndexBuilder
getNotification
INFO: Progress is at 71.42857142857143
attached message is: Now indexing file dof01\C_272161.tif
18.9.2006 15:57:40 it.geosolutions.utils.imagemosaic.MosaicIndexBuilder
getNotification
INFO: Progress is at 84.85714285714286
attached message is: Done with file D:\Dof_77\dof01\C_272161.tif

18.9.2006 15:57:40 it.geosolutions.utils.imagemosaic.MosaicIndexBuilder
getNotification
INFO: Progress is at 85.71428571428571
attached message is: Now indexing file dof02\qqC272261.tif
18.9.2006 15:57:40 it.geosolutions.utils.imagemosaic.MosaicIndexBuilder
getNotification
INFO: Progress is at 99.0
attached message is: Done with file D:\Dof_77\dof02\qqC272261.tif

18.9.2006 15:57:40 it.geosolutions.utils.imagemosaic.MosaicIndexBuilder
getNotification
INFO: Progress is at 99.9
attached message is: Creating final properties file
18.9.2006 15:57:40 it.geosolutions.utils.imagemosaic.MosaicIndexBuilder
getNotification
INFO: Progress is at 100.0
attached message is: Done!!!

the dbf file looks like this:

location
C272161.tif
C272261.tif
C272361.tif
C272461.tif
dof00\eC272161.tif
dof01\C_272161.tif
dof02\qqC272261.tif

properties like this:

#
#Mon Sep 18 15:57:40 CEST 2006
Name=index
Levels=0.5,0.5
LevelsNum=1
Envelope2D=410000.0,5151000.0 419000.0,5154000.0
NumFiles=7

then i loaded the data into geoserver and it works/displays perfectly
--
View this message in context: http://www.nabble.com/ImageMosaic-plugin-update-tf2177970.html#a6364452
Sent from the GeoServer - User mailing list archive at Nabble.com.