I want do contribute the imagemoasic-jdbc plugin to geoserver.
I tested the module with geoserver 1.7.2 and it works as expected.
The info page
http://geotools.codehaus.org/Image+Mosaicing+Pyramidal+JDBC+Plugin
The geotools documentation is here
http://docs.codehaus.org/display/GEOTDOC/Image+Mosaicing+Pyramidal+JDBC+Plu gin
Question: Should there be a tutorial like
http://geoserver.org/display/GEOSDOC/Using+the+ImageMosaic+plugin
Your opinions
christian
Hi Christian, sounds like a great candidate for a new extension!
http://geoserver.org/display/GEOS/GSIP+22+-+Community+Modules
It looks like it meets the requirements of an extension. What is the test coverage of the module?
And any tutorial you could contribute would just be an added bonus, and very much welcomed.
-Justin
Christian Müller wrote:
I want do contribute the imagemoasic-jdbc plugin to geoserver.
I tested the module with geoserver 1.7.2 and it works as expected.
The info page
http://geotools.codehaus.org/Image+Mosaicing+Pyramidal+JDBC+PluginThe geotools documentation is here
http://docs.codehaus.org/display/GEOTDOC/Image+Mosaicing+Pyramidal+JDBC+Plu ginQuestion: Should there be a tutorial like
http://geoserver.org/display/GEOSDOC/Using+the+ImageMosaic+pluginYour opinions
christian
------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel
--
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.
Stupid question for you Justin; do we need a community module or can this just be added to libs?
Jody
Justin Deoliveira wrote:
Hi Christian, sounds like a great candidate for a new extension!
http://geoserver.org/display/GEOS/GSIP+22+-+Community+Modules
It looks like it meets the requirements of an extension. What is the test coverage of the module?
And any tutorial you could contribute would just be an added bonus, and very much welcomed.
-Justin
Christian Müller wrote:
I want do contribute the imagemoasic-jdbc plugin to geoserver.
I tested the module with geoserver 1.7.2 and it works as expected.
The info page
http://geotools.codehaus.org/Image+Mosaicing+Pyramidal+JDBC+PluginThe geotools documentation is here
http://docs.codehaus.org/display/GEOTDOC/Image+Mosaicing+Pyramidal+JDBC+Plu ginQuestion: Should there be a tutorial like
http://geoserver.org/display/GEOSDOC/Using+the+ImageMosaic+pluginYour opinions
christian
------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel
Well what we have been doing for extensions in geoserver which are simply made up of geotools libs is adding a stub module in geoserver, which declares the dependency. That way it can (in geoserver) be activated via profile, and also selectively built at release time as an extension.
So yes, a community module or extension will be required.
-Justin
Jody Garnett wrote:
Stupid question for you Justin; do we need a community module or can this just be added to libs?
JodyJustin Deoliveira wrote:
Hi Christian, sounds like a great candidate for a new extension!
http://geoserver.org/display/GEOS/GSIP+22+-+Community+Modules
It looks like it meets the requirements of an extension. What is the test coverage of the module?
And any tutorial you could contribute would just be an added bonus, and very much welcomed.
-Justin
Christian Müller wrote:
I want do contribute the imagemoasic-jdbc plugin to geoserver.
I tested the module with geoserver 1.7.2 and it works as expected.
The info page
http://geotools.codehaus.org/Image+Mosaicing+Pyramidal+JDBC+Plugin
The geotools documentation is here
http://docs.codehaus.org/display/GEOTDOC/Image+Mosaicing+Pyramidal+JDBC+Plu gin
Question: Should there be a tutorial like
http://geoserver.org/display/GEOSDOC/Using+the+ImageMosaic+plugin
Your opinions
christian------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel
--
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.
Concerning the requirements
1. The module has at least a "handful" of users.
Yes, I have 3 deployments, one in an EJB Container and two in geoserver 1.6.x series.
The module is part of an production environment for fighting against animal diseases in Austria,
and yes, it works. The only point is , as far as I know, I am the only one admin who configures
the module and I want to change that becoming part of geoserver.
The module is part of the geotools plugin directory.
2. The module has a designated and active maintainer.
Yes, me
3. The module is considered "stable" by the majority of the PSC.
Yes it is stable, in the last 6 month I only did some minor improvements.
4. The module maintains 40% test coverage.
Yes, I remember I had over 60 & percent test coverage when migrating from unsupported to
the geotools plugin directory
5. The module has no IP violations.
Yes, we checked that
The module must not contain any code with a license or copyright that violates the GPL.
Yes, I did not include foreign modules. Needed JDBC Drivers have to be installed by the user.
6. The module has a page on the Wiki.
Yes, there is a documentation. A tutorial for geoserver users would be useful,
but it does not make sense without a "go" from the geoserver community
7. The maintainer has signed the GeoServer Contributor Agreement.
Yes, I signed the Contributer Agreement for geotools. I hope this is enough.
I offered geoserver as an alternative to ARCSDE for my customer and storing the images in a db is essential.
christian
Justin Deoliveira writes:
Well what we have been doing for extensions in geoserver which are simply made up of geotools libs is adding a stub module in geoserver, which declares the dependency. That way it can (in geoserver) be activated via profile, and also selectively built at release time as an extension.
So yes, a community module or extension will be required.
-Justin
Jody Garnett wrote:
Stupid question for you Justin; do we need a community module or can this just be added to libs?
JodyJustin Deoliveira wrote:
Hi Christian, sounds like a great candidate for a new extension!
http://geoserver.org/display/GEOS/GSIP+22+-+Community+Modules
It looks like it meets the requirements of an extension. What is the test coverage of the module?
And any tutorial you could contribute would just be an added bonus, and very much welcomed.
-Justin
Christian Müller wrote:
I want do contribute the imagemoasic-jdbc plugin to geoserver.
I tested the module with geoserver 1.7.2 and it works as expected.
The info page
http://geotools.codehaus.org/Image+Mosaicing+Pyramidal+JDBC+Plugin
The geotools documentation is here
http://docs.codehaus.org/display/GEOTDOC/Image+Mosaicing+Pyramidal+JDBC +Plu gin
Question: Should there be a tutorial like
http://geoserver.org/display/GEOSDOC/Using+the+ImageMosaic+plugin
Your opinions
christian----------------------------------------------------------------------- -------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel--
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.
Christian Müller ha scritto:
6. The module has a page on the Wiki.
Yes, there is a documentation. A tutorial for geoserver users would be useful,
but it does not make sense without a "go" from the geoserver community
Yet it would be very nice to have 
7. The maintainer has signed the GeoServer Contributor Agreement.
Yes, I signed the Contributer Agreement for geotools. I hope this is enough.
No, GeoServer has a separate contribution agreement, the GeoServer one
demands you give the copyright assignment to TOPP, not to OSGEO.
Yet, in this case, I guess it might not be a problem, as in GeoServer
you would not have to manipulate java source files (and we don't have
copyright headers in pom files).
Having a tutorial starting from a
publicly available image to storing images in postgis (or mysql) would
be useful for us to check the extension is working (I know for you
it's almost natural to setup the whole thing, but for us it might be
a bit of a hassle, at least the first time).
I offered geoserver as an alternative to ARCSDE for my customer and storing the images in a db is essential.
Yup, I understand, in some environments storing data (or config) on the
file system is a big no-no. It would be interesting to compare the
module performance against the file based mosaic module, there is quite
some debate in OSGEO on which model delivers better performance.
On one side there is the database access overhead, on the other there
is the file opening and metadata reading overhead, which many forget
about. Such a comparison (especially with some very big amounts of
data) would make for a very interesting FOSS4G 2009 presentation, too.
Cheers
Andrea
--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.
1)
Andrea, should I add a tutorial like
http://geoserver.org/display/GEOSDOC/Using+the+ImageMosaic+plugin ?
2)
I think it would be a good idea to sign the contributer agreement. Since I want to deploy geoserver as a strategic component, it is very likely that I will do some contributions.
Would that be ok ?
Andrea Aime writes:
Christian Müller ha scritto:
6. The module has a page on the Wiki.
Yes, there is a documentation. A tutorial for geoserver users would be useful,
but it does not make sense without a "go" from the geoserver communityYet it would be very nice to have
7. The maintainer has signed the GeoServer Contributor Agreement.
Yes, I signed the Contributer Agreement for geotools. I hope this is enough.No, GeoServer has a separate contribution agreement, the GeoServer one
demands you give the copyright assignment to TOPP, not to OSGEO.Yet, in this case, I guess it might not be a problem, as in GeoServer
you would not have to manipulate java source files (and we don't have
copyright headers in pom files).Having a tutorial starting from a
publicly available image to storing images in postgis (or mysql) would
be useful for us to check the extension is working (I know for you
it's almost natural to setup the whole thing, but for us it might be
a bit of a hassle, at least the first time).I offered geoserver as an alternative to ARCSDE for my customer and storing the images in a db is essential.
Yup, I understand, in some environments storing data (or config) on the
file system is a big no-no. It would be interesting to compare the
module performance against the file based mosaic module, there is quite
some debate in OSGEO on which model delivers better performance.
On one side there is the database access overhead, on the other there
is the file opening and metadata reading overhead, which many forget
about. Such a comparison (especially with some very big amounts of
data) would make for a very interesting FOSS4G 2009 presentation, too.Cheers
Andrea--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.
Christian Müller ha scritto:
1)
Andrea, should I add a tutorial like
http://geoserver.org/display/GEOSDOC/Using+the+ImageMosaic+plugin ?
Yep, something along those lines. Ideally, something using a
public data set, step by step, would be best, because it would
allow everybody to reproduce a known set of steps, with a known
dataset, and known (good) results.
2)
I think it would be a good idea to sign the contributer agreement. Since I want to deploy geoserver as a strategic component, it is very likely that I will do some contributions.
Would that be ok ?
That would be more than ok, it would be very much welcomed! 
Cheers
Andrea
--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.
I added a tutorial
http://geoserver.org/display/GEOSDOC/Using+the+ImageMosaic+JDBC+Plugin
to get a step further towards including the module into geoserver.
Attention: Anyone who tries this tutorial must use a gt2-imagemosaic-jdbc-2.5-SNAPSHOT.jar from 2.5.x. I tested with geoserver 1.7.2.
Waiting for feedback and looking forward to geoserver 1.7.3 for storing tiles in a jdbc database
Christian Müller ha scritto:
I added a tutorial
http://geoserver.org/display/GEOSDOC/Using+the+ImageMosaic+JDBC+Plugin
to get a step further towards including the module into geoserver.
Attention: Anyone who tries this tutorial must use a gt2-imagemosaic-jdbc-2.5-SNAPSHOT.jar from 2.5.x. I tested with geoserver 1.7.2.
Waiting for feedback and looking forward to geoserver 1.7.3 for storing tiles in a jdbc database
Hi Christian,
so I went through the tutorial and collected some feedback.
Here goes:
- at the very beginning of the procedure you say to create a "working
dictionary"... I guess you mean a working directory?
- I'm on windows with fwtools, even in the fwtools console
gdal_retile.py does not work, but calling gdal_retile (without .py)
does work.
- to get a nice result on the tiles it would be nice to add
-r bilinear to the command line. I guess you want to add that
every time anyways, so it's better to add it to the tutorial,
otherwise people might spend hours building a pyramid that
does not look good
- a section talking about the pyramid would be nice. Stuff like,
telling people how many levels to use. If you want to see your
images only fairly zoomed in, a handful of levels are sufficient
(but remember to declare a maxScaleDenominator in the SLD Rule),
but if you want to make a full pyramid, the number of levels
needed should be something like log(size)/log(2) where size is
the largest side of the original image
- any reason for using 128 pixels tiles? I guess it's just because
the input image is small, but maybe there is a deeper reason?
- the configUrl parameter requires a url, and might have to wait
for a time out if you don't type it properly.
What about supporting also a straight path on the file system,
without the protocol, to be chekced with new File(path).exists()?
- in the osm.postgis.xml, 2 = bipolar, but it should actually
be "bilinear"
- any reason why the dll command creates many little sql statements?
It would be handier to have just two, one to create the tables,
the other to drop them... or not?
- the request to put the drivers in lib/ext does not look like a
very good idea, people might have to use different versions of
the same jdbc driver in different applications,
and putting one in lib/ext prevents it. Any way to get the
same result by using -cp and leaving the jdbc drivers out
of lib/ext?
- the import is tedious if one has many levels. Any chance to
create some convention so that a single command can import
the whole pyramid?
- when looking at the result in GeoServer one can see the image
is stretched compared to the original png. This is actually due
to the projection, and the fact the original image has probably
been rendered in a different one (google mercator I guess).
It would be a good idea to explain the reason for the stretch,
it might save you some user questions on the topic.
Ok, now I'm moving to try an import a bluemarble image
(single ecw file, 86400x43200) into the database (with 16
levels pyramid, bilinear interpolation, jpeg format).
Well, actually, it's been creating the base level since
15 minutes ago, I guess I'll have to wait quite a bit more
(I guess, a few hours... a nice improvement could be to parallelize
gdal_retile so that it can leverage multiple cores).
Will let you know when/if I finish and get it working
Another (easy) idea that came to mind is that it would be
nice to have GWC learn how to read the files of this pyramid
directly. Grab a big image that one cannot server due to
licensing constraints (I'm thinking bluemarble), retile,
serve directly out with the tile cache (eventually learn
how to grab the tiles from the db if the organisation
does not allow data on disk).
Cheers
Andrea
--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.
Andrea Aime ha scritto:
Ok, now I'm moving to try an import a bluemarble image
(single ecw file, 86400x43200) into the database (with 16
levels pyramid, bilinear interpolation, jpeg format).
Well, actually, it's been creating the base level since
15 minutes ago, I guess I'll have to wait quite a bit more
(I guess, a few hours... a nice improvement could be to parallelize
gdal_retile so that it can leverage multiple cores).
Will let you know when/if I finish and get it working
Hum, that actually did not go so well. After creating some
56 thousands files in the root directory the command bombed out:
gdal_retile -co "WORLDFILE=YES" -ps 256 256 -of JPEG -levels 16 -targetDir tiles -r bilinear world-topo-bathy-200408-3x86400x43200.ecw
ERROR 2: CPLRealloc(): Out of memory allocating 131080 bytes
Hum... memory leak? It does not look like it was done with the
base level, that would have required the generation of
(86400 / 256) *(43200 / 256) = 338 * 169 = 57122 jpeg
files, whilst only 18972 where created (for each jpg file
there is a .wld and a .jpg.aux.xml too).
I'm using fwtools 2.2.8, gdalinfo reports:
C:\progetti\gisData\bluemarble\tiles>gdalinfo --version
GDAL 1.6.0dev, FWTools 2.2.8, released 2008/10/29
(funny how a FWTools official release uses a
1.6.0dev GDAL).
Cheers
Andrea
--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.
fwtools usually tracks the best development snapshot of gdal.
Simone.
-------------------------------------------------------
Ing. Simone Giannecchini
GeoSolutions S.A.S.
Owner - Software Engineer
Via Carignoni 51
55041 Camaiore (LU)
Italy
phone: +39 0584983027
fax: +39 0584983027
mob: +39 333 8128928
http://www.geo-solutions.it
http://simboss.blogspot.com/
http://www.linkedin.com/in/simonegiannecchini
-------------------------------------------------------
On Wed, Feb 18, 2009 at 5:44 PM, Andrea Aime <aaime@anonymised.com> wrote:
Andrea Aime ha scritto:
Ok, now I'm moving to try an import a bluemarble image
(single ecw file, 86400x43200) into the database (with 16
levels pyramid, bilinear interpolation, jpeg format).
Well, actually, it's been creating the base level since
15 minutes ago, I guess I'll have to wait quite a bit more
(I guess, a few hours... a nice improvement could be to parallelize
gdal_retile so that it can leverage multiple cores).
Will let you know when/if I finish and get it workingHum, that actually did not go so well. After creating some
56 thousands files in the root directory the command bombed out:gdal_retile -co "WORLDFILE=YES" -ps 256 256 -of JPEG -levels 16
-targetDir tiles -r bilinear world-topo-bathy-200408-3x86400x43200.ecw
ERROR 2: CPLRealloc(): Out of memory allocating 131080 bytesHum... memory leak? It does not look like it was done with the
base level, that would have required the generation of
(86400 / 256) *(43200 / 256) = 338 * 169 = 57122 jpeg
files, whilst only 18972 where created (for each jpg file
there is a .wld and a .jpg.aux.xml too).I'm using fwtools 2.2.8, gdalinfo reports:
C:\progetti\gisData\bluemarble\tiles>gdalinfo --version
GDAL 1.6.0dev, FWTools 2.2.8, released 2008/10/29(funny how a FWTools official release uses a
1.6.0dev GDAL).Cheers
Andrea--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel
Hi Andrea, thanks for your feedback, please read below. Lines marked with an * are my answers.
- at the very beginning of the procedure you say to create a "working
dictionary"... I guess you mean a working directory?
* Yes, of course --> fixed in the tutorial
- I'm on windows with fwtools, even in the fwtools console
gdal_retile.py does not work, but calling gdal_retile (without .py)
does work.
* gdal_retile is gdal_retile.bat which in turn calls "python gdal_retile.py"
* --> added the windows stuff to the tutorial and the geotools users guide
- to get a nice result on the tiles it would be nice to add
-r bilinear to the command line. I guess you want to add that
every time anyways, so it's better to add it to the tutorial,
otherwise people might spend hours building a pyramid that
does not look good
* Hmm, I always use nearest neighbour (defautl for gdal_retile), but to be safe, it is ok
* --> added "-r bilinear" to the tutorial
- a section talking about the pyramid would be nice. Stuff like,
telling people how many levels to use. If you want to see your
images only fairly zoomed in, a handful of levels are sufficient
(but remember to declare a maxScaleDenominator in the SLD Rule),
but if you want to make a full pyramid, the number of levels
needed should be something like log(size)/log(2) where size is
the largest side of the original image
* --> added the following link to the tutorial http://star.pst.qub.ac.uk/idl/Image_Tiling.html
* --> added calculation hints to the tutorial
* I am confused about maxScaleDenominator. I looked at raster.sld and it is not used here.
* I think it is only needed if you want to stop zoom out at a given denominater ?
- any reason for using 128 pixels tiles? I guess it's just because
the input image is small, but maybe there is a deeper reason?
* No there is no deeper reason, its because of the small input image
* --> added a note about tile size considerations to the tutorial
- the configUrl parameter requires a url, and might have to wait
for a time out if you don't type it properly.
What about supporting also a straight path on the file system,
without the protocol, to be chekced with new File(path).exists()?
* --> Changed param name from configUrl to config and accept files and URLs
* --> Doing the same for cvsUrl and shapeUrl params (not used in the tutorial) to be consistent
* --> Updated geotoools user guide and tutorial
- in the osm.postgis.xml, 2 = bipolar, but it should actually
be "bilinear"
* --> fixed in tutorial
- any reason why the dll command creates many little sql statements?
It would be handier to have just two, one to create the tables,
the other to drop them... or not?
* Yes, beyond a tutorial, there are many reasons.
* 1) fillmeta.sql is only useable if you use a third party utility to import your image data
* 2) For some database systems it is simply not possible (db2 requires grid sizes when creating a spatial index,
* there is an own advise tool for calculating these sizes, useable after importing the image data)
* http://docs.codehaus.org/display/GEOTDOC/Creating+indexes+for+performance
* 3) which tablespaces for physical data layout ?
* 4) The intension is that these sql files help the user to create his own scripts, fitting for his needs
* 5) It is better to force the user to think over the db design, I want to avoid a disappointed users because of a bad db design
and they do not be aware of it.
- the request to put the drivers in lib/ext does not look like a
very good idea, people might have to use different versions of
the same jdbc driver in different applications,
and putting one in lib/ext prevents it. Any way to get the
same result by using -cp and leaving the jdbc drivers out
of lib/ext?
* Look here for the reason: http://docs.codehaus.org/display/GEOTDOC/Using+the+java+import+utility.
* Using the -Xbootclasspath/a: option for a tutorial is unsave, since it is an extended option not guaranteed to work on any java runtime.
- the import is tedious if one has many levels. Any chance to
create some convention so that a single command can import
the whole pyramid?
* Not really, for an import with world files it would be manageable.
* But if you look at the other possibilities, having a shape file or csv file
* for each pyramid, things get complicated, look here for the possibilities
* http://docs.codehaus.org/display/GEOTDOC/Using+the+java+import+utility
- when looking at the result in GeoServer one can see the image
is stretched compared to the original png. This is actually due
to the projection, and the fact the original image has probably
been rendered in a different one (google mercator I guess).
It would be a good idea to explain the reason for the stretch,
it might save you some user questions on the topic.
* It seems that the image uses EPSG:900913, wich is supported in geotools, but not in postgis
* --> added an explanaition to the tutorial
Ok, now I'm moving to try an import a bluemarble image
(single ecw file, 86400x43200) into the database (with 16
levels pyramid, bilinear interpolation, jpeg format).
Well, actually, it's been creating the base level since
15 minutes ago, I guess I'll have to wait quite a bit more
(I guess, a few hours... a nice improvement could be to parallelize
gdal_retile so that it can leverage multiple cores).
Will let you know when/if I finish and get it working
* A full retile job on 80 GB Erdas image on a ppc with 4.2 Ghz needs also some hours
* Parallelize is a challenge, i did some investigations, heavy stuff
Another (easy) idea that came to mind is that it would be
nice to have GWC learn how to read the files of this pyramid
directly. Grab a big image that one cannot server due to
licensing constraints (I'm thinking bluemarble), retile,
serve directly out with the tile cache (eventually learn
how to grab the tiles from the db if the organisation
does not allow data on disk).
* I had a similiar idea, writing an optional file backend and reading from http://wiki.osgeo.org/wiki/Tile_Map_Service_Specification
Cheers
Chrsitian
Hmm, I did some bigger images producing some hundred thousand files. To be fair, I used linux on ppc and linux on intel, so I have no idea about running gdal_retile on windows.
Andrea, how can I get your test image for trying on my equipment.
christian
Andrea Aime writes:
Andrea Aime ha scritto:
Ok, now I'm moving to try an import a bluemarble image
(single ecw file, 86400x43200) into the database (with 16
levels pyramid, bilinear interpolation, jpeg format).
Well, actually, it's been creating the base level since
15 minutes ago, I guess I'll have to wait quite a bit more
(I guess, a few hours... a nice improvement could be to parallelize
gdal_retile so that it can leverage multiple cores).
Will let you know when/if I finish and get it workingHum, that actually did not go so well. After creating some
56 thousands files in the root directory the command bombed out:gdal_retile -co "WORLDFILE=YES" -ps 256 256 -of JPEG -levels 16 -targetDir tiles -r bilinear world-topo-bathy-200408-3x86400x43200.ecw
ERROR 2: CPLRealloc(): Out of memory allocating 131080 bytesHum... memory leak? It does not look like it was done with the
base level, that would have required the generation of
(86400 / 256) *(43200 / 256) = 338 * 169 = 57122 jpeg
files, whilst only 18972 where created (for each jpg file
there is a .wld and a .jpg.aux.xml too).I'm using fwtools 2.2.8, gdalinfo reports:
C:\progetti\gisData\bluemarble\tiles>gdalinfo --version
GDAL 1.6.0dev, FWTools 2.2.8, released 2008/10/29(funny how a FWTools official release uses a
1.6.0dev GDAL).Cheers
Andrea--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.
Christian Müller ha scritto:
* --> added the following link to the tutorial http://star.pst.qub.ac.uk/idl/Image_Tiling.html
* --> added calculation hints to the tutorial
Nice
* I am confused about maxScaleDenominator. I looked at raster.sld and it is not used here.
* I think it is only needed if you want to stop zoom out at a given denominater ?
It's used when you want the layer not to be draw after a certain
scale. If you set maxScaleDenominator = 100.000, the layer will be
drawn at 1:1000, 1:10000, but not at 1:200000.
- the configUrl parameter requires a url, and might have to wait
for a time out if you don't type it properly.
What about supporting also a straight path on the file system,
without the protocol, to be chekced with new File(path).exists()?
* --> Changed param name from configUrl to config and accept files and URLs
* --> Doing the same for cvsUrl and shapeUrl params (not used in the tutorial) to be consistent
* --> Updated geotoools user guide and tutorial
Nice.
- in the osm.postgis.xml, 2 = bipolar, but it should actually
be "bilinear"
* --> fixed in tutorial
- any reason why the dll command creates many little sql statements?
It would be handier to have just two, one to create the tables,
the other to drop them... or not?
* Yes, beyond a tutorial, there are many reasons.
* 1) fillmeta.sql is only useable if you use a third party utility to import your image data
* 2) For some database systems it is simply not possible (db2 requires grid sizes when creating a spatial index,
* there is an own advise tool for calculating these sizes, useable after importing the image data)
* http://docs.codehaus.org/display/GEOTDOC/Creating+indexes+for+performance
So these commands are not part of the standard sql generated?
Hmm... I understand the DB2 case, in which you have to hand tune,
but for the other databases it would be safe and simple to just
generate the index code as well.
* 3) which tablespaces for physical data layout ?
* 4) The intension is that these sql files help the user to create his own scripts, fitting for his needs
The typical user I know from the GeoServer ml does not want to
think this much, many are looking for a "one button import"
* 5) It is better to force the user to think over the db design, I want to avoid a disappointed users because of a bad db design
and they do not be aware of it.
Ok. But you're forgetting about the typical user that is disappointed
because he cannot manage to get through the procedure and gives
up, frustrated for the time spend without results.
- the request to put the drivers in lib/ext does not look like a
very good idea, people might have to use different versions of
the same jdbc driver in different applications,
and putting one in lib/ext prevents it. Any way to get the
same result by using -cp and leaving the jdbc drivers out
of lib/ext?
* Look here for the reason: http://docs.codehaus.org/display/GEOTDOC/Using+the+java+import+utility.
* Using the -Xbootclasspath/a: option for a tutorial is unsave, since it is an extended option not guaranteed to work on any java runtime.
But it would still work on the most common java runtime
Ideally it would be nice to have the code dynamically load the
classes, a bit like SquirrelSQL does (you just point it to the
driver jar, it does the classloading at runtime).
I know that is going to be hard, so take it just as an
option for a future improvement.
- the import is tedious if one has many levels. Any chance to
create some convention so that a single command can import
the whole pyramid?
* Not really, for an import with world files it would be manageable.
* But if you look at the other possibilities, having a shape file or csv file
* for each pyramid, things get complicated, look here for the possibilities
* http://docs.codehaus.org/display/GEOTDOC/Using+the+java+import+utility
Hmmm... I don't believe many people will ever manage to get thru
all these commands properly. Just look at how many problems have
been reported by the users on the plain mosaic plugin just because
the property file was not correctly setup (the reader for that
property file is fragile, but still, many people get simply lost
in the process). I've been telling Simone the property file should
be auto-generated to avoid this hassle on the users list, I guess
after some tens of mails about that file config he started to
agree with me
Anyways, better to have one more mosaic storage option that's
hard to get started with, but which in the end works, than
no option at all
Cheers
Andrea
--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.
Hi Andrea, I think the problem is that the typical user is a Windows user :-).
But I know,
1) everything should happen with one click,
2) nobody takes care what is happening
3) anybody will see the result at once
4) if there are problems,nobody has an idea and the software is bad
Ok, some proposals
Import:
I can introduce a new switch "-auto" which will do the following:
1) Try a full import including all pyramids
2) For each directory, look for a csv file and if found, use it
3) No csv file found, look for a shape file and use the first string attribute as tilename
4) No shape file found, search for world files
5) No world files found -> abort
6) Import and anybody is happy
Oh Good, it is starting to get sick.
DDL: Have you ever setup a db2 or oracle db for ARCSDE. That is a challenge. I think I kept it simple as far as I can,
we are not talking about the users address book storing in MS Access
I think, i will leave it for the moment waiting for troubles reported from users.
christian
Andrea Aime writes:
Christian Müller ha scritto:
* --> added the following link to the tutorial http://star.pst.qub.ac.uk/idl/Image_Tiling.html
* --> added calculation hints to the tutorialNice
* I am confused about maxScaleDenominator. I looked at raster.sld and it is not used here.
* I think it is only needed if you want to stop zoom out at a given denominater ?It's used when you want the layer not to be draw after a certain
scale. If you set maxScaleDenominator = 100.000, the layer will be
drawn at 1:1000, 1:10000, but not at 1:200000.- the configUrl parameter requires a url, and might have to wait
for a time out if you don't type it properly.
What about supporting also a straight path on the file system,
without the protocol, to be chekced with new File(path).exists()?
* --> Changed param name from configUrl to config and accept files and URLs
* --> Doing the same for cvsUrl and shapeUrl params (not used in the tutorial) to be consistent
* --> Updated geotoools user guide and tutorialNice.
- in the osm.postgis.xml, 2 = bipolar, but it should actually
be "bilinear"
* --> fixed in tutorial
- any reason why the dll command creates many little sql statements?
It would be handier to have just two, one to create the tables,
the other to drop them... or not?
* Yes, beyond a tutorial, there are many reasons.
* 1) fillmeta.sql is only useable if you use a third party utility to import your image data
* 2) For some database systems it is simply not possible (db2 requires grid sizes when creating a spatial index,
* there is an own advise tool for calculating these sizes, useable after importing the image data)
* http://docs.codehaus.org/display/GEOTDOC/Creating+indexes+for+performanceSo these commands are not part of the standard sql generated?
Hmm... I understand the DB2 case, in which you have to hand tune,
but for the other databases it would be safe and simple to just
generate the index code as well.* 3) which tablespaces for physical data layout ?
* 4) The intension is that these sql files help the user to create his own scripts, fitting for his needsThe typical user I know from the GeoServer ml does not want to
think this much, many are looking for a "one button import"* 5) It is better to force the user to think over the db design, I want to avoid a disappointed users because of a bad db design
and they do not be aware of it.Ok. But you're forgetting about the typical user that is disappointed
because he cannot manage to get through the procedure and gives
up, frustrated for the time spend without results.- the request to put the drivers in lib/ext does not look like a
very good idea, people might have to use different versions of
the same jdbc driver in different applications,
and putting one in lib/ext prevents it. Any way to get the
same result by using -cp and leaving the jdbc drivers out
of lib/ext?
* Look here for the reason: http://docs.codehaus.org/display/GEOTDOC/Using+the+java+import+utility.
* Using the -Xbootclasspath/a: option for a tutorial is unsave, since it is an extended option not guaranteed to work on any java runtime.But it would still work on the most common java runtime
Ideally it would be nice to have the code dynamically load the
classes, a bit like SquirrelSQL does (you just point it to the
driver jar, it does the classloading at runtime).
I know that is going to be hard, so take it just as an
option for a future improvement.- the import is tedious if one has many levels. Any chance to
create some convention so that a single command can import
the whole pyramid?
* Not really, for an import with world files it would be manageable.
* But if you look at the other possibilities, having a shape file or csv file
* for each pyramid, things get complicated, look here for the possibilities
* http://docs.codehaus.org/display/GEOTDOC/Using+the+java+import+utilityHmmm... I don't believe many people will ever manage to get thru
all these commands properly. Just look at how many problems have
been reported by the users on the plain mosaic plugin just because
the property file was not correctly setup (the reader for that
property file is fragile, but still, many people get simply lost
in the process). I've been telling Simone the property file should
be auto-generated to avoid this hassle on the users list, I guess
after some tens of mails about that file config he started to
agree with meAnyways, better to have one more mosaic storage option that's
hard to get started with, but which in the end works, than
no option at allCheers
Andrea--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.
Christian Müller ha scritto:
Hi Andrea, I think the problem is that the typical user is a Windows user :-).
Hmmm... I would not count on it, Ubuntu has lowered the bar enough
that you'll get Ubuntu users that are just not much interested
in the details as a Windows one
But I know,
1) everything should happen with one click,
2) nobody takes care what is happening
3) anybody will see the result at once
4) if there are problems,nobody has an idea and the software is bad
Yeah, that's pretty much the picture I see, and more often than
I'd like to.
Ok, some proposals
Import:
I can introduce a new switch "-auto" which will do the following:
1) Try a full import including all pyramids
2) For each directory, look for a csv file and if found, use it
3) No csv file found, look for a shape file and use the first string attribute as tilename
4) No shape file found, search for world files
5) No world files found -> abort
6) Import and anybody is happy
Oh Good, it is starting to get sick.
I guess looking for whatever gdal_retile generates is enough
DDL: Have you ever setup a db2 or oracle db for ARCSDE. That is a challenge.
Usually I call that un-necessary evil (every time I use Oracle).
I think I kept it simple as far as I can,
we are not talking about the users address book storing in MS Access
I think, i will leave it for the moment waiting for troubles reported from users.
Sure, no problem
Cheers
Andrea
--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.