[Geoserver-users] Serving a large mosaic

Jon_Britton2 · January 14, 2008, 3:11pm

Hi,
I've got spot data for the whole of the UK. The full country consists of 35
tiff tiles, each around 55mb (without overviews etc). I've added overviews
to them and I've tried creating a mosaic of a few (using just 14 images),
however this works VERY slowly. I'm on an Intel Xeon 1.80GHz machine and
don't really have access to anything better for the minute.

Would it be even remotely possible to display this data locally on this
machine through GeoServer? Or am I wasting my time?

What would it take to handle this sort of data?

Thanks!
Jon.
--
View this message in context: http://www.nabble.com/Serving-a-large-mosaic-tp14803179p14803179.html
Sent from the GeoServer - User mailing list archive at Nabble.com.

Andrea_Aime3 · January 14, 2008, 3:15pm

Jon Britton ha scritto:

Hi,
I've got spot data for the whole of the UK. The full country consists of 35
tiff tiles, each around 55mb (without overviews etc). I've added overviews
to them and I've tried creating a mosaic of a few (using just 14 images),
however this works VERY slowly. I'm on an Intel Xeon 1.80GHz machine and
don't really have access to anything better for the minute.

Would it be even remotely possible to display this data locally on this
machine through GeoServer? Or am I wasting my time?

What would it take to handle this sort of data?

Well, afaik Simone is serving some tens of gigabytes worth of mosaics,
and it's working for him (caveat emptor, he's the one that made the
mosaic plugin). To have good performance you need to:
* install native jai and native jai image i/o
* make sure tiff have overviews
* make sure they are inner tiled (this is good only when zoomin in)
* double check the xxx.properties file for your mosaic is
properly setup, that is, it lists the resolutions for all overviews
and the like.

Cheers
Andrea

Simone3 · January 14, 2008, 4:41pm

Ciao Jon,
quick question since I am about to leave for today.

How did you add overviews? How did you build the index file? Did you
create a .properties file for the mosaic? My guess is that overviews
are not picked up correctly, but of course I might be wrong :-).

Ciao,
Simone.

On Jan 14, 2008 4:11 PM, Jon Britton <man_kills_everything@anonymised.com> wrote:

Hi,
I've got spot data for the whole of the UK. The full country consists of 35
tiff tiles, each around 55mb (without overviews etc). I've added overviews
to them and I've tried creating a mosaic of a few (using just 14 images),
however this works VERY slowly. I'm on an Intel Xeon 1.80GHz machine and
don't really have access to anything better for the minute.

Would it be even remotely possible to display this data locally on this
machine through GeoServer? Or am I wasting my time?

What would it take to handle this sort of data?

Thanks!
Jon.
--
View this message in context: http://www.nabble.com/Serving-a-large-mosaic-tp14803179p14803179.html
Sent from the GeoServer - User mailing list archive at Nabble.com.

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
-------------------------------------------------------
Eng. Simone Giannecchini
President /CEO GeoSolutions S.A.S.
Via Carignoni 51
55041 Camaiore (LU)
Italy

phone: +39 0584983027
fax: +39 0584983027
mob: +39 333 8128928

http://www.geo-solutions.it

-------------------------------------------------------

Jon_Britton2 · January 15, 2008, 9:55am

I added overviews with GDAL and created a mosaic using GDAL and the tutorial
on the wiki. I've got overviews of 1/2, 1/4, 1/8 and 1/16
Thanks,
Jon

Simone.Giannecchini wrote:

Ciao Jon,
quick question since I am about to leave for today.

How did you add overviews? How did you build the index file? Did you
create a .properties file for the mosaic? My guess is that overviews
are not picked up correctly, but of course I might be wrong :-).

Ciao,
Simone.

On Jan 14, 2008 4:11 PM, Jon Britton <man_kills_everything@anonymised.com>
wrote:

Hi,
I've got spot data for the whole of the UK. The full country consists of
35
tiff tiles, each around 55mb (without overviews etc). I've added
overviews
to them and I've tried creating a mosaic of a few (using just 14 images),
however this works VERY slowly. I'm on an Intel Xeon 1.80GHz machine and
don't really have access to anything better for the minute.

Would it be even remotely possible to display this data locally on this
machine through GeoServer? Or am I wasting my time?

What would it take to handle this sort of data?

Thanks!
Jon.
--
View this message in context:
http://www.nabble.com/Serving-a-large-mosaic-tp14803179p14803179.html
Sent from the GeoServer - User mailing list archive at Nabble.com.

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
-------------------------------------------------------
Eng. Simone Giannecchini
President /CEO GeoSolutions S.A.S.
Via Carignoni 51
55041 Camaiore (LU)
Italy

phone: +39 0584983027
fax: +39 0584983027
mob: +39 333 8128928

http://www.geo-solutions.it

-------------------------------------------------------

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
View this message in context: http://www.nabble.com/Serving-a-large-mosaic-tp14803179p14835837.html
Sent from the GeoServer - User mailing list archive at Nabble.com.

Jon_Britton2 · January 15, 2008, 9:58am

aaime wrote:

double check the xxx.properties file for your mosaic is properly setup,
that is, it lists the resolutions for all overviews

How should the .properties file be set up for overviews? I followed the
tutorial on the wiki and can't remember it mentioned overviews. Here is
what my file is like:

Name=2006mos
Levels=20.0,20.0
LevelsNum=1
Envelope2D=48760.0,153700.0 500940.0,1267240.0
NumFiles=14

I assume then that I've missed something?

Thanks,
Jon
--
View this message in context: http://www.nabble.com/Serving-a-large-mosaic-tp14803179p14835838.html
Sent from the GeoServer - User mailing list archive at Nabble.com.

Andrea_Aime3 · January 15, 2008, 10:06am

Jon Britton ha scritto:

aaime wrote:

double check the xxx.properties file for your mosaic is properly setup,
that is, it lists the resolutions for all overviews

How should the .properties file be set up for overviews? I followed the
tutorial on the wiki and can't remember it mentioned overviews. Here is
what my file is like:

Name=2006mos
Levels=20.0,20.0
LevelsNum=1
Envelope2D=48760.0,153700.0 500940.0,1267240.0
NumFiles=14

I assume then that I've missed something?

Definitely... the overviews levels are not there so the mosaic plugin
is not using them.
If you have 1/2 1/4 1/8 and 1/16 the file should look like:

Name=2006mos
Levels=20.0,20.0 40.0,40.0 80.0,80.0 160.0,160.0 320.0,320.0
LevelsNum=5
Envelope2D=48760.0,153700.0 500940.0,1267240.0
NumFiles=14

(that is, native, 20m per pixel, then 40 and so on)

Hope this helps
Cheers
Andrea

Jon_Britton2 · January 15, 2008, 10:31am

Thankyou! That's solved it, working nice and fast now (but still, my SPOT
images look funny )
Jon
--
View this message in context: http://www.nabble.com/Serving-a-large-mosaic-tp14803179p14836541.html
Sent from the GeoServer - User mailing list archive at Nabble.com.

Simone3 · January 15, 2008, 10:42am

Ciao Jon,
I understand the licensing problems on providing a some of you data.
However I would be great to have at least a gdalinfo for one of them
as well as the SLD you want to apply.
Are you sure you cannot even share a small subsampled subset with a cc
overlayed on top of it ?

Ciao,
Simone.

On Jan 15, 2008 11:31 AM, Jon Britton <man_kills_everything@anonymised.com> wrote:

Thankyou! That's solved it, working nice and fast now (but still, my SPOT
images look funny )
Jon
--
View this message in context: http://www.nabble.com/Serving-a-large-mosaic-tp14803179p14836541.html
Sent from the GeoServer - User mailing list archive at Nabble.com.

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
-------------------------------------------------------
Eng. Simone Giannecchini
President /CEO GeoSolutions S.A.S.
Via Carignoni 51
55041 Camaiore (LU)
Italy

phone: +39 0584983027
fax: +39 0584983027
mob: +39 333 8128928

http://www.geo-solutions.it

-------------------------------------------------------

Jon_Britton2 · January 15, 2008, 11:54am

Hi,
Sorry, are you talking about my problems with creating the mosaic (which is
solved now) or my problems in styling my SPOT 4 data?

For now I'll give you some background on it. I originally got the data in
.bil format and converted it to GeoTiff using GDAL and added tiling and
overviews. As far as styling goes, I just want to display it as R=3, G=2 and
B=3 (referring to the band numbers) preferrably with some contrast
enhancement since the image is a bit dim. The data displays fine in gvSIG,
which is only desktop software I have available at the moment.

I don't really have a specific SLD I want to use, I was just dabbling with
the ChannelSelection tag which GeoServer seems to ignore.

Then, there's the strange pixelation problem I get when I zoom in:
http://s256.photobucket.com/albums/hh190/dead_passive/?action=view&current=wierd.jpg

Here is a small section of one of my files.
http://download.yousendit.com/5F06A4C450D57959

I haven't tried it in GeoServer yet.

Thanks,
Jon

EDIT: here's the gdalinfo for the original file...

Driver: GTiff/GeoTIFF
Files: imagery1.tif
Size is 3839, 3675
Coordinate System is:
PROJCS["unnamed",
    GEOGCS["OSGB 1936",
        DATUM["unknown",
            SPHEROID["unretrievable - using WGS84",6378137,298.257223563]],
        PRIMEM["Greenwich",0],
        UNIT["degree",0.0174532925199433]],
    UNIT["metre",1,
        AUTHORITY["EPSG","9001"]],
    AUTHORITY["EPSG","27700"]]
Origin = (383360.000000000000000,557880.000000000000000)
Pixel Size = (20.000000000000000,-20.000000000000000)
Metadata:
  AREA_OR_POINT=Area
Corner Coordinates:
Upper Left ( 383360.000, 557880.000)
Lower Left ( 383360.000, 484380.000)
Upper Right ( 460140.000, 557880.000)
Lower Right ( 460140.000, 484380.000)
Center ( 421750.000, 521130.000)
Band 1 Block=256x256 Type=Byte, ColorInterp=Red
  Overviews: 1920x1838, 960x919, 480x460, 240x230
  Mask Flags: PER_DATASET ALPHA
Band 2 Block=256x256 Type=Byte, ColorInterp=Green
  Overviews: 1920x1838, 960x919, 480x460, 240x230
  Mask Flags: PER_DATASET ALPHA
Band 3 Block=256x256 Type=Byte, ColorInterp=Blue
  Overviews: 1920x1838, 960x919, 480x460, 240x230
  Mask Flags: PER_DATASET ALPHA
Band 4 Block=256x256 Type=Byte, ColorInterp=Alpha
  Overviews: 1920x1838, 960x919, 480x460, 240x230

Simone.Giannecchini wrote:

Ciao Jon,
I understand the licensing problems on providing a some of you data.
However I would be great to have at least a gdalinfo for one of them
as well as the SLD you want to apply.
Are you sure you cannot even share a small subsampled subset with a cc
overlayed on top of it ?

Ciao,
Simone.

On Jan 15, 2008 11:31 AM, Jon Britton <man_kills_everything@anonymised.com>
wrote:

Thankyou! That's solved it, working nice and fast now (but still, my SPOT
images look funny )
Jon
--
View this message in context:
http://www.nabble.com/Serving-a-large-mosaic-tp14803179p14836541.html
Sent from the GeoServer - User mailing list archive at Nabble.com.

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
-------------------------------------------------------
Eng. Simone Giannecchini
President /CEO GeoSolutions S.A.S.
Via Carignoni 51
55041 Camaiore (LU)
Italy

phone: +39 0584983027
fax: +39 0584983027
mob: +39 333 8128928

http://www.geo-solutions.it

-------------------------------------------------------

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
View this message in context: http://www.nabble.com/Serving-a-large-mosaic-tp14803179p14838195.html
Sent from the GeoServer - User mailing list archive at Nabble.com.