[Geoserver-devel] Image pyramid improvements and tutorial

GeoServer GeoServer Developer
,
1

Hi Andrea,
      I've been working on giving it a spin with our own data. The tutorial
is excellent. Easy to read and follow.
      I'm currently testing it on a Windows 2003 server with nightly build
geoserver-2.0.x-2010-03-03-bin.zip, and
geoserver-2.0.2-SNAPSHOT-pyramid-plugin.zip
      The test I've been doing is with 0.5ft (0.15m) color orthophotos that
sum to about 150GB as TIFFs. They are currently tiled as 5000x4000 images.
I've retiled them with gdal_retile as 2048x2048 images with overviews with
the following command:

gdal_retile -v -s_srs "EPSG:102722" -levels 5 -ps 2048 2048 -targetdir test3
--optfile tilelist.txt

I'm currently working on a subset of the data for testing.
Checking all the tiles generated from gdal_retile, they all look fine in an
image viewer, and check out (with a spot check) with gdalinfo.

When I load them in GeoServer, I get a strange effect. Some of the tiles
never render at their full resolution for a given zoom level, but at a
reduced resolution. If I load ~16 tiles vs. loading e.g. 40+ tiles the
places where this happens shifts, i.e. tiles that were a problem before stop
being a problem, while other tiles exhibit the same behavior.

Does this make sense, is this an effect you've seen, and would images and
log files be helpful?

Thanks,
Steve

Andrea Aime-4 wrote:

Hi all,
lately I've been working, under OpenGeo request, to improve
the situation vs image pyramids, in particular, making it
easier to configure a file based one.

...

Is anyone interested to give the tutorial a spin, a review?
Is there anything missing or that could be done better?

Cheers
Andrea

--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.

------------------------------------------------------------------------------
SOLARIS 10 is the OS for Data Centers - provides features such as DTrace,
Predictive Self Healing and Award Winning ZFS. Get Solaris 10 NOW
http://p.sf.net/sfu/solaris-dev2dev
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

--
View this message in context: http://old.nabble.com/Image-pyramid-improvements-and-tutorial-tp27608654p27910740.html
Sent from the GeoServer - Dev mailing list archive at Nabble.com.

2

Stephen Mather ha scritto:

Hi Andrea,
      I've been working on giving it a spin with our own data. The tutorial
is excellent. Easy to read and follow.
      I'm currently testing it on a Windows 2003 server with nightly build
geoserver-2.0.x-2010-03-03-bin.zip, and
geoserver-2.0.2-SNAPSHOT-pyramid-plugin.zip
      The test I've been doing is with 0.5ft (0.15m) color orthophotos that
sum to about 150GB as TIFFs. They are currently tiled as 5000x4000 images. I've retiled them with gdal_retile as 2048x2048 images with overviews with
the following command:

gdal_retile -v -s_srs "EPSG:102722" -levels 5 -ps 2048 2048 -targetdir test3
--optfile tilelist.txt

I'm currently working on a subset of the data for testing.
Checking all the tiles generated from gdal_retile, they all look fine in an
image viewer, and check out (with a spot check) with gdalinfo.

When I load them in GeoServer, I get a strange effect. Some of the tiles
never render at their full resolution for a given zoom level, but at a
reduced resolution. If I load ~16 tiles vs. loading e.g. 40+ tiles the
places where this happens shifts, i.e. tiles that were a problem before stop
being a problem, while other tiles exhibit the same behavior.

Does this make sense, is this an effect you've seen, and would images and
log files be helpful?

I actually never seen this effect.
Can you open a report on Jira?
To be effective we should also be able to reproduce the problem.
Can you generate it with any freely available imagery set?

Cheers
Andrea

--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.

3

Hi All,

Does anyone know if there's been some change in the way HTTP GET WMS
requests are done to Geoserver when combining LAYERS, STYLES and CQL_FILTERS
?

When i do the request:
http://localhost:8080/Geoserver/wms? (...) CQL_FILTER=TYPE=13& (...)
LAYERS=layer_test1;layer_test2& (...) STYLES=sld_test1;sld_test2& (...)
upon Geoserver 1.7.4 it returns an image. Ok.

But instead, when i do the exact same request on Geoserver 1.7.7 it returns
an exception:

<?xml version="1.0" encoding="UTF-8" standalone="no"?><!DOCTYPE
ServiceExceptionReport SYSTEM
"http://schemas.opengis.net/wms/1.1.1/WMS_exception_1_1_1.dtd&quot;&gt;
<ServiceExceptionReport version="1.1.1" > <ServiceException>
      java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
Index: 1, Size: 1
</ServiceException></ServiceExceptionReport>

It only works if I define 3 filters, one for each layer.
Wasn't this supposed to assume the same filter to all layers without having
do repeat them?

Thanks in advance.

_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

4

Hi Andrea,
  I've submitted a bug report on JIRA. I have a ~80MB example zip.
How would you prefer I send that to you?

Steve

Stephen Mather, GIS Manager
Cleveland Metroparks
4101 Fulton Pkwy
Cleveland, OH 44144
svm@anonymised.com
Phone: (216) 635-3243
FAX: (216) 635-3286

-----Original Message-----
From: Andrea Aime [mailto:aaime@anonymised.com]
Sent: Thursday, March 18, 2010 11:22 AM
To: Stephen Mather
Cc: geoserver-devel@lists.sourceforge.net
Subject: Re: [Geoserver-devel] Image pyramid improvements and tutorial

Stephen Mather ha scritto:

Hi Andrea,
      I've been working on giving it a spin with our own data. The
tutorial is excellent. Easy to read and follow.
      I'm currently testing it on a Windows 2003 server with nightly
build geoserver-2.0.x-2010-03-03-bin.zip, and
geoserver-2.0.2-SNAPSHOT-pyramid-plugin.zip
      The test I've been doing is with 0.5ft (0.15m) color orthophotos
that sum to about 150GB as TIFFs. They are currently tiled as
5000x4000 images. I've retiled them with gdal_retile as 2048x2048
images with overviews with the following command:

gdal_retile -v -s_srs "EPSG:102722" -levels 5 -ps 2048 2048 -targetdir
test3 --optfile tilelist.txt

I'm currently working on a subset of the data for testing. Checking
all the tiles generated from gdal_retile, they all look fine in an
image viewer, and check out (with a spot check) with gdalinfo.

When I load them in GeoServer, I get a strange effect. Some of the
tiles never render at their full resolution for a given zoom level,
but at a reduced resolution. If I load ~16 tiles vs. loading e.g. 40+
tiles the places where this happens shifts, i.e. tiles that were a
problem before stop being a problem, while other tiles exhibit the
same behavior.

Does this make sense, is this an effect you've seen, and would images
and log files be helpful?

I actually never seen this effect.
Can you open a report on Jira?
To be effective we should also be able to reproduce the problem. Can you
generate it with any freely available imagery set?

Cheers
Andrea

--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.

5

Qcuik questions:
- are you adding a reprojection in the mix while you are doing this tests?
- are all your granules of the same resolution?

Simone.
-------------------------------------------------------
Ing. Simone Giannecchini
GeoSolutions S.A.S.
Founder - Software Engineer
Via Carignoni 51
55041 Camaiore (LU)
Italy

phone: +39 0584983027
fax: +39 0584983027
mob: +39 333 8128928

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.linkedin.com/in/simonegiannecchini
http://twitter.com/simogeo

-------------------------------------------------------

On Tue, Mar 16, 2010 at 1:13 PM, Stephen Mather
<svm@anonymised.com> wrote:

Hi Andrea,
I've been working on giving it a spin with our own data. The tutorial
is excellent. Easy to read and follow.
I'm currently testing it on a Windows 2003 server with nightly build
geoserver-2.0.x-2010-03-03-bin.zip, and
geoserver-2.0.2-SNAPSHOT-pyramid-plugin.zip
The test I've been doing is with 0.5ft (0.15m) color orthophotos that
sum to about 150GB as TIFFs. They are currently tiled as 5000x4000 images.
I've retiled them with gdal_retile as 2048x2048 images with overviews with
the following command:

gdal_retile -v -s_srs "EPSG:102722" -levels 5 -ps 2048 2048 -targetdir test3
--optfile tilelist.txt

I'm currently working on a subset of the data for testing.
Checking all the tiles generated from gdal_retile, they all look fine in an
image viewer, and check out (with a spot check) with gdalinfo.

When I load them in GeoServer, I get a strange effect. Some of the tiles
never render at their full resolution for a given zoom level, but at a
reduced resolution. If I load ~16 tiles vs. loading e.g. 40+ tiles the
places where this happens shifts, i.e. tiles that were a problem before stop
being a problem, while other tiles exhibit the same behavior.

Does this make sense, is this an effect you've seen, and would images and
log files be helpful?

Thanks,
Steve

Andrea Aime-4 wrote:

Hi all,
lately I've been working, under OpenGeo request, to improve
the situation vs image pyramids, in particular, making it
easier to configure a file based one.

...

Is anyone interested to give the tutorial a spin, a review?
Is there anything missing or that could be done better?

Cheers
Andrea

--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.

------------------------------------------------------------------------------
SOLARIS 10 is the OS for Data Centers - provides features such as DTrace,
Predictive Self Healing and Award Winning ZFS. Get Solaris 10 NOW
http://p.sf.net/sfu/solaris-dev2dev
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

--
View this message in context: http://old.nabble.com/Image-pyramid-improvements-and-tutorial-tp27608654p27910740.html
Sent from the GeoServer - Dev mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

6

Qcuik questions:
- are you adding a reprojection in the mix while you are doing this tests?

I have tried it both with and without reprojection, to the same effect.

- are all your granules of the same resolution?

Yes, they are all 0.15m pixels in 5000x4000 images. I have tried both
leaving the existing 0 level images at 5000x4000 (-pyramidOnly) and retiling
to 2048x2048, both with the same result.

Steve

On Tue, Mar 16, 2010 at 1:13 PM, Stephen Mather
<svm@anonymised.com> wrote:

Hi Andrea,
I've been working on giving it a spin with our own data. The
tutorial is excellent. Easy to read and follow.
I'm currently testing it on a Windows 2003 server with nightly
build geoserver-2.0.x-2010-03-03-bin.zip, and
geoserver-2.0.2-SNAPSHOT-pyramid-plugin.zip
The test I've been doing is with 0.5ft (0.15m) color orthophotos
that sum to about 150GB as TIFFs. They are currently tiled as
5000x4000 images. I've retiled them with gdal_retile as 2048x2048
images with overviews with the following command:

gdal_retile -v -s_srs "EPSG:102722" -levels 5 -ps 2048 2048 -targetdir
test3 --optfile tilelist.txt

I'm currently working on a subset of the data for testing. Checking
all the tiles generated from gdal_retile, they all look fine in an
image viewer, and check out (with a spot check) with gdalinfo.

When I load them in GeoServer, I get a strange effect. Some of the
tiles never render at their full resolution for a given zoom level,
but at a reduced resolution. If I load ~16 tiles vs. loading e.g. 40+
tiles the places where this happens shifts, i.e. tiles that were a
problem before stop being a problem, while other tiles exhibit the
same behavior.

Does this make sense, is this an effect you've seen, and would images
and log files be helpful?

Thanks,
Steve

Andrea Aime-4 wrote:

Hi all,
lately I've been working, under OpenGeo request, to improve the
situation vs image pyramids, in particular, making it easier to
configure a file based one.

...

Is anyone interested to give the tutorial a spin, a review? Is there
anything missing or that could be done better?

Cheers
Andrea

--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.

---------------------------------------------------------------------
---------
SOLARIS 10 is the OS for Data Centers - provides features such as DTrace,
Predictive Self Healing and Award Winning ZFS. Get Solaris 10 NOW
http://p.sf.net/sfu/solaris-dev2dev
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

--
View this message in context:

http://old.nabble.com/Image-pyramid-improvements-and-tutorial-tp27608654p279
10740.html

Sent from the GeoServer - Dev mailing list archive at Nabble.com.

----------------------------------------------------------------------
--------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

7

I guess that doing some testing with your data could be helpful.
I have some uncommitted fixes/improvements that might solve these issues.

Simone

On 3/19/10, Stephen V. Mather <svm@anonymised.com> wrote:

Qcuik questions:
- are you adding a reprojection in the mix while you are doing this tests?

I have tried it both with and without reprojection, to the same effect.

- are all your granules of the same resolution?

Yes, they are all 0.15m pixels in 5000x4000 images. I have tried both
leaving the existing 0 level images at 5000x4000 (-pyramidOnly) and retiling
to 2048x2048, both with the same result.

Steve

On Tue, Mar 16, 2010 at 1:13 PM, Stephen Mather
<svm@anonymised.com> wrote:

Hi Andrea,
     I've been working on giving it a spin with our own data. The
tutorial is excellent. Easy to read and follow.
     I'm currently testing it on a Windows 2003 server with nightly
build geoserver-2.0.x-2010-03-03-bin.zip, and
geoserver-2.0.2-SNAPSHOT-pyramid-plugin.zip
     The test I've been doing is with 0.5ft (0.15m) color orthophotos
that sum to about 150GB as TIFFs. They are currently tiled as
5000x4000 images. I've retiled them with gdal_retile as 2048x2048
images with overviews with the following command:

gdal_retile -v -s_srs "EPSG:102722" -levels 5 -ps 2048 2048 -targetdir
test3 --optfile tilelist.txt

I'm currently working on a subset of the data for testing. Checking
all the tiles generated from gdal_retile, they all look fine in an
image viewer, and check out (with a spot check) with gdalinfo.

When I load them in GeoServer, I get a strange effect. Some of the
tiles never render at their full resolution for a given zoom level,
but at a reduced resolution. If I load ~16 tiles vs. loading e.g. 40+
tiles the places where this happens shifts, i.e. tiles that were a
problem before stop being a problem, while other tiles exhibit the
same behavior.

Does this make sense, is this an effect you've seen, and would images
and log files be helpful?

Thanks,
Steve

Andrea Aime-4 wrote:

Hi all,
lately I've been working, under OpenGeo request, to improve the
situation vs image pyramids, in particular, making it easier to
configure a file based one.

...

Is anyone interested to give the tutorial a spin, a review? Is there
anything missing or that could be done better?

Cheers
Andrea

--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.

---------------------------------------------------------------------
---------
SOLARIS 10 is the OS for Data Centers - provides features such as DTrace,
Predictive Self Healing and Award Winning ZFS. Get Solaris 10 NOW
http://p.sf.net/sfu/solaris-dev2dev
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

--
View this message in context:

http://old.nabble.com/Image-pyramid-improvements-and-tutorial-tp27608654p279
10740.html

Sent from the GeoServer - Dev mailing list archive at Nabble.com.

----------------------------------------------------------------------
--------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

--
-------------------------------------------------------
Ing. Simone Giannecchini
GeoSolutions S.A.S.
Founder - Software Engineer
Via Carignoni 51
55041 Camaiore (LU)
Italy

phone: +39 0584983027
fax: +39 0584983027
mob: +39 333 8128928

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.linkedin.com/in/simonegiannecchini
http://twitter.com/simogeo

-------------------------------------------------------

8

Simone Giannecchini ha scritto:

I guess that doing some testing with your data could be helpful.
I have some uncommitted fixes/improvements that might solve these issues.

Steve was offering to make a 80MB sample data set available.
I guess you and Steve can look into how to transfer it? :slight_smile:

Cheers
Andrea

--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.

9

The test data are a little bigger than expected-- 195MB. They will be
available shortly at the following FTP site:

ftp://ftp.clevelandmetroparks.com/pdnr/out

Thanks,
Steve

Stephen Mather, GIS Manager
Cleveland Metroparks
4101 Fulton Pkwy
Cleveland, OH 44144
svm@anonymised.com
Phone: (216) 635-3243
FAX: (216) 635-3286

-----Original Message-----
From: simboss1@anonymised.com [mailto:simboss1@anonymised.com] On Behalf Of Simone
Giannecchini
Sent: Saturday, March 20, 2010 10:16 AM
To: svm@anonymised.com
Cc: geoserver-devel@lists.sourceforge.net
Subject: Re: [Geoserver-devel] Image pyramid improvements and tutorial

I guess that doing some testing with your data could be helpful. I have some
uncommitted fixes/improvements that might solve these issues.

Simone

On 3/19/10, Stephen V. Mather <svm@anonymised.com> wrote:

Qcuik questions:
- are you adding a reprojection in the mix while you are doing this
tests?

I have tried it both with and without reprojection, to the same
effect.

- are all your granules of the same resolution?

Yes, they are all 0.15m pixels in 5000x4000 images. I have tried both
leaving the existing 0 level images at 5000x4000 (-pyramidOnly) and
retiling to 2048x2048, both with the same result.

Steve

On Tue, Mar 16, 2010 at 1:13 PM, Stephen Mather
<svm@anonymised.com> wrote:

Hi Andrea,
     I've been working on giving it a spin with our own data. The
tutorial is excellent. Easy to read and follow.
     I'm currently testing it on a Windows 2003 server with nightly
build geoserver-2.0.x-2010-03-03-bin.zip, and
geoserver-2.0.2-SNAPSHOT-pyramid-plugin.zip
     The test I've been doing is with 0.5ft (0.15m) color orthophotos
that sum to about 150GB as TIFFs. They are currently tiled as
5000x4000 images. I've retiled them with gdal_retile as 2048x2048
images with overviews with the following command:

gdal_retile -v -s_srs "EPSG:102722" -levels 5 -ps 2048 2048
-targetdir test3 --optfile tilelist.txt

I'm currently working on a subset of the data for testing. Checking
all the tiles generated from gdal_retile, they all look fine in an
image viewer, and check out (with a spot check) with gdalinfo.

When I load them in GeoServer, I get a strange effect. Some of the
tiles never render at their full resolution for a given zoom level,
but at a reduced resolution. If I load ~16 tiles vs. loading e.g.
40+ tiles the places where this happens shifts, i.e. tiles that were
a problem before stop being a problem, while other tiles exhibit the
same behavior.

Does this make sense, is this an effect you've seen, and would images
and log files be helpful?

Thanks,
Steve

Andrea Aime-4 wrote:

Hi all,
lately I've been working, under OpenGeo request, to improve the
situation vs image pyramids, in particular, making it easier to
configure a file based one.

...

Is anyone interested to give the tutorial a spin, a review? Is there
anything missing or that could be done better?

Cheers
Andrea

--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.

--------------------------------------------------------------------
-
---------
SOLARIS 10 is the OS for Data Centers - provides features such as

DTrace,

Predictive Self Healing and Award Winning ZFS. Get Solaris 10 NOW
http://p.sf.net/sfu/solaris-dev2dev
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

--
View this message in context:

http://old.nabble.com/Image-pyramid-improvements-and-tutorial-tp276086
54p279
10740.html

Sent from the GeoServer - Dev mailing list archive at Nabble.com.

---------------------------------------------------------------------
-
--------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

----------------------------------------------------------------------
--------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

--
-------------------------------------------------------
Ing. Simone Giannecchini
GeoSolutions S.A.S.
Founder - Software Engineer
Via Carignoni 51
55041 Camaiore (LU)
Italy

phone: +39 0584983027
fax: +39 0584983027
mob: +39 333 8128928

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.linkedin.com/in/simonegiannecchini
http://twitter.com/simogeo

-------------------------------------------------------