[Geoserver-users] Why does image pyramid take so long?

Hi,

I have been following these guidelines - http://elogeo.nottingham.ac.uk/xmlui/bitstream/handle/url/226/gs_steroids_sgiannec_foss4g2013_01.03.pdf?sequence=1
So I have ended up with a test directory (1.3TB) that includes the tiles in folders and sub-folders.
I used the geoserver web interface to create a new image pyramid coverage and itv ran for over 30 hours. It has created shapefiles for every level and pyramid prj and property files. When tested in layer preview, it loads up a black tile in wrong coordinates.

is this time consuming behaviour normal?
Any ideas why the pyramid turned out to be a black image with wrong coords?
My total data set size is over 5TB. So does that mean that I will have to make the 1.3TB super fast in order for the total data to be served fast enough?

My geo-server version is 2.4.1 and the hardware meets suggested requirements as well.

Any advice would be greatly appreciated if you have experience in similar size data sets.

All the best,
Antony

Hi Antony,

can you provide us a few gdalinfo on the files contained in a couple of sub-folders?

(so we can take a look at the images structure)

Cheers,
Daniele

ยทยทยท

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Daniele Romagnoli
Senior Software Engineer

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272

http://www.geo-solutions.it
http://twitter.com/geosolutions_it


On Mon, Nov 4, 2013 at 1:03 PM, antchrysochoou <antchrysochoou@anonymised.com> wrote:

Hi,

I have been following these guidelines - http://elogeo.nottingham.ac.uk/xmlui/bitstream/handle/url/226/gs_steroids_sgiannec_foss4g2013_01.03.pdf?sequence=1
So I have ended up with a test directory (1.3TB) that includes the tiles in folders and sub-folders.
I used the geoserver web interface to create a new image pyramid coverage and itv ran for over 30 hours. It has created shapefiles for every level and pyramid prj and property files. When tested in layer preview, it loads up a black tile in wrong coordinates.

is this time consuming behaviour normal?
Any ideas why the pyramid turned out to be a black image with wrong coords?
My total data set size is over 5TB. So does that mean that I will have to make the 1.3TB super fast in order for the total data to be served fast enough?

My geo-server version is 2.4.1 and the hardware meets suggested requirements as well.

Any advice would be greatly appreciated if you have experience in similar size data sets.

All the best,
Antony


November Webinars for C, C++, Fortran Developers
Accelerate application performance with scalable programming models. Explore
techniques for threading, error checking, porting, and tuning. Get the most
from the latest Intel processors and coprocessors. See abstracts and register
http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk


Geoserver-users mailing list
Geoserver-users@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Hi Daniele,

I found out why it was so slow and it had to do with the server settings. Tomcat was set to use a max of 1 GB memory and it was using 128MB !!!!
So now, we configurated it properly and it is working faster.

Let me know if I should post this somehwere to close the question.

thank you for your reply!

Kind regards,
Antony

On 07/11/2013 13:57, Daniele Romagnoli wrote:

Hi Antony,
can you provide us a few gdalinfo on the files contained in a couple of sub-folders?
(so we can take a look at the images structure)

Cheers,
Daniele

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Daniele Romagnoli
Senior Software Engineer

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

On Mon, Nov 4, 2013 at 1:03 PM, antchrysochoou <antchrysochoou@anonymised.com <mailto:antchrysochoou@anonymised.com>> wrote:

    Hi,

    I have been following these guidelines -
    http://elogeo.nottingham.ac.uk/xmlui/bitstream/handle/url/226/gs_steroids_sgiannec_foss4g2013_01.03.pdf?sequence=1
    So I have ended up with a test directory (1.3TB) that includes the
    tiles in folders and sub-folders.
    I used the geoserver web interface to create a new image pyramid
    coverage and itv ran for over 30 hours. It has created shapefiles
    for every level and pyramid prj and property files. When tested in
    layer preview, it loads up a black tile in wrong coordinates.
    is this time consuming behaviour normal?
    Any ideas why the pyramid turned out to be a black image with
    wrong coords?
    My total data set size is over 5TB. So does that mean that I will
    have to make the 1.3TB super fast in order for the total data to
    be served fast enough?

    My geo-server version is 2.4.1 and the hardware meets suggested
    requirements as well.

    Any advice would be greatly appreciated if you have experience in
    similar size data sets.

    All the best,
    Antony

    ------------------------------------------------------------------------------
    November Webinars for C, C++, Fortran Developers
    Accelerate application performance with scalable programming
    models. Explore
    techniques for threading, error checking, porting, and tuning. Get
    the most
    from the latest Intel processors and coprocessors. See abstracts
    and register
    http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk
    _______________________________________________
    Geoserver-users mailing list
    Geoserver-users@lists.sourceforge.net
    <mailto:Geoserver-users@lists.sourceforge.net>
    https://lists.sourceforge.net/lists/listinfo/geoserver-users