[Geoserver-users] image mosaic performance

All,

Following suggestions I have tried the following.

  • translated my tiffs to geotiffs (gdal_translate -of GTiff inFile outFile)
  • built overviews on the geotiffs (gdaladdo -r average file.tif 2 4 8 16)
  • file size increased from 5.8mb to 7.9mb (about 250 files)
  • rebuild the index shapefile
  • completely recreate the image mosaic in geoserver

RESULTS
A minor improvement; I was hoping for more.
Rendering speeds at the browser:
scale time
1:8K 10s
1:4K 10s
1:2K 6s
1:1K 2s

PREVIOUS RESULTS
secs secs
to to
render render
scale raster vector
1:6000 15 3
1:3000 15 3
1:1500 5 3
1:800 3 2
1:400 3 2
1:200 2 2

Did I build the overviews properly (with the correct parms?)
Any recomendations on my next step?

Thanks
Paul

Hi,

There has been a lot of discussion about raster image performance on UMN Mapserver user list. The situation with Geoserver is mostly the same so I believe making a search on Mapserver archive would be useful for you.
Briefly, you should first have a look at your imagery and think about what happens with it at different zoom levels. Your main problem is with the far-away zoom levels, and in that case the main reason for the slow response can not be removed by making overviews for individual images.
When you have zoomed out very far, the server must open many physical image files to fill in the requested window. Having images with overviews do help in getting out the low-resolution version of each file that must be opened, but just accessing the files on a disk system for reading can take most of time. The key to speed in this case is to create a separate physical low resolution overview image that is covering larger area or perhaps the whole image layer. Then it will be enough to open just one image file and it will be much faster. When the user has zoomed in so close that the resolusion of the overview image is not good enough the system should be configured to switch to use the original high resolution images through image mosaic. Because the requested area is not so large anymore, it will be enough to access just a few images on disk. This way you might get 2 second response time at each zoom level. That is not especially good timing for my mind either, but if you are measuring times to get single requests visible on screen with browser it is a bit difficult to say if there is something strange in the chain between your Geoserver and browser. Making for example a hundred request with changing bounding boxes with wget or some performance monitor program like jmeter would give more accurate results.

Your images are also rather small as geospatial images, and they could be mosaiced even all together into one single 2 gigabyte Geotiff file with internal overviews. By doing so the performance should be good without having to create two image layers and scale dependent switching system between them.

-Jukka Rahkonen-


Lähettäjä: geoserver-users-bounces@lists.sourceforge.net [mailto:geoserver-users-bounces@lists.sourceforge.net] Puolesta Paul McCullough
Lähetetty: 7. joulukuuta 2007 21:32
Vastaanottaja: geoserver-users@lists.sourceforge.net
Kopio: geoserver-users@lists.sourceforge.net; geoserver-users-bounces@lists.sourceforge.net
Aihe: Re: [Geoserver-users] image mosaic performance

All,

Following suggestions I have tried the following.

  • translated my tiffs to geotiffs (gdal_translate -of GTiff inFile outFile)
  • built overviews on the geotiffs (gdaladdo -r average file.tif 2 4 8 16)
  • file size increased from 5.8mb to 7.9mb (about 250 files)
  • rebuild the index shapefile
  • completely recreate the image mosaic in geoserver

RESULTS
A minor improvement; I was hoping for more.
Rendering speeds at the browser:
scale time
1:8K 10s
1:4K 10s
1:2K 6s
1:1K 2s

PREVIOUS RESULTS
secs secs
to to
render render
scale raster vector
1:6000 15 3
1:3000 15 3
1:1500 5 3
1:800 3 2
1:400 3 2
1:200 2 2

Did I build the overviews properly (with the correct parms?)
Any recomendations on my next step?

Thanks
Paul

I am seeing the same issue. We have RPF data (1000 300kb images) that is
slow to render with ImageMosaic at far zoomed out levels.

Can we use controlFlow extension to block any rendering that takes more than
X seconds?

I have tried control flow with timeout=10 and also GeoServer WMS
configuration "max rendering time"=10seconds.

Neither seem to work but maybe it's because I am just constantly sending in
too many requests (from my CesiumJS client map).

Any advice? Thank you!

Rahkonen Jukka (Tike) wrote

Hi,

There has been a lot of discussion about raster image performance on UMN
Mapserver user list. The situation with Geoserver is mostly the same so I
believe making a search on Mapserver archive would be useful for you.
Briefly, you should first have a look at your imagery and think about what
happens with it at different zoom levels. Your main problem is with the
far-away zoom levels, and in that case the main reason for the slow
response can not be removed by making overviews for individual images.
When you have zoomed out very far, the server must open many physical
image files to fill in the requested window. Having images with overviews
do help in getting out the low-resolution version of each file that must
be opened, but just accessing the files on a disk system for reading can
take most of time. The key to speed in this case is to create a separate
physical low resolution overview image that is covering larger area or
perhaps the whole image layer. Then it will be enough to open just one
image file and it will be much faster. When the user has zoomed in so
close that the resolusion of the overview image is not good enough the
system should be configured to switch to use the original high resolution
images through image mosaic. Because the requested area is not so large
anymore, it will be enough to access just a few images on disk. This way
you might get 2 second response time at each zoom level. That is not
especially good timing for my mind either, but if you are measuring times
to get single requests visible on screen with browser it is a bit
difficult to say if there is something strange in the chain between your
Geoserver and browser. Making for example a hundred request with changing
bounding boxes with wget or some performance monitor program like jmeter
would give more accurate results.

Your images are also rather small as geospatial images, and they could be
mosaiced even all together into one single 2 gigabyte Geotiff file with
internal overviews. By doing so the performance should be good without
having to create two image layers and scale dependent switching system
between them.

-Jukka Rahkonen-

________________________________

Lähettäjä:

geoserver-users-bounces@anonymised.com

[mailto:

geoserver-users-bounces@anonymised.com

] Puolesta Paul McCullough
Lähetetty: 7. joulukuuta 2007 21:32
Vastaanottaja:

geoserver-users@anonymised.com

Kopio:

geoserver-users@anonymised.com

;

geoserver-users-bounces@anonymised.com

Aihe: Re: [Geoserver-users] image mosaic performance

  All,
  
  Following suggestions I have tried the following.
   - translated my tiffs to geotiffs (gdal_translate -of GTiff inFile
outFile)
   - built overviews on the geotiffs (gdaladdo -r average file.tif 2 4 8
16)
   - file size increased from 5.8mb to 7.9mb (about 250 files)
   - rebuild the index shapefile
   - completely recreate the image mosaic in geoserver
  
  RESULTS
  A minor improvement; I was hoping for more.
  Rendering speeds at the browser:
  scale time
  1:8K 10s
  1:4K 10s
  1:2K 6s
  1:1K 2s
  
  PREVIOUS RESULTS
          secs secs
         to to
         render render
  scale raster vector
  1:6000 15 3
  1:3000 15 3
  1:1500 5 3
  1:800 3 2
  1:400 3 2
  1:200 2 2
  
  Did I build the overviews properly (with the correct parms?)
  Any recomendations on my next step?
  
  Thanks
  Paul
  
-------------------------------------------------------------------------
SF.Net email is sponsored by:
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
Geoserver-users mailing list

Geoserver-users@anonymised.com

https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
Sent from: http://osgeo-org.1560.x6.nabble.com/GeoServer-User-f3786390.html

I am seeing the same issue. We have RPF data (1000 300kb images) that is
slow to render with ImageMosaic at far zoomed out levels.

Can we use controlFlow extension to block any rendering that takes more than
X seconds?

I have tried control flow with timeout=10 and also GeoServer WMS
configuration “max rendering time”=10seconds.

The raster operations cannot be blocked by the rendering timeout, only the vector rendering (feature by feature)
is currently easy to block.

Neither seem to work but maybe it’s because I am just constantly sending in
too many requests (from my CesiumJS client map).

Any advice? Thank you!

Set a scale dependency so that you don’t end up opening that many files,
eventually replacing them with a seamless scaled down mosaic (single file, made
with GDAL) at those zoom levels.

Cheers
Andrea

···

GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.