Hi,
If you want to keep all 60000 layers which is not optimal as Andrea just wrote, the most obvious thing to do is to create overviews for the images.
gdaladdo –r average 2 4 8 16 32 64 image.tif --config COMPRESS_OVERVIEW DEFLATE
It is worth making a test before reprojecting images physically into EPSG:3857. On-the-fly reprojecting is not necessarily so slow.
I saw that you have used the BIGTIFF=YES option. It is not needed for your small images and it probably makes reading the images a little bit slower but I guess that the difference is marginal. Tile size 256x256 could be a bit faster than 512x512 but that’t also marginal.
-Jukka Rahkonen-
basiliosz wrote:
As 60000 layers. They are not contiguous, are completely separate and are viewed strictly one at a time per user.
The scenario: user selects a city and a couple parameters, the JS code figures out what layer to request, then submits the request for that layer to Geoserver. Eventually the layer is added to the map.
Max
···
On 17 Mar 2016 13:44, “Andrea Aime” <andrea.aime@…1107…> wrote:
Hi,
just to be sure, how have you published them?
As a single image mosaic or as 60000 separate layers?
Cheers
Andrea
On Thu, Mar 17, 2016 at 12:56 PM, Max <basiliosz@…84…> wrote:
Thanks - because of the origin of the data the Geotiffs are not
adjacent, there’s a little group for each of about 500 cities, but
strictly one of them at a time is shown, so I’m not sure how much
that’s going to help. If there’s something computationally intensive
going on at the server’s end, that’s going to be a drag in any case.Going through the Geoserver on Steroids presentation now, thanks!
Max
On 17 March 2016 at 11:16, Ian Turton <ijturton@…84…> wrote:
Those sound like small tiffs so you would be better combining them together
to avoid opening too many files at a time. Have a look at gdalbuildvrt to
make a virtual raster catalogue that you can then convert into a tiled
compressed geotiff that is in your main output projection.Have a read through the annual “GeoServer on Steroids” presentation
(http://www.slideshare.net/geosolutions/geoserver-on-steroids-foss4g-2015 )
for more ideas.#Ian
On 17 March 2016 at 09:51, Max <basiliosz@…84…> wrote:
I have a large number - more than 60 thousand - of relatively small
Geotiffs, usually from 2 to 12 Mb. I have a web client that uses
Leaflet to view them, but things are quite slow even inside our own
network. I have a hunch that both Geoserver and these Geotiffs are not
configured in the best way possible.The tiffs were generated from a bunch of EPSG:3035 Ascii grid files
using this command:for f in *.rsl; do gdal_translate -a_srs EPSG:3035 -co “TILED=YES”
-co “BLOCKXSIZE=512” -co “BLOCKYSIZE=512” -co “COMPRESS=DEFLATE” -co
“ZLEVEL=9” -co “BIGTIFF=YES” $f $f.tif; doneThen I wrote a Python script that created a store and a layer for each
of the tiffs.Here is the output from gdalinfo for a typical one:
Driver: GTiff/GeoTIFF
Files: se502c_R9_C1_3_T_2531_sec.rsl.tif
Size is 3396, 2271
Coordinate System is:
PROJCS[“ETRS89 / LAEA Europe”,
GEOGCS[“ETRS89”,
DATUM[“European_Terrestrial_Reference_System_1989”,
SPHEROID[“GRS 1980”,6378137,298.2572221010002,
AUTHORITY[“EPSG”,“7019”]],
TOWGS84[0,0,0,0,0,0,0],
AUTHORITY[“EPSG”,“6258”]],
PRIMEM[“Greenwich”,0],
UNIT[“degree”,0.0174532925199433],
AUTHORITY[“EPSG”,“4258”]],
PROJECTION[“Lambert_Azimuthal_Equal_Area”],
PARAMETER[“latitude_of_center”,52],
PARAMETER[“longitude_of_center”,10],
PARAMETER[“false_easting”,4321000],
PARAMETER[“false_northing”,3210000],As you can see they are EPSG:3035. They should be internally tiled. In
the Coordinate Reference Systems for Geoserver, it says that both
Native SRS and Declared SRS are EPSG:3035, and that the handling
should be to reproject native to declared.My web client overlays these Geotiffs on a standard OpenStreetMap
layer in Web Mercator. All the geotiffs we have tried appear
correctly, so I guess reprojection is still happening at some stage.My gut feeling is that I might gain some speed by reprojecting the
original Geotiffs to Web Mercator, or at least changing the declared
SRS to Web Mercator in Geoserver. Not keen on it, it’s going to take
days. What other properties of the Geotiffs could I tinker with?Would caching with GeoWebCache help, given the file size?
Bonus question - if a Geotiff is changed, by reprojecting or retiling
or what have you, does the Geoserver layer associated with it get
invalidated? It took days to create them all.I’d really appreciate your feeback.
Thanks in advance,
Max
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.
http://pubads.g.doubleclick.net/gampad/clk?id=278785231&iu=/4140
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users–
Ian Turton
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.
http://pubads.g.doubleclick.net/gampad/clk?id=278785231&iu=/4140
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users–
==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.
==
Ing. Andrea Aime
@geowolf
Technical Lead
GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549
http://twitter.com/geosolutions_it
AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.
The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.