Hi,
We are running into scalability issues with geoserver. We have terabytes of GeoTIFF files, and its getting larger and larger. As far as we know, Geoserver needs to place all the files locally to produce visualizations on the fly, requiring a huge server.
Are there any solutions that allow Geoserver to be scalable, to meet the needs of many tif files?
Best,
JC
On Tue, Jan 26, 2016 at 7:38 AM, Jiehua Chen <j@anonymised.com> wrote:
Hi,
We are running into scalability issues with geoserver. We have terabytes
of GeoTIFF files, and its getting larger and larger. As far as we know,
Geoserver needs to place all the files locally to produce visualizations
on the fly, requiring a huge server.
Terabytes of geotiffs is not a problem, there are properly optimized
installations running off hundreds of terabytes, and some ranging in the
petabytes too,
often the trick is setting up the data properly and do not pretend
sub-second response times if you're trying to make GeoServer open hundreds
of files (optimize
the data so that this is not the case instead).
I believe we need a bit more information, please answer the following basic
questions:
* What OS and GeoServer version are you using?
* Is it a image mosaic?
* Are you routinely trying to display all of the mosaic, or just parts of
it?
* Do you have the tiffs already retiled and with overviews?
This presentation also gives some pointers on how to optimize raster data,
and which data structures
to use depending on your input size:
http://www.slideshare.net/geosolutions/gs-steroids-sgiannecfoss4g20130103
Cheers
Andrea
Ing. Andrea Aime
@geowolf
Technical Lead
GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549
http://www.geo-solutions.it
http://twitter.com/geosolutions_it
*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*
Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.
The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.
-------------------------------------------------------
Hi Andrea,
We use ubuntu, and geoserver 2.8.1
we have hundreds of tif files, and the largest tif file is about 800M.
Each file will be displayed individually.
We are going to visualize the maps using a Django framework:
http://maps-test.qed.ai/layers/?limit=100&offset=0
Since geoserver requires all the tif files are saved locally in the server, we then need a huge server. Usually we use S3 to store data. If we do that for geoserver, the visualization in the site will be too slow?
On Tue, Jan 26, 2016 at 12:17 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:
Jiehua Chen, Ph.D.
QED | http://qed.ai
On Tue, Jan 26, 2016 at 7:38 AM, Jiehua Chen <j@anonymised.com428…> wrote:
Hi,
We are running into scalability issues with geoserver. We have terabytes of GeoTIFF files, and its getting larger and larger. As far as we know, Geoserver needs to place all the files locally to produce visualizations on the fly, requiring a huge server.
Terabytes of geotiffs is not a problem, there are properly optimized installations running off hundreds of terabytes, and some ranging in the petabytes too,
often the trick is setting up the data properly and do not pretend sub-second response times if you’re trying to make GeoServer open hundreds of files (optimize
the data so that this is not the case instead).
I believe we need a bit more information, please answer the following basic questions:
This presentation also gives some pointers on how to optimize raster data, and which data structures
to use depending on your input size:
http://www.slideshare.net/geosolutions/gs-steroids-sgiannecfoss4g20130103
Cheers
Andrea
–
Ing. Andrea Aime
@geowolf
Technical Lead
GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549
http://www.geo-solutions.it
http://twitter.com/geosolutions_it
AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.
The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.