Hi Jukka,
completing a bit what Even and Jody already said, GeoServer WCS can already deal with images
larger than memory as we try not to allocate them in a single surface, however we are writing
the output to a local file because most format cannot be written “from beginning to end” but
requires one to go back and forth.
One example is tiled and compressed TIFF, the tile directory needs to contain the offsets of all
the tiles, but those are known only when all of them are written, since we cannot predict how
big they will be (each will compress by a different ratio).
With some modifications we should be able to write uncompressed GeoTiff directly on the
output stream, since its contents are fully predictable, we’ll just have to make a special case for
it so that we don’t rely on the generic JAI ImageIO architecture, which instead assumes one
might have to go back and forth in the output file.
About NetCDF, Jody mentions DAP, but I believe you are interested in streaming from the INSPIRE
point of view, where DAP would not be an option… or would it? In any case, it would be
a different protocol, I did search a bit on the internet, and could fine some experiences with
people implementing a WCS fronting a DAP server, but not a merger between the two protocols.
Looking at the UCAR NetCDF library we are using, it demands a file as a target, which makes
me assume the file structure is not suitable for direct streaming.
Looks like the NetCDF have been experimenting with a streamable format, called ncstream,
but I’m unclear if this got any traction:
https://www.unidata.ucar.edu/software/thredds/v4.3/netcdf-java/stream/NcStream.html
As an aside, a format that is ugly and big, but that I believe we can stream directly today, is the GML
coverage format. Mind, pure GML, not GML/JP2, which we do not support today (although
it would be an interesting development, but looking at its spec, it does not seem like
they are offering a tile-able approach that would make streaming possible).
Hope this helps
Cheers
Andrea
···
On Wed, Feb 10, 2016 at 8:41 PM, Rahkonen Jukka (MML) <jukka.rahkonen@anonymised.com> wrote:
Hi,
A couple of hours ago I sent this question to mapserver-devel list:
“Some WCS users would prefer if large output from WCS (large=a few gigabytes or more) could be sent streaming with a relatively short lag between the request and the start of response. If the output image is made complete on the server side before the transfer starts the lag can be some minutes. Is it reasonable idea to stream 1-4 GB GeoTIFFs from Mapserver? What restrictions would it put to the possible file formats, compression etc?”
Even’s answer about streaming rasters is at https://lists.osgeo.org/pipermail/mapserver-dev/2016-February/014742.html.
What would Geoserver developers say, is streaming output reasonable with Geoserver? Does NetCDF have some specific good or bad features for streaming?
-Jukka Rahkonen-
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel
–
==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.
Ing. Andrea Aime
@geowolf
Technical Lead
GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549
http://www.geo-solutions.it
http://twitter.com/geosolutions_it
AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.
The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.