[Geoserver-devel] Considering a new WMS/WMTS output format: jpeg or png (JPENG for friends)

GeoServer GeoServer Developer
1

Hi all,
when working with imagery it happens often enough that one wants the compression
factor of a JPEG, but also transparency at the raster data edges.
Unfortunately one cannot have both… or can she?

With tiled maps we can introduce a new output format, which I nicknamed JPENG
(JPEg or pNG) that would return the right compression depending on the tile content:
if it’s fully opaque, JPEG, if there is any transparency, PNG instead, of course setting
the right return type headers so that the browser can do the right decoding.

It’s simple enough and would save significant bandwidth (and also CPU, JPEG is
faster to compress) for imagery that also needs occasional transparency support.

I’d add this at a WMS output format, and if time allows, WMTS too… but no guarantees,
I have actually no funding backing any of this (I’m doing it for another system), but it seems
quick to implement in GeoServer too and would be a pity to lose it.

Objections? Suggestions for a better name (I checked a few other variants, but they seem to be
already in use on the internet with a different meaning)?

Cheers
Andrea

···

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313

fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

2

Heh,
our mapserver friends are discussion the option of adding it to mapserver too, one feedback in the discussion:

“We might consider adding to mapserver if there’s a need for it. FWIW, this feature has been in mapproxy since end of 2012, and mapcache since mid 2011.”

Cheers
Andrea

···

On Tue, Jun 7, 2016 at 6:50 PM, Andrea Aime <andrea.aime@anonymised.com> wrote:

Hi all,
when working with imagery it happens often enough that one wants the compression
factor of a JPEG, but also transparency at the raster data edges.
Unfortunately one cannot have both… or can she?

With tiled maps we can introduce a new output format, which I nicknamed JPENG
(JPEg or pNG) that would return the right compression depending on the tile content:
if it’s fully opaque, JPEG, if there is any transparency, PNG instead, of course setting
the right return type headers so that the browser can do the right decoding.

It’s simple enough and would save significant bandwidth (and also CPU, JPEG is
faster to compress) for imagery that also needs occasional transparency support.

I’d add this at a WMS output format, and if time allows, WMTS too… but no guarantees,
I have actually no funding backing any of this (I’m doing it for another system), but it seems
quick to implement in GeoServer too and would be a pity to lose it.

Objections? Suggestions for a better name (I checked a few other variants, but they seem to be
already in use on the internet with a different meaning)?

Cheers
Andrea

–

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313

fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

–

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313

fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

3

I'm always watching :wink:

Cheers from an old friend.

-jeff

--
Jeff McKenna
MapServer Consulting and Training Services
http://www.gatewaygeomatics.com/

On 2016-06-07 3:03 PM, Andrea Aime wrote:

Heh,
our mapserver friends are discussion the option of adding it to
mapserver too, one feedback in the discussion:

"We might consider adding to mapserver if there's a need for it. FWIW,
this feature has been in mapproxy since end of 2012, and mapcache since
mid 2011."

Cheers
Andrea

On Tue, Jun 7, 2016 at 6:50 PM, Andrea Aime
<andrea.aime@anonymised.com <mailto:andrea.aime@anonymised.com>> wrote:

    Hi all,
    when working with imagery it happens often enough that one wants the
    compression
    factor of a JPEG, but also transparency at the raster data edges.
    Unfortunately one cannot have both... or can she?

    With tiled maps we can introduce a new output format, which I
    nicknamed JPENG
    (JPEg or pNG) that would return the right compression depending on
    the tile content:
    if it's fully opaque, JPEG, if there is any transparency, PNG
    instead, of course setting
    the right return type headers so that the browser can do the right
    decoding.

    It's simple enough and would save significant bandwidth (and also
    CPU, JPEG is
    faster to compress) for imagery that also needs occasional
    transparency support.

    I'd add this at a WMS output format, and if time allows, WMTS too...
    but no guarantees,
    I have actually no funding backing any of this (I'm doing it for
    another system), but it seems
    quick to implement in GeoServer too and would be a pity to lose it.

    Objections? Suggestions for a better name (I checked a few other
    variants, but they seem to be
    already in use on the internet with a different meaning)?

    Cheers
    Andrea

    --
    ==
    GeoServer Professional Services from the experts! Visit
    http://goo.gl/it488V for more information.
    ==

    Ing. Andrea Aime
    @geowolf
    Technical Lead

    GeoSolutions S.A.S.
    Via di Montramito 3/A
    55054 Massarosa (LU)
    phone: +39 0584 962313 <tel:%2B39%200584%20962313>
    fax: +39 0584 1660272 <tel:%2B39%200584%201660272>
    mob: +39 339 8844549 <tel:%2B39%20%C2%A0339%208844549>

    http://www.geo-solutions.it
    http://twitter.com/geosolutions_it

    *AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

    Le informazioni contenute in questo messaggio di posta elettronica
    e/o nel/i file/s allegato/i sono da considerarsi strettamente
    riservate. Il loro utilizzo è consentito esclusivamente al
    destinatario del messaggio, per le finalità indicate nel messaggio
    stesso. Qualora riceviate questo messaggio senza esserne il
    destinatario, Vi preghiamo cortesemente di darcene notizia via
    e-mail e di procedere alla distruzione del messaggio stesso,
    cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
    divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo,
    od utilizzarlo per finalità diverse, costituisce comportamento
    contrario ai principi dettati dal D.Lgs. 196/2003.

    The information in this message and/or attachments, is intended
    solely for the attention and use of the named addressee(s) and may
    be confidential or proprietary in nature or covered by the
    provisions of privacy act (Legislative Decree June, 30 2003, no.196
    - Italy's New Data Protection Code).Any use not in accord with its
    purpose, any disclosure, reproduction, copying, distribution, or
    either dissemination, either whole or partial, is strictly forbidden
    except previous formal approval of the named addressee(s). If you
    are not the intended recipient, please contact immediately the
    sender by telephone, fax or e-mail and delete the information in
    this message that has been received in error. The sender does not
    give any warranty or accept liability as the content, accuracy or
    completeness of sent messages and accepts no responsibility for
    changes made after they were sent or for other risks which arise as
    a result of e-mail transmission, viruses, etc.

    -------------------------------------------------------

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio
stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti,
copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento
contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely
for the attention and use of the named addressee(s) and may be
confidential or proprietary in nature or covered by the provisions of
privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data
Protection Code).Any use not in accord with its purpose, any disclosure,
reproduction, copying, distribution, or either dissemination, either
whole or partial, is strictly forbidden except previous formal approval
of the named addressee(s). If you are not the intended recipient, please
contact immediately the sender by telephone, fax or e-mail and delete
the information in this message that has been received in error. The
sender does not give any warranty or accept liability as the content,
accuracy or completeness of sent messages and accepts no responsibility
for changes made after they were sent or for other risks which arise as
a result of e-mail transmission, viruses, etc.

-------------------------------------------------------

------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity
planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e

_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

4

Andrea,

I like the idea but would add that my decision to use png or jpeg also depends on the type of data, not just transparency. If vectors, like OSM, then png looks much better. If raster, then jpeg is typically fine.

I have also thought about a similar concept when exposing tiled layers to our applications. Its more of a configuration issue, but we always pre-configure layers in our apps. It tends to become a hassle for developers to know which format the layer is available in. I usually only publish tiled layers in 1 particular format based on the decision above. It would be nice if there was an alias that could be used which basically means either/or, since web clients don’t really care. This would probably be more of a GWC feature.

···

On Tue, Jun 7, 2016 at 12:50 PM, Andrea Aime <andrea.aime@anonymised.com> wrote:

Hi all,
when working with imagery it happens often enough that one wants the compression
factor of a JPEG, but also transparency at the raster data edges.
Unfortunately one cannot have both… or can she?

With tiled maps we can introduce a new output format, which I nicknamed JPENG
(JPEg or pNG) that would return the right compression depending on the tile content:
if it’s fully opaque, JPEG, if there is any transparency, PNG instead, of course setting
the right return type headers so that the browser can do the right decoding.

It’s simple enough and would save significant bandwidth (and also CPU, JPEG is
faster to compress) for imagery that also needs occasional transparency support.

I’d add this at a WMS output format, and if time allows, WMTS too… but no guarantees,
I have actually no funding backing any of this (I’m doing it for another system), but it seems
quick to implement in GeoServer too and would be a pity to lose it.

Objections? Suggestions for a better name (I checked a few other variants, but they seem to be
already in use on the internet with a different meaning)?

Cheers
Andrea

–

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313

fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity
planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e

Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

Jason Newmoyer
Newmoyer Geospatial Solutions
843.606.0424
jason@anonymised.com

5

+1 Andrea, I’d love to see this in GeoServer. At Planet Labs we’ve been thinking about this as well. Esri supports it in their stuff, not sure when they added it, but I see it in their tiles.

Jason, at least for my purposes this is geared for raster, and indeed as Andrea points out, when you want transparency for your raster. The reason that jpeg is not fine in that case is that it doesn’t support transparency. JPEG is by far the best for raster, but if you’re using it as an overlay instead of a base layer then jpeg doesn’t work at the edges. So PNG at the edges lets you use jpeg for every tile that is fully covered, but do the edges transparent.

···

On Wed, Jun 8, 2016 at 1:12 PM, Jason Newmoyer <jason@anonymised.com> wrote:

Andrea,

I like the idea but would add that my decision to use png or jpeg also depends on the type of data, not just transparency. If vectors, like OSM, then png looks much better. If raster, then jpeg is typically fine.

I have also thought about a similar concept when exposing tiled layers to our applications. Its more of a configuration issue, but we always pre-configure layers in our apps. It tends to become a hassle for developers to know which format the layer is available in. I usually only publish tiled layers in 1 particular format based on the decision above. It would be nice if there was an alias that could be used which basically means either/or, since web clients don’t really care. This would probably be more of a GWC feature.

What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity
planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e

Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

Jason Newmoyer
Newmoyer Geospatial Solutions
843.606.0424
jason@anonymised.com

On Tue, Jun 7, 2016 at 12:50 PM, Andrea Aime <andrea.aime@anonymised.com> wrote:

Hi all,
when working with imagery it happens often enough that one wants the compression
factor of a JPEG, but also transparency at the raster data edges.
Unfortunately one cannot have both… or can she?

With tiled maps we can introduce a new output format, which I nicknamed JPENG
(JPEg or pNG) that would return the right compression depending on the tile content:
if it’s fully opaque, JPEG, if there is any transparency, PNG instead, of course setting
the right return type headers so that the browser can do the right decoding.

It’s simple enough and would save significant bandwidth (and also CPU, JPEG is
faster to compress) for imagery that also needs occasional transparency support.

I’d add this at a WMS output format, and if time allows, WMTS too… but no guarantees,
I have actually no funding backing any of this (I’m doing it for another system), but it seems
quick to implement in GeoServer too and would be a pity to lose it.

Objections? Suggestions for a better name (I checked a few other variants, but they seem to be
already in use on the internet with a different meaning)?

Cheers
Andrea

–

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313

fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity
planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e

Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel