[Geoserver-users] NetCDF Compression Settings

GeoServer GeoServer User
1

Hello,

Sorry for posting this twice, but I think my first attempt was hidden due to Outlook's preference of sending HTML mail.

I'm trying to enable NetCDF compression via the NetCDF Output plugin in GeoServer 2.8.1. I have set a compression value (5) in both the individual layer settings as well as the global settings; however, all WCS 2.0.1 NetCDF4-Classic responses appear to be the uncompressed file size.

I believe I have verified that they are uncompressed by running the following command:
`ncdump -h -s <netcdf file> | grep DeflateLevel`

Sample files which are known to be compressed respond with their compression level, but the GeoServer responses do not display anything.

Is there a step I could be missing to enable compression? Has anyone else gotten this working?

Thanks,

Kevin M. Weiss
Software Engineer
CRITICAL NETWORKS / HARRIS CORPORATION
harris.com / kevin.weiss@anonymised.com

2

Hi Kevin,
We had this working when specifying NetCDF4-classic output format.
I will check that in the next days looking for any regression occurred in the meantime.

Cheers,
Daniele

···

On Wed, Feb 3, 2016 at 3:51 PM, kweiss01@anonymised.com <kweiss01@anonymised.com> wrote:

Hello,

Sorry for posting this twice, but I think my first attempt was hidden due to Outlook’s preference of sending HTML mail.

I’m trying to enable NetCDF compression via the NetCDF Output plugin in GeoServer 2.8.1. I have set a compression value (5) in both the individual layer settings as well as the global settings; however, all WCS 2.0.1 NetCDF4-Classic responses appear to be the uncompressed file size.

I believe I have verified that they are uncompressed by running the following command:
ncdump -h -s <netcdf file> | grep DeflateLevel

Sample files which are known to be compressed respond with their compression level, but the GeoServer responses do not display anything.

Is there a step I could be missing to enable compression? Has anyone else gotten this working?

Thanks,

Kevin M. Weiss
Software Engineer
CRITICAL NETWORKS / HARRIS CORPORATION
harris.com / kevin.weiss@anonymised.com0…

Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140

Geoserver-users mailing list
Geoserver-users@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

–

==

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Daniele Romagnoli
Senior Software Engineer

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

3

Hi Kevin,
I have found the issue and prepared a pull request containing the very trivial fix.
I will merge it in the next couple of hours.

Cheers,
Daniele

···

On Wed, Feb 3, 2016 at 6:38 PM, Daniele Romagnoli <daniele.romagnoli@anonymised.com107…> wrote:

Hi Kevin,
We had this working when specifying NetCDF4-classic output format.
I will check that in the next days looking for any regression occurred in the meantime.

Cheers,
Daniele

–

On Wed, Feb 3, 2016 at 3:51 PM, kweiss01@anonymised.com <kweiss01@anonymised.com> wrote:

Hello,

Sorry for posting this twice, but I think my first attempt was hidden due to Outlook’s preference of sending HTML mail.

I’m trying to enable NetCDF compression via the NetCDF Output plugin in GeoServer 2.8.1. I have set a compression value (5) in both the individual layer settings as well as the global settings; however, all WCS 2.0.1 NetCDF4-Classic responses appear to be the uncompressed file size.

I believe I have verified that they are uncompressed by running the following command:
ncdump -h -s <netcdf file> | grep DeflateLevel

Sample files which are known to be compressed respond with their compression level, but the GeoServer responses do not display anything.

Is there a step I could be missing to enable compression? Has anyone else gotten this working?

Thanks,

Kevin M. Weiss
Software Engineer
CRITICAL NETWORKS / HARRIS CORPORATION
harris.com / kevin.weiss@anonymised.com

Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140

Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

–

==

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Daniele Romagnoli
Senior Software Engineer

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

==

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Daniele Romagnoli
Senior Software Engineer

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

4

Hi Daniele,
I got finally got a chance to look into the NetCDF Output code. It appears the issue is due to the writer being created before the parseParams call in the NetCDFOutputManager constructor. The private attribute “compressionLevel” is defaulted to “0” during instantiation and then used in the getWriter call. However the “parseParams” call which immediately follows the creation of the writer object is where the system configuration is read and the “compressionLevel” value is properly set.

I see you created a bug ticket. I’ll add a comment with my findings.

Thanks,

Kevin M. Weiss
Software Engineer
CRITICAL NETWORKS / HARRIS CORPORATION
harris.com / kevin.weiss@...5740...

From: dany.geotools@...84... [mailto:dany.geotools@…84…] On Behalf Of Daniele Romagnoli
Sent: Wednesday, February 03, 2016 11:38 AM
To: Weiss, Kevin <kweiss01@...5740...>
Cc: geoserver-users@lists.sourceforge.net
Subject: Re: [Geoserver-users] NetCDF Compression Settings

Hi Kevin,
We had this working when specifying NetCDF4-classic output format.
I will check that in the next days looking for any regression occurred in the meantime.

Cheers,
Daniele

On Wed, Feb 3, 2016 at 3:51 PM, kweiss01@...5740... <kweiss01@...5740...> wrote:
Hello,

Sorry for posting this twice, but I think my first attempt was hidden due to Outlook's preference of sending HTML mail.

I'm trying to enable NetCDF compression via the NetCDF Output plugin in GeoServer 2.8.1. I have set a compression value (5) in both the individual layer settings as well as the global settings; however, all WCS 2.0.1 NetCDF4-Classic responses appear to be the uncompressed file size.

I believe I have verified that they are uncompressed by running the following command:
`ncdump -h -s <netcdf file> | grep DeflateLevel`

Sample files which are known to be compressed respond with their compression level, but the GeoServer responses do not display anything.

Is there a step I could be missing to enable compression? Has anyone else gotten this working?

Thanks,

Kevin M. Weiss
Software Engineer
CRITICAL NETWORKS / HARRIS CORPORATION
harris.com / kevin.weiss@...5740...

------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Daniele Romagnoli
Senior Software Engineer

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------
AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

5

Hi Kevin,
I have already fixed that on master a couple of days ago.
I have also created the pull request containing the backport to 2.8.x but it’s failing on travis (tests). I didn’t investigated on that failure yet but I think it’s unrelated with my changes.

I will check it again tomorrow to see if I can backport that and close the ticket.
Cheers,
Daniele

···

On Thu, Feb 11, 2016 at 6:03 PM, kweiss01@anonymised.com5740… <kweiss01@anonymised.com> wrote:

Hi Daniele,
I got finally got a chance to look into the NetCDF Output code. It appears the issue is due to the writer being created before the parseParams call in the NetCDFOutputManager constructor. The private attribute “compressionLevel” is defaulted to “0” during instantiation and then used in the getWriter call. However the “parseParams” call which immediately follows the creation of the writer object is where the system configuration is read and the “compressionLevel” value is properly set.

I see you created a bug ticket. I’ll add a comment with my findings.

Thanks,

Kevin M. Weiss
Software Engineer
CRITICAL NETWORKS / HARRIS CORPORATION
harris.com / kevin.weiss@anonymised.com0…

From: dany.geotools@anonymised.com… [mailto:dany.geotools@anonymised.com4…] On Behalf Of Daniele Romagnoli
Sent: Wednesday, February 03, 2016 11:38 AM
To: Weiss, Kevin <kweiss01@anonymised.com0…>
Cc: geoserver-users@lists.sourceforge.net
Subject: Re: [Geoserver-users] NetCDF Compression Settings

Hi Kevin,
We had this working when specifying NetCDF4-classic output format.
I will check that in the next days looking for any regression occurred in the meantime.

Cheers,
Daniele

On Wed, Feb 3, 2016 at 3:51 PM, kweiss01@anonymised.com <kweiss01@anonymised.com5740…> wrote:
Hello,

Sorry for posting this twice, but I think my first attempt was hidden due to Outlook’s preference of sending HTML mail.

I’m trying to enable NetCDF compression via the NetCDF Output plugin in GeoServer 2.8.1. I have set a compression value (5) in both the individual layer settings as well as the global settings; however, all WCS 2.0.1 NetCDF4-Classic responses appear to be the uncompressed file size.

I believe I have verified that they are uncompressed by running the following command:
ncdump -h -s <netcdf file> | grep DeflateLevel

Sample files which are known to be compressed respond with their compression level, but the GeoServer responses do not display anything.

Is there a step I could be missing to enable compression? Has anyone else gotten this working?

Thanks,

Kevin M. Weiss
Software Engineer
CRITICAL NETWORKS / HARRIS CORPORATION
harris.com / kevin.weiss@anonymised.com0…

Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140

Geoserver-users mailing list
Geoserver-users@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

–

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Daniele Romagnoli
Senior Software Engineer

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

–

==

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Daniele Romagnoli
Senior Software Engineer

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.