[Geoserver-devel] Faster PNG encoder: some WMS tests

[Andrea_Aime4]

Hi,
the faster PNG encoder module in community is ready for people to try out: I’ve tested it with a number of demo layers,
seems to be working, so I though it was time to try out its performance (get the popcorn and have a seat).

The comparisons involve 3 PNG encoders:

• The JDK one, which it not tunable (it actually ignores our quality/compression indications)
  and compresses everything to the fullest, delivering consistently the smallest PNG in
  exchange for a very slow performance.
• The native ImageIO one, which you can get only if you have properly installed ImageIO,
  which honors the compression settings by altering the level of effort involved in encoding
  a PNG image, delivering larger images but also providing better performance.
  This is the encoder you cannot have on Windows 64 and OSX, since there is no native ImageIO for
  those platforms
• The new encoder, based on PNGJ, a open source, java based, low level PNG encoder,
  coupled with high performance scanline extractors from yours truly, that efficiently feed
  the image bytes into PNGJ

First I did a very silly test against topp:states, usual style, map size 780x330, using ab (apache bench) with 1 and 4 threads,
comparing the throughput and output size for a compression level of 25 (the default one in the WMS panel).
Results:

Threads JDK Native ImageIO PNGJ based
1 11.7 25.4 36.9
4 38.9 75.11 94.5
Size 39KB 55KB 45KB

Well, as you can see the improvement over JDK is very significant, but the one over
NativeIO is not to throw away either.
The output size is good news too, the new encoder manages to be faster than the CLib one
and yet provides a output size closer to the JDK one (which, as said, uses every possible effort
to have the smallest possible output).

Yet, this use case is a toy, the map is small, the vectors are simple.

So I’ve decided to try something bigger, the FOSS4G 2010 benchmark, the vector
map of the whole spain with roads, buildings, contour lines, labels.
I’ve collected the full results in this Google Spreadsheet:
https://docs.google.com/spreadsheet/ccc?key=0Aq3GF1EnUyHEdHVodEpjNHM0MVhEMlhPcmJHTTFCTWc&usp=sharing

Here is just the comparison chart:

Well, I’d say the speedup is again rather good compared to the plain JDK encoder, and not to throw
away compared to the ImageIO native encoder.
Again, I’ve reached for one of the output maps of the benchmark and compared sizes (this is a 904x764 image
with a dense set of countour lines, road network, some labels, some polygons):

• JDK → 338KB
• Native ImageIO → 509KB
• PNGJ based → 367KB

Again, close to the JDK size, yet faster than the native ImageIO encoder in speed.

Not too bad, if I may say! (Windows Server users, likely using 64 bits JVMs, should pop the champagne :-p )

I’ve added the module to the nightly build on the master branch, and if you are a developer,
you can load it by using the -Ppng profile with your maven commands.

I’d like to get some feedback from the early testers, and then push this module for extension
status in the 2.4.x series to have a larger user base test it.

Closing remark for users that might be reading this: before you run downloading the new module,
remember that the speedup is solely in the PNG encoding phase.
If your map takes 1 second or more to draw, you probably have troubles in data setup, don’t waste your time
trying a difference PNG encoder, optimize your data or styles instead.
If instead your current maps already get returned in less than, say, 0.3 seconds then yes, give the new PNG
encoder a try.

Cheers
Andrea

–

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

---

[Jonathan_Moules2]

Hi Andrea,
As a windows 64bit user, I’d be happy to test this. Especially given my slow rendering speeds; it’ll be interesting to see if it makes any difference to my issue.

This transmission is intended for the named addressee(s) only and may contain sensitive or protectively marked material up to RESTRICTED and should be handled accordingly. Unless you are the named addressee (or authorised to receive it for the addressee) you may not copy or use it, or disclose it to anyone else. If you have received this transmission in error please notify the sender immediately. All email traffic sent to or from us, including without limitation all GCSX traffic, may be subject to recording and/or monitoring in accordance with relevant legislation.

(attachments)

···

One question though: Where can I acquire it? I’m using 2.4.0 and have tried various directories in - http://gridlock.opengeo.org/geoserver/ but can’t see an obvious contender.

Cheers,
Jonathan

On 13 October 2013 18:25, Andrea Aime <andrea.aime@anonymised.com> wrote:

Hi,
the faster PNG encoder module in community is ready for people to try out: I’ve tested it with a number of demo layers,
seems to be working, so I though it was time to try out its performance (get the popcorn and have a seat).

The comparisons involve 3 PNG encoders:

• The JDK one, which it not tunable (it actually ignores our quality/compression indications)
  and compresses everything to the fullest, delivering consistently the smallest PNG in
  exchange for a very slow performance.
• The native ImageIO one, which you can get only if you have properly installed ImageIO,
  which honors the compression settings by altering the level of effort involved in encoding
  a PNG image, delivering larger images but also providing better performance.
  This is the encoder you cannot have on Windows 64 and OSX, since there is no native ImageIO for
  those platforms
• The new encoder, based on PNGJ, a open source, java based, low level PNG encoder,
  coupled with high performance scanline extractors from yours truly, that efficiently feed
  the image bytes into PNGJ

First I did a very silly test against topp:states, usual style, map size 780x330, using ab (apache bench) with 1 and 4 threads,
comparing the throughput and output size for a compression level of 25 (the default one in the WMS panel).
Results:

Threads JDK Native ImageIO PNGJ based
1 11.7 25.4 36.9
4 38.9 75.11 94.5
Size 39KB 55KB 45KB

Well, as you can see the improvement over JDK is very significant, but the one over
NativeIO is not to throw away either.
The output size is good news too, the new encoder manages to be faster than the CLib one
and yet provides a output size closer to the JDK one (which, as said, uses every possible effort
to have the smallest possible output).

Yet, this use case is a toy, the map is small, the vectors are simple.

So I’ve decided to try something bigger, the FOSS4G 2010 benchmark, the vector
map of the whole spain with roads, buildings, contour lines, labels.
I’ve collected the full results in this Google Spreadsheet:
https://docs.google.com/spreadsheet/ccc?key=0Aq3GF1EnUyHEdHVodEpjNHM0MVhEMlhPcmJHTTFCTWc&usp=sharing

Here is just the comparison chart:

Well, I’d say the speedup is again rather good compared to the plain JDK encoder, and not to throw
away compared to the ImageIO native encoder.
Again, I’ve reached for one of the output maps of the benchmark and compared sizes (this is a 904x764 image
with a dense set of countour lines, road network, some labels, some polygons):

• JDK → 338KB
• Native ImageIO → 509KB
• PNGJ based → 367KB

Again, close to the JDK size, yet faster than the native ImageIO encoder in speed.

Not too bad, if I may say! (Windows Server users, likely using 64 bits JVMs, should pop the champagne :-p )

I’ve added the module to the nightly build on the master branch, and if you are a developer,
you can load it by using the -Ppng profile with your maven commands.

I’d like to get some feedback from the early testers, and then push this module for extension
status in the 2.4.x series to have a larger user base test it.

Closing remark for users that might be reading this: before you run downloading the new module,
remember that the speedup is solely in the PNG encoding phase.
If your map takes 1 second or more to draw, you probably have troubles in data setup, don’t waste your time
trying a difference PNG encoder, optimize your data or styles instead.
If instead your current maps already get returned in less than, say, 0.3 seconds then yes, give the new PNG
encoder a try.

Cheers
Andrea

–

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

---

---

October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60134071&iu=/4140/ostg.clktrk

---

Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

[Jonathan_Moules2]

Thanks Michael,
The Help is still pointing to the opengeo server - http://docs.geoserver.org/stable/en/user/community/index.html?highlight=community

This transmission is intended for the named addressee(s) only and may contain sensitive or protectively marked material up to RESTRICTED and should be handled accordingly. Unless you are the named addressee (or authorised to receive it for the addressee) you may not copy or use it, or disclose it to anyone else. If you have received this transmission in error please notify the sender immediately. All email traffic sent to or from us, including without limitation all GCSX traffic, may be subject to recording and/or monitoring in accordance with relevant legislation.

(attachments)

···

Jonathan

On 14 October 2013 13:41, Räder, Michael <Michael.Raeder@anonymised.com> wrote:

Look here:

http://ares.boundlessgeo.com/geoserver/2.4.x/

Mit freundlichen Grüßen

Im Auftrag

Michael Räder

---

Michael Räder

Nationalpark-Verwaltung Niedersächsisches Wattenmeer
Virchowstr. 1 / 26382 Wilhelmshaven
Tel.: +49 (0) 4421 911 295 / Fax.: +49 (0) 4421 911 280

Email: michael.raeder@anonymised.com

║ [MDI - NI] ║ [NLPV] ║ [MDI - DE] ║

---

---

Von: Jonathan Moules [mailto:jonathanmoules@anonymised.com]
Gesendet: Montag, 14. Oktober 2013 14:32
An: Andrea Aime
Cc: Geoserver-devel
Betreff: Re: [Geoserver-devel] Faster PNG encoder: some WMS tests

Hi Andrea,

As a windows 64bit user, I’d be happy to test this. Especially given my slow rendering speeds; it’ll be interesting to see if it makes any difference to my issue.

One question though: Where can I acquire it? I’m using 2.4.0 and have tried various directories in - http://gridlock.opengeo.org/geoserver/ but can’t see an obvious contender.

Cheers,

Jonathan

On 13 October 2013 18:25, Andrea Aime <andrea.aime@anonymised.com> wrote:

Hi,

the faster PNG encoder module in community is ready for people to try out: I’ve tested it with a number of demo layers,

seems to be working, so I though it was time to try out its performance (get the popcorn and have a seat).

The comparisons involve 3 PNG encoders:

• The JDK one, which it not tunable (it actually ignores our quality/compression indications)

and compresses everything to the fullest, delivering consistently the smallest PNG in

exchange for a very slow performance.

• The native ImageIO one, which you can get only if you have properly installed ImageIO,

which honors the compression settings by altering the level of effort involved in encoding

a PNG image, delivering larger images but also providing better performance.

This is the encoder you cannot have on Windows 64 and OSX, since there is no native ImageIO for

those platforms

• The new encoder, based on PNGJ, a open source, java based, low level PNG encoder,

coupled with high performance scanline extractors from yours truly, that efficiently feed

the image bytes into PNGJ

First I did a very silly test against topp:states, usual style, map size 780x330, using ab (apache bench) with 1 and 4 threads,

comparing the throughput and output size for a compression level of 25 (the default one in the WMS panel).

Results:

Threads JDK Native ImageIO PNGJ based

1 11.7 25.4 36.9

4 38.9 75.11 94.5

Size 39KB 55KB 45KB

Well, as you can see the improvement over JDK is very significant, but the one over

NativeIO is not to throw away either.

The output size is good news too, the new encoder manages to be faster than the CLib one

and yet provides a output size closer to the JDK one (which, as said, uses every possible effort

to have the smallest possible output).

Yet, this use case is a toy, the map is small, the vectors are simple.

So I’ve decided to try something bigger, the FOSS4G 2010 benchmark, the vector

map of the whole spain with roads, buildings, contour lines, labels.

I’ve collected the full results in this Google Spreadsheet:

https://docs.google.com/spreadsheet/ccc?key=0Aq3GF1EnUyHEdHVodEpjNHM0MVhEMlhPcmJHTTFCTWc&usp=sharing

Here is just the comparison chart:

Well, I’d say the speedup is again rather good compared to the plain JDK encoder, and not to throw

away compared to the ImageIO native encoder.

Again, I’ve reached for one of the output maps of the benchmark and compared sizes (this is a 904x764 image

with a dense set of countour lines, road network, some labels, some polygons):

• JDK → 338KB

• Native ImageIO → 509KB

• PNGJ based → 367KB

Again, close to the JDK size, yet faster than the native ImageIO encoder in speed.

Not too bad, if I may say! (Windows Server users, likely using 64 bits JVMs, should pop the champagne :-p )

I’ve added the module to the nightly build on the master branch, and if you are a developer,

you can load it by using the -Ppng profile with your maven commands.

I’d like to get some feedback from the early testers, and then push this module for extension

status in the 2.4.x series to have a larger user base test it.

Closing remark for users that might be reading this: before you run downloading the new module,

remember that the speedup is solely in the PNG encoding phase.

If your map takes 1 second or more to draw, you probably have troubles in data setup, don’t waste your time

trying a difference PNG encoder, optimize your data or styles instead.

If instead your current maps already get returned in less than, say, 0.3 seconds then yes, give the new PNG

encoder a try.

Cheers

Andrea

–

==

Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

==

Ing. Andrea Aime

@geowolf

Technical Lead

GeoSolutions S.A.S.

Via Poggio alle Viti 1187

55054 Massarosa (LU)

Italy

phone: +39 0584 962313

fax: +39 0584 1660272

mob: +39 339 8844549

http://www.geo-solutions.it

http://twitter.com/geosolutions_it

---

---

October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60134071&iu=/4140/ostg.clktrk

---

Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

This transmission is intended for the named addressee(s) only and may contain sensitive or protectively marked material up to RESTRICTED and should be handled accordingly. Unless you are the named addressee (or authorised to receive it for the addressee) you may not copy or use it, or disclose it to anyone else. If you have received this transmission in error please notify the sender immediately. All email traffic sent to or from us, including without limitation all GCSX traffic, may be subject to recording and/or monitoring in accordance with relevant legislation.

[Jonathan_Moules2]

Although that said, I can’t seem to find any obvious candidate for it on that server either.

Jonathan

This transmission is intended for the named addressee(s) only and may contain sensitive or protectively marked material up to RESTRICTED and should be handled accordingly. Unless you are the named addressee (or authorised to receive it for the addressee) you may not copy or use it, or disclose it to anyone else. If you have received this transmission in error please notify the sender immediately. All email traffic sent to or from us, including without limitation all GCSX traffic, may be subject to recording and/or monitoring in accordance with relevant legislation.

(attachments)

···

On 14 October 2013 13:43, Jonathan Moules <jonathanmoules@anonymised.com> wrote:

Thanks Michael,
The Help is still pointing to the opengeo server - http://docs.geoserver.org/stable/en/user/community/index.html?highlight=community

Jonathan

On 14 October 2013 13:41, Räder, Michael <Michael.Raeder@anonymised.com> wrote:

Look here:

http://ares.boundlessgeo.com/geoserver/2.4.x/

Mit freundlichen Grüßen

Im Auftrag

Michael Räder

---

Michael Räder

Nationalpark-Verwaltung Niedersächsisches Wattenmeer
Virchowstr. 1 / 26382 Wilhelmshaven
Tel.: +49 (0) 4421 911 295 / Fax.: +49 (0) 4421 911 280

Email: michael.raeder@anonymised.com

║ [MDI - NI] ║ [NLPV] ║ [MDI - DE] ║

---

---

Von: Jonathan Moules [mailto:jonathanmoules@anonymised.com]
Gesendet: Montag, 14. Oktober 2013 14:32
An: Andrea Aime
Cc: Geoserver-devel
Betreff: Re: [Geoserver-devel] Faster PNG encoder: some WMS tests

Hi Andrea,

As a windows 64bit user, I’d be happy to test this. Especially given my slow rendering speeds; it’ll be interesting to see if it makes any difference to my issue.

One question though: Where can I acquire it? I’m using 2.4.0 and have tried various directories in - http://gridlock.opengeo.org/geoserver/ but can’t see an obvious contender.

Cheers,

Jonathan

On 13 October 2013 18:25, Andrea Aime <andrea.aime@anonymised.com> wrote:

Hi,

the faster PNG encoder module in community is ready for people to try out: I’ve tested it with a number of demo layers,

seems to be working, so I though it was time to try out its performance (get the popcorn and have a seat).

The comparisons involve 3 PNG encoders:

• The JDK one, which it not tunable (it actually ignores our quality/compression indications)

and compresses everything to the fullest, delivering consistently the smallest PNG in

exchange for a very slow performance.

• The native ImageIO one, which you can get only if you have properly installed ImageIO,

which honors the compression settings by altering the level of effort involved in encoding

a PNG image, delivering larger images but also providing better performance.

This is the encoder you cannot have on Windows 64 and OSX, since there is no native ImageIO for

those platforms

• The new encoder, based on PNGJ, a open source, java based, low level PNG encoder,

coupled with high performance scanline extractors from yours truly, that efficiently feed

the image bytes into PNGJ

First I did a very silly test against topp:states, usual style, map size 780x330, using ab (apache bench) with 1 and 4 threads,

comparing the throughput and output size for a compression level of 25 (the default one in the WMS panel).

Results:

Threads JDK Native ImageIO PNGJ based

1 11.7 25.4 36.9

4 38.9 75.11 94.5

Size 39KB 55KB 45KB

Well, as you can see the improvement over JDK is very significant, but the one over

NativeIO is not to throw away either.

The output size is good news too, the new encoder manages to be faster than the CLib one

and yet provides a output size closer to the JDK one (which, as said, uses every possible effort

to have the smallest possible output).

Yet, this use case is a toy, the map is small, the vectors are simple.

So I’ve decided to try something bigger, the FOSS4G 2010 benchmark, the vector

map of the whole spain with roads, buildings, contour lines, labels.

I’ve collected the full results in this Google Spreadsheet:

https://docs.google.com/spreadsheet/ccc?key=0Aq3GF1EnUyHEdHVodEpjNHM0MVhEMlhPcmJHTTFCTWc&usp=sharing

Here is just the comparison chart:

Well, I’d say the speedup is again rather good compared to the plain JDK encoder, and not to throw

away compared to the ImageIO native encoder.

Again, I’ve reached for one of the output maps of the benchmark and compared sizes (this is a 904x764 image

with a dense set of countour lines, road network, some labels, some polygons):

• JDK → 338KB

• Native ImageIO → 509KB

• PNGJ based → 367KB

Again, close to the JDK size, yet faster than the native ImageIO encoder in speed.

Not too bad, if I may say! (Windows Server users, likely using 64 bits JVMs, should pop the champagne :-p )

I’ve added the module to the nightly build on the master branch, and if you are a developer,

you can load it by using the -Ppng profile with your maven commands.

I’d like to get some feedback from the early testers, and then push this module for extension

status in the 2.4.x series to have a larger user base test it.

Closing remark for users that might be reading this: before you run downloading the new module,

remember that the speedup is solely in the PNG encoding phase.

If your map takes 1 second or more to draw, you probably have troubles in data setup, don’t waste your time

trying a difference PNG encoder, optimize your data or styles instead.

If instead your current maps already get returned in less than, say, 0.3 seconds then yes, give the new PNG

encoder a try.

Cheers

Andrea

–

==

Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

==

Ing. Andrea Aime

@geowolf

Technical Lead

GeoSolutions S.A.S.

Via Poggio alle Viti 1187

55054 Massarosa (LU)

Italy

phone: +39 0584 962313

fax: +39 0584 1660272

mob: +39 339 8844549

http://www.geo-solutions.it

http://twitter.com/geosolutions_it

---

---

October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60134071&iu=/4140/ostg.clktrk

---

Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

This transmission is intended for the named addressee(s) only and may contain sensitive or protectively marked material up to RESTRICTED and should be handled accordingly. Unless you are the named addressee (or authorised to receive it for the addressee) you may not copy or use it, or disclose it to anyone else. If you have received this transmission in error please notify the sender immediately. All email traffic sent to or from us, including without limitation all GCSX traffic, may be subject to recording and/or monitoring in accordance with relevant legislation.

[Andrea_Aime4]

On Mon, Oct 14, 2013 at 2:32 PM, Jonathan Moules <
jonathanmoules@anonymised.com> wrote:

Hi Andrea,
As a windows 64bit user, I'd be happy to test this. Especially given my
slow rendering speeds; it'll be interesting to see if it makes any
difference to my issue.

One question though: Where can I acquire it? I'm using 2.4.0 and have
tried various directories in - http://gridlock.opengeo.org/geoserver/ but
can't see an obvious contender.

Hopefully you noticed my comment at the end, yes? The one saying that if
your maps are returned in more than 0.3s this will do little or nothing for
you, and you
should look at optimizing your data/styles instead?

Anyways, the plugin is available only for the master build at the moment,
but it _should_ still be compatible with 2.4.0,
you can download it from here:

http://ares.boundlessgeo.com/geoserver/master/community-latest/geoserver-2.5-SNAPSHOT-png-plugin.zip

Just drop the jars in web-inf/lib and restart geoserver.

Cheers
Andrea

--

Our support, Your Success! Visit http://opensdi.geo-solutions.it for more
information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

[Jonathan_Moules2]

Hi Andrea,
Thanks for the link.

I did indeed notice that comment which is what prompted my post in the user list. But given that the rendering portion is fairly slow itself (or appears to be) and I’m using 64bit windows, I figured I’d give it a spin; what’s the worst that can happen right?

I guess it doesn’t need any special setting to turn it on? My log from a getmap request is slightly different now and ends shorter with only one line (no DEBUG level events?):

2013-10-14 13:23:56,384 INFO [org.geoserver.flow] - Running requests: 0, processing through flow controllers: 0

(attachments)

request-png-plugin.zip (4.16 KB)

···

On 14 October 2013 13:57, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Mon, Oct 14, 2013 at 2:32 PM, Jonathan Moules <jonathanmoules@anonymised.com> wrote:

Hi Andrea,
As a windows 64bit user, I’d be happy to test this. Especially given my slow rendering speeds; it’ll be interesting to see if it makes any difference to my issue.

Hopefully you noticed my comment at the end, yes? The one saying that if your maps are returned in more than 0.3s this will do little or nothing for you, and you
should look at optimizing your data/styles instead?

Anyways, the plugin is available only for the master build at the moment, but it should still be compatible with 2.4.0,
you can download it from here:

http://ares.boundlessgeo.com/geoserver/master/community-latest/geoserver-2.5-SNAPSHOT-png-plugin.zip

Just drop the jars in web-inf/lib and restart geoserver.

Cheers

Andrea

–

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

---

One question though: Where can I acquire it? I’m using 2.4.0 and have tried various directories in - http://gridlock.opengeo.org/geoserver/ but can’t see an obvious contender.

[Andrea_Aime4]

On Mon, Oct 14, 2013 at 3:35 PM, Jonathan Moules <
jonathanmoules@anonymised.com> wrote:

Hi Andrea,
Thanks for the link.

I did indeed notice that comment which is what prompted my post in the
user list. But given that the rendering portion is fairly slow itself (or
appears to be) and I'm using 64bit windows, I figured I'd give it a spin;
what's the worst that can happen right?

I guess it doesn't need any special setting to turn it on? My log from a
getmap request is slightly different now and ends shorter with only one
line (no DEBUG level events?):

2013-10-14 13:23:56,384 INFO [org.geoserver.flow] - Running requests: 0,
processing through flow controllers: 0

---
Previously it had about 100 lines of:
...
2013-10-14 11:29:57,022 DEBUG [org.geotools.image] - Encoded input image
for png writer
2013-10-14 11:29:57,022 DEBUG [org.geotools.image] - Getting a writer
2013-10-14 11:29:57,022 DEBUG [org.geotools.image] - Unable to use Native
ImageIO PNG writer.
2013-10-14 11:29:57,022 DEBUG [org.geotools.image] - Using ImageIO Writer
with SPI: com.sun.imageio.plugins.png.PNGImageWriterSpi
...
----

As a result, assuming I'm reading the logs right (big if!), the previous
tile took about 1.2 seconds from "gettile" to the "running requests", this
one takes about 0.4seconds, which implies a big speedup!
I've attached the zip file of the request so you can check/compare with
the ones in my user-list thread.

However the new tile appears to be a larger file. 121KB versus the
previous 119KB.

Well, 3 times faster and a 1% increase in size, it seems it's working well
for you
File size is generally big though... is that really a tile?
Have you tried the image/png8 output format? It can do transparency and
translucency,
so it can be used for overlays too.

Cheers
Andrea

--

Our support, Your Success! Visit http://opensdi.geo-solutions.it for more
information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

[Jonathan_Moules2]

My files are png/8 with transparency. You can see the tiles here (being served as a regular image, not TMS):
http://maps.warwickshire.gov.uk/new.png
http://maps.warwickshire.gov.uk/old.png

The largest tile we have on our production system for this layergroup is 149KB. The median (of 270,000 tiles) is 10KB. Just under 1% are >= 100KB.

This transmission is intended for the named addressee(s) only and may contain sensitive or protectively marked material up to RESTRICTED and should be handled accordingly. Unless you are the named addressee (or authorised to receive it for the addressee) you may not copy or use it, or disclose it to anyone else. If you have received this transmission in error please notify the sender immediately. All email traffic sent to or from us, including without limitation all GCSX traffic, may be subject to recording and/or monitoring in accordance with relevant legislation.

(attachments)

···

---

I’ve done a further test, this time WMS using layerpreview; the styling was fairly complex, but simpler than the TMS stuff. The exact same requests to a 2.3.5 instance and this 2.4.0 with the new module.

2.3.5:

On 14 October 2013 14:41, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Mon, Oct 14, 2013 at 3:35 PM, Jonathan Moules <jonathanmoules@anonymised.com> wrote:

Compared to 2.4.0:

Size easily went to the old one (about 4% smaller). Speed shouldn’t be compared here (most of it is the Oracle-GeoServer connection). Looking in the logs, rendering times:

2.3.5 - 0.05s to 0.15s
2.4.0 - 0.01s to 0.3s

So speedwise the new plugin wins, but the difference is negligible as you predicted.

I’ll try large WMS requests for that complex style next.

Jonathan

Hi Andrea,

Thanks for the link.

I did indeed notice that comment which is what prompted my post in the user list. But given that the rendering portion is fairly slow itself (or appears to be) and I’m using 64bit windows, I figured I’d give it a spin; what’s the worst that can happen right?

I guess it doesn’t need any special setting to turn it on? My log from a getmap request is slightly different now and ends shorter with only one line (no DEBUG level events?):

2013-10-14 13:23:56,384 INFO [org.geoserver.flow] - Running requests: 0, processing through flow controllers: 0

---

Previously it had about 100 lines of:
…

2013-10-14 11:29:57,022 DEBUG [org.geotools.image] - Encoded input image for png writer
2013-10-14 11:29:57,022 DEBUG [org.geotools.image] - Getting a writer
2013-10-14 11:29:57,022 DEBUG [org.geotools.image] - Unable to use Native ImageIO PNG writer.
2013-10-14 11:29:57,022 DEBUG [org.geotools.image] - Using ImageIO Writer with SPI: com.sun.imageio.plugins.png.PNGImageWriterSpi

…

As a result, assuming I’m reading the logs right (big if!), the previous tile took about 1.2 seconds from “gettile” to the “running requests”, this one takes about 0.4seconds, which implies a big speedup!
I’ve attached the zip file of the request so you can check/compare with the ones in my user-list thread.

However the new tile appears to be a larger file. 121KB versus the previous 119KB.

Well, 3 times faster and a 1% increase in size, it seems it’s working well for you
File size is generally big though… is that really a tile?
Have you tried the image/png8 output format? It can do transparency and translucency,
so it can be used for overlays too.

Cheers

Andrea

–

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

---

[Jonathan_Moules2]

My last WMS requests were all quite small (using the Layerpreview), about 275*512.

I made the same requests but for a larger area (2000*1000). Approximate rendering times:
2.3.5 - 0.7 to 1.7s
2.4.0 - 0.15 to 0.25

This transmission is intended for the named addressee(s) only and may contain sensitive or protectively marked material up to RESTRICTED and should be handled accordingly. Unless you are the named addressee (or authorised to receive it for the addressee) you may not copy or use it, or disclose it to anyone else. If you have received this transmission in error please notify the sender immediately. All email traffic sent to or from us, including without limitation all GCSX traffic, may be subject to recording and/or monitoring in accordance with relevant legislation.

(attachments)

···

---

For the complex style, large (2000*1000) image in a very busy area (80,000+ features):
2.3.5 - 1.6s
2.4.0 - 0.4s

So the speed improvements scale up and are thus much more noticeable in larger and/or busier (like my TMS tile) images.

Short version - good work Andrea!

Jonathan

On 14 October 2013 15:13, Jonathan Moules <jonathanmoules@anonymised.com> wrote:

My files are png/8 with transparency. You can see the tiles here (being served as a regular image, not TMS):
http://maps.warwickshire.gov.uk/new.png
http://maps.warwickshire.gov.uk/old.png

The largest tile we have on our production system for this layergroup is 149KB. The median (of 270,000 tiles) is 10KB. Just under 1% are >= 100KB.

---

I’ve done a further test, this time WMS using layerpreview; the styling was fairly complex, but simpler than the TMS stuff. The exact same requests to a 2.3.5 instance and this 2.4.0 with the new module.

2.3.5:

On 14 October 2013 14:41, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Mon, Oct 14, 2013 at 3:35 PM, Jonathan Moules <jonathanmoules@anonymised.com> wrote:

Compared to 2.4.0:

Size easily went to the old one (about 4% smaller). Speed shouldn’t be compared here (most of it is the Oracle-GeoServer connection). Looking in the logs, rendering times:

2.3.5 - 0.05s to 0.15s
2.4.0 - 0.01s to 0.3s

So speedwise the new plugin wins, but the difference is negligible as you predicted.

I’ll try large WMS requests for that complex style next.

Jonathan

Hi Andrea,

Thanks for the link.

I did indeed notice that comment which is what prompted my post in the user list. But given that the rendering portion is fairly slow itself (or appears to be) and I’m using 64bit windows, I figured I’d give it a spin; what’s the worst that can happen right?

I guess it doesn’t need any special setting to turn it on? My log from a getmap request is slightly different now and ends shorter with only one line (no DEBUG level events?):

2013-10-14 13:23:56,384 INFO [org.geoserver.flow] - Running requests: 0, processing through flow controllers: 0

---

Previously it had about 100 lines of:
…

2013-10-14 11:29:57,022 DEBUG [org.geotools.image] - Encoded input image for png writer
2013-10-14 11:29:57,022 DEBUG [org.geotools.image] - Getting a writer
2013-10-14 11:29:57,022 DEBUG [org.geotools.image] - Unable to use Native ImageIO PNG writer.
2013-10-14 11:29:57,022 DEBUG [org.geotools.image] - Using ImageIO Writer with SPI: com.sun.imageio.plugins.png.PNGImageWriterSpi

…

As a result, assuming I’m reading the logs right (big if!), the previous tile took about 1.2 seconds from “gettile” to the “running requests”, this one takes about 0.4seconds, which implies a big speedup!
I’ve attached the zip file of the request so you can check/compare with the ones in my user-list thread.

However the new tile appears to be a larger file. 121KB versus the previous 119KB.

Well, 3 times faster and a 1% increase in size, it seems it’s working well for you
File size is generally big though… is that really a tile?
Have you tried the image/png8 output format? It can do transparency and translucency,
so it can be used for overlays too.

Cheers

Andrea

–

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

---

[Andrea_Aime4]

Ah, one thing,
it occurred to me you were reporting some quite detailed logging output.
You’re aware that all those logs are going to kill performance dead right?
For performance tests always run with “PRODUCTION” logging settings on.
Which you probably are, just making sure.

Cheers
Andrea

(attachments)

···

On Mon, Oct 14, 2013 at 4:46 PM, Jonathan Moules <jonathanmoules@anonymised.com> wrote:

My last WMS requests were all quite small (using the Layerpreview), about 275*512.

I made the same requests but for a larger area (2000*1000). Approximate rendering times:
2.3.5 - 0.7 to 1.7s
2.4.0 - 0.15 to 0.25

This transmission is intended for the named addressee(s) only and may contain sensitive or protectively marked material up to RESTRICTED and should be handled accordingly. Unless you are the named addressee (or authorised to receive it for the addressee) you may not copy or use it, or disclose it to anyone else. If you have received this transmission in error please notify the sender immediately. All email traffic sent to or from us, including without limitation all GCSX traffic, may be subject to recording and/or monitoring in accordance with relevant legislation.

–

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

---

---

For the complex style, large (2000*1000) image in a very busy area (80,000+ features):
2.3.5 - 1.6s
2.4.0 - 0.4s

So the speed improvements scale up and are thus much more noticeable in larger and/or busier (like my TMS tile) images.

Short version - good work Andrea!

Jonathan

On 14 October 2013 15:13, Jonathan Moules <jonathanmoules@anonymised.com> wrote:

My files are png/8 with transparency. You can see the tiles here (being served as a regular image, not TMS):
http://maps.warwickshire.gov.uk/new.png
http://maps.warwickshire.gov.uk/old.png

The largest tile we have on our production system for this layergroup is 149KB. The median (of 270,000 tiles) is 10KB. Just under 1% are >= 100KB.

---

I’ve done a further test, this time WMS using layerpreview; the styling was fairly complex, but simpler than the TMS stuff. The exact same requests to a 2.3.5 instance and this 2.4.0 with the new module.

2.3.5:

On 14 October 2013 14:41, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Mon, Oct 14, 2013 at 3:35 PM, Jonathan Moules <jonathanmoules@anonymised.com> wrote:

Compared to 2.4.0:

Size easily went to the old one (about 4% smaller). Speed shouldn’t be compared here (most of it is the Oracle-GeoServer connection). Looking in the logs, rendering times:

2.3.5 - 0.05s to 0.15s
2.4.0 - 0.01s to 0.3s

So speedwise the new plugin wins, but the difference is negligible as you predicted.

I’ll try large WMS requests for that complex style next.

Jonathan

Hi Andrea,

Thanks for the link.

I did indeed notice that comment which is what prompted my post in the user list. But given that the rendering portion is fairly slow itself (or appears to be) and I’m using 64bit windows, I figured I’d give it a spin; what’s the worst that can happen right?

I guess it doesn’t need any special setting to turn it on? My log from a getmap request is slightly different now and ends shorter with only one line (no DEBUG level events?):

2013-10-14 13:23:56,384 INFO [org.geoserver.flow] - Running requests: 0, processing through flow controllers: 0

---

Previously it had about 100 lines of:
…

2013-10-14 11:29:57,022 DEBUG [org.geotools.image] - Encoded input image for png writer
2013-10-14 11:29:57,022 DEBUG [org.geotools.image] - Getting a writer
2013-10-14 11:29:57,022 DEBUG [org.geotools.image] - Unable to use Native ImageIO PNG writer.
2013-10-14 11:29:57,022 DEBUG [org.geotools.image] - Using ImageIO Writer with SPI: com.sun.imageio.plugins.png.PNGImageWriterSpi

…

As a result, assuming I’m reading the logs right (big if!), the previous tile took about 1.2 seconds from “gettile” to the “running requests”, this one takes about 0.4seconds, which implies a big speedup!
I’ve attached the zip file of the request so you can check/compare with the ones in my user-list thread.

However the new tile appears to be a larger file. 121KB versus the previous 119KB.

Well, 3 times faster and a 1% increase in size, it seems it’s working well for you
File size is generally big though… is that really a tile?
Have you tried the image/png8 output format? It can do transparency and translucency,
so it can be used for overlays too.

Cheers

Andrea

–

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

---

[Jonathan_Moules2]

Ah…

But if I run it as “PRODUCTION” I don’t get any actually feedback in my log allowing me to judge how long a request took to render. Thus I had to use GEOTOOLS_DEVELOPER as the logging level otherwise I would have had no results at all. I guess I’m missing some fancy tool to judge how long rendering takes, so have just been doing the maths in my head from the logs. Totally scientific!

Cheers,
Jonathan

This transmission is intended for the named addressee(s) only and may contain sensitive or protectively marked material up to RESTRICTED and should be handled accordingly. Unless you are the named addressee (or authorised to receive it for the addressee) you may not copy or use it, or disclose it to anyone else. If you have received this transmission in error please notify the sender immediately. All email traffic sent to or from us, including without limitation all GCSX traffic, may be subject to recording and/or monitoring in accordance with relevant legislation.

(attachments)

···

On 14 October 2013 15:54, Andrea Aime <andrea.aime@anonymised.com> wrote:

Ah, one thing,
it occurred to me you were reporting some quite detailed logging output.
You’re aware that all those logs are going to kill performance dead right?
For performance tests always run with “PRODUCTION” logging settings on.
Which you probably are, just making sure.

Cheers

Andrea

On Mon, Oct 14, 2013 at 4:46 PM, Jonathan Moules <jonathanmoules@anonymised.com> wrote:

My last WMS requests were all quite small (using the Layerpreview), about 275*512.

I made the same requests but for a larger area (2000*1000). Approximate rendering times:
2.3.5 - 0.7 to 1.7s
2.4.0 - 0.15 to 0.25

This transmission is intended for the named addressee(s) only and may contain sensitive or protectively marked material up to RESTRICTED and should be handled accordingly. Unless you are the named addressee (or authorised to receive it for the addressee) you may not copy or use it, or disclose it to anyone else. If you have received this transmission in error please notify the sender immediately. All email traffic sent to or from us, including without limitation all GCSX traffic, may be subject to recording and/or monitoring in accordance with relevant legislation.

–

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

---

---

For the complex style, large (2000*1000) image in a very busy area (80,000+ features):
2.3.5 - 1.6s
2.4.0 - 0.4s

So the speed improvements scale up and are thus much more noticeable in larger and/or busier (like my TMS tile) images.

Short version - good work Andrea!

Jonathan

On 14 October 2013 15:13, Jonathan Moules <jonathanmoules@anonymised.com> wrote:

My files are png/8 with transparency. You can see the tiles here (being served as a regular image, not TMS):
http://maps.warwickshire.gov.uk/new.png
http://maps.warwickshire.gov.uk/old.png

The largest tile we have on our production system for this layergroup is 149KB. The median (of 270,000 tiles) is 10KB. Just under 1% are >= 100KB.

---

I’ve done a further test, this time WMS using layerpreview; the styling was fairly complex, but simpler than the TMS stuff. The exact same requests to a 2.3.5 instance and this 2.4.0 with the new module.

2.3.5:

On 14 October 2013 14:41, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Mon, Oct 14, 2013 at 3:35 PM, Jonathan Moules <jonathanmoules@anonymised.com> wrote:

Compared to 2.4.0:

Size easily went to the old one (about 4% smaller). Speed shouldn’t be compared here (most of it is the Oracle-GeoServer connection). Looking in the logs, rendering times:

2.3.5 - 0.05s to 0.15s
2.4.0 - 0.01s to 0.3s

So speedwise the new plugin wins, but the difference is negligible as you predicted.

I’ll try large WMS requests for that complex style next.

Jonathan

Hi Andrea,

Thanks for the link.

I did indeed notice that comment which is what prompted my post in the user list. But given that the rendering portion is fairly slow itself (or appears to be) and I’m using 64bit windows, I figured I’d give it a spin; what’s the worst that can happen right?

I guess it doesn’t need any special setting to turn it on? My log from a getmap request is slightly different now and ends shorter with only one line (no DEBUG level events?):

2013-10-14 13:23:56,384 INFO [org.geoserver.flow] - Running requests: 0, processing through flow controllers: 0

---

Previously it had about 100 lines of:
…

2013-10-14 11:29:57,022 DEBUG [org.geotools.image] - Encoded input image for png writer
2013-10-14 11:29:57,022 DEBUG [org.geotools.image] - Getting a writer
2013-10-14 11:29:57,022 DEBUG [org.geotools.image] - Unable to use Native ImageIO PNG writer.
2013-10-14 11:29:57,022 DEBUG [org.geotools.image] - Using ImageIO Writer with SPI: com.sun.imageio.plugins.png.PNGImageWriterSpi

…

As a result, assuming I’m reading the logs right (big if!), the previous tile took about 1.2 seconds from “gettile” to the “running requests”, this one takes about 0.4seconds, which implies a big speedup!
I’ve attached the zip file of the request so you can check/compare with the ones in my user-list thread.

However the new tile appears to be a larger file. 121KB versus the previous 119KB.

Well, 3 times faster and a 1% increase in size, it seems it’s working well for you
File size is generally big though… is that really a tile?
Have you tried the image/png8 output format? It can do transparency and translucency,
so it can be used for overlays too.

Cheers

Andrea

–

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

---