[Geoserver-users] gif performance

All...

Is there a known issue with WMS performance in GeoServer v1.4.0 when using
gif as the output format?

I have been trying to get acceptable performance out of GeoServer without
much success by following the instructions at
http://docs.codehaus.org/display/GEOSDOC/4+GeoServer+in+Production+Environment
but even after following the (seemingly) relevant steps from that document,
performance remained abysmal. I was finally struck with the inspiration of
trying a different output format and low and behold performance was MUCH
better. Tiles that were taking 6-7 seconds to render to a gif are taking
less than a second to render in either jpeg or png format! Does anyone have
an explanation for this large discrepency in rendering time? Is there any
way to improve gif performance? I would rather keep using gif if at all
possible as the output format because it currently offers the best tradeoff
between image size and quality. (PNGs are anywhere from 2 to 5 times larger
in size while JPEGs suffer from the usual JPEG artifacts.)

I installed JAI with the native libraries in the hopes that this would
improve performance but it appears to have had no impact. Is there anyway
to verify that GeoServer is using the JAI native libraries? I can't find
anything in the logs that sheds any light on the matter either way.

My environment:

CentOS 4 (RHEL4 clone)
JDK v1.5.0_07/v1.6.0 (tried both... Java 6 is slightly faster, but not
enough to matter)
Tomcat v5.5.17
GeoServer v1.4.0

Thanks,
Corey
--
View this message in context: http://www.nabble.com/gif-performance-tf3574011.html#a9986880
Sent from the GeoServer - User mailing list archive at Nabble.com.

Yeroc ha scritto:

All...

Is there a known issue with WMS performance in GeoServer v1.4.0 when using
gif as the output format?

Nothing specific to GeoServer 1.4

I have been trying to get acceptable performance out of GeoServer without
much success by following the instructions at
http://docs.codehaus.org/display/GEOSDOC/4+GeoServer+in+Production+Environment
but even after following the (seemingly) relevant steps from that document,
performance remained abysmal. I was finally struck with the inspiration of
trying a different output format and low and behold performance was MUCH
better. Tiles that were taking 6-7 seconds to render to a gif are taking
less than a second to render in either jpeg or png format! Does anyone have
an explanation for this large discrepency in rendering time?

I think I do. Images are always rendered at full color depth, and only
during the GIF encoding are reduced to 256 colors. This color reduction
is usually very time consuming, and thus the bigger rendering times.

I've been working on PNG8 output and reported some findings here:
http://docs.codehaus.org/display/GEOS/Paletted+images

Basically the idea is that if we want good performance we need to render
the image directly at 256 colors, this gives both a size and performance
improvement. Unfortunately, it's not easy as it seems, to have good output we have to refer to a pre-computed palette, and even with that,
the results are less than stellar in quality since apparently I cannot
kill the dithering in the output...

Is there any
way to improve gif performance? I would rather keep using gif if at all
possible as the output format because it currently offers the best tradeoff
between image size and quality. (PNGs are anywhere from 2 to 5 times larger
in size while JPEGs suffer from the usual JPEG artifacts.)

Long story short, without changing the code, no, there is no specific
GIF improvement we can sport. I guess it's something we may want to handle for the Geoserver 1.5.1 release, and in this case, you may be
interested in trying out 1.5.1 nightlies (we do provide a night build for the in-development branches now).

I installed JAI with the native libraries in the hopes that this would
improve performance but it appears to have had no impact. Is there anyway
to verify that GeoServer is using the JAI native libraries? I can't find
anything in the logs that sheds any light on the matter either way.

JAI is not used at all by Geoserver 1.4.0, you have to upgrade to Geoserver 1.5.0 (that will be announced on Monday) to benefit from it.
I also suggest to have a look again at that page, which I have modified
two days ago, and set the fastest output strategy. This will give you
some more improvement.

Cheers
Andrea

  >> I installed JAI with the native libraries in the hopes that this would

improve performance but it appears to have had no impact. Is there anyway
to verify that GeoServer is using the JAI native libraries? I can't find
anything in the logs that sheds any light on the matter either way.

JAI is not used at all by Geoserver 1.4.0, you have to upgrade to Geoserver 1.5.0 (that will be announced on Monday) to benefit from it.
I also suggest to have a look again at that page, which I have modified
two days ago, and set the fastest output strategy. This will give you
some more improvement.

I think there may be some chance that 1.5.0 will make things a bit faster (though it could change in other directions too), as I believe it's a new GIF output writer, using JAI instead of our own. And it may benefit from native acceleration. So at least try that one out and let us know.

Chris

Cheers
Andrea

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
Chris Holmes
The Open Planning Project
http://topp.openplans.org

Thanks for your response Andrea.

I will try GeoServer v1.5.0 when it's released in the hopes that performance
is improved. I'm surprised this hasn't come up before. Am I the only one
using GeoServer as a WMS with GIF output? It might be worth documenting
this issue somewhere on the website. I've switched to PNG output for now.
Another thing that occurred to me is that JPEG output would be usable if it
were possible to tell GeoServer to generate JPEGs using less compression.
Is it possible to control this somehow? Most applications that generate
JPEGs let you choose a quality setting... something like this would be very
nice in GeoServer. Currently, the JPEGs generated by GeoServer contain a
lot of compression artifacts.

aaime wrote:

JAI is not used at all by Geoserver 1.4.0, you have to upgrade to
Geoserver 1.5.0 (that will be announced on Monday) to benefit from it.
I also suggest to have a look again at that page, which I have modified
two days ago, and set the fastest output strategy. This will give you
some more improvement.

When you say "set the fastest output strategy" I assume you mean the service
strategy? I had already tried setting this to "SPEED" in the hopes that it
would improve performance. Unfortunately, as with most of the settings I
changed, it had negligible impact. Actually, the only change that had a
measurable impact (in my case) was switching to using the server JVM.

Corey
--
View this message in context: http://www.nabble.com/gif-performance-tf3574011.html#a9998104
Sent from the GeoServer - User mailing list archive at Nabble.com.

Yeroc ha scritto:

Thanks for your response Andrea.

I will try GeoServer v1.5.0 when it's released in the hopes that performance
is improved. I'm surprised this hasn't come up before. Am I the only one
using GeoServer as a WMS with GIF output? It might be worth documenting
this issue somewhere on the website.

I do believe there are very few, if any, using that output format because of the dithering which makes maps look bad.

I've switched to PNG output for now. Another thing that occurred to me is that JPEG output would be usable if it
were possible to tell GeoServer to generate JPEGs using less compression. Is it possible to control this somehow? Most applications that generate
JPEGs let you choose a quality setting... something like this would be very
nice in GeoServer. Currently, the JPEGs generated by GeoServer contain a
lot of compression artifacts.

In Geoserver 1.5 it's controlled programmaticaly, but we don't expose it
to the user level (in 1.4 I do think the default compression was used).
We have currently troubles modifying configuration and UI because people
that made those subsystems are no more with us, and to add a single
configuration option (like the jpeg compression level) we have to modify
some 10-12 files. We are investigating a new UI and configuration subsystem, but the thing it's taking a lot longer than anticipated.
I've opened a new issue for this, tentatively scheduled for 1.5.1
http://jira.codehaus.org/browse/GEOS-1041

aaime wrote:

JAI is not used at all by Geoserver 1.4.0, you have to upgrade to Geoserver 1.5.0 (that will be announced on Monday) to benefit from it.
I also suggest to have a look again at that page, which I have modified
two days ago, and set the fastest output strategy. This will give you
some more improvement.

When you say "set the fastest output strategy" I assume you mean the service
strategy? I had already tried setting this to "SPEED" in the hopes that it
would improve performance. Unfortunately, as with most of the settings I
changed, it had negligible impact. Actually, the only change that had a
measurable impact (in my case) was switching to using the server JVM.

Sigh... that's because color reduction process is dominating the image production chain afaik. With other benchmarks changing the output strategy almost halved the response time.

Anyways, I just had a quick check with Geoserver 1.5.0 using the sample
request and FasterFox to time it. I made a few requests before reporting
the timings, so that what needed to be JIT compiled is:
http://localhost:8080/geoserver/wms?bbox=-130,24,-66,50&styles=population&Format=image/png&request=GetMap&layers=topp:states&width=1100&height=500&srs=EPSG:4326
0.359s
http://localhost:8080/geoserver/wms?bbox=-130,24,-66,50&styles=population&Format=image/gif&request=GetMap&layers=topp:states&width=1100&height=500&srs=EPSG:4326
0.422s

This is with jdk 1.6.0, with jdk 1.5.0_09 both image generation takes
around 0.5s (I haven't used the server VM in neither of the tests).

Cheers
Andrea

Yeroc ha scritto:

Thanks for your response Andrea.

I will try GeoServer v1.5.0 when it's released in the hopes that performance
is improved. I'm surprised this hasn't come up before. Am I the only one
using GeoServer as a WMS with GIF output?

Yeroc,
I've successfully implemented a png8 output that does not look bad, and it's good looking too (that is, no dithering).
Here you can find some details:
http://docs.codehaus.org/display/GEOS/Paletted+images

At the moment the code is in geoserver trunk (1.6.x series) but I'll back port it shortly. Would you be interested in trying it out using
a Geoserver 1.5.1 nightly?

Cheers
Andrea

Andrea....

I just tried v1.5.0 but so far it has been a real step backwards for me. :frowning:
Maybe I'm doing something(s) wrong. I've got both v1.4.0 and v1.5.0
installed as separate contexts in a single Tomcat v5.5.17 instance running
under JDK v1.6.0 with native JAI/ImageIO. Both versions are configured to
use the same data directory (and thus have identical configurations).

The first issue is that the rendering under v1.5.0 is all messed up. I'm
using the layer grouping functionality to create a virtual basemap layer but
v1.5.0 doesn't seem to handle it properly. It looks like it might be
rendering the layers using the default style associated with the feature
type rather than the one specified with the virtual layer.

The second issue (as noted below) is that the GIF output produced by v1.5.0
is badly dithered (I haven't noticed any dithering in v1.4.0). Do you know
what caused the output quality regression between v1.4.0 and v1.5.0?

The third issue is that the JPEGs produced by v1.5.0 seem to have colour
issues. The background colour is defined as a blue but becomes a rather
bright orange with v1.5.0. v1.4.0 gets the colour right.

I hope someone can shed some light on these issues (I can send sample before
& after images if that would be helpful.) Until I can resolve the first
issue I can't comment further on v1.5.0 performance relative to v1.4.0.

See further responses inline...

aaime wrote:

I do believe there are very few, if any, using that output format
because of the dithering which makes maps look bad.

I hadn't noticed any dithering when using GIF output with GeoServer v1.4.0
but I just gave v1.5.0 a try and I see that the output is badly dithered as
you say.

aaime wrote:

In Geoserver 1.5 it's controlled programmaticaly, but we don't expose it
to the user level (in 1.4 I do think the default compression was used).
We have currently troubles modifying configuration and UI because people
that made those subsystems are no more with us, and to add a single
configuration option (like the jpeg compression level) we have to modify
some 10-12 files. We are investigating a new UI and configuration
subsystem, but the thing it's taking a lot longer than anticipated.
I've opened a new issue for this, tentatively scheduled for 1.5.1
http://jira.codehaus.org/browse/GEOS-1041

That would be great... Maybe a stopgap solution would be to default to
higher-quality output? Or would this negatively affect existing users?

Corey
--
View this message in context: http://www.nabble.com/gif-performance-tf3574011.html#a10066983
Sent from the GeoServer - User mailing list archive at Nabble.com.

Ciao Yeroc,
as of the issue with JPEGs, can you check that the native acceleration is
switched off for JPEG generation? Pure java jpeg encoder is safer even if
slightly slower.
Could you also, please, report the version number of JAI and ImageIO you are
using as well as the platform you are working on? Some sample data would
also help.

Ciao,
Simone.

-------------------------------------------------------
Eng. Simone Giannecchini
President /CEO GeoSolutions S.A.S.
Via Carignoni 51
55041 Camaiore (LU)
Italy

phone: +39 0584983027
fax: +39 0584983027
mob: +39 333 8128928

http://www.geo-solutions.it

-------------------------------------------------------

-----Original Message-----
From: geoserver-users-bounces@lists.sourceforge.net
[mailto:geoserver-users-bounces@lists.sourceforge.net] On Behalf Of Yeroc
Sent: giovedì 19 aprile 2007 0.17
To: geoserver-users@lists.sourceforge.net
Subject: Re: [Geoserver-users] gif performance

Andrea....

I just tried v1.5.0 but so far it has been a real step backwards for me. :frowning:
Maybe I'm doing something(s) wrong. I've got both v1.4.0 and v1.5.0
installed as separate contexts in a single Tomcat v5.5.17 instance running
under JDK v1.6.0 with native JAI/ImageIO. Both versions are configured to
use the same data directory (and thus have identical configurations).

The first issue is that the rendering under v1.5.0 is all messed up. I'm
using the layer grouping functionality to create a virtual basemap layer but
v1.5.0 doesn't seem to handle it properly. It looks like it might be
rendering the layers using the default style associated with the feature
type rather than the one specified with the virtual layer.

The second issue (as noted below) is that the GIF output produced by v1.5.0
is badly dithered (I haven't noticed any dithering in v1.4.0). Do you know
what caused the output quality regression between v1.4.0 and v1.5.0?

The third issue is that the JPEGs produced by v1.5.0 seem to have colour
issues. The background colour is defined as a blue but becomes a rather
bright orange with v1.5.0. v1.4.0 gets the colour right.

I hope someone can shed some light on these issues (I can send sample before
& after images if that would be helpful.) Until I can resolve the first
issue I can't comment further on v1.5.0 performance relative to v1.4.0.

See further responses inline...

aaime wrote:

I do believe there are very few, if any, using that output format
because of the dithering which makes maps look bad.

I hadn't noticed any dithering when using GIF output with GeoServer v1.4.0
but I just gave v1.5.0 a try and I see that the output is badly dithered as
you say.

aaime wrote:

In Geoserver 1.5 it's controlled programmaticaly, but we don't expose it
to the user level (in 1.4 I do think the default compression was used).
We have currently troubles modifying configuration and UI because people
that made those subsystems are no more with us, and to add a single
configuration option (like the jpeg compression level) we have to modify
some 10-12 files. We are investigating a new UI and configuration
subsystem, but the thing it's taking a lot longer than anticipated.
I've opened a new issue for this, tentatively scheduled for 1.5.1
http://jira.codehaus.org/browse/GEOS-1041

That would be great... Maybe a stopgap solution would be to default to
higher-quality output? Or would this negatively affect existing users?

Corey
--
View this message in context:
http://www.nabble.com/gif-performance-tf3574011.html#a10066983
Sent from the GeoServer - User mailing list archive at Nabble.com.

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Simone...

Thanks for your reply. I was using the default (native acceleration on)
settings. Turning off native acceleration for JPEG generation did fix the
wonky colour problem though. Should the production environment page
(http://docs.codehaus.org/display/GEOSDOC/4+GeoServer+in+Production+Environment)
be revised? It currently recommends JAI/ImageIO with native
acceleration...

I have JAI v1.1.3 and ImageIO v1.1 installed on RedHat Enterprise Linux v4
under JDK v1.6.0.

Corey

Simone.Giannecchini wrote:

Ciao Yeroc,
as of the issue with JPEGs, can you check that the native acceleration is
switched off for JPEG generation? Pure java jpeg encoder is safer even if
slightly slower.
Could you also, please, report the version number of JAI and ImageIO you
are
using as well as the platform you are working on? Some sample data would
also help.

--
View this message in context: http://www.nabble.com/gif-performance-tf3574011.html#a10069077
Sent from the GeoServer - User mailing list archive at Nabble.com.

Ciao Corey,
There are some combinations of OS and ImageIO JPEG native encoder versions
that cause problems. I think it is worth to warn people about this by
updating the production environment page, if you could that yourself it
would be just great.
What type of data are you rendering? Could you, please, provide a sample for
testing purposes and error reporting?

Moreover, would you be willing to try out a newer binary build of ImageIO to
see if the problem with the JPEG encoder disappears? You can find it here:

https://jai-imageio.dev.java.net/binary-builds.html#Daily_builds

Take the right zip for you, put the .so under JDK_HOME/lib/i386 or somewhere
else using the LD_LIBRARY_APTH env var, then place the jars under
JDK_HOME/jre/lib/ext.

Could you also provide some more info about your testing of GIFs, I might
try to find some time to improve the dithering code.

Ciao,
Simone.

PS:
Sorry about calling you yeroc, but it is getting late here and lack of sleep
time it is jeopardizing my brain's capabilities :slight_smile:

-------------------------------------------------------
Eng. Simone Giannecchini
President /CEO GeoSolutions S.A.S.
Via Carignoni 51
55041 Camaiore (LU)
Italy

phone: +39 0584983027
fax: +39 0584983027
mob: +39 333 8128928

http://www.geo-solutions.it

-------------------------------------------------------

-----Original Message-----
From: geoserver-users-bounces@lists.sourceforge.net
[mailto:geoserver-users-bounces@lists.sourceforge.net] On Behalf Of Yeroc
Sent: giovedì 19 aprile 2007 1.18
To: geoserver-users@lists.sourceforge.net
Subject: Re: [Geoserver-users] gif performance

Simone...

Thanks for your reply. I was using the default (native acceleration on)
settings. Turning off native acceleration for JPEG generation did fix the
wonky colour problem though. Should the production environment page
(http://docs.codehaus.org/display/GEOSDOC/4+GeoServer+in+Production+Environm
ent)
be revised? It currently recommends JAI/ImageIO with native
acceleration...

I have JAI v1.1.3 and ImageIO v1.1 installed on RedHat Enterprise Linux v4
under JDK v1.6.0.

Corey

Simone.Giannecchini wrote:

Ciao Yeroc,
as of the issue with JPEGs, can you check that the native acceleration is
switched off for JPEG generation? Pure java jpeg encoder is safer even if
slightly slower.
Could you also, please, report the version number of JAI and ImageIO you
are
using as well as the platform you are working on? Some sample data would
also help.

--
View this message in context:
http://www.nabble.com/gif-performance-tf3574011.html#a10069077
Sent from the GeoServer - User mailing list archive at Nabble.com.

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Yeroc ha scritto:

Andrea....

I just tried v1.5.0 but so far it has been a real step backwards for me. :frowning: Maybe I'm doing something(s) wrong. I've got both v1.4.0 and v1.5.0
installed as separate contexts in a single Tomcat v5.5.17 instance running
under JDK v1.6.0 with native JAI/ImageIO. Both versions are configured to
use the same data directory (and thus have identical configurations).

The first issue is that the rendering under v1.5.0 is all messed up. I'm
using the layer grouping functionality to create a virtual basemap layer but
v1.5.0 doesn't seem to handle it properly. It looks like it might be
rendering the layers using the default style associated with the feature
type rather than the one specified with the virtual layer.

Hum...

The second issue (as noted below) is that the GIF output produced by v1.5.0
is badly dithered (I haven't noticed any dithering in v1.4.0). Do you know
what caused the output quality regression between v1.4.0 and v1.5.0?

Moving from Geoserver 1.4.0 to Geoserver 1.5.0 we moved from the classic
Java encoder to a JAI based one, that uses dithering to simulate colors
it does not have in its palette.
I tried out removing that code and inserting back the old ImageIO based
one, rendering times are 50-100% bigger, but the image quality is definitely superior (it's taking the time to figure out an optimal palette at runtime, now I see why it was so slow with 1.4.0).

The third issue is that the JPEGs produced by v1.5.0 seem to have colour
issues. The background colour is defined as a blue but becomes a rather
bright orange with v1.5.0. v1.4.0 gets the colour right.

I noticed this one too, after releasing 1.5.0 thought...
http://jira.codehaus.org/browse/GEOS-1042

And yeah, turning native acceleration off makes the image appear fine. Btw, I'm on Windows, so it does not seem to be a platform dependent issue to me...
The Goeserver 1.4.0 was using JAI too, but without native acceleration,
I guess that's why we never saw this issue.

aaime wrote:

In Geoserver 1.5 it's controlled programmaticaly, but we don't expose it
to the user level (in 1.4 I do think the default compression was used).
We have currently troubles modifying configuration and UI because people
that made those subsystems are no more with us, and to add a single
configuration option (like the jpeg compression level) we have to modify
some 10-12 files. We are investigating a new UI and configuration subsystem, but the thing it's taking a lot longer than anticipated.
I've opened a new issue for this, tentatively scheduled for 1.5.1
http://jira.codehaus.org/browse/GEOS-1041

That would be great... Maybe a stopgap solution would be to default to
higher-quality output? Or would this negatively affect existing users?

It would, for sure, since it would generate bigger images too.
We have to find a way to expose this settings to the user, either
with configuration, or with request parameters (so that you can use different compression on different layers).

In any case, I have completed the first part of my png8 work, that
renders both fast and with decent quality (but you'll have to give up antialiasing, sorry, there's nothing I can do about it).
Here is some sample output:
http://docs.codehaus.org/display/GEOS/Paletted+images

You can try it out when today's nightly build is out at:
http://geo.openplans.org/nightly/1.5.x/
(the server is in Canada, so it should be a matter of a few hours
before the Apr 19 nightly is out).

Then, add &palette=safe to your GetMap requests to work against the
internet safe palette, or if you want better quality, create a
"palettes" subfodler inside your data directory, and drop a .png
or .gif image with the desired palette there, then user "palette=xxx"
where xxx is the name of the file with the palette, minus the extension
(that is, if you have palettes/mypalette.png, use palette=mypalette).

In my experiments pngs and gifs generated with such a fixed palette
are both good looking and very fast to generate (faster than 24bit
png's).

Cheers
Andrea

Simone,

Simone.Giannecchini wrote:

Ciao Corey,
There are some combinations of OS and ImageIO JPEG native encoder versions
that cause problems. I think it is worth to warn people about this by
updating the production environment page, if you could that yourself it
would be just great.
What type of data are you rendering? Could you, please, provide a sample
for
testing purposes and error reporting?

Andrea's bug report (http://jira.codehaus.org/browse/GEOS-1042) contains (I
think) all the information required to reproduce the JPEG colour problem. I
added a warning note to the production page though I couldn't figure out how
to link directly to the JIRA issue (is there not a WIKI macro for linking to
JIRA issues?) Let me know if you still need more information. I'm not sure
what you mean by type of data? It's a mixture of shapefiles and postgres
data sources derived from sources like VMAP0 for a basemap. In particular
the one colour that causes the problem is the map background colour which is
set to BGCOLOR=0x6496FF.

Moreover, would you be willing to try out a newer binary build of ImageIO
to
see if the problem with the JPEG encoder disappears? You can find it here:

https://jai-imageio.dev.java.net/binary-builds.html#Daily_builds

Take the right zip for you, put the .so under JDK_HOME/lib/i386 or
somewhere
else using the LD_LIBRARY_APTH env var, then place the jars under
JDK_HOME/jre/lib/ext.

I don't have extra time to spend on this right now. Until my first issue is
resolved I'll be sticking with GeoServer v1.4.0.

Could you also provide some more info about your testing of GIFs, I might
try to find some time to improve the dithering code.

It looks like Andrea is aware of the issue and has written some code that
will improve this situation in v1.5.1.

Thanks,
Corey
--
View this message in context: http://www.nabble.com/gif-performance-tf3574011.html#a10081320
Sent from the GeoServer - User mailing list archive at Nabble.com.