[Geoserver-users] Adding PNG8 format and fixed palette parameters

Hi,
the new Geoserver site gives me an opportunity to
bring back on the table the PNG8 issue.
When I do open the new home page, the map takes
some 10-20 seconds to be fully loaded. Tilecache
is serving the pages, so I guess not much computation
is performed, the issue is more one of the bandwidth/latency
needed to gather the tiles.

Using PNG 256 colors images usually cuts the image size
by 50% or more on vector style layers, see for example:
http://mapserver.gis.umn.edu/docs/faq/faqsection_view?section=Map%20Output

Adding a PNG8 format is not difficult, the issue in tiled
rendering is that we want the same palette for all images,
so we do have to use a fixed one. At the same time, those not
using tiled rendering may want to have an optimal palette
generated at run time.

The palette could be a new vendor parameter in wms requests.
The rationale here is that palette is really dependent on the
request (what layers you put in). We could start with a
fixed palette=geoserver parameter, to enable the usage of a
pre-configure palette, and then decide whether we do want to
support user configurable ones.

What do you think?
Cheers
Andrea

Hi,
actually enabling PNG8 in the actual GeoServer 1.5 woul be a matter of
10 minutes since the ImageWorker that me and Martin developed (and
that Andrea Tested a quite a this summer) hall all the methods we need
for supporting the PNG8 creation.

Don't get too excited, there are a couple of cons to consider.

1>There are a couple algorithms already implemented in JAI to do the
dithering or the error diffusion to build a palette for an rgb image,
doing the same on an RGBA image is almost impossible but in the
ImageWorker I implemented some methods that do this which took me a
while to figure out actually (I am complimenting myself I know :slight_smile: ).
The problem is that to achieve good quality this is not enough as one
user (Vincent Schut) I believe) pointed out already. This algorithms
requires the user to provide a palette otherwise they use a fixed one
and they try to minimize the error between the actual color and the
final one leading to some visual artifacts (do you remember waves on
the north sea Vincent? :slight_smile: ).
Well there is a way to prevent this which is by doing a previous
precomputation on the image to convert looking for the best fitting
palette but this process can be computational heavy.

I am sure we can find a way to workaround this problem (well, I have a
couple of ideas that should do the trick but they need to be tweaked a
bit) but it would require some good thinking.

2>Loading an externally defined palette would be pretty easy but I
guess we are referring to a palette to actually dither the image we
get from the streaming renderer. In this case we could try to draw
directly on a paletted image which would make things MUCH faster
instead of drawing in RGBA and then doing a dither.

This is not going to be that easy though, but it would worth giving it a try.

3>In case we would want to specify a different palette for each layer
we would need to modify a bit the streaming renderer using more than
just a single buffered image as he target graphics allowing each layer
to draw on its own graphics (which btw is way better and we could use
some parallelism to speed things up).

<<Summarising>>
Implementing the vendor param for giving a palette could be pretty
easy. I would also like to see an option in GeoServer UI for loading a
palette from a given pal file.

We could implement this with the actual configuration based on the
ImageWorker capabilities in 1 day.

The things I would like to see some experimenting on asap would be:

1>make possible to choose as a config option from the UI the dithering
algorithm from the outside in order to give users the possibility to
pick the bet one for them up. (easy task)

2>make possible to ask the geoserver to look for the best fitting
palette possibly using some optimization. This would create better
looking images.

3>render directly on paletted images using a supplied or default palette.

I stop myself here since I am realizing that I already lost the
audience, but I am excited about this topic (as Andrea knows well)
since I think we can find a way to conjugate quality and speed.

Simone.

On 1/25/07, Andrea Aime <aaime@anonymised.com> wrote:

Hi,
the new Geoserver site gives me an opportunity to
bring back on the table the PNG8 issue.
When I do open the new home page, the map takes
some 10-20 seconds to be fully loaded. Tilecache
is serving the pages, so I guess not much computation
is performed, the issue is more one of the bandwidth/latency
needed to gather the tiles.

Using PNG 256 colors images usually cuts the image size
by 50% or more on vector style layers, see for example:
http://mapserver.gis.umn.edu/docs/faq/faqsection_view?section=Map%20Output

Adding a PNG8 format is not difficult, the issue in tiled
rendering is that we want the same palette for all images,
so we do have to use a fixed one. At the same time, those not
using tiled rendering may want to have an optimal palette
generated at run time.

The palette could be a new vendor parameter in wms requests.
The rationale here is that palette is really dependent on the
request (what layers you put in). We could start with a
fixed palette=geoserver parameter, to enable the usage of a
pre-configure palette, and then decide whether we do want to
support user configurable ones.

What do you think?
Cheers
Andrea

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

--
-------------------------------------------------------
Eng. Simone Giannecchini
President /CEO GeoSolutions

http://www.geo-solutions.it

-------------------------------------------------------

When considering your vendor specific parameter I have a couple of ideas.
- Reuse the color map elements from the SLD document.
- Consider the use of an environment variable so that the SLD rendering has a chance to map from the color it ended up with to an entry in your shared palette

Finally are we sure the SLD specification does not allow for this sort of use? It strikes me as more the concern of the SLD specification then something for WMS GetMap. I know SLD itself is an extenstion for WMS GetMap but you know what I am talking about here - let's see if we can solve this problem for rendering (rather then just for WMS).

Cheers
Jody

Hi,
the new Geoserver site gives me an opportunity to
bring back on the table the PNG8 issue.
When I do open the new home page, the map takes
some 10-20 seconds to be fully loaded. Tilecache
is serving the pages, so I guess not much computation
is performed, the issue is more one of the bandwidth/latency
needed to gather the tiles.

Using PNG 256 colors images usually cuts the image size
by 50% or more on vector style layers, see for example:
MapServer 8.2.2 Documentation — MapServer 8.2.2 documentation

Adding a PNG8 format is not difficult, the issue in tiled
rendering is that we want the same palette for all images,
so we do have to use a fixed one. At the same time, those not
using tiled rendering may want to have an optimal palette
generated at run time.

The palette could be a new vendor parameter in wms requests.
The rationale here is that palette is really dependent on the
request (what layers you put in). We could start with a
fixed palette=geoserver parameter, to enable the usage of a
pre-configure palette, and then decide whether we do want to
support user configurable ones.

What do you think?
Cheers
Andrea

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
geoserver-devel List Signup and Options
  

Jody Garnett ha scritto:

When considering your vendor specific parameter I have a couple of ideas.
- Reuse the color map elements from the SLD document.
- Consider the use of an environment variable so that the SLD rendering has a chance to map from the color it ended up with to an entry in your shared palette

Finally are we sure the SLD specification does not allow for this sort of use? It strikes me as more the concern of the SLD specification then something for WMS GetMap. I know SLD itself is an extenstion for WMS GetMap but you know what I am talking about here - let's see if we can solve this problem for rendering (rather then just for WMS).

There is an SLD extension all right, but not everyone is using it.
The palette depends on what layers you asked for and the styles you
used, and I don't feel like forcing people to use SLD just for that.
Adding a vendor parameter plays well with clients such as OpenLayers.
Oh, I see your point now, maybe it does not play so well with uDig?

Using the colormap for palette definition is a bit heavyweight, but
it could do. I would like to support palette definition files from
some widely used image software, I don't know, the Gimp.

Cheers
Andrea

Simone Giannecchini ha scritto:

Hi,
actually enabling PNG8 in the actual GeoServer 1.5 woul be a matter of
10 minutes since the ImageWorker that me and Martin developed (and
that Andrea Tested a quite a this summer) hall all the methods we need
for supporting the PNG8 creation.

Don't get too excited, there are a couple of cons to consider.

1>There are a couple algorithms already implemented in JAI to do the
dithering or the error diffusion to build a palette for an rgb image,
doing the same on an RGBA image is almost impossible but in the
ImageWorker I implemented some methods that do this which took me a
while to figure out actually (I am complimenting myself I know :slight_smile: ).
The problem is that to achieve good quality this is not enough as one
user (Vincent Schut) I believe) pointed out already. This algorithms
requires the user to provide a palette otherwise they use a fixed one
and they try to minimize the error between the actual color and the
final one leading to some visual artifacts (do you remember waves on
the north sea Vincent? :slight_smile: ).
Well there is a way to prevent this which is by doing a previous
precomputation on the image to convert looking for the best fitting
palette but this process can be computational heavy.

I am sure we can find a way to workaround this problem (well, I have a
couple of ideas that should do the trick but they need to be tweaked a
bit) but it would require some good thinking.

Is there any way to avoid dithering, and use "nearest color" instead?

2>Loading an externally defined palette would be pretty easy but I
guess we are referring to a palette to actually dither the image we
get from the streaming renderer. In this case we could try to draw
directly on a paletted image which would make things MUCH faster
instead of drawing in RGBA and then doing a dither.

This is not going to be that easy though, but it would worth giving it a try.

That was my point, use fixed or user provide palettes and 256 color
images to start with.

3>In case we would want to specify a different palette for each layer
we would need to modify a bit the streaming renderer using more than
just a single buffered image as he target graphics allowing each layer
to draw on its own graphics (which btw is way better and we could use
some parallelism to speed things up).

I'd say, one request, one palette. You get one image from getMap anyways. Lets leave speeding up things further with parallel rendering
threads for another day, shall we? :-p (I'm not saying it's a bad idea,
I just want to get things done a bit a time, and start attacking
this issue quick).

<<Summarising>>
Implementing the vendor param for giving a palette could be pretty
easy. I would also like to see an option in GeoServer UI for loading a
palette from a given pal file.

Pal file? Do you have a file format in mind? As I said with Jody,
I would like to have the ability to load common palette files (if
a common one exists). Well, we could load the pallette out of a gif
or a png file... now that's something any software can generate :slight_smile:
"Palette by example", LOL!

That would be even easier. Set up a "palette" directory in the data dir,
fill it with gif and png8 files, when the request contains a "palette=shiny" just look for shiny.png or shiny.gif and suck
the palette out of it. Hopefully this can be done without
loading the file in memory right?

I stop myself here since I am realizing that I already lost the
audience, but I am excited about this topic (as Andrea knows well)
since I think we can find a way to conjugate quality and speed.

Very much agree :slight_smile:
Cheers
Andrea

Jody Garnett ha scritto:

When considering your vendor specific parameter I have a couple of ideas.
- Reuse the color map elements from the SLD document.
- Consider the use of an environment variable so that the SLD rendering has a chance to map from the color it ended up with to an entry in your shared palette

Finally are we sure the SLD specification does not allow for this sort of use? It strikes me as more the concern of the SLD specification then something for WMS GetMap. I know SLD itself is an extenstion for WMS GetMap but you know what I am talking about here - let's see if we can solve this problem for rendering (rather then just for WMS).

Oh, btw, there is one conceputal mix between palettes and SLD...
scale ranges. Different scales would ask for different rendering,
and thus potentially, different palettes.

There are issues thought:
* the usual wms request names styles, and styles do not know about
each other, where do I put the palette definition?
* with multiple layers, they can change styles at different scales,
   thus mixing things quite a bit
* an SLD may contain various rules, would I need to specify a palette
   name in a vendor option for each?
   I see that we are using <VendorOption> for extra tags, but looking
   at the SLD spec, it seems this tag is not cited anywhere. Well,
   I guess in this case, I could add it everywhere, even in Rule...

So, I see a need for a way to specify scale-palette dependency,
not sure where to put it thought. Maybe, if we go the "palette
by example" route, but having "shiny-10000.gif" for scale denominator < 10.000, "shiny-1000000.gif" for scale denominator > 10.000, < 1.000.000
and "shiny.gif" for everything else.

Cheers
Andrea