[Geoserver-users] On tiling, rich clients, and cache

Folks,

since I probably missed something in recent weeks, I'd like to know what's the state of the play regarding cache support for tiling on rich clients.

I'd like to use OpenLayers client with Google Maps imagery and a WMS overlay, which means WMS images has to be tiled and cached to have a good performance.

My understanding is that, so far, GeoServer hasn't a working cache, let alone a cache able to serve tiled data (which is harder to make, due to issues with cross-tyles symbols and text), though some other products may be used, like TileCache.

Any timeframe for this tiled-cache to come along ?

If time allows I'd love to contribute, by the way.

Regards,

--------------------
    Luca Morandini
www.lucamorandini.it
--------------------

Hi Luca,

You are right, at this time GeoServer itself has no cache built in. However users have succesfully use it with TileCache.

For some time we have been talking about a "JTileCache", that is a port of TileCache to java that we could bundle directly in GeoServer. Someone was recently done a jython based one but I dont think anyone has gotten around to trying it out.

Also, there is a Summer of Code project ( which is looking pretty good ) that is to do a complete port to java ( non-jython based ). So we are hoping that comes through.

So... excuse the rant, the upshot is that you should see caching built into GeoServer in the next couple of months.

Aside, I believe there are a still a few issues with the renderer when it comes to rendering tiles. Andrea is this is true?

-Justin

Luca Morandini wrote:

Folks,

since I probably missed something in recent weeks, I'd like to know what's the state of the play regarding cache support for tiling on rich clients.

I'd like to use OpenLayers client with Google Maps imagery and a WMS overlay, which means WMS images has to be tiled and cached to have a good performance.

My understanding is that, so far, GeoServer hasn't a working cache, let alone a cache able to serve tiled data (which is harder to make, due to issues with cross-tyles symbols and text), though some other products may be used, like TileCache.

Any timeframe for this tiled-cache to come along ?

If time allows I'd love to contribute, by the way.

Regards,

--------------------
    Luca Morandini
www.lucamorandini.it
--------------------

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
Justin Deoliveira
The Open Planning Project
http://topp.openplans.org

Justin Deoliveira ha scritto:

Hi Luca,

You are right, at this time GeoServer itself has no cache built in. However users have succesfully use it with TileCache.

For some time we have been talking about a "JTileCache", that is a port of TileCache to java that we could bundle directly in GeoServer. Someone was recently done a jython based one but I dont think anyone has gotten around to trying it out.

Also, there is a Summer of Code project ( which is looking pretty good ) that is to do a complete port to java ( non-jython based ). So we are hoping that comes through.

So... excuse the rant, the upshot is that you should see caching built into GeoServer in the next couple of months.

Aside, I believe there are a still a few issues with the renderer when it comes to rendering tiles. Andrea is this is true?

I do believe there is still one regarding point symbolizers, but I'm
not sure wheter I already fixed it or not. Anyways, any time you
see something wrong with rendering and tiling, file an issue :slight_smile:

Besides that, I do believe that tile caching won't improve things so much unless you have a ton of bandwidth, since we're still producing
24bit pngs, whilst for many applications and layers 8bit png's would
do. Stated in other words, I do believe that for internet facing applications the bottleneck is still more on the bandwidth side than
on the CPU side (which is what tile caching speeds up). This is just
a gut feeling, but have a look at this:
http://mapserver.gis.umn.edu/docs/faq/output_image_formats
Using PNG8 instead of PNG24 allows to double or more the number of clients a given internet connection can serve. The catch is having
a good palette for each layer, and I already posted a proposal about
this on this mailing list.
Using 256 colors rendering would speed up a lot image encoding
too, making us scale a lot better on the CPU side of the fence too.

Cheers
Andrea

Andrea Aime wrote:

Using 256 colors rendering would speed up a lot image encoding
too, making us scale a lot better on the CPU side of the fence too.

I know this is rather naive, but... what prevents GeoServer to produce 8bit PNGs (I used this format for ArcIMS, and appreciated its small footprint) ?

Regards,

--------------------
    Luca Morandini
www.lucamorandini.it
--------------------

Andrea Aime wrote:

Using 256 colors rendering would speed up a lot image encoding
too, making us scale a lot better on the CPU side of the fence too.

I know this is rather naive, but... what prevents GeoServer to produce 8bit PNGs (I used this format for ArcIMS, and appreciated its small footprint) ?

Regards,

--------------------
    Luca Morandini
www.lucamorandini.it
--------------------

Luca Morandini ha scritto:

Andrea Aime wrote:

Using 256 colors rendering would speed up a lot image encoding
too, making us scale a lot better on the CPU side of the fence too.

I know this is rather naive, but... what prevents GeoServer to produce 8bit PNGs (I used this format for ArcIMS, and appreciated its small footprint) ?

To just generate PNG8 would be relatively trivial in fact... it's just
the result would be like gifs, that is, usually pretty ugly.
To get good looking results you need a custom palette. Otherwise, you
end up having a different palette in each tile, and the result won't
be pleasing.

Cheers
Andrea

Justin Deoliveira wrote:

You are right, at this time GeoServer itself has no cache built in. However users have succesfully use it with TileCache.

The people I work for manage a strictly Java shop, I'm afraid this is not a good option.

So... excuse the rant, the upshot is that you should see caching built into GeoServer in the next couple of months.

Wow ! Exactly when my project is supposed to start :slight_smile:

Regards,

--------------------
    Luca Morandini
www.lucamorandini.it
--------------------

Luca Morandini ha scritto:

Andrea Aime wrote:

Using 256 colors rendering would speed up a lot image encoding
too, making us scale a lot better on the CPU side of the fence too.

I know this is rather naive, but... what prevents GeoServer to produce 8bit PNGs (I used this format for ArcIMS, and appreciated its small footprint) ?

Well, I added a png8 output format to Geoserver trunk, and can be used by asking for the "image/png8" format. It's new functionality, so we won't back port it to 1.5.0, but it will be available along with 1.5.1.

Anyways, the output quality is not stellar. I have summarized some
of the work I have done today on this page:
http://docs.codehaus.org/display/GEOS/Paletted+images

Cheers
Andrea

Luca,

Another alternative is to use the Java OSCache library. It implements a
servlet filter which is fairly straightforward to configure. If you're a
Java shop you should be able figure out what you have to add to the
GeoServer web.xml (just add the oscache stuff as a servlet filter on /wms*)
in order to get it working...

Andre, my experience has been the opposite (regarding bandwidth being the
bottleneck). Maybe we're doing something wrong but we're finding that
implementing a basemap which extracts from multiple shape files is too slow
without caching. (This is with GeoServer v1.4.0... if v1.5 contains
tremendous performance improvements I'd like to hear about it.)

Can you elaborate on the rendering fixes (which I assume are implemented in
v1.5)? The biggest problem for me is the rendering of labels. Particularly
when rendering a label for a polygon (say State or Province labels). Once
you zoom in you end up the same label on multiple tiles on your display
which looks quite goofy. Do you know if this particular issue has been
fixed? Is the information at
http://docs.codehaus.org/display/GEOS/Rendering+and+tiling+issues+research
up-to-date?

Thanks,
Corey

aaime wrote:

Justin Deoliveira ha scritto:
[...]

Aside, I believe there are a still a few issues with the renderer when
it comes to rendering tiles. Andrea is this is true?

I do believe there is still one regarding point symbolizers, but I'm
not sure wheter I already fixed it or not. Anyways, any time you
see something wrong with rendering and tiling, file an issue :slight_smile:

Besides that, I do believe that tile caching won't improve things so
much unless you have a ton of bandwidth, since we're still producing
24bit pngs, whilst for many applications and layers 8bit png's would
do. Stated in other words, I do believe that for internet facing
applications the bottleneck is still more on the bandwidth side than
on the CPU side (which is what tile caching speeds up). This is just
a gut feeling, but have a look at this:
http://mapserver.gis.umn.edu/docs/faq/output_image_formats
Using PNG8 instead of PNG24 allows to double or more the number of
clients a given internet connection can serve. The catch is having
a good palette for each layer, and I already posted a proposal about
this on this mailing list.
Using 256 colors rendering would speed up a lot image encoding
too, making us scale a lot better on the CPU side of the fence too.

--
View this message in context: http://www.nabble.com/On-tiling%2C-rich-clients%2C-and-cache-tf3485934.html#a9827487
Sent from the GeoServer - User mailing list archive at Nabble.com.

Yeroc ha scritto:

Luca,

Another alternative is to use the Java OSCache library. It implements a
servlet filter which is fairly straightforward to configure. If you're a
Java shop you should be able figure out what you have to add to the
GeoServer web.xml (just add the oscache stuff as a servlet filter on /wms*)
in order to get it working...

Andre, my experience has been the opposite (regarding bandwidth being the
bottleneck). Maybe we're doing something wrong but we're finding that
implementing a basemap which extracts from multiple shape files is too slow
without caching. (This is with GeoServer v1.4.0... if v1.5 contains
tremendous performance improvements I'd like to hear about it.)

Not tremendous, just a little bit. Have a look at the Geoserver in production environemnt guide:
http://docs.codehaus.org/display/GEOSDOC/GeoServer+in+Production+Environment

One of the thing we state is to never use shapefiles for production, but
postgis, if you can. Oh, have you tried to use indexing?
Finally, just for shapefiles, there's a custom renderer we could use that's quite a lot faster (it's the one used by uDig) but not as well
tested as the one we're using now (and would speed up just shapefiles).

Can you elaborate on the rendering fixes (which I assume are implemented in
v1.5)? The biggest problem for me is the rendering of labels. Particularly
when rendering a label for a polygon (say State or Province labels). Once
you zoom in you end up the same label on multiple tiles on your display
which looks quite goofy. Do you know if this particular issue has been
fixed?

No, it hasn't. Once we have TMS (tiled map server) we'll be able to pre-render tiles on a cache, using meta-tiles for rendering which we then split up. A meta tile is a group of tiles, say 3x3, and this will
mitigate the effect you see since labels will repeat only in each meta-tile.
To avoid completely the effect we would have to choose a fixed labeling point per geometry instead of trying to adjust it to the current rendering area. Hmm.... in fact it's a rendering option we could add, provided you're not using group labeling at the same time.
The risk is, that if a label falls in between two tiles, it'll never be
rendered at all (rendering labels that fall on tile edges causes issues
with label conflict resolution and label grouping, as noted in the
research doc you're referring to).

The only real solution I can think of to have good consistent labeling is to preprocess the whole set of labelled layers in one go, and create a new data set that contains labels for the current resolution.
Then render that dataset without any extra magic, no conflict resolution
and the like. It'll be fast and good, the catch being that you'll need
one of such datasets per layer and per zoom level.

Is the information at
http://docs.codehaus.org/display/GEOS/Rendering+and+tiling+issues+research
up-to-date?

As far as I know, yes. Are you referring to something in particular?
Cheers
Andrea

Andrea,

Thanks for your response.

On 4/4/07, Andrea Aime <aaime@anonymised.com> wrote:

[…]
Not tremendous, just a little bit. Have a look at the Geoserver in
production environemnt guide:
http://docs.codehaus.org/display/GEOSDOC/GeoServer+in+Production+Environment

One of the thing we state is to never use shapefiles for production, but
postgis, if you can. Oh, have you tried to use indexing?
Finally, just for shapefiles, there’s a custom renderer we could use
that’s quite a lot faster (it’s the one used by uDig) but not as well
tested as the one we’re using now (and would speed up just shapefiles).

Thanks for the pointer to that page. I am using indexing (on the shapefiles) or at least I enabled that option in GeoServer and it did create files with .fix and .qix extensions that look suspiciously like index files. Is there a significant performance gap between postgis & indexed shapefile performance? (Note that I’m not exposing these shapefiles via wfs or performing any editing.)

Can you elaborate on the rendering fixes (which I assume are implemented in
v1.5)? The biggest problem for me is the rendering of labels. Particularly
when rendering a label for a polygon (say State or Province labels). Once
you zoom in you end up the same label on multiple tiles on your display
which looks quite goofy. Do you know if this particular issue has been
fixed?

No, it hasn’t. Once we have TMS (tiled map server) we’ll be able to
pre-render tiles on a cache, using meta-tiles for rendering which we
then split up. A meta tile is a group of tiles, say 3x3, and this will
mitigate the effect you see since labels will repeat only in each meta-tile.
To avoid completely the effect we would have to choose a fixed labeling
point per geometry instead of trying to adjust it to the current
rendering area. Hmm… in fact it’s a rendering option we could add,
provided you’re not using group labeling at the same time.
The risk is, that if a label falls in between two tiles, it’ll never be
rendered at all (rendering labels that fall on tile edges causes issues
with label conflict resolution and label grouping, as noted in the
research doc you’re referring to).

Meta-tiles still leave you with sub-optimal imagery. I looked at the rendering code briefly and I think the solution (or part of the solution) is as you just mentioned to pass in a rendering hint via the SLD or some other means so that the renderer can operate differently when it knows tiling is being done. In this case choosing a fixed label per geometry rather than per image bounds makes all the difference in the world. I know this doesn’t solve all the problems but it would be great to have this rendering option even if it had a few caveats like not working with grouping and conflict resolution. Right now, I’m unable to render an adequate tiled (using OpenLayers) basemap with labels using GeoServer. :frowning: I did spend some time trying to configure TileCache with its meta-tiling option but wasn’t successful in the end and I would prefer not to have the added complexity of another layer in front of GeoServer.

[…]

Is the information at
http://docs.codehaus.org/display/GEOS/Rendering+and+tiling+issues+research
up-to-date?

As far as I know, yes. Are you referring to something in particular?

Nothing in particular. Just wanted to know if further analysis had been done and if some of the issues mentioned there had already been resolved.

Thanks,
Corey

Corey Puffalt ha scritto:

Andrea,

Thanks for your response.

On 4/4/07, *Andrea Aime* <aaime@anonymised.com <mailto:aaime@anonymised.com>> wrote:

    [...]
    Not tremendous, just a little bit. Have a look at the Geoserver in
    production environemnt guide:
    http://docs.codehaus.org/display/GEOSDOC/GeoServer+in+Production+Environment
    <http://docs.codehaus.org/display/GEOSDOC/GeoServer+in+Production+Environment&gt;

    One of the thing we state is to never use shapefiles for production, but
    postgis, if you can. Oh, have you tried to use indexing?
    Finally, just for shapefiles, there's a custom renderer we could use
    that's quite a lot faster (it's the one used by uDig) but not as well
    tested as the one we're using now (and would speed up just shapefiles).

Thanks for the pointer to that page. I am using indexing (on the shapefiles) or at least I enabled that option in GeoServer and it did create files with .fix and .qix extensions that look suspiciously like index files. Is there a significant performance gap between postgis & indexed shapefile performance? (Note that I'm not exposing these shapefiles via wfs or performing any editing.)

Ok, I've done some little testing prompted by another user that had
performance issues with shapefiles.
My limited testing shows that when rendering the full data set shapefiles have a slight edge, but when zoomed in, postgis beats them
hands down (5 times faster at least).

Now, it may be that with that shapefile with just hit a bug, but
I'd suggest you to install postgis on the same box as your Geoserver
and do some quick tests. It may be well worth the time.
If you do so, I'm curious to hear what your results are.

Cheers
Andrea