[Geoserver-devel] Direct GWC/WMS integration

GeoServer GeoServer Developer
1

Hi all,

as per http://jira.codehaus.org/browse/GEOS-4163, there's now the
possibility of directly integrating GWC with the regular WMS, providing
WMS-C capabilities directly out of the GeoServer WMS:

      * There's the possibility to enable direct GWC WMS integration
        through the UI (disabled by default)
      * The gwc module intercepts regular WMS calls, and if direct WMS
        integration is enabled, it will check whether the request comes
        with TILED=true AND matches a cached tile, and if to, will serve
        the cached copy. Otherwise it defers to the regular WMS call.
      * if the getcapabilities request comes with TILED=true, the gwc
        module contributes the WMS-C VendorSpecificCapabilities, adding
        an internal DTD definition for TileSet/Resolutions/etc and one
        TileSet per cached layer/CRS/output format combination.
      * the GetCapabilities namespace vendor specific parameter is
        respected, so that if the request comes with namespace=topp,
        only TileSets for layers in the topp namespace will be added.

Note, however, that the default geoserver layer preview demos won't hit
the cache because they calculate the tile origin based on the layer's
bounding box, which generally doesn't match the GWC's gridset tile
origin. But the WMS caps providing the WMS-C extended capabilities
(specifically the TileSet definitions) do make it possible to create OL
apps that match the layer's gridsets.

It would be nice to have the layer preview match the gwc gridsets by
default, but so far it's not such an easy task as the openlayers map
output format is decoupled from the gwc module.
Also, OL lacks WMS-C parsing support so far, so it usually reprojects
the layer's bbox in its native CRS to the map's CRS, leading to
mismatched tile origins. But that's hopefuly gonna be fixed shortly by
the OL experts.

Cheers,
Gabriel

--
Gabriel Roldan
groldan@anonymised.com
Expert service straight from the developers

2

On Wed, Jan 5, 2011 at 6:30 PM, Gabriel Roldán <groldan@anonymised.com> wrote:

Hi all,

as per http://jira.codehaus.org/browse/GEOS-4163, there's now the
possibility of directly integrating GWC with the regular WMS, providing
WMS-C capabilities directly out of the GeoServer WMS:

 \* There&#39;s the possibility to enable direct GWC WMS integration
   through the UI \(disabled by default\)

Nice

 \* The gwc module intercepts regular WMS calls, and if direct WMS
   integration is enabled, it will check whether the request comes
   with TILED=true AND matches a cached tile, and if to, will serve
   the cached copy\. Otherwise it defers to the regular WMS call\.

I see this has been done using Spring AOP, good.
I hope this means it's still possible to remove all GWC related jars
from GeoServer
and have it working fine.
This is imperative for the WMS performance shootout, always removed GWC entirely
to prove that no caching was going on (or was possible) in the tests.

Also, there is a concern that GWC embedded in GeoServer can turn into a disk
killing bomb: the tile caches can become rather large, as an administrator I'd
like to have GWC around only if I really need it, and to configure it tightly
to avoid misuse.
The default GWC configuration instead does not prevent that, which is one reason
why I normally get rid of it unless I'm actually going to use it and
configure it
in detail (btw, is the quota functionality ever going to be included
in GeoServer
GWC? That's something every conscious admin should configure, along with
the other service limits).

 \* if the getcapabilities request comes with TILED=true, the gwc
   module contributes the WMS\-C VendorSpecificCapabilities, adding
   an internal DTD definition for TileSet/Resolutions/etc and one
   TileSet per cached layer/CRS/output format combination\.
 \* the GetCapabilities namespace vendor specific parameter is
   respected, so that if the request comes with namespace=topp,
   only TileSets for layers in the topp namespace will be added\.

Nice and nice.

Note, however, that the default geoserver layer preview demos won't hit
the cache because they calculate the tile origin based on the layer's
bounding box, which generally doesn't match the GWC's gridset tile
origin. But the WMS caps providing the WMS-C extended capabilities
(specifically the TileSet definitions) do make it possible to create OL
apps that match the layer's gridsets.

It would be nice to have the layer preview match the gwc gridsets by
default, but so far it's not such an easy task as the openlayers map
output format is decoupled from the gwc module.

I guess it would be nice, but it would be better to do that only if GWC
is around, and without introducing a dependency. Some sort of
extension point?
Wondering if the extension point that is used to declare the tilesets
extensions in the capabilities could be also used to drive the OL
preview configuration.

Cheers
Andrea

-----------------------------------------------------
Ing. Andrea Aime
Senior Software Engineer

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584962313
fax: +39 0584962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

-----------------------------------------------------

3

On Wed, 2011-01-05 at 18:54 +0100, Andrea Aime wrote:

On Wed, Jan 5, 2011 at 6:30 PM, Gabriel Roldán <groldan@anonymised.com> wrote:
> Hi all,
>
> as per http://jira.codehaus.org/browse/GEOS-4163, there's now the
> possibility of directly integrating GWC with the regular WMS, providing
> WMS-C capabilities directly out of the GeoServer WMS:
>
> * There's the possibility to enable direct GWC WMS integration
> through the UI (disabled by default)

Nice

> * The gwc module intercepts regular WMS calls, and if direct WMS
> integration is enabled, it will check whether the request comes
> with TILED=true AND matches a cached tile, and if to, will serve
> the cached copy. Otherwise it defers to the regular WMS call.

I see this has been done using Spring AOP, good.
I hope this means it's still possible to remove all GWC related jars
from GeoServer
and have it working fine.

right.

This is imperative for the WMS performance shootout, always removed GWC entirely
to prove that no caching was going on (or was possible) in the tests.

yes, it was a design decision to keep it decoupled.

Also, there is a concern that GWC embedded in GeoServer can turn into a disk
killing bomb: the tile caches can become rather large, as an administrator I'd
like to have GWC around only if I really need it, and to configure it tightly
to avoid misuse.
The default GWC configuration instead does not prevent that, which is one reason
why I normally get rid of it unless I'm actually going to use it and
configure it
in detail (btw, is the quota functionality ever going to be included
in GeoServer
GWC? That's something every conscious admin should configure, along with
the other service limits).

Agreed. The GWC integration story so far has been kind of weak IMHO. I'm
working towards improving it. Started with truncating the cache based on
transaction's affected bounds, etc. Now this direct WMS integration,
which's disabled by default. And next diskquota may arrive late this
week or the next top, with ability to set a global cache quota limit and
an expiration policy so that tiles least used get removed to keep up
with the quota.
A per-layer quota limit is also possible, but configuring that from the
GeoServer UI is part of a wider plan, so it'll take longer to arrive.

> * if the getcapabilities request comes with TILED=true, the gwc
> module contributes the WMS-C VendorSpecificCapabilities, adding
> an internal DTD definition for TileSet/Resolutions/etc and one
> TileSet per cached layer/CRS/output format combination.
> * the GetCapabilities namespace vendor specific parameter is
> respected, so that if the request comes with namespace=topp,
> only TileSets for layers in the topp namespace will be added.

Nice and nice.

> Note, however, that the default geoserver layer preview demos won't hit
> the cache because they calculate the tile origin based on the layer's
> bounding box, which generally doesn't match the GWC's gridset tile
> origin. But the WMS caps providing the WMS-C extended capabilities
> (specifically the TileSet definitions) do make it possible to create OL
> apps that match the layer's gridsets.
>
> It would be nice to have the layer preview match the gwc gridsets by
> default, but so far it's not such an easy task as the openlayers map
> output format is decoupled from the gwc module.

I guess it would be nice, but it would be better to do that only if GWC
is around, and without introducing a dependency. Some sort of
extension point?

right. Reason why I didn't make it so far, just ran out of time to keep
things so nice and didn't want to hack in.

Wondering if the extension point that is used to declare the tilesets
extensions in the capabilities could be also used to drive the OL
preview configuration.

Guess it has to be a separate one. That one is intended for capabilities
only and used by the inspire and gwc so far.

Thanks for the feedback Andrea.
Gabriel.

Cheers
Andrea

-----------------------------------------------------
Ing. Andrea Aime
Senior Software Engineer

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584962313
fax: +39 0584962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

-----------------------------------------------------

--
Gabriel Roldan
groldan@anonymised.com
Expert service straight from the developers

4

On Wed, Jan 5, 2011 at 7:46 PM, Gabriel Roldán <groldan@anonymised.com> wrote:

This is imperative for the WMS performance shootout, always removed GWC entirely
to prove that no caching was going on (or was possible) in the tests.

yes, it was a design decision to keep it decoupled.

Yup, good good.

Also, there is a concern that GWC embedded in GeoServer can turn into a disk
killing bomb: the tile caches can become rather large, as an administrator I'd
like to have GWC around only if I really need it, and to configure it tightly
to avoid misuse.
The default GWC configuration instead does not prevent that, which is one reason
why I normally get rid of it unless I'm actually going to use it and
configure it
in detail (btw, is the quota functionality ever going to be included
in GeoServer
GWC? That's something every conscious admin should configure, along with
the other service limits).

Agreed. The GWC integration story so far has been kind of weak IMHO. I'm
working towards improving it. Started with truncating the cache based on
transaction's affected bounds, etc. Now this direct WMS integration,
which's disabled by default. And next diskquota may arrive late this
week or the next top, with ability to set a global cache quota limit and
an expiration policy so that tiles least used get removed to keep up
with the quota.

Aaah, very good.

Wondering, would it be possible (and easy/quick) to have a flag disabling
GWC entirely just like we can disable the OGC services?
Getting rid of the jars is a more permanent solution, but one that
might get into trouble people not familiar with java webapps
(not to mention there is no official list of jar to remove)

A per-layer quota limit is also possible, but configuring that from the
GeoServer UI is part of a wider plan, so it'll take longer to arrive.

Good to know it's in the plans anyways.

I guess it would be nice, but it would be better to do that only if GWC
is around, and without introducing a dependency. Some sort of
extension point?

right. Reason why I didn't make it so far, just ran out of time to keep
things so nice and didn't want to hack in.

Wondering if the extension point that is used to declare the tilesets
extensions in the capabilities could be also used to drive the OL
preview configuration.

Guess it has to be a separate one. That one is intended for capabilities
only and used by the inspire and gwc so far.

Yeah, I understand

Thanks for the feedback Andrea.

Thanks for the improvements :wink:

Cheers
Andrea

-----------------------------------------------------
Ing. Andrea Aime
Senior Software Engineer

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584962313
fax: +39 0584962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

-----------------------------------------------------

5

On Wed, 2011-01-05 at 21:38 +0100, Andrea Aime wrote:

On Wed, Jan 5, 2011 at 7:46 PM, Gabriel Roldán <groldan@anonymised.com> wrote:
>> This is imperative for the WMS performance shootout, always removed GWC entirely
>> to prove that no caching was going on (or was possible) in the tests.
>
> yes, it was a design decision to keep it decoupled.

Yup, good good.

>> Also, there is a concern that GWC embedded in GeoServer can turn into a disk
>> killing bomb: the tile caches can become rather large, as an administrator I'd
>> like to have GWC around only if I really need it, and to configure it tightly
>> to avoid misuse.
>> The default GWC configuration instead does not prevent that, which is one reason
>> why I normally get rid of it unless I'm actually going to use it and
>> configure it
>> in detail (btw, is the quota functionality ever going to be included
>> in GeoServer
>> GWC? That's something every conscious admin should configure, along with
>> the other service limits).
> Agreed. The GWC integration story so far has been kind of weak IMHO. I'm
> working towards improving it. Started with truncating the cache based on
> transaction's affected bounds, etc. Now this direct WMS integration,
> which's disabled by default. And next diskquota may arrive late this
> week or the next top, with ability to set a global cache quota limit and
> an expiration policy so that tiles least used get removed to keep up
> with the quota.

Aaah, very good.

Wondering, would it be possible (and easy/quick) to have a flag disabling
GWC entirely just like we can disable the OGC services?
Getting rid of the jars is a more permanent solution, but one that
might get into trouble people not familiar with java webapps
(not to mention there is no official list of jar to remove)

I found myself in the need/wish to do that, and more, like
enabling/disabling TMS, WMTS, etc, but every time hit my head against
the wall due to lack of time to improve GWC itself enough as to being
able to do that (properly). But yeah, it's also on my radar of things to
improve, like in GWC is not a service by itself but provides a bunch of
services, and I'd like a config page for each just like for
WMS/WFS/etc., where you can enable/disable them and set other
properties. By the other side there's the general GWC settings page for
things like wms integration, global quota, etc.

Cheers,
Gabriel.

> A per-layer quota limit is also possible, but configuring that from the
> GeoServer UI is part of a wider plan, so it'll take longer to arrive.

Good to know it's in the plans anyways.

>> I guess it would be nice, but it would be better to do that only if GWC
>> is around, and without introducing a dependency. Some sort of
>> extension point?
> right. Reason why I didn't make it so far, just ran out of time to keep
> things so nice and didn't want to hack in.
>
>> Wondering if the extension point that is used to declare the tilesets
>> extensions in the capabilities could be also used to drive the OL
>> preview configuration.
> Guess it has to be a separate one. That one is intended for capabilities
> only and used by the inspire and gwc so far.

Yeah, I understand

> Thanks for the feedback Andrea.

Thanks for the improvements :wink:

Cheers
Andrea

-----------------------------------------------------
Ing. Andrea Aime
Senior Software Engineer

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584962313
fax: +39 0584962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

-----------------------------------------------------

--
Gabriel Roldan
groldan@anonymised.com
Expert service straight from the developers