[Geoserver-users] How to deal with large numbers of WMS layers

GeoServer GeoServer User
1

Hello list,

are there any best practices on offering a large number of WMS layers with geoserver? We are operating a geoserver instance that has about 600 time-enabled vector layers (loaded from a postgres database) which causes two problems:

1) The capabilities take a very long time (>5 Minutes) to load
2) The capabilities are becoming quite big (Currently about one 1MB) so we might run into issues with clients

So are there any solutions for these problems? Can we speed up the Capabilities response? Can we maybe assign layers to different WMS endpoints?

Thanks,
Matthias

2

On Wed, Feb 20, 2013 at 3:46 PM, Matthias Müller <Matthias_Mueller@anonymised.com> wrote:

Hello list,

are there any best practices on offering a large number of WMS layers with geoserver? We are operating a geoserver instance that has about 600 time-enabled vector layers (loaded from a postgres database) which causes two problems:

  1. The capabilities take a very long time (>5 Minutes) to load
  2. The capabilities are becoming quite big (Currently about one 1MB) so we might run into issues with clients

So are there any solutions for these problems? Can we speed up the Capabilities response? Can we maybe assign layers to different WMS endpoints?

You should probably partition the stores/layers in different workspaces, and use the worskpace specific
OGC services.

The other thing that slows things down is that for each time enabled layer a query is done against the
database to retrieve fresh time values… we don’t have a way to register them statically, but
I’m thinking that one potential improvement to GeoServer (which requires coding) could
be to submit these lists to some caching, so that they are not requested every time (pretty much
like we cache the structure of feature types today)… the significant downside of this approach
is that the list of times can be very large, caching it would eat memory easily…

Cheers
Andrea

–

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

3

Thanks for the hints, Andrea.

If we provide fixed values for time and spatial extent when creating the layer - would that avoid the execssive data base queries?

Am 20.02.2013 16:26, schrieb Andrea Aime:

On Wed, Feb 20, 2013 at 3:46 PM, Matthias Müller
<Matthias_Mueller@anonymised.com <mailto:Matthias_Mueller@anonymised.com>>
wrote:

    Hello list,

    are there any best practices on offering a large number of WMS
    layers with geoserver? We are operating a geoserver instance that
    has about 600 time-enabled vector layers (loaded from a postgres
    database) which causes two problems:

    1) The capabilities take a very long time (>5 Minutes) to load
    2) The capabilities are becoming quite big (Currently about one 1MB)
    so we might run into issues with clients

    So are there any solutions for these problems? Can we speed up the
    Capabilities response? Can we maybe assign layers to different WMS
    endpoints?

You should probably partition the stores/layers in different workspaces,
and use the worskpace specific
OGC services.

The other thing that slows things down is that for each time enabled
layer a query is done against the
database to retrieve fresh time values... we don't have a way to
register them statically, but
I'm thinking that one potential improvement to GeoServer (which requires
coding) could
be to submit these lists to some caching, so that they are not requested
every time (pretty much
like we cache the structure of feature types today).... the significant
downside of this approach
is that the list of times can be very large, caching it would eat memory
easily...

Cheers
Andrea

--

Our support, Your Success! Visit http://opensdi.geo-solutions.it for
more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb

_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
Matthias Müller
Dipl.-Geogr. | Research Associate

Technische Universität Dresden
Geoinformation Systems
01062 Dresden

Phone: +49 351 463-31953
Fax: +49 351 463-35879
Mail: Matthias_Mueller@anonymised.com

www: http://tu-dresden.de/fgh/geo/gis

4

On Thu, Feb 21, 2013 at 1:19 PM, Matthias Müller <Matthias_Mueller@anonymised.com> wrote:

Thanks for the hints, Andrea.

If we provide fixed values for time and spatial extent when creating the layer - would that avoid the execssive data base queries?

The spatial extent used is the one already set in the configuration page.
The fixed time list is an idea, it has to be implemented in code by someone, but yes, getting
the domain (list, or min/max) of the times is nowadays the primary reason why a
getCapabilities issues queries on a database.

The rationale is simple, while the spatial extent of most data is roughly static, the
list of times normally changes multiple times a day.
As such, the fixed list of times in the config would have to be developed as something
optional, and whatever software updates the datasets adding/removing times would
also have to update the configured ones.

Cheers
Andrea

–

==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

5

So we'll try to implement some logic calulates the WMS layer time based on the preconfigured interval / resolution. I haven't yet hacked in the geoserver project. Can you point me to a particular module, package or class that requires tweaking?

Thanks,
Matthias

Am 21.02.2013 14:12, schrieb Andrea Aime:

On Thu, Feb 21, 2013 at 1:19 PM, Matthias Müller
<Matthias_Mueller@anonymised.com <mailto:Matthias_Mueller@anonymised.com>>
wrote:

    Thanks for the hints, Andrea.

    If we provide fixed values for time and spatial extent when creating
    the layer - would that avoid the execssive data base queries?

The spatial extent used is the one already set in the configuration page.
The fixed time list is an idea, it has to be implemented in code by
someone, but yes, getting
the domain (list, or min/max) of the times is nowadays the primary
reason why a
getCapabilities issues queries on a database.

The rationale is simple, while the spatial extent of most data is
roughly static, the
list of times normally changes multiple times a day.
As such, the fixed list of times in the config would have to be
developed as something
optional, and whatever software updates the datasets adding/removing
times would
also have to update the configured ones.

Cheers
Andrea
--

Our support, Your Success! Visit http://opensdi.geo-solutions.it for
more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb

_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users