Hi,
like I’ve said in previous threads, we have with GWC and large catalogs, in case
all these layers are cached (which is true by default, even if having 100k layers
and have the also be cached might not be realistic).
The issue here is that there are places during the startup that enumerate all
GWC tile layers, and instantiating GeoServerTileLayer requires fetching the
associated GeoServer layer.
One of such examples is the GeoRssPoller, which on startup does something like:
private void findEnabledPolls() {
final Iterable layers = seeder.getLayers();
and another example is the disk quota subsystem, UsageStatsMonitor:
Iterable allLayers = tileLayerDispatcher.getLayerList();
for (TileLayer layer : allLayers) {
layer.addLayerListener(usageStatsProducer);
}
Both end up listing all layers in the CatalogConfiguration in GeoServer, which has a
loader for layers doing the following:
@Override
public GeoServerTileLayer load(String layerId) throws Exception {
GeoServerTileLayer tileLayer = null;
final GridSetBroker gridSetBroker = CatalogConfiguration.this.gridSetBroker;
lock.readLock().lock();
try {
if (pendingDeletes.contains(layerId)) {
throw new IllegalArgumentException(“Tile layer '” + layerId + “’ was deleted.”);
}
GeoServerTileLayerInfo tileLayerInfo = pendingModications.get(layerId);
if (tileLayerInfo == null) {
tileLayerInfo = tileLayerCatalog.getLayerById(layerId);
}
if (tileLayerInfo == null) {
throw new IllegalArgumentException(“GeoServerTileLayerInfo '” + layerId
- “’ does not exist.”);
}
LayerInfo layerInfo = geoServerCatalog.getLayer(layerId);
if (layerInfo != null) {
tileLayer = new GeoServerTileLayer(layerInfo, gridSetBroker, tileLayerInfo);
} else {
LayerGroupInfo lgi = geoServerCatalog.getLayerGroup(layerId);
if (lgi != null) {
tileLayer = new GeoServerTileLayer(lgi, gridSetBroker, tileLayerInfo);
}
}
} finally {
lock.readLock().unlock();
}
if (null == tileLayer) {
throw new IllegalArgumentException(“GeoServer layer or layer group '” + layerId
- “’ does not exist”);
}
return tileLayer;
}
Now, long story short, loading a GeoServerTileLayer for any reason equates to
one query to JDBCConfig.
I was thinking, well, what if we delay that? Not everythig GeoServerTileLayer does
requires the associated layer/layer group no?
We could just give it the id, and have it lazy load the info when actually needed.
Unfortunately at the very end of the constructor we have:
TileLayerInfoUtil.checkAutomaticStyles(layerInfo, state);
which actually needs the list of styles to update the StyleParameterFilter, which
is used to automatically synch the allowed values for STYLES with
the current values of the layer own styles.
I’ll see if I can invert this relationship, and have everything loaded on demand.
I’m betting that these two large scans at startup actually do not need any
information out of LayerInfo/LayerGroupInfo (well, it does not look like they
do, as far as I can see).
There is however some source of concern… as far as I can see, the
CatalogConfiguration is trying to build GeoServerTileLayer on demand,
and has a cache of those.
Other parts of GWC are instead assuming the layers are going to be static/long lived.
The UsageStatsMonitor is one such case, I don’t see it being re-created over
time, and as far as I can see it only gets the lists of layers to be updated in
its startUp() method, attaching a listener to them… I haven’t checked yet,
but it would seem new layers are not actually going to be taken into account
until a restart of either GWC or GeoServer happens…
Cheers
Andrea
···
==
GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.
==
Ing. Andrea Aime
@geowolf
Technical Lead
GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549
http://www.geo-solutions.it
http://twitter.com/geosolutions_it
AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.
The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.