[Geoserver-devel] Multithreading Catalog Builder?

Hello,

I’m trying to publish a large amount of layers using the CatalogBuilder instead of REST but this effort takes too long.

I’ve been publishing layers using the CatalogBuilder as such:

CatalogBuilder builder = new CatalogBuilder(catalog);

CoverageStoreInfo info = catalog.getCoverageStoreByName(storeName);

builder.setStore(info);

CoverageInfo ci = builder.buildCoverage(coverageName);

LayerInfo layer = builder.buildLayer(ci);

layer.setResource(ci);

catalog.add(ci);

catalog.add(layer);

I’ve read that the CatalogBuilder is not thread safe, is there a better way to programmatically publish a large amount of layers in geoserver without the REST API?

Best,

Sylvia

On Fri, Jan 9, 2015 at 4:26 PM, Sylvia Charchut <
Sylvia.Charchut@anonymised.com> wrote:

Hello,

I’m trying to publish a large amount of layers using the CatalogBuilder
instead of REST but this effort takes too long.

I’ve been publishing layers using the CatalogBuilder as such:

CatalogBuilder builder = new CatalogBuilder(catalog);

CoverageStoreInfo info = catalog.getCoverageStoreByName(storeName);

builder.setStore(info);

CoverageInfo ci = builder.buildCoverage(coverageName);

LayerInfo layer = builder.buildLayer(ci);

layer.setResource(ci);

catalog.add(ci);

catalog.add(layer);

I’ve read that the CatalogBuilder is not thread safe, is there a better
way to programmatically publish a large amount of layers in geoserver
without the REST API?

The problem is not much that the CatalogBuilder is not thread safe, it's
that the Catalog itself is not thread safe,
at least the default one.
The JDBC based one, contained in a community module, might be, but I'm not
100% sure about it

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------

I found a lot of the issues take place in the Catalog “ResourcePool”. The GeoServer REST API has a few hacks to try and make interacting with Catalolog+ResourcePool thread safe - but it is really the wrong spot.

Jody

···

On 9 January 2015 at 07:37, Andrea Aime <andrea.aime@anonymised.com> wrote:


Dive into the World of Parallel Programming! The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net


Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel


Jody Garnett

On Fri, Jan 9, 2015 at 4:26 PM, Sylvia Charchut <Sylvia.Charchut@anonymised.com> wrote:

Hello,

I’m trying to publish a large amount of layers using the CatalogBuilder instead of REST but this effort takes too long.

I’ve been publishing layers using the CatalogBuilder as such:

CatalogBuilder builder = new CatalogBuilder(catalog);

CoverageStoreInfo info = catalog.getCoverageStoreByName(storeName);

builder.setStore(info);

CoverageInfo ci = builder.buildCoverage(coverageName);

LayerInfo layer = builder.buildLayer(ci);

layer.setResource(ci);

catalog.add(ci);

catalog.add(layer);

I’ve read that the CatalogBuilder is not thread safe, is there a better way to programmatically publish a large amount of layers in geoserver without the REST API?

The problem is not much that the CatalogBuilder is not thread safe, it’s that the Catalog itself is not thread safe,
at least the default one.
The JDBC based one, contained in a community module, might be, but I’m not 100% sure about it

Cheers
Andrea

==

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

==

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.