There’s an optimized data-directory-specific catalog and config loader plugin developed under geoserver-cloud’s code base that we’d like to contribute upstream as a community module.
Its goal is to improve the startup time of GeoServer when configured with a data directory that has thousands of layers, stores, etc., and config objects such as workspace services and settings.
On my laptop’s SSD, running off a cold disk cache, for a data-dir with 100K layers over 100 workspaces, the performance improvement is ~8.5 seconds instead of ~44.5 seconds, with 2.23.1 as a baseline.
But the most important difference is when you need to load the data dir from a shared drive, especially NFS, where it regularly takes over 40 minutes, and with this plugin around 4 minutes.
Hence I’m asking the PSC for permission to push this plugin under community/datadir-catalog-loader
Here’s a PR for your reference.
Cheers,
···
camptocamp
INNOVATIVE SOLUTIONS
BY OPEN SOURCE EXPERTS
Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.
This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail
Thanks Andrea.
Sorry guys if eventually an exactly equal message arrives from my @anonymised.com account, I initially sent it from there but it wasn’t coming through for some reason.
···
camptocamp
INNOVATIVE SOLUTIONS
BY OPEN SOURCE EXPERTS
Gabriel Roldán
Geospatial Developer
Gabriel Roldán
Regards,
Andrea Aime
==
GeoServer Professional Services from the experts!
Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.
This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail
Lähettäjä: Gabriel Roldan <gabriel.roldan@…403…> Lähetetty: lauantai 1. heinäkuuta 2023 17.25 Vastaanottaja: Geoserver-devel geoserver-devel@lists.sourceforge.net Aihe: [Geoserver-devel] New community module “datadir-catalog-loader”
Hi, list,
There’s an optimized data-directory-specific catalog and config loader plugin developed under geoserver-cloud’s code base that we’d like to contribute upstream as a community module.
Its goal is to improve the startup time of GeoServer when configured with a data directory that has thousands of layers, stores, etc., and config objects such as workspace services and settings.
On my laptop’s SSD, running off a cold disk cache, for a data-dir with 100K layers over 100 workspaces, the performance improvement is ~8.5 seconds instead of ~44.5 seconds, with 2.23.1 as a baseline.
But the most important difference is when you need to load the data dir from a shared drive, especially NFS, where it regularly takes over 40 minutes, and with this plugin around 4 minutes.
Hence I’m asking the PSC for permission to push this plugin under community/datadir-catalog-loader
I see that it is not documented in Community modules — GeoServer 2.27.x User Manual. Is there some documentation that you can provide (in absolutely any format), and I will see about adding it to increase the visibility of your contribution.