I’m afraid this would solve the packaging issue but not the problem of deploying both the COG and the Azure blobstore extensions, as you’d end up with duplicate netty dependencies with incompatible versions.
In GeoServer Cloud I had to deal with this specific issue, using the “dependencyConvergence” maven enforcer plugin rule 1, and ensuring a single version of each duplicate dependency is in place. With the additional overhead of ensuring the plugins whose dependencies are overridden still work.
For the specific case of the netty version, I’m pegging it to 4.1.41.Final. Both COG and Azure blobstore work with that one.
The libs are in the pom’s “dependencyConvergence” maven profile [3].
So probably we could do something similar in geoserver to ensure a single version of each dependency is ever present.
Though that’d probably open its own can of worms. Perhaps just making sure both COG and Azure blobstore carry over netty 4.1.41.Final would suffice for the time being, without a rewrite of the Azure blobstore?
Sounds good, dependency sets do provide include/exclude filtering1 (or if the pom already has things correctly scoped as provided vs compile then you can easily filter out provided), but I’m all for easy wins!
Thanks,
Emilio
Emilio Lahr-Vivaz
General Atomics, CCRi
From: Andrea Aime <andrea.aime@anonymised.com>
Sent: Wednesday, May 29, 2024 11:15 AM
To: Lahr-Vivaz, Emilio <Emilio.Lahr-Vivaz@anonymised.com>
Cc: Jody Garnett <jody.garnett@anonymised.com>; GeoServer <geoserver-devel@lists.sourceforge.net>
Subject: Re: [Geoserver-devel] Thinking about community modules packaging
Hi Emilio,
I’m aware of it, but there are issues… one one side, we want to package in the zip file only the jars that are not already covered by core.
The other bit is, even if there was a way to achieve the above, all 74 community module packaging files would have to be individually rewritten.
What I’m proposing is simple, mechanical, and can be at least partially automated, which means, it fits a small budget.
If we can party up with several people and attempt a technically better approach, so that my isolated effort is more or less the same,
I’m all for it!
Regards,
Andrea Aime
==
GeoServer Professional Services from the experts!
Ing. Andrea Aime
@geowolf
Technical Lead
GeoSolutions Group
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549
https://www.geosolutionsgroup.com/
http://twitter.com/geosolutions_it
Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.
This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail
On Wed, May 29, 2024 at 5:06 PM Lahr-Vivaz, Emilio <Emilio.Lahr-Vivaz@anonymised.com> wrote:
If it’s helpful, the maven assembly plugin has a concept of ‘dependencySets’, which could replace some of the fileSets and allow you to avoid running dependencies:dependency-copy.
Thanks,
Emilio
From: Andrea Aime <andrea.aime@anonymised.com>
Sent: Wednesday, May 29, 2024 10:40 AM
To: Jody Garnett <jody.garnett@anonymised.com>
Cc: GeoServer <geoserver-devel@lists.sourceforge.net>
Subject: Re: [Geoserver-devel] Thinking about community modules packaging
Hi Jody.
yes each module would have the assembly configuration local, besides its pom.xml.
I’ve made a small experiment, on the COG module alone, this should work:
cog-plugin
zip
false
target/dependency
annotations-2.*.jar
auth-2.*.jar
aws-*-2.*.jar
azure-storage*.jar
checksums-*2*.jar
client-runtime*.jar
crt-core-*-2*.jar
endpoints-spi-2*.jar
gs-cog*.jar
http-auth-*2*.jar
http-client-spi-2*.jar
http*-osgi*.jar
imageio-ext-*rangereader*.jar
identity-spi*.jar
jackson-datatype*.jar
jackson-module*.jar
kotlin-stdlib*.jar
metrics-spi-2.*.jar
*netty*.jar
okhttp*.jar
okio*.jar
profiles-2.*.jar
protocol-core-2.*.jar
reactive-streams*.jar
regions-2.*.jar
rxjava*.jar
s3-2.*.jar
sdk-core-2.*.jar
utils-2.*.jar
target
gs-cog*.jar
To work, it still requires dependencies:dependency-copy to run, which we can either have running
subject to a flag, or move it as part of the explicit build instructions.
Regards,
Andrea Aime
==
GeoServer Professional Services from the experts!
Ing. Andrea Aime
@geowolf
Technical Lead
GeoSolutions Group
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549
https://www.geosolutionsgroup.com/
http://twitter.com/geosolutions_it
Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.
This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail
On Wed, May 29, 2024 at 4:35 PM Jody Garnett <jody.garnett@anonymised.com> wrote:
I would like to see each module contains its own assembly, it would make things more normal.
Initially we made the release module to force assembly to occur at the end, and to avoid taking the time to build download artifacts during day to day development.
We could keep the release profile and only build zip download bundles when it is used.
Jody Garnett
On Wed, May 29, 2024 at 1:15 AM Andrea Aime <andrea.aime@anonymised.com> wrote:
Hi all,
I’ve got reports that the COG community module packaged zip bundles are not working any
longer. Looking into it, it seems there are conflicting versions of dependencies that are causing the trouble.
As you probably know, the packaging is currently done this way:
- A “release” module depends on all the modules that want to be packages
- The release module dumps all these module dependencies into a single directory
- Each module packaging descriptor picks from that pile of jars using globs
Now, we already had some issues with incompatible dependency versions, but we more or less managed. Now there is a group of 3 different community modules that depend, in turn, to AWS and Azure cloud libraries, which in turn depend on Netty. Unfortunately, two different and incompatible versions of it.
Ideally, we could just upgrade the Azure blob storage library in GWC, but that amounts to a rewrite of the module.
Rather than doing that, I’d suggest to untangle community module packaging in the simplest possible way:
- Have each module collect its own dependencies in target/dependencies when a specific profile is used (e.g., -Passembly?)
- Have the assembly descriptor point at its own module
- Get rid of the “release” module
Ideally we could do that for extensions as well, but I’d try that with community modules first.
Thoughts?
Regards,
Andrea Aime
==
GeoServer Professional Services from the experts!
Ing. Andrea Aime
@geowolf
Technical Lead
GeoSolutions Group
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549
https://www.geosolutionsgroup.com/
http://twitter.com/geosolutions_it
Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.
This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail
Geoserver-devel mailing list
Geoserver-devel@anonymised.comurceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel
Geoserver-devel mailing list
Geoserver-devel@anonymised.comurceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel