Hello All,
I’m going to go into a fair bit of detail relative to previous updates, so that everyone has a clearer idea of exactly where we are at, the current state, and what still needs to be done.
Server Configuration Changes
First off, the new Jenkins instance has some differences from the old one:
- It is, of course, running on a newer version of Jenkins, with security updates installed, so some things may look a little different.
- Worker nodes now get spun up (and down) dynamically (via Amazon SpotFleet). Ideally, this should give decent build performance without having a large number of resources idling when there are no builds in progress. This does result in a couple of compromises:
- If there are no jobs running for a while, all the worker nodes will get shut down, and will take a little bit of time to start up when the next job is started
- It makes copying nightly artifacts back onto the master a little more complicated, although this should be (mostly) working now. At current, artifacts get archived to the master node by Jenkins, then copied to the location we use to host the nightly builds via a nightly cron job (so they will not show up immediately). I hope to change this to just ssh from the worker node to the hosting location on master if I can figure out how. One benefit of the new configuration is that artifacts are now also linked directly from the nightly build that produced them.
- Configuration can not be assumed to be persistent between builds. This will likely require the release jobs to be reconfigured somewhat.
To help deal with this, some configuration which used to just be assumed to be on the build server is now managed by Jenkins (which is overall, probably a good thing). This currently includes: Java versions/installers, Git identity, and the Maven settings.xml, and will eventually include keys for GitHub, SourceForge, and GeoTools.org, once those get added.
Job Status:
I have been focusing on getting the main jobs for GeoTools, GeoWebCache, and GeoServer working.
-
geoserver-, geoserver--nightly and geoserver--live-docs are all working (except for geoserver-master-live-docs, but this appears to be caused by a legitimate error in the docs on master
-
geowebcache- jobs are currently working (and deploying to maven). geowebcache--nightly jobs are not, as they assume they will have access to the results of geowebcache-, which is no longer true. They will need to be reconfigured. I’m currently looking into this.
-
geotools- and geotools--docs builds are building successfully, but failing when they try to copy javadocs/docs (respectively) to geotools.org. We need to aquire the key to geotools.org and add it to jenkins before we can get these jobs working (or disable doc uploads until we have the key).
-
Most *-java11 jobs are working, since they generally don’t need to copy artifacts anywhere (java 11 has been configured in jenkins)
I haven’t looked at any of the other jobs yet -
Cite and Online jobs aren’t going to work until PostGIS is configured on the worker nodes
-
Release jobs require github and sourceforge keys, and may need some other modifications
-
I haven’t looked at the geofence/geogig/geoscript jobs, and don’t know what is needed to get them working.
Mailing list notifications also aren’t working yet - a bit of configuration still needs to be done on the build server before those will be working (and its probably best to get most of the jobs working first anyways, to avoid excess noise).
How you can help
-
If you are a PSC member and would like to be able to start/stop/configure jobs on the new jenkins server, please create a new Jenkins account by visiting build.geoserver.org please click “sign up” in the top right corner. You will initially have the same permissions as if you were not logged in - after creating an account please email me with your account name and I will elevate your permissions.
All of following points will only be applicable to members who currently have a Jenkins account. -
If you are familiar with the geofence/geogig/geoscript jobs, please take a look at the recent failures to see what is going wrong there (and maybe see if you can fix it)
-
If you are familiar with geowebcache, any modifications to the geowebcache--nightly jobs so that they don’t rely on the corresponding geowebcache- job being built in the same workspace would be appreciated (although if you do make changes, please contact me so that we aren’t both changing the same job at the same time)
The following points will require some coordination between various project administrators: -
If you have access to the geotools.org ssh key, please contact me so we can add it to Jenkins.
-
If you are an administrator for both the GeoTools and GeoServer projects in GitHub, try and create a separate CI account with push permissions to those projects and generate an ssh key for it, so that the release jobs will be able to push artifacts to GitHub
-
If you are an administrator for both the GeoTools and GeoServer projects in SourceForge, try and create a separate CI account with push permissions to those projects and generate an ssh key for it, so that the release jobs will be able to push artifacts to SourceForge
Thanks,
Torben
On Thu, May 23, 2019 at 11:36 AM Andrea Aime <andrea.aime@anonymised.com> wrote:
Hi Torben,
please let us know where you’re at at the end of your day. I’m not sure I can help tomorrow but
if worst come to worst I could give it a try during the weekendCheers
AndreaOn Thu, May 23, 2019 at 7:08 PM Torben Barsballe <tbarsballe@anonymised.com6…> wrote:
The main thing blocking functionality at this point is adding keys for all the things Jenkins needs to talk to.
A lot of the original keys used before were set up by Justin using his accounts back when we were still using Ares; given that there was a security breach I’d like us to generate new keys, and ideally use a shared / bot account for them. If someone is able to create such generic accounts and generate ssh keys for them, that would help a lot.Keys/accounts needed:
- github (For releases; commit access to GeoTools, GeoServer)
- sourceforge (For releases, push access to GeoTools, GeoServer)
I’ll also need a copy of the key that is used to push docs to geotools.org in order to get the geotools builds working.The artifactory key is already set up, and I am currently working on setting up a key so the worker nodes can talk to master to push nightly builds (again).
If there are any other keys I’m forgetting about, that would be good to know as well.Thanks,
TorbenOn Wed, May 22, 2019 at 11:24 PM Andrea Aime <andrea.aime@anonymised.com…> wrote:
Hi Torben,
thanks!Is there anything we can do to help getting the rest of the functionality up and running?
Cheers
AndreaOn Wed, May 22, 2019 at 10:23 PM Torben Barsballe <tbarsballe@anonymised.com> wrote:
On Wed, May 22, 2019 at 9:13 AM Torben Barsballe <tbarsballe@anonymised.com5476…> wrote:
On Tue, May 21, 2019 at 11:47 PM Andrea Aime <andrea.aime@anonymised.com> wrote:
Hi Torben,
thanks for the update! Nice to see it online.
It seems that nightly build results are not accessible though, even going without https, e.g.: http://build.geoserver.org/geoserver
won’t respond.This was working yesterday, but isn’t working for me now either; I’ll look into what might have changed.
I found out the issue - there was an error with how redirects were handled when the trailing slash was missing from the URL (e.g. http://build.geoserver.org/geoserver vs http://build.geoserver.org/geoserver/ - It should be fixed now.
Torben
Torben
–
Regards, Andrea Aime == GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.
–
Regards, Andrea Aime == GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.
