[Geoserver-devel] Setting up a Windows Build Server for GeoTools and GeoServer

GeoServer GeoServer Developer
1

Dear All,
first of all sorry for cross-posting but I believe this topic is
important for both lists of developers.

We have been fighting lately to keep GS and GT (GWC as well btw) build
happily on Windows with partial success; everytime we fixed things
someone committed something that broke the build again.
I had a discussion with Andrea about this (he is not really a Windows
guy :slight_smile: ) but we actually got to the same conclusion: the build on
windows breaks mostly because
of wrong assumptions (i.e. linux path when looking for files), wrong
handling of paths with spaces or even not properly closing files and
streams. The point is that these are not Windows problems (i.e.
something to blame
Windows and forget about) but are real problems hidden by the fact
that we build on Linux _only_. Then my point is, why we give away
linux exe and bin if we don't make any effort to have GS and GT build
on Win?

Here is what I suggest. We ( as in GeoSolutions) will set up an
official build server with Windows and dedicate it to the projects
(giving everybody access to Jenkins on it).
At that point all developers should make sure that they don't break
the build anymore on Windows, which basically mean taking extra care
when writing code and tests, which can do only good to our beloved
projects!

What do you think?

Regards,
Simone Giannecchini

GeoServer training in Milan, 6th & 7th June 2013! Visit
http://geoserver.geo-solutions.it for more information.

Ing. Simone Giannecchini
@simogeo
Founder/Director

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 333 8128928

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

2

On Mon, Jun 3, 2013 at 4:09 PM, Simone Giannecchini <
simone.giannecchini@anonymised.com> wrote:

Here is what I suggest. We ( as in GeoSolutions) will set up an
official build server with Windows and dedicate it to the projects
(giving everybody access to Jenkins on it).
At that point all developers should make sure that they don't break
the build anymore on Windows, which basically mean taking extra care
when writing code and tests, which can do only good to our beloved
projects!

While it's no secret that I hate Windows I'd be very much +1 on this one.
Like it or not, Windows is still an important platform, both on the desktop,
where it's still the dominator, and on the server, where it still has a
clear
lead in several states.

While I cannot say I have seen many Windows servers in the last years,
it's also most likely an effect of me being involved in the open source
projects,
and some self-selection effect, in that people using Linux are probably more
likely to interact with the projects and/or fund them to get improvements.

Yet, I have the feeling there is still a large untapped basin of Windows
users
that we're not reaching to, and getting the build to work flawlessly on
Windows
for both GeoServer and GeoTools would imho go a long way to help us
cross that chasm.

Cheers
Andrea

--

GeoServer training in Milan, 6th & 7th June 2013! Visit
http://geoserver.geo-solutions.it for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

3

Hello,

Good proposal but this can only work if code is put in a intermediate stage and only code
that can build on the target systems will be accepted as a commit.

If this last is not done only a restricted number of commiters will feel responsible for
fixing the issues.

regards,

Eric Smets

Op 3/06/2013 16:09, Simone Giannecchini schreef:

Dear All,
first of all sorry for cross-posting but I believe this topic is
important for both lists of developers.

We have been fighting lately to keep GS and GT (GWC as well btw) build
happily on Windows with partial success; everytime we fixed things
someone committed something that broke the build again.
I had a discussion with Andrea about this (he is not really a Windows
guy :slight_smile: ) but we actually got to the same conclusion: the build on
windows breaks mostly because
of wrong assumptions (i.e. linux path when looking for files), wrong
handling of paths with spaces or even not properly closing files and
streams. The point is that these are not Windows problems (i.e.
something to blame
Windows and forget about) but are real problems hidden by the fact
that we build on Linux _only_. Then my point is, why we give away
linux exe and bin if we don't make any effort to have GS and GT build
on Win?

Here is what I suggest. We ( as in GeoSolutions) will set up an
official build server with Windows and dedicate it to the projects
(giving everybody access to Jenkins on it).
At that point all developers should make sure that they don't break
the build anymore on Windows, which basically mean taking extra care
when writing code and tests, which can do only good to our beloved
projects!

What do you think?

Regards,
Simone Giannecchini

GeoServer training in Milan, 6th & 7th June 2013! Visit
http://geoserver.geo-solutions.it for more information.

Ing. Simone Giannecchini
@simogeo
Founder/Director

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 333 8128928

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite
It's a free troubleshooting tool designed for production
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://p.sf.net/sfu/appdyn_d2d_ap2
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

--
Eric Smets eric.smets@anonymised.com
FKS bvba - Formal and Knowledge Systems http://www.fks.be/
Schampbergstraat 32 Tel: ++32-(0)11-21 49 11
B-3511 Hasselt Fax: ++32-(0)11-22 04 19

4

On Mon, Jun 3, 2013 at 4:23 PM, Eric Smets <eric.smets@anonymised.com> wrote:

Hello,

Good proposal but this can only work if code is put in a intermediate
stage and
only code
that can build on the target systems will be accepted as a commit.

If this last is not done only a restricted number of commiters will feel
responsible for
fixing the issues.

I don't believe going this lenght is necessary. The official build server
on Linux
rarely breaks and when it does it's fixed quickly without the need of
staging
servers in the middle.
We're not elementary school kids, the issue here is that there is no
official
agreement to support a Windows build environment, if we get that,
I'm positive it will be treated just like the Linux one

Cheers
Andrea

--

GeoServer training in Milan, 6th & 7th June 2013! Visit
http://geoserver.geo-solutions.it for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

5

The point is, as Andrea said:

"it will be treated just like the Linux one"

if the build breaks, its breaks regardless of the fact that itr breaks
on Win or Linux and shall be fixed or the push will be reverted.

Regards,
Simone Giannecchini

GeoServer training in Milan, 6th & 7th June 2013! Visit
http://geoserver.geo-solutions.it for more information.

Ing. Simone Giannecchini
@simogeo
Founder/Director

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 333 8128928

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

On Mon, Jun 3, 2013 at 4:29 PM, Andrea Aime
<andrea.aime@anonymised.com> wrote:

On Mon, Jun 3, 2013 at 4:23 PM, Eric Smets <eric.smets@anonymised.com> wrote:

Hello,

Good proposal but this can only work if code is put in a intermediate
stage and
only code
that can build on the target systems will be accepted as a commit.

If this last is not done only a restricted number of commiters will feel
responsible for
fixing the issues.

I don't believe going this lenght is necessary. The official build server on
Linux
rarely breaks and when it does it's fixed quickly without the need of
staging
servers in the middle.
We're not elementary school kids, the issue here is that there is no
official
agreement to support a Windows build environment, if we get that,
I'm positive it will be treated just like the Linux one

Cheers
Andrea

--

GeoServer training in Milan, 6th & 7th June 2013! Visit
http://geoserver.geo-solutions.it for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite
It's a free troubleshooting tool designed for production
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://p.sf.net/sfu/appdyn_d2d_ap2
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

6

Hi,

I have never used Geoserver on anything else than Windows and I have always considered that Geoserver is extraordinary easy to install, including the nightly builds. But if you think that the system can be made even better then I have nothing against it.

-Jukka Rahkonen-

________________________________
Andrea Aime wrote:

On Mon, Jun 3, 2013 at 4:09 PM, Simone Giannecchini <simone.giannecchini@anonymised.com.1268...<mailto:simone.giannecchini@anonymised.com>> wrote:
Here is what I suggest. We ( as in GeoSolutions) will set up an
official build server with Windows and dedicate it to the projects
(giving everybody access to Jenkins on it).
At that point all developers should make sure that they don't break
the build anymore on Windows, which basically mean taking extra care
when writing code and tests, which can do only good to our beloved
projects!

While it's no secret that I hate Windows I'd be very much +1 on this one.
Like it or not, Windows is still an important platform, both on the desktop,
where it's still the dominator, and on the server, where it still has a clear
lead in several states.

While I cannot say I have seen many Windows servers in the last years,
it's also most likely an effect of me being involved in the open source projects,
and some self-selection effect, in that people using Linux are probably more
likely to interact with the projects and/or fund them to get improvements.

Yet, I have the feeling there is still a large untapped basin of Windows users
that we're not reaching to, and getting the build to work flawlessly on Windows
for both GeoServer and GeoTools would imho go a long way to help us
cross that chasm.

Cheers
Andrea

--

GeoServer training in Milan, 6th & 7th June 2013! Visit http://geoserver.geo-solutions.it/&gt; for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

7

That's the point Jukka,
making things better :slight_smile:

This attempt can succeed only if everybody agrees on keeping the
windows build happy ( which means, fix it if you broke it).
Having some devs responsible for fixing the build on windows while
others don't care won't work IMHO.

Regards,
Simone Giannecchini

GeoServer training in Milan, 6th & 7th June 2013! Visit
http://geoserver.geo-solutions.it for more information.

Ing. Simone Giannecchini
@simogeo
Founder/Director

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 333 8128928

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

On Mon, Jun 3, 2013 at 7:20 PM, Rahkonen Jukka
<jukka.rahkonen@anonymised.com> wrote:

Hi,

I have never used Geoserver on anything else than Windows and I have always considered that Geoserver is extraordinary easy to install, including the nightly builds. But if you think that the system can be made even better then I have nothing against it.

-Jukka Rahkonen-

________________________________
Andrea Aime wrote:

On Mon, Jun 3, 2013 at 4:09 PM, Simone Giannecchini <simone.giannecchini@...1268...<mailto:simone.giannecchini@anonymised.com>> wrote:
Here is what I suggest. We ( as in GeoSolutions) will set up an
official build server with Windows and dedicate it to the projects
(giving everybody access to Jenkins on it).
At that point all developers should make sure that they don't break
the build anymore on Windows, which basically mean taking extra care
when writing code and tests, which can do only good to our beloved
projects!

While it's no secret that I hate Windows I'd be very much +1 on this one.
Like it or not, Windows is still an important platform, both on the desktop,
where it's still the dominator, and on the server, where it still has a clear
lead in several states.

While I cannot say I have seen many Windows servers in the last years,
it's also most likely an effect of me being involved in the open source projects,
and some self-selection effect, in that people using Linux are probably more
likely to interact with the projects and/or fund them to get improvements.

Yet, I have the feeling there is still a large untapped basin of Windows users
that we're not reaching to, and getting the build to work flawlessly on Windows
for both GeoServer and GeoTools would imho go a long way to help us
cross that chasm.

Cheers
Andrea

--

GeoServer training in Milan, 6th & 7th June 2013! Visit http://geoserver.geo-solutions.it/&gt; for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite
It's a free troubleshooting tool designed for production
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://p.sf.net/sfu/appdyn_d2d_ap2
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

8

On Mon, Jun 3, 2013 at 7:20 PM, Rahkonen Jukka <jukka.rahkonen@anonymised.com>wrote:

Hi,

I have never used Geoserver on anything else than Windows and I have
always considered that Geoserver is extraordinary easy to install,
including the nightly builds. But if you think that the system can be made
even better then I have nothing against it.

My main concern comes from seeing developers. At the Vienna code sprint
I've seen devs using windows struggle to work on the
code sprint struggle because the build was not working there.
Then I've seen colleagues of mine struggle in a similar way with modules
that would not build on Windows.
And then you get mails from people trying to contribute patches or getting
into development, using Windows, and getting stuck
on build failures too.

We're bleeding resources and piling frustration bit by bit....

As for Windows specific bugs, yep, we already fixed some, and we already
found leaks (unclosed files) that
broke Windows, but would have been leaks nevertheless on Linux as well,
just not as immediately noticeable
(they took time to build up).

Cheers
Andrea

--

GeoServer training in Milan, 6th & 7th June 2013! Visit
http://geoserver.geo-solutions.it for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

9

On Mon, Jun 3, 2013 at 8:53 PM, Frank Warmerdam <warmerdam@anonymised.com> wrote:

Simone,

Is the proposed process that we commit changes and then check in with he
windows Jenkins build an hour or two later to see if we broke them? Or is
there a way to pre-test on windows? Or will we be automatically notified
if our change breaks the tests?

The workflow would be the same as the linux build server: it checks at
fixed intervals if the code changed (once per hour?),
if so it triggers the build and notifies the devel mailing list and the
person that did the commit about build failures.
If within a few hours (12? we don't have a fixed amount, it's more of an
empirical observation) the build is not fixed people
affected by it are at liberty of reverting the commit to restore the build.
It's a loose process, but so far it has worked pretty well I'd say... it's
not like we have 20 commits a day, or 20 people
working in parallel on code, so a somewhat relaxed workflow as the above
works fine in practice.

I know of places in which the build triggers within 10 seconds of your
commit, no one can commit if the build is broken,
a special commit message has to be setup in order to be able to commit at
all when the build is broken
(commit hooks synched with the build server), and if you don't say you're
working on it to fix it, the commit
is reverted within the hour. But these are also places in which 20-40
developers are working in parallel
over the same code base during the same working hours, so the impact of a
broken build is a lot heavier.

Cheers
Andrea

--

GeoServer training in Milan, 6th & 7th June 2013! Visit
http://geoserver.geo-solutions.it for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

10

Thanks to GeoSolutions for additional project resources, I am all in favour of our Jenkins overlords.

Is there any artefact that the windows build server can produce that would put it more seriously in the line of fire? This would allow us as a community
to take a windows build failure more seriously…

Perhaps it could be responsible for nightly windows installer creation?

–
Jody Garnett

On Tuesday, 4 June 2013 at 12:09 AM, Simone Giannecchini wrote:

Dear All,
first of all sorry for cross-posting but I believe this topic is
important for both lists of developers.

We have been fighting lately to keep GS and GT (GWC as well btw) build
happily on Windows with partial success; everytime we fixed things
someone committed something that broke the build again.
I had a discussion with Andrea about this (he is not really a Windows
guy :slight_smile: ) but we actually got to the same conclusion: the build on
windows breaks mostly because
of wrong assumptions (i.e. linux path when looking for files), wrong
handling of paths with spaces or even not properly closing files and
streams. The point is that these are not Windows problems (i.e.
something to blame
Windows and forget about) but are real problems hidden by the fact
that we build on Linux only. Then my point is, why we give away
linux exe and bin if we don’t make any effort to have GS and GT build
on Win?

Here is what I suggest. We ( as in GeoSolutions) will set up an
official build server with Windows and dedicate it to the projects
(giving everybody access to Jenkins on it).
At that point all developers should make sure that they don’t break
the build anymore on Windows, which basically mean taking extra care
when writing code and tests, which can do only good to our beloved
projects!

What do you think?

Regards,
Simone Giannecchini

GeoServer training in Milan, 6th & 7th June 2013! Visit
http://geoserver.geo-solutions.it for more information.

Ing. Simone Giannecchini
@simogeo
Founder/Director

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 333 8128928

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

Get 100% visibility into Java/.NET code with AppDynamics Lite
It’s a free troubleshooting tool designed for production
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://p.sf.net/sfu/appdyn_d2d_ap2

GeoTools-Devel mailing list
GeoTools-Devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel

11

Ciao Jody,
I would say we can make it perform whichever task we want and creating
a Windows installer is surely a good one. I will create a VM with
enough cores/RAM/disk and badwidth to do that we need once we have a
clear idea about it.
I am also open to give VNC/Remove Desktop access to the devs on the
machine to create deploy scripts and so on.

That said, before committing any resources I would like to have
general consensus on considering the Windows build server as important
as the Linux one, otherwise I am afraid
the effort will fail.

Regards,
Simone Giannecchini

GeoServer training in Milan, 6th & 7th June 2013! Visit
http://geoserver.geo-solutions.it for more information.

Ing. Simone Giannecchini
@simogeo
Founder/Director

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 333 8128928

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

On Tue, Jun 4, 2013 at 6:43 AM, Jody Garnett <jody.garnett@anonymised.com> wrote:

Thanks to GeoSolutions for additional project resources, I am all in favour
of our Jenkins overlords.

Is there any artefact that the windows build server can produce that would
put it more seriously in the line of fire? This would allow us as a
community
to take a windows build failure more seriously...

Perhaps it could be responsible for nightly windows installer creation?
--
Jody Garnett

On Tuesday, 4 June 2013 at 12:09 AM, Simone Giannecchini wrote:

Dear All,
first of all sorry for cross-posting but I believe this topic is
important for both lists of developers.

We have been fighting lately to keep GS and GT (GWC as well btw) build
happily on Windows with partial success; everytime we fixed things
someone committed something that broke the build again.
I had a discussion with Andrea about this (he is not really a Windows
guy :slight_smile: ) but we actually got to the same conclusion: the build on
windows breaks mostly because
of wrong assumptions (i.e. linux path when looking for files), wrong
handling of paths with spaces or even not properly closing files and
streams. The point is that these are not Windows problems (i.e.
something to blame
Windows and forget about) but are real problems hidden by the fact
that we build on Linux _only_. Then my point is, why we give away
linux exe and bin if we don't make any effort to have GS and GT build
on Win?

Here is what I suggest. We ( as in GeoSolutions) will set up an
official build server with Windows and dedicate it to the projects
(giving everybody access to Jenkins on it).
At that point all developers should make sure that they don't break
the build anymore on Windows, which basically mean taking extra care
when writing code and tests, which can do only good to our beloved
projects!

What do you think?

Regards,
Simone Giannecchini

GeoServer training in Milan, 6th & 7th June 2013! Visit
http://geoserver.geo-solutions.it for more information.

Ing. Simone Giannecchini
@simogeo
Founder/Director

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 333 8128928

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite
It's a free troubleshooting tool designed for production
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://p.sf.net/sfu/appdyn_d2d_ap2
_______________________________________________
GeoTools-Devel mailing list
GeoTools-Devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel

12

+1 for the Windows build server to be considered as important as Linux, with notifications sent to the respective developer lists.

On 04/06/13 16:02, Simone Giannecchini wrote:

That said, before committing any resources I would like to have
general consensus on considering the Windows build server as important
as the Linux one, otherwise I am afraid
the effort will fail.

--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineer
CSIRO Earth Science and Resource Engineering
Australian Resources Research Centre

13

On Tue, Jun 4, 2013 at 10:27 AM, Ben Caradoc-Davies <
Ben.Caradoc-Davies@anonymised.com> wrote:

+1 for the Windows build server to be considered as important as Linux,
with notifications sent to the respective developer lists.

+1 here too

Cheers
Andrea

--

GeoServer training in Milan, 6th & 7th June 2013! Visit
http://geoserver.geo-solutions.it for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------