[SAC] QGIS servers

Committees SAC mailing list
1

Hi List,

last year the QGIS virtual machine has been heavily used for:
- our issuetracking system (redmine)
- wiki (redmine)
- building documentation (sphinx)
- bulding website (sphinx based)
- building nightlies
etc etc
- our jenkins server is run by somebody else

Currently disk is full (this will be handled today, hopefully), but in general we suffered from limited resources on that server.
Eg, hanging redmine processes, made our site unavailable etc etc
(not talking about outages from last days, that is just the 2.0 downloads)

Ideally all this different beasts would be separate processes OR machines:
- one java one for jenkins et al.
- one ruby one for redmine et al
- one python one for sphinx (building/serving docs and sites 2 times a day)
- one dev one (for building/nightlies etc)

We have been investigating 'Dockers' to ease this separation, and make it easier for (future) QGIS devs to create a local development environment. Nice thing about docker (if I understand correctly), is that it runs beautifully on on a big server, and can be tweaked and configured from all sides.
I also investigated the costs of running a rented server (eg at hetzner: 2x2Tb disk, 32 gb RAM, i7-8core etc for 49 euro).

Question to OSGeo SAC:
- what is the position/idea of OSGeo in this?

Should a (pretty demanding?) question like this being served by OSGeo server(s). Or should the OSGeo hardware be reserved for smaller/starting projects.

Or practical:

Can OSGeo provide us either
one big bare-metal machine, so we could try/do the Docker scenario (preferred option!!)
OR
provide us with enough virtual machines so we could partition different beasts better
OR
should we try to fund a hetzner server for QGIS (actually: "should OSGeo try compete with commercial services like Hetzner for this kind of questions").

Please let us know what you think so we can proceed our quest for world domination :wink:

Regards,

Richard Duivenvoorde
Infrastructure Manager QGIS Project Steering Committee

[0] https://www.docker.io/learn_more/

2

On Tue, Sep 24, 2013 at 7:10 AM, Richard Duivenvoorde <richard@duif.net>wrote:

Question to OSGeo SAC:
- what is the position/idea of OSGeo in this?

Should a (pretty demanding?) question like this being served by OSGeo
server(s). Or should the OSGeo hardware be reserved for smaller/starting
projects.

Richard,

I'm not speaking for the committee, just myself.

I'd like to serve most project needs, even for large projects. However,
OSGeo/SAC assistance to projects works best when there is some commonality
with other projects so that SAC itself can provide some of the system
support. To the extent that QGIS does not seem interested in using any of
the services (LDAP, trac, svn) that we provide to other projects, and if
QGIS wants to personally managed servers for other services as opposed to
hosting them in OSGeo managed servers (ie. many projects keep various
activities on the Projects VM, or even using the VM approach used for other
OSGeo VMs), then I start to doubt the value we add.

Or practical:

Can OSGeo provide us either
one big bare-metal machine, so we could try/do the Docker scenario
(preferred option!!)
OR
provide us with enough virtual machines so we could partition different
beasts better
OR
should we try to fund a hetzner server for QGIS (actually: "should OSGeo
try compete with commercial services like Hetzner for this kind of
questions").

It seems you have already made up your mind to ask the question in this way.

All that said, I'm not adverse to OSGeo purchasing an additional server to
place at OSU OSL and making it available. This likely would take a number
of weeks to complete.

Best regards,
Frank

Please let us know what you think so we can proceed our quest for world
domination :wink:

Regards,

Richard Duivenvoorde
Infrastructure Manager QGIS Project Steering Committee

[0] https://www.docker.io/learn_**more/&lt;https://www.docker.io/learn_more/&gt;

______________________________**_________________
Sac mailing list
Sac@lists.osgeo.org
http://lists.osgeo.org/**mailman/listinfo/sac&lt;http://lists.osgeo.org/mailman/listinfo/sac&gt;

--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up | Frank Warmerdam,
warmerdam@pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush | Geospatial Software Developer

3

Richard,
I concur with Frank and suggest you run with a bare bone box (which
provider is then also up to you). Question is whether the cost of a
decent server is something that OSGeo can and wants to invest or
whether this is up to the project (or a Local Chapter like FOSSGIS who
have already contributed to QGIS code sprints).

Cheers,
Arnulf

PS:
btw: "Infrastructure Manager QGIS Project Steering Committee" is an
impressive title. Have you considered to contribute to the OSGeo SAC
team directly? Then more projects could profit from your
competencies...

On Tue, Sep 24, 2013 at 4:37 PM, Frank Warmerdam <warmerdam@pobox.com> wrote:

On Tue, Sep 24, 2013 at 7:10 AM, Richard Duivenvoorde <richard@duif.net>
wrote:

Question to OSGeo SAC:
- what is the position/idea of OSGeo in this?

Should a (pretty demanding?) question like this being served by OSGeo
server(s). Or should the OSGeo hardware be reserved for smaller/starting
projects.

Richard,

I'm not speaking for the committee, just myself.

I'd like to serve most project needs, even for large projects. However,
OSGeo/SAC assistance to projects works best when there is some commonality
with other projects so that SAC itself can provide some of the system
support. To the extent that QGIS does not seem interested in using any of
the services (LDAP, trac, svn) that we provide to other projects, and if
QGIS wants to personally managed servers for other services as opposed to
hosting them in OSGeo managed servers (ie. many projects keep various
activities on the Projects VM, or even using the VM approach used for other
OSGeo VMs), then I start to doubt the value we add.

Or practical:

Can OSGeo provide us either
one big bare-metal machine, so we could try/do the Docker scenario
(preferred option!!)
OR
provide us with enough virtual machines so we could partition different
beasts better
OR
should we try to fund a hetzner server for QGIS (actually: "should OSGeo
try compete with commercial services like Hetzner for this kind of
questions").

It seems you have already made up your mind to ask the question in this way.

All that said, I'm not adverse to OSGeo purchasing an additional server to
place at OSU OSL and making it available. This likely would take a number
of weeks to complete.

Best regards,
Frank

Please let us know what you think so we can proceed our quest for world
domination :wink:

Regards,

Richard Duivenvoorde
Infrastructure Manager QGIS Project Steering Committee

[0] https://www.docker.io/learn_more/

_______________________________________________
Sac mailing list
Sac@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/sac

--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up | Frank Warmerdam,
warmerdam@pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush | Geospatial Software Developer

_______________________________________________
Sac mailing list
Sac@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/sac

--
Making sense with your spatial data
http://www.metaspatial.net
http://arnulf.us

4

On 24-09-13 16:37, Frank Warmerdam wrote:

    Can OSGeo provide us either
    one big bare-metal machine, so we could try/do the Docker scenario
    (preferred option!!)
    OR
    provide us with enough virtual machines so we could partition
    different beasts better
    OR
    should we try to fund a hetzner server for QGIS (actually: "should
    OSGeo try compete with commercial services like Hetzner for this
    kind of questions").

It seems you have already made up your mind to ask the question in this way.

Honestly: no

/me is a pretty loyal guy. So I prefer a nice server in the OSGeo rack to use. But I'm also aware that we are pretty demanding, and think that it is pretty hard to run this kind of servers that cheap ...

All that said, I'm not adverse to OSGeo purchasing an additional server
to place at OSU OSL and making it available. This likely would take a
number of weeks to complete.

Ok, let's wait for some other input/opinions.

Regards,

Richard Duivenvoorde

5

On 24-09-13 16:53, Arnulf Christl (Seven) wrote:

PS:
btw: "Infrastructure Manager QGIS Project Steering Committee" is an
impressive title. Have you considered to contribute to the OSGeo SAC
team directly? Then more projects could profit from your
competencies...

That title came with the responsibilities I accepted when asked for by the QGIS project. While the title is maybe impressive, my experience is NOT.

So bare with me to invest my prescious time in QGIS only for now...

Regards,

Richard D

6

On 09/24/2013 07:10 AM, Richard Duivenvoorde wrote:

Hi List,

last year the QGIS virtual machine has been heavily used for:
- our issuetracking system (redmine)
- wiki (redmine)
- building documentation (sphinx)
- bulding website (sphinx based)
- building nightlies
etc etc
- our jenkins server is run by somebody else

Currently disk is full (this will be handled today, hopefully), but in
general we suffered from limited resources on that server.
Eg, hanging redmine processes, made our site unavailable etc etc
(not talking about outages from last days, that is just the 2.0 downloads)

I'll point out, we have increased the disk before. It's simply been a
matter of waiting for QGIS to ask. Turns out we have ~200G disk, 50% of
our RAM on osgeo4 unused. 60% of the QGIS disk is actually for builds,
so if we move that off, there shouldn't be a disk crisis for years.

Ideally all this different beasts would be separate processes OR machines:
- one java one for jenkins et al.

Falls in line with a build server that has been discussed recently with
postgis team et al (M. Smith offered one)

- one ruby one for redmine et al

I'd like to see more work done to optimize this setup and keep it up to
date. I see a value here because it keeps some expertise on alternates
to trac available and there is much discussion of git that could be
shared. I briefly looked into it and passenger does have some parameters
similar to mod_wsgi Daemon mode and we haven't tried them yet. Note we
are also increasing RAM on QGIS this week, I think this will help keep
it from crashing as often.

- one python one for sphinx (building/serving docs and sites 2 times a day)

Many other projects are also using sphinx: OSGeoLive, Mapserver, etc....
A whole machine just for building 2x a days seems a little much. OSGeo
Live builds once an hour when getting close to release, though are docs
are smaller and slightly fewer languages. Seems like plenty of overlap
with existing infrastructure.

- one dev one (for building/nightlies etc)

See comments about build server above. I think other projects could
benefit from a shared service.

We have been investigating 'Dockers' to ease this separation, and make

https://www.docker.io/ ah very similar to vagrant and other VM creation
tools. Copying a whole VM in and out is not hard.

it easier for (future) QGIS devs to create a local development
environment. Nice thing about docker (if I understand correctly), is
that it runs beautifully on on a big server, and can be tweaked and
configured from all sides.
I also investigated the costs of running a rented server (eg at hetzner:
2x2Tb disk, 32 gb RAM, i7-8core etc for 49 euro).

1 year of hosting on this = purchase price of such a machine.

Question to OSGeo SAC:
- what is the position/idea of OSGeo in this?

Should a (pretty demanding?) question like this being served by OSGeo
server(s). Or should the OSGeo hardware be reserved for smaller/starting
projects.

Or practical:

Can OSGeo provide us either
one big bare-metal machine, so we could try/do the Docker scenario
(preferred option!!)
OR
provide us with enough virtual machines so we could partition different
beasts better
OR
should we try to fund a hetzner server for QGIS (actually: "should OSGeo
try compete with commercial services like Hetzner for this kind of
questions").

Please let us know what you think so we can proceed our quest for world
domination :wink:

I actually think the current OSGeo machines have the capacity to handle
all of this and the discussion over build machines might actually pan
out if a couple more people (perhaps from qgis) volunteer to help
maintain build services. I'm going to propose once the backups are moved
to the new Backup machine, that we use the freed up resources for a
general building machine.

Thanks,
Alex

7

Hi Alex,

thanks for your thourough answers, some comments from my side:

We have been investigating 'Dockers' to ease this separation, and make
https://www.docker.io/

ah very similar to vagrant and other VM creation
tools. Copying a whole VM in and out is not hard.

Please someone correct me if I'm wrong, but Vagrant is about juggling around VM's, while Docker is more lightweight, more like separating environments with cgroups or virtualenv (but then on OS level)?

Pzl have a look [0] and [1] and correct me if wrong.

I also investigated the costs of running a rented server (eg at hetzner:
2x2Tb disk, 32 gb RAM, i7-8core etc for 49 euro).

1 year of hosting on this = purchase price of such a machine.

Is this really so? 600 euro (incl 19% tax) for such a server? I would think you needed at least two years (but then not having the support and harddisk guarantees).

To me it seemed that it was hard to win from this Hetzner guys, but I'm fine with OSGeo providing such a server. It's not that we want to flee from the flock or so. Should I officially 'order' it here? Please let me know the exact costs (for specs see [2] as an example), and how to proceed then.

Anyway, first let's upgrade current VM, so I can split up the documentation/website builds from the serving dirs (currently there is no diskspace for this).

Regards and thanks for the work you do here,

Richard Duivenvoorde

[0] http://stackoverflow.com/questions/16647069/should-i-use-vagrant-or-docker-io-for-creating-an-isolated-envinronment
[1] https://www.slideshare.net/dotCloud/docker-at-djangocon-kencochranedockertalk2013
[2] http://www.hetzner.de/en/hosting/produkte_rootserver/ex40

8

On 09/25/2013 10:08 AM, Richard Duivenvoorde wrote:

Hi Alex,

thanks for your thourough answers, some comments from my side:

We have been investigating 'Dockers' to ease this separation, and make
https://www.docker.io/

ah very similar to vagrant and other VM creation
tools. Copying a whole VM in and out is not hard.

Please someone correct me if I'm wrong, but Vagrant is about juggling
around VM's, while Docker is more lightweight, more like separating
environments with cgroups or virtualenv (but then on OS level)?

Pzl have a look [0] and [1] and correct me if wrong.

If I'm reading right docker is kind of like
http://en.wikipedia.org/wiki/Virtual_private_server

Which means you could actually run it inside a VM if you want, but I
don't think that will have the results you're aiming for as ram/cpu
isn't specifically allocated, one process out of control could still
bring down other services. Where as running 2 VMs means you can crash
one without crashing the other because their ram is allocated separately
though they may share some cpu cycles.

I also investigated the costs of running a rented server (eg at hetzner:
2x2Tb disk, 32 gb RAM, i7-8core etc for 49 euro).

1 year of hosting on this = purchase price of such a machine.

Is this really so? 600 euro (incl 19% tax) for such a server? I would
think you needed at least two years (but then not having the support and
harddisk guarantees).

Tax on the servers isn't that much, but you're right I forgot to check
current exchange rates, 1.5-2 years of hosting is closer to the cost of
a server. Feel free to poke around and see if you like something on
http://www.siliconmechanics.com/

We have not raised the question yet if anyone else would like to split
such a purchase we can also look at bigger hardware (still 1-2 U).

To me it seemed that it was hard to win from this Hetzner guys, but I'm
fine with OSGeo providing such a server. It's not that we want to flee
from the flock or so. Should I officially 'order' it here? Please let me
know the exact costs (for specs see [2] as an example), and how to
proceed then.

Anyway, first let's upgrade current VM, so I can split up the
documentation/website builds from the serving dirs (currently there is
no diskspace for this).

Can we run this on another VM that already has sphinx like Projects or
Webextra?

Regards and thanks for the work you do here,

Richard Duivenvoorde

[0]
http://stackoverflow.com/questions/16647069/should-i-use-vagrant-or-docker-io-for-creating-an-isolated-envinronment

[1]
https://www.slideshare.net/dotCloud/docker-at-djangocon-kencochranedockertalk2013

[2] http://www.hetzner.de/en/hosting/produkte_rootserver/ex40

Thanks,
Alex

9

On Wed, Sep 25, 2013 at 7:45 PM, Alex Mandel <tech_dev@wildintellect.com> wrote:

On 09/25/2013 10:08 AM, Richard Duivenvoorde wrote:

...

Anyway, first let's upgrade current VM, so I can split up the
documentation/website builds from the serving dirs (currently there is
no diskspace for this).

Can we run this on another VM that already has sphinx like Projects or
Webextra?

Please don't overload the ProjectsVM...:
http://webextra.osgeo.osuosl.org/munin/osgeo.org/projects.osgeo.org.html#System
(see the iowait)

Webextra looks better:
http://webextra.osgeo.osuosl.org/munin/osgeo.org/webextra.osgeo.org.html#System

thanks
Markus