Folks,
In the preliminary board budgeting process we included a line item of $2500
to purchase a new server to be hosted at Telascience. Markus can perhaps
add a bit, but I believe the idea is to provide a more muscular server that
we could move some of the stuff now hosted on xblade14-2. From my perspective
I'm interested in us having a server with enough memory (and perhaps cores)
that we could run several VMs on it as we have been talking about trying to
move some services into VMs for greater portability, isolation and
managability.
Alternatively Tyler has also been suggesting we establish some services
at http://osuosl.org with the intent of reducing our hosting at Peer1.
http://lists.osgeo.org/pipermail/sac/2008-October/001486.html
I would like SAC to consider this problem and within the next month or
so come back to the Finance committee with a moderately detailed budget
proposal for the coming year (2009). This should indicate the resources
needed, and the rough plan for how they would be spent.
Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up | Frank Warmerdam, warmerdam@pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush | Geospatial Programmer for Rent
On Thu, Oct 9, 2008 at 4:19 PM, Frank Warmerdam <warmerdam@pobox.com> wrote:
Folks,
In the preliminary board budgeting process we included a line item of $2500
to purchase a new server to be hosted at Telascience. Markus can perhaps
add a bit, but I believe the idea is to provide a more muscular server that
we could move some of the stuff now hosted on xblade14-2.
We have the situation that xblade14-2 is regularly saturated (Apache takes
1min to respond, ssh login as well). The blade is currently used at
least by:
- buildbot
- www.GDAL.org
- GRASS.osgeo.org
- remotesensing.org
- QGIS.org (with CMS)
Hence the wish to move some of these to a more powerful machine.
Best regards,
Markus
Markus Neteler wrote:
On Thu, Oct 9, 2008 at 4:19 PM, Frank Warmerdam <warmerdam@pobox.com> wrote:
Folks,
In the preliminary board budgeting process we included a line item of $2500
to purchase a new server to be hosted at Telascience. Markus can perhaps
add a bit, but I believe the idea is to provide a more muscular server that
we could move some of the stuff now hosted on xblade14-2.
We have the situation that xblade14-2 is regularly saturated (Apache takes
1min to respond, ssh login as well). The blade is currently used at
least by:
- buildbot
- www.GDAL.org
- GRASS.osgeo.org
- remotesensing.org
- QGIS.org (with CMS)
Hence the wish to move some of these to a more powerful machine.
Markus,
I would note that given sufficient manpower, we can migrate some of
the buildbot slaves to xblade11-2 (buildtest.osgeo.org). However, we
seem to have a limited number of people with the expertise and time to
do this migration. I think this would help alot.
Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up | Frank Warmerdam, warmerdam@pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush | Geospatial Programmer for Rent
Frank Warmerdam kirjoitti:
I would note that given sufficient manpower, we can migrate some of
the buildbot slaves to xblade11-2 (buildtest.osgeo.org). However, we
seem to have a limited number of people with the expertise and time to
do this migration. I think this would help alot.
I could do that. I don't have the expertise yet, but I'd like to do it and I believe I can do it.
I believe also that I don't have an account on the machines.
Regards,
Ari
Best regards,
--
Prof. Ari Jolma
Environmental Management Information Technology
Teknillinen korkeakoulu / Helsinki University of Technology
tel: +358 9 451 3812 address: POBox 5300, 02015 TKK, Finland
Email: ari.jolma at tkk.fi http://geoinformatics.tkk.fi/twiki/bin/view/Main/AriJolmaHomePage
Frank Warmerdam pisze:
Markus Neteler wrote:
On Thu, Oct 9, 2008 at 4:19 PM, Frank Warmerdam <warmerdam@pobox.com> wrote:
Folks,
In the preliminary board budgeting process we included a line item of $2500
to purchase a new server to be hosted at Telascience. Markus can perhaps
add a bit, but I believe the idea is to provide a more muscular server that
we could move some of the stuff now hosted on xblade14-2.
We have the situation that xblade14-2 is regularly saturated (Apache takes
1min to respond, ssh login as well). The blade is currently used at
least by:
- buildbot
- www.GDAL.org
- GRASS.osgeo.org
- remotesensing.org
- QGIS.org (with CMS)
Hence the wish to move some of these to a more powerful machine.
Markus,
I would note that given sufficient manpower, we can migrate some of
the buildbot slaves to xblade11-2 (buildtest.osgeo.org). However, we
seem to have a limited number of people with the expertise and time to
do this migration. I think this would help alot.
Frank,
Am I correct this migration would be planned for Nov/Dec?
I think I'm available to do this migration.
Best regards,
--
Mateusz Loskot, http://mateusz.loskot.net
Charter Member of OSGeo, http://osgeo.org
Hi,
2008/10/9 Mateusz Łoskot <mateusz@loskot.net>:
Am I correct this migration would be planned for Nov/Dec?
I think I'm available to do this migration.
if needed, I can help too...
Martin
--
Martin Landa <landa.martin gmail.com> * http://gama.fsv.cvut.cz/~landa *
Mateusz Łoskot wrote:
Frank,
Am I correct this migration would be planned for Nov/Dec?
I think I'm available to do this migration.
Mateusz,
November would be great!
Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up | Frank Warmerdam, warmerdam@pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush | Geospatial Programmer for Rent
Ari Jolma wrote:
Frank Warmerdam kirjoitti:
I would note that given sufficient manpower, we can migrate some of
the buildbot slaves to xblade11-2 (buildtest.osgeo.org). However, we
seem to have a limited number of people with the expertise and time to
do this migration. I think this would help alot.
I could do that. I don't have the expertise yet, but I'd like to do it and I believe I can do it.
I believe also that I don't have an account on the machines.
Ari,
Ah cool - I read the reply from Mateusz out of order.
From my perspective it would be great if the two of you could cooperate
on this a bit so there are more people who understand how the buildbot
instances work. November would be soon enough. You and Mateusz could
start with the two gdal slaves currently running on the 14-2 blade.
I'd also like some review of what would happen if the buildbot master
died suddenly. That is, are things being backed up in any fashion?
How long would it take to reconstitute the services?
Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up | Frank Warmerdam, warmerdam@pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush | Geospatial Programmer for Rent
Frank Warmerdam pisze:
Ari Jolma wrote:
Frank Warmerdam kirjoitti:
I would note that given sufficient manpower, we can migrate some of
the buildbot slaves to xblade11-2 (buildtest.osgeo.org). However, we
seem to have a limited number of people with the expertise and time to
do this migration. I think this would help alot.
I could do that. I don't have the expertise yet, but I'd like to do it and I believe I can do it.
I believe also that I don't have an account on the machines.
Ari,
Ah cool - I read the reply from Mateusz out of order.
From my perspective it would be great if the two of you could cooperate
on this a bit so there are more people who understand how the buildbot
instances work. November would be soon enough. You and Mateusz could
start with the two gdal slaves currently running on the 14-2 blade.
Frank,
Works for me too.
I'd also like some review of what would happen if the buildbot master
died suddenly. That is, are things being backed up in any fashion?
How long would it take to reconstitute the services?
I do backups of config files of all BB instances we have
and all executable scripts we use to maintain BB (start, stop, clean logs). I put all backups and important stuff under /osgeo/buildbot/admin
Generally, the procedure is:
1. Install Buildbot engine
2. Unpack backup package with config stuff
3. Tweak if needed (some paths and parameters etc.)
4. Run
Best regards,
--
Mateusz Loskot, http://mateusz.loskot.net
Charter Member of OSGeo, http://osgeo.org
Mateusz Łoskot kirjoitti:
I'd also like some review of what would happen if the buildbot master
died suddenly. That is, are things being backed up in any fashion?
How long would it take to reconstitute the services?
I do backups of config files of all BB instances we have
and all executable scripts we use to maintain BB (start, stop, clean logs). I put all backups and important stuff under /osgeo/buildbot/admin
Generally, the procedure is:
1. Install Buildbot engine
2. Unpack backup package with config stuff
3. Tweak if needed (some paths and parameters etc.)
4. Run
Mateusz,
Is the OSGeo BB this one: http://buildbot.net/trac?
Could you send me a copy of the gdal config file, or better if I can grab it from someplace?
What Linux distros are the osgeo blades using?
Regards,
Ari
--
Prof. Ari Jolma
Environmental Management Information Technology
Teknillinen Korkeakoulu / Helsinki University of Technology
tel: +358 9 4511 address: POBox 5300, 02015 TKK, Finland
Email: ari.jolma at tkk.fi URL: http://geoinformatics.tkk.fi
Ari Jolma pisze:
Mateusz Łoskot kirjoitti:
I'd also like some review of what would happen if the buildbot master
died suddenly. That is, are things being backed up in any fashion?
How long would it take to reconstitute the services?
I do backups of config files of all BB instances we have
and all executable scripts we use to maintain BB (start, stop, clean logs). I put all backups and important stuff under /osgeo/buildbot/admin
Generally, the procedure is:
1. Install Buildbot engine
2. Unpack backup package with config stuff
3. Tweak if needed (some paths and parameters etc.)
4. Run
Mateusz,
Is the OSGeo BB this one: http://buildbot.net/trac?
Ari,
Yes, this is the engine we use.
Could you send me a copy of the gdal config file, or better if I can grab it from someplace?
I think that best is to grab current version from Buildbot machine directly:
Connect to xblade14-2
Go to /osgeo/buildbot
Go to subdirectory ./gdal/buildmaster
Grab file ./master.cfg
Grab directory ./config
The master.cfg is the main Buildbot config file.
According to the manual, Buildbot uses single monolithic configuration file (master.cfg) where you put settings for master machine and all slaves.
For easier maintenance, I redesigned it into pluggable config files:
1. master.cfg stores common config and is responsible to find & load configuration for all slaves
2. "config" directory consists of _common.py + number of config files for slaves (each slave has its own config, ie. a_telascience.py is for all builders running on telascience server, etc.)
3. If you want to disable some slave, rename config file, so it does not end with ".py", ie. a_telascience.py-disabled
4. If you want to add new slave, create new file ie. h_ari.py inside config directory and fill with all your settings. The master.cfg will load it automagically.
5. As you see, slaves config files are named as X_name.py where X is a letter. The X is used to sort slaves in the Waterfall table (http://buildbot.osgeo.org:8500/waterfall). If you rename a_telascience.py to ie. z_telascience.py then all builders on telascience slave will be loaded as last and displayed at the end of table (horizontally).
I hope it makes sense 
What Linux distros are the osgeo blades using?
We host all masters on Fedora Core 4.
It also host some slaves (telacience). Other slaves are hosted on different OS'es - this is a distributed structure.
Best regards,
--
Mateusz Loskot, http://mateusz.loskot.net
Charter Member of OSGeo, http://osgeo.org
Ari Jolma wrote:
Could you send me a copy of the gdal config file, or better if I can grab it from someplace?
Ari,
I have created an account "ajolma" on xblade14 (buildbot.osgeo.org,
upload.osgeo.org, www.gdal.org, etc) and on xblade11 (buildtest.osgeo.org).
I'll email your password separately. You also have sudo access.
The existing GDAL buildbot stuff can be found on xblade14 in /osgeo/buildbot.
What Linux distros are the osgeo blades using?
Fedora Core 4.
The goal (or one goal at least) is the migration of the GDAL buildbot
slaves from xblade14 to xblade11 to reduce load on xblade14, and to save
disk space.
I'd add that xblade14 can also be used to place stuff on the OSGeo download
site (/osgeo/download) - for instance if you wanted to setup a
geoinformatica download subdirectory go ahead.
Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up | Frank Warmerdam, warmerdam@pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush | Geospatial Programmer for Rent
Frank Warmerdam kirjoitti:
Fedora Core 4.
uh... I recently tried to build Geoinformatica on a Fedora 4 but it turned out to be not practical since all Gnome code were sooo old 
ok, I'll play a bit with Buildbot on my machine which I use for do-it-yourself buildbot (results are published at http://map.hut.fi/files/Geoinformatica/snapshots/ ) and which has Fedora 8. There's already Fedora 9 also but it's too new for GEOS (it has gcc 4)
Thanks,
Ari
--
Prof. Ari Jolma
Environmental Management Information Technology
Teknillinen Korkeakoulu / Helsinki University of Technology
tel: +358 9 4511 address: POBox 5300, 02015 TKK, Finland
Email: ari.jolma at tkk.fi URL: http://geoinformatics.tkk.fi
Ari Jolma wrote:
Frank Warmerdam kirjoitti:
Fedora Core 4.
uh... I recently tried to build Geoinformatica on a Fedora 4 but it turned out to be not practical since all Gnome code were sooo old
Ari,
Understood. There is no problem with hosting buildbot slaves in
other locations as well as long as the "slave" machines have bidirectional
internet access.
ok, I'll play a bit with Buildbot on my machine which I use for do-it-yourself buildbot (results are published at http://map.hut.fi/files/Geoinformatica/snapshots/ ) and which has Fedora 8. There's already Fedora 9 also but it's too new for GEOS (it has gcc 4)
I presume the gcc4 issues are being resolved?
Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up | Frank Warmerdam, warmerdam@pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush | Geospatial Programmer for Rent
Ari Jolma pisze:
There's already Fedora 9 also but it's too new for GEOS (it has gcc
4)
Ari,
GEOS trunk works well with GCC 4.x, including 4.3
Best regards,
--
Mateusz Loskot, http://mateusz.loskot.net
Charter Member of OSGeo, http://osgeo.org
Mateusz Łoskot kirjoitti:
Ari Jolma pisze:
There's already Fedora 9 also but it's too new for GEOS (it has gcc
4)
Ari,
GEOS trunk works well with GCC 4.x, including 4.3
Good to know. I used the patch that I found on the GEOS list since I'm using GEOS releases, not trunk. There is a GEOS rpm for Fedora 9, I don't know if the packagers are using the trunk version but a new release would be a good thing.
Regards,
Ari
Best regards,
--
Prof. Ari Jolma
Environmental Management Information Technology
Teknillinen korkeakoulu / Helsinki University of Technology
tel: +358 9 451 3812 address: POBox 5300, 02015 TKK, Finland
Email: ari.jolma at tkk.fi http://geoinformatics.tkk.fi/twiki/bin/view/Main/AriJolmaHomePage