New Blades - Initialization and tasking

Committees SAC mailing list
1

Folks,

John has a bunch of shiny new blades setup now that kickstart is working
again. /me ^5's John!

Anyways, I have LDAP enabled and yum updated 198.202.74.219 so I think it is
ready for folks with ldap shell access to log into.

I'd be happy to do the same for .216 and .217 as needed.

John has added lots of info in the system status page, and I've been
trying to keep some notes on what I've done.

   http://wiki.osgeo.org/index.php/SAC_Service_Status

I'm seeing a whole bunch of services (and DNS entries) for 198.202.74.220,
including the LDAP server. This server also seems to be ldap enabled now.
I had been under the impression that we were planning to keep the LDAP
server as "most secure", with essentially no other services on it. Has
that changed? I do think we need to work out some policy on what goes
where, with security and stability vs. flexibility in mind.

For instance, we might designate something like:

  198.202.74.218 (shell.telascience.org)
     - Low security.
     - anyone with shell access can ssh login.
     - anyone with Admin access can sudo.
     - used by geodata for processing
     - setting up ephemeral services for experiments, etc.
     - buildbot slave.

  198.202.74.219
      - Medium security
      - anyone with Admin access can login, and sudo.
      - Various services we want to be dependable placed here like
          FOSS GIS Book Wiki
          SVN servers
          OSGeo Membership Application
          Buildbot master

  198.202.74.220 (ldap.osgeo.org)
      - Most secure.
      - does not use ldap service for authentication - only give out root as
        needed.
      - for now, just ldap server. Perhaps a few other "high security"
        services later.

Other servers might be used for experiments, or possibly "live backups"
for now, with the intent to deploy stuff to them in the future as our
needs evolve.

Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up | Frank Warmerdam, warmerdam@pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush | President OSGF, http://osgeo.org

2

Hello Frank,

Frank Warmerdam schrieb:

Folks,

John has a bunch of shiny new blades setup now that kickstart is working
again. /me ^5's John!

What is kickstart? I found this on wikipedia
http://de.wikipedia.org/wiki/Kickstart. You'll directly see that
telascience seems to use Amiga ;-))!?

Well, it must have something to do with bootstrapping?

Anyways, I have LDAP enabled and yum updated 198.202.74.219 so I think it is
ready for folks with ldap shell access to log into.

I'd be happy to do the same for .216 and .217 as needed.

You suggest to install "FOSS GIS Book Wiki" on .219. I'd like to "resuggest" - you didn't have a real chance to read my last mail, yet - to use one machine, e.g .217 for all the wikis. Or are the blades to powerful, that it would be wasted power to use just one blade for some simple wikis? Don't know.

I assume that "FOSS GIS Book Wiki" is the same as "freegis book wiki", which was asked by edu@osgeo.org to setup?

I wonder, how the backup works for all these blades. Is there already a solution?

Is it possible to install a FreeBSD on one blade? That really would be cool. I'd like to setup WMS services, mapserver, GeoServer on it.

Benjamin

John has added lots of info in the system status page, and I've been
trying to keep some notes on what I've done.

  http://wiki.osgeo.org/index.php/SAC_Service_Status

I'm seeing a whole bunch of services (and DNS entries) for 198.202.74.220,
including the LDAP server. This server also seems to be ldap enabled now.
I had been under the impression that we were planning to keep the LDAP
server as "most secure", with essentially no other services on it. Has
that changed? I do think we need to work out some policy on what goes
where, with security and stability vs. flexibility in mind.

For instance, we might designate something like:

198.202.74.218 (shell.telascience.org)
    - Low security.
    - anyone with shell access can ssh login.
    - anyone with Admin access can sudo.
    - used by geodata for processing
    - setting up ephemeral services for experiments, etc.
    - buildbot slave.

198.202.74.219
     - Medium security
     - anyone with Admin access can login, and sudo.
     - Various services we want to be dependable placed here like
         FOSS GIS Book Wiki
         SVN servers
         OSGeo Membership Application
         Buildbot master

198.202.74.220 (ldap.osgeo.org)
     - Most secure.
     - does not use ldap service for authentication - only give out root as
       needed.
     - for now, just ldap server. Perhaps a few other "high security"
       services later.

Other servers might be used for experiments, or possibly "live backups"
for now, with the intent to deploy stuff to them in the future as our
needs evolve.

Best regards,

3

It sounds to me like a very short introduction to what the blade servers are, what kinds of CPUs, disks, etc. are available might be a good idea. Maybe there's even already a web page someone could direct is to that has this information.

  Allan

On Jul 25, 2006, at 08:32, Benjamin Thelen wrote:

Hello Frank,

Frank Warmerdam schrieb:

Folks,
John has a bunch of shiny new blades setup now that kickstart is working
again. /me ^5's John!

What is kickstart? I found this on wikipedia
http://de.wikipedia.org/wiki/Kickstart. You'll directly see that
telascience seems to use Amiga ;-))!?

Well, it must have something to do with bootstrapping?

Anyways, I have LDAP enabled and yum updated 198.202.74.219 so I think it is
ready for folks with ldap shell access to log into.
I'd be happy to do the same for .216 and .217 as needed.

You suggest to install "FOSS GIS Book Wiki" on .219. I'd like to "resuggest" - you didn't have a real chance to read my last mail, yet - to use one machine, e.g .217 for all the wikis. Or are the blades to powerful, that it would be wasted power to use just one blade for some simple wikis? Don't know.

I assume that "FOSS GIS Book Wiki" is the same as "freegis book wiki", which was asked by edu@osgeo.org to setup?

I wonder, how the backup works for all these blades. Is there already a solution?

Is it possible to install a FreeBSD on one blade? That really would be cool. I'd like to setup WMS services, mapserver, GeoServer on it.

Benjamin

John has added lots of info in the system status page, and I've been
trying to keep some notes on what I've done.
  http://wiki.osgeo.org/index.php/SAC_Service_Status
I'm seeing a whole bunch of services (and DNS entries) for 198.202.74.220,
including the LDAP server. This server also seems to be ldap enabled now.
I had been under the impression that we were planning to keep the LDAP
server as "most secure", with essentially no other services on it. Has
that changed? I do think we need to work out some policy on what goes
where, with security and stability vs. flexibility in mind.
For instance, we might designate something like:
198.202.74.218 (shell.telascience.org)
    - Low security.
    - anyone with shell access can ssh login.
    - anyone with Admin access can sudo.
    - used by geodata for processing
    - setting up ephemeral services for experiments, etc.
    - buildbot slave.
198.202.74.219
     - Medium security
     - anyone with Admin access can login, and sudo.
     - Various services we want to be dependable placed here like
         FOSS GIS Book Wiki
         SVN servers
         OSGeo Membership Application
         Buildbot master
198.202.74.220 (ldap.osgeo.org)
     - Most secure.
     - does not use ldap service for authentication - only give out root as
       needed.
     - for now, just ldap server. Perhaps a few other "high security"
       services later.
Other servers might be used for experiments, or possibly "live backups"
for now, with the intent to deploy stuff to them in the future as our
needs evolve.
Best regards,

--
Allan Doyle
+1.781.433.2695
adoyle@eogeo.org

4

Allan Doyle wrote:

It sounds to me like a very short introduction to what the blade servers are, what kinds of CPUs, disks, etc. are available might be a good idea. Maybe there's even already a web page someone could direct is to that has this information.

Allan,

The blades are listed as "Mobile AMD Athlon(tm) XP-M 1800+" (single cpu) in
/proc/cpuinfo. It looks like they have 1GB of RAM. Each blade has a local
25GB disk. Substantially more space seems to be available via NFS from
"bucket". The shared home directories are on a 34GB drive (shared between
all the blades for home directories) and perhaps some other purposes. On
.218 I also see a 400GB drive mounted as /mnt/data3. I imagine some of this
will be available for holding larger geodata datasets.

John, perhaps you can give a bit more detail on what disk resources are
available for geodata? What spots you are ok with us making use of?

There is also a big multi-cpu Itanium server that could potentially be used
for heavy processing with John's permission though this machine isn't delgated
to OSGeo for our exclusive use.

Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up | Frank Warmerdam, warmerdam@pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush | President OSGF, http://osgeo.org

5

Benjamin Thelen wrote:

Hello Frank,

Frank Warmerdam schrieb:

Folks,

John has a bunch of shiny new blades setup now that kickstart is working
again. /me ^5's John!

What is kickstart? I found this on wikipedia
http://de.wikipedia.org/wiki/Kickstart. You'll directly see that
telascience seems to use Amiga ;-))!?

Well, it must have something to do with bootstrapping?

Benjamin,

The kickstart problems seem to have been fixed. I don't know any details
about it though.

You suggest to install "FOSS GIS Book Wiki" on .219. I'd like to "resuggest" - you didn't have a real chance to read my last mail, yet - to use one machine, e.g .217 for all the wikis. Or are the blades to powerful, that it would be wasted power to use just one blade for some simple wikis? Don't know.

I hadn't expected that we would use a whole blade exclusively for just
wikis. We basically have five blades, and so I'm hopeful we can run
serveral "medium security" services like wiki, svn, etc off one server.

I assume that "FOSS GIS Book Wiki" is the same as "freegis book wiki", which was asked by edu@osgeo.org to setup?

Sorry ... my mixup. I mean the freegis book wiki.

I wonder, how the backup works for all these blades. Is there already a solution?

I don't believe there is. I am thinking we could use a two part
strategy. One is to have a frequent backup of some services to another
blade, or to the big NFS drive on "bucket". But I also think we ought
to identify key services and data to backup off-site. I wonder if
"rsync" would be useful for network backup or if someone has a better
idea?

Is it possible to install a FreeBSD on one blade? That really would be cool. I'd like to setup WMS services, mapserver, GeoServer on it.

I'm not sure how easy this would be. Would FreeBSD be better because
of better "jail" support"? Generally speaking, I would prefer to keep to
one platform as much as possible so it is easier to move stuff around.

> What about something like jails, used in FreeBSD, XEN or something like
> that, to segregate things? I don't know, if there is something else
> besides XEN similar like jails in Linux.

I have little experience with virtualization solutions like XEN or
jails, so I'm open to some advice. It *seems* like XEN might quickly
use up memory and disk space to operate several virtual machines at
once. My understanding of jails was that they were primarily useful
to prevent users from being able to impact each other by having almost
distinct directory trees for the jails. How does this work in practice?

> Could you find a possibility to use LDAP together with mediawiki? I
> could't at first sight.

I haven't had a look yet, so I have no suggestions.

Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up | Frank Warmerdam, warmerdam@pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush | President OSGF, http://osgeo.org

6

All

Page 38 explains the differences between Kickstart and Jumpstart (Solaris os install)
http://www.sun.com/blueprints/0205/819-1692.pdf

I had a bad macro in the DHCP server that needed deleting.

These blades are pretty snappy they are on a GigE on a clean network.
I am moving storage around on bucket ... there will be ~1.2 TB of fast storage.
There is an additional 18 TB storage not linked into the system yet...

Backups can be improved.. the data center has a backup scheduling system i can take advantage of.. in the past i have just kept duplicates of data on several machines and on the blades there is a mkflash script that snapshots the blade and stores it in a way that the system can be restored to that point in about 45 min. these flash archives were stored on bucket also.

The Agami has been touchy going across the network... I will look into why NFS mounts were getting stale.

John

Frank Warmerdam wrote:

Benjamin Thelen wrote:

Hello Frank,

Frank Warmerdam schrieb:

Folks,

John has a bunch of shiny new blades setup now that kickstart is working
again. /me ^5's John!

What is kickstart? I found this on wikipedia
http://de.wikipedia.org/wiki/Kickstart. You'll directly see that
telascience seems to use Amiga ;-))!?

Well, it must have something to do with bootstrapping?

Benjamin,

The kickstart problems seem to have been fixed. I don't know any details
about it though.

You suggest to install "FOSS GIS Book Wiki" on .219. I'd like to "resuggest" - you didn't have a real chance to read my last mail, yet - to use one machine, e.g .217 for all the wikis. Or are the blades to powerful, that it would be wasted power to use just one blade for some simple wikis? Don't know.

I hadn't expected that we would use a whole blade exclusively for just
wikis. We basically have five blades, and so I'm hopeful we can run
serveral "medium security" services like wiki, svn, etc off one server.

I assume that "FOSS GIS Book Wiki" is the same as "freegis book wiki", which was asked by edu@osgeo.org to setup?

Sorry ... my mixup. I mean the freegis book wiki.

I wonder, how the backup works for all these blades. Is there already a solution?

I don't believe there is. I am thinking we could use a two part
strategy. One is to have a frequent backup of some services to another
blade, or to the big NFS drive on "bucket". But I also think we ought
to identify key services and data to backup off-site. I wonder if
"rsync" would be useful for network backup or if someone has a better
idea?

Is it possible to install a FreeBSD on one blade? That really would be cool. I'd like to setup WMS services, mapserver, GeoServer on it.

I'm not sure how easy this would be. Would FreeBSD be better because
of better "jail" support"? Generally speaking, I would prefer to keep to
one platform as much as possible so it is easier to move stuff around.

> What about something like jails, used in FreeBSD, XEN or something like
> that, to segregate things? I don't know, if there is something else
> besides XEN similar like jails in Linux.

I have little experience with virtualization solutions like XEN or
jails, so I'm open to some advice. It *seems* like XEN might quickly
use up memory and disk space to operate several virtual machines at
once. My understanding of jails was that they were primarily useful
to prevent users from being able to impact each other by having almost
distinct directory trees for the jails. How does this work in practice?

> Could you find a possibility to use LDAP together with mediawiki? I
> could't at first sight.

I haven't had a look yet, so I have no suggestions.

Best regards,