[SAC] [[support.osuosl.org #11303] OSGeos New Servers]

Committees SAC mailing list
1

FYI:
-------- Original Message --------
Subject: [support.osuosl.org #11303] OSGeos New Servers
Date: Tue, 09 Mar 2010 00:03:09 +0000

On Tue Feb 23 21:17:53 2010, tech@wildintellect.com wrote:

Lance,

I have coded the new machines OSGeo3 and OSGeo4 (Bigger Drives and
RAID 6). We'd like OSGeo4 up 1st, and you can delay OSGeo3 a couple of
weeks if you need to or want to wait for the new Ganeti.

Finally have an update!

I have both machines up and already have Ganeti setup between both of
them. I'm not quite ready to go live with the system yet but I'm hoping
by the end of the week (if not sooner) we will. I need to finalize
testing on the guest image deployment scripts and then we can start
deploying new VMs.

I probably won't have much of the documentation ready by then but it is
a high priority for me. I can at least get the basics online. The
physical machines will reside on our backend network so you'll need to
get access via our jumphost.

As for the VMs you need to decide which ones you'd like to have redudant
storage via drbd and which you don't. Since osgeo3 has much less disk
space its something to consider since it'll limit how much you can
deploy on osgeo3. I'd recommend you make a decision on whether you the
VM to always be online in case we need to reboot the machine for
security updates, etc. We can perform live migrations but only on VMs
which have drbd backed storage. Otherwise they will just use simple lvm
storage.

I haven't actually tested plain lvm storage yet so hopefully we don't
run into any major issues. Ganeti just released 2.1.0 a few days ago so
we're running the latest stable version.

It might be a great idea if you had one or a couple of VMs you'd like to
have setup ASAP that way I can set them up and make usre everything is
working properly. Afterwards I can give you documentation on how to
deploy new VMs on your own then.

I didn't check to see how the RAID was setup on the machines but I'm
assuming they are ok. Would like me to double check to ensure its in
RAID6?

Here is the list of our Admin team which will need shell accounts to
manage the virtual machines via Ganeti, and potentially need shell
accounts on the virtual machines in order to setup LDAP access to our
VMs (OSGeo LDAP).

Can you provide perferred usernames for each user and an ssh public key
for each as well?

I'm trying my best to get this done as soon as possible. Hopefully
everything will be ready very soon.

Thanks-

--
Lance Albertson
Systems Administrator / Architect
Open Source Lab

2

Hi folks,

On Mon, Mar 08, 2010 at 04:35:58PM -0800, Alex Mandel wrote:

I didn't check to see how the RAID was setup on the machines but I'm
assuming they are ok. Would like me to double check to ensure its in
RAID6?

Did "we" finally buy RAID6 for both machines ?

Can you provide perferred usernames for each user and an ssh public key
for each as well?

I guess this could be easily picked/copied from 'osgeo1', doesn't it ?

Cheers,
  Martin.
--
Unix _IS_ user friendly - it's just selective about who its friends are !
--------------------------------------------------------------------------

3

On 03/15/2010 03:27 PM, Martin Spott wrote:

Hi folks,

On Mon, Mar 08, 2010 at 04:35:58PM -0800, Alex Mandel wrote:

I didn't check to see how the RAID was setup on the machines but I'm
assuming they are ok. Would like me to double check to ensure its in
RAID6?

Did "we" finally buy RAID6 for both machines ?

Can you provide perferred usernames for each user and an ssh public key
for each as well?

I guess this could be easily picked/copied from 'osgeo1', doesn't it ?

Cheers,
  Martin.

osgeo3 is raid 5
osgeo4 is raid 6

I was unaware of osgeo1 using key based authentication. If it is the yes
we just need to copy the same public keys over.

Thanks,
Alex

4

SAC,

I talked with Lance(OSL) this afternoon. It sounds like a single account
with multiple ssh keys is the best/easiest way to go.

That means I need ssh public keys from:
Frank
Howard
Martin
Tyler
Shawn
Markus
Wolf

You can send me armored ascii file of your public key or your public key
id if you've uploaded to a public keyserver. Assuming those are the
people who we want to be able to create/manage virtual machines.

This same user can also be used as a non LDAP way to access the virtual
machines in case LDAP is having issues. We will be able to add/remove
systems administrators ourselves this way simply by managing the keys on
the osl end and sync all of our servers at osl to the main account.

For the initial tests I was going to ask for them to create Wiki and
Backup which are both currently functions of osgeo2. What we need to
decide today(~next 16h) is on how much ram and disk space to give to
each and if we want to have drbd turned on for them.

Drbd is an uptime/failover cloning of a virtual machine, it requires
double the hard drive space as the original virtual machine but allows
for applying security updates or fail over service without downtime. The
only vm I can think of that we need this on for sure is Secure(LDAP). I
think it would be a waste of space on Backup since that vm is already on
a RAID6 is a copy of data from all the other vms and isn't in constant
use (it's only in use while things are being backed up, and it's config
ie Bacula etc could be implemented on another machine rather quickly).

There is potential in the near future to enable/disable this feature for
a given vm. So do we want to try it on Wiki?

Thanks,
Alex

5

Alex Mandel wrote:

For the initial tests I was going to ask for them to create Wiki and
Backup which are both currently functions of osgeo2. What we need to
decide today(~next 16h) is on how much ram and disk space to give to
each and if we want to have drbd turned on for them.

Drbd is an uptime/failover cloning of a virtual machine, it requires
double the hard drive space as the original virtual machine but allows
for applying security updates or fail over service without downtime. The
only vm I can think of that we need this on for sure is Secure(LDAP). I
think it would be a waste of space on Backup since that vm is already on
a RAID6 is a copy of data from all the other vms and isn't in constant
use (it's only in use while things are being backed up, and it's config
ie Bacula etc could be implemented on another machine rather quickly).

I've got no opinion on the ram/space question or the drbd idea - I'm
sure whatever you pick will be fine. :slight_smile:

Tyler

6

Martin,

Since you will be the primary admin on Wiki (and are currently on osgeo2
Mediawiki) could you let us know how big the database is now and
estimate how much disc space we should allocate to allow adequate room
for growth and room for backup file moving/tmp files.

Does 2 or 4GB of RAM sounds adequate to allow backups to happen without
hindering access of the site.

I also assume a single core is adequate for this vm?

Thanks,
Alex

Alex Mandel wrote:

SAC,

I talked with Lance(OSL) this afternoon. It sounds like a single account
with multiple ssh keys is the best/easiest way to go.

That means I need ssh public keys from:
Frank
Howard
Martin
Tyler
Shawn
Markus
Wolf

You can send me armored ascii file of your public key or your public key
id if you've uploaded to a public keyserver. Assuming those are the
people who we want to be able to create/manage virtual machines.

This same user can also be used as a non LDAP way to access the virtual
machines in case LDAP is having issues. We will be able to add/remove
systems administrators ourselves this way simply by managing the keys on
the osl end and sync all of our servers at osl to the main account.

For the initial tests I was going to ask for them to create Wiki and
Backup which are both currently functions of osgeo2. What we need to
decide today(~next 16h) is on how much ram and disk space to give to
each and if we want to have drbd turned on for them.

Drbd is an uptime/failover cloning of a virtual machine, it requires
double the hard drive space as the original virtual machine but allows
for applying security updates or fail over service without downtime. The
only vm I can think of that we need this on for sure is Secure(LDAP). I
think it would be a waste of space on Backup since that vm is already on
a RAID6 is a copy of data from all the other vms and isn't in constant
use (it's only in use while things are being backed up, and it's config
ie Bacula etc could be implemented on another machine rather quickly).

There is potential in the near future to enable/disable this feature for
a given vm. So do we want to try it on Wiki?

Thanks,
Alex
_______________________________________________
Sac mailing list
Sac@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/sac

7

Alex Mandel wrote:

Martin,

Since you will be the primary admin on Wiki (and are currently on osgeo2
Mediawiki) could you let us know how big the database is now

SELECT sum(round(data_length/1024/1024,2) ) total_size_mb FROM
information_schema.tables ;

gives me: 1.55GB for osgeo_wiki db.

File space used for the wiki tree is about 280MB.

and
estimate how much disc space we should allocate to allow adequate room
for growth and room for backup file moving/tmp files.

Does 2 or 4GB of RAM sounds adequate to allow backups to happen without
hindering access of the site.

Seems to be running fine currently with 2GB (with ~200MB free) but not
sure about any of the other variables.

I'm sure others have comments, but thought I'd throw that out there in
case it helps.

Tyler

8

Well 10GB sounds a little too small. So how about 20GB HD, 4GB ram -
that should give us plenty of room to grow, still have space for backup
and not worry about a backup bringing the wiki to a halt?

As for Backup, how about 500GB HD (I'll check if this really needs to be
in the vm or if it should me a mounted partition on the host), 4 GB ram
to start(We can bump it up later if we need to)?

Thanks,
Alex

On 03/23/2010 10:14 AM, Tyler Mitchell (OSGeo) wrote:

Alex Mandel wrote:

Martin,

Since you will be the primary admin on Wiki (and are currently on osgeo2
Mediawiki) could you let us know how big the database is now

SELECT sum(round(data_length/1024/1024,2) ) total_size_mb FROM
information_schema.tables ;

gives me: 1.55GB for osgeo_wiki db.

File space used for the wiki tree is about 280MB.

and
estimate how much disc space we should allocate to allow adequate room
for growth and room for backup file moving/tmp files.

Does 2 or 4GB of RAM sounds adequate to allow backups to happen without
hindering access of the site.

Seems to be running fine currently with 2GB (with ~200MB free) but not
sure about any of the other variables.

I'm sure others have comments, but thought I'd throw that out there in
case it helps.

Tyler

9

All,

Here's the plan - OSL has made a base virtual machine image. Those who
sent me their ssh keys will have access to it right away (if we could
get the keys from the rest of you that would be very helpful).

What we need to do with this base image is install and configure
anything that will be the same on all of the future virtual machines.
So far the only things I have listed are to install Open-ssh server(must
already be done), add backports to the repo list and to configure LDAP
against the OSGeo LDAP service.

Is there anything else? A standard firewall config? A standard emergency
user account with sudo? A backup client application?

Also who is going to be in charge of setting the base image up?

Once it's done, I'll notify OSL and they will create/give us
instructions on how to create clones of the virtual machine and adjust
the ram and disk space. The 1st 2 machines will be Wiki and Backup
according to the specs on:
http://wiki.osgeo.org/wiki/Infrastructure_Transition_Plan_2010#Final_Plan

Thanks,
Alex