[SAC] [[support.osuosl.org #11303] OSGeos New Servers]

Committees SAC mailing list
1

Ok, 3 new servers are up and ready for login attempts(access info
below). I have verified that I can login with ldap to Backup no problem.

There's not much written on
http://wiki.osgeo.org/wiki/Infrastructure_Transition_Plan_2010#osgeo3_.26_osgeo4

for specifics of what's being setup on each other than the general
ideas, so please document what you do.

One note, we had to go with ext3 instead of ext4 because the current
Debian Stable kernels don't support ext4 well. We may want to consider a
different version of Debian/Ubuntu for the download mirror later on to
take advantage of the speed increase of ext4 (I've heard it rivals XFS)
Luckily if a newer kernel becomes available migrating from ext3 to ext4
is easy (We might want to keep this in mind for Backup).

Thanks,
Alex

-------- Original Message --------
Subject: [support.osuosl.org #11303] OSGeos New Servers
Date: Tue, 30 Mar 2010 22:49:08 +0000

On Mon Mar 29 16:40:18 2010, tech@wildintellect.com wrote:

So now we're ready for 3 Vms:
wiki.osgeo.org(aka Wiki)
secure.osgeo.org(aka Secure)
backup.osgeo.org(aka Backup)

All three of these should be online as $hostname.osgeo.osuosl.org.
Please verify that everything looks ok. Don't freak out when you do a df
on backup and it says 493G, trust me its 504G :slight_smile:

Thanks-

--
Lance Albertson
Systems Administrator / Architect
Open Source Lab

2

Quick follow-up, we could move everything to ext4 quickly (ie OSL can
update our base image and remake the current VMs in a matter of minutes)
If we want to use a kernel from backports that fixes known bugs in the
ext4 support.
http://packages.debian.org/lenny-backports/linux-image-2.6-amd64

We should probably decide on this before embarking on work in the new
VMs, or could piecemeal upgrade any particular VM later.

Why ext4: (Anyone have con arguments?)
https://ext4.wiki.kernel.org/index.php/Ext4_Howto#For_people_who_are_running_Debian
http://en.wikipedia.org/wiki/Ext4

Thanks,
Alex

Alex Mandel wrote:

Ok, 3 new servers are up and ready for login attempts(access info
below). I have verified that I can login with ldap to Backup no problem.

There's not much written on
http://wiki.osgeo.org/wiki/Infrastructure_Transition_Plan_2010#osgeo3_.26_osgeo4

for specifics of what's being setup on each other than the general
ideas, so please document what you do.

One note, we had to go with ext3 instead of ext4 because the current
Debian Stable kernels don't support ext4 well. We may want to consider a
different version of Debian/Ubuntu for the download mirror later on to
take advantage of the speed increase of ext4 (I've heard it rivals XFS)
Luckily if a newer kernel becomes available migrating from ext3 to ext4
is easy (We might want to keep this in mind for Backup).

Thanks,
Alex

-------- Original Message --------
Subject: [support.osuosl.org #11303] OSGeos New Servers
Date: Tue, 30 Mar 2010 22:49:08 +0000

On Mon Mar 29 16:40:18 2010, tech@wildintellect.com wrote:

So now we're ready for 3 Vms:
wiki.osgeo.org(aka Wiki)
secure.osgeo.org(aka Secure)
backup.osgeo.org(aka Backup)

All three of these should be online as $hostname.osgeo.osuosl.org.
Please verify that everything looks ok. Don't freak out when you do a df
on backup and it says 493G, trust me its 504G :slight_smile:

Thanks-

3

On Tue, Mar 30, 2010 at 06:18:09PM -0700, Alex Mandel wrote:

One note, we had to go with ext3 instead of ext4 because the current
Debian Stable kernels don't support ext4 well. We may want to consider a
different version of Debian/Ubuntu for the download mirror later on to
take advantage of the speed increase of ext4 (I've heard it rivals XFS)

I wonder why we then didn't go for XFS right from the beginning, if XFS
is the reference anyway !?

Cheers,
  Martin.
--
Unix _IS_ user friendly - it's just selective about who its friends are !
--------------------------------------------------------------------------

4

Martin Spott wrote:

On Tue, Mar 30, 2010 at 06:18:09PM -0700, Alex Mandel wrote:

One note, we had to go with ext3 instead of ext4 because the current
Debian Stable kernels don't support ext4 well. We may want to consider a
different version of Debian/Ubuntu for the download mirror later on to
take advantage of the speed increase of ext4 (I've heard it rivals XFS)

I wonder why we then didn't go for XFS right from the beginning, if XFS
is the reference anyway !?

Cheers,
  Martin.

XFS has been the benchmark for speed (On large files afaik, not lots of
small files), but from what I've read ext is considered safer in the
event of power loss etc. ext4 seems to combine the specs of both. I know
XFS can't shrink, but I guess that's unlikely to be an issue for us. So
the only other question is if OSL's tools and setup supports XFS. I'm
also unsure if XFS requires tweaking to achieve it performance, so
familiarity may be a factor here.

There must be some reason why most Linux distros have not made it their
default filesystem. Ah trolling around it actually looks like
historically Redhat/Centos line didn't include XFS in it's
kernels(supposedly added in 5.4, I think that's their latest version).

I really had only been thinking about XFS when dealing with the download
mirror where we could potentially have lots of 4 GB iso images of the
Live DVD/Virtual Machines. There doesn't seem to be any reason to use it
for anything else. ext4 however is soon to be the standard default for
more distros and by moving now we could start taking advantage of
performance over ext3 while still expecting it and it's tools to be the
same for the most part.

Thanks,
Alex

(Anyone on the list used XFS before, I've read about it plenty but never
used it)

5

On Wed, Mar 31, 2010 at 01:07:47AM -0700, Alex Mandel wrote:

(Anyone on the list used XFS before, I've read about it plenty but never
used it)

Yup, I've been using XFS on Linux starting with the first 'official'
patch against kernel 2.4.4 (and earlier on IRIX).

According to my experience with various filesystems which have been
available with the stock linux kernel over the years, it would have
been the primary choice for heavily loaded, 'unattended' servers - at
least from a technical point of view (thus ignoring distro politics).

Cheers,
  Martin.
--
Unix _IS_ user friendly - it's just selective about who its friends are !
--------------------------------------------------------------------------

6

Alex Mandel wrote:
> One note, we had to go with ext3 instead of ext4 because
> the current Debian Stable kernels don't support ext4 well.
> We may want to consider a different version of Debian/Ubuntu
> for the download mirror later on to take advantage of the
> speed increase of ext4 (I've heard it rivals XFS)

Martin Spott wrote:

I wonder why we then didn't go for XFS right from the
beginning, if XFS is the reference anyway !?

I agree with Martin. XFS is proven and well tested technology
which requires no special updates to bleeding-edge hotfixes.
Ext4 does not have the track record and does require custom
package installs (which should be avoided as much as possible).
Bugfixes and tuning are still dribbling into the newest Kernel
releases. In a year or two it could be the FS of choice but
for now, IMpersonalO it is still too immature.

a worthwhile read, more FS comparisons linked from the comments
section:

http://www.debian-administration.org/articles/388
(written for Etch == Lenny-1 aka "oldstable")

IMHO backports.org should be used as sparingly as possible, it
contains anything any DebDev decided to upload to it, has not
been through the QA gauntlet as such, and does not get formal
or timely security updates by the Debian Security Team (AFAIK).

The default should be to build the install using default stable
distro packages + security + volatile updates.

deb http://security.debian.org/ lenny/updates main
deb http://volatile.debian.org/debian-volatile lenny/volatile main

(volatile contains things like daylight savings database updates,
etc. which are not strictly security fixes)

After the install is complete, and only then, add backports.org
as a package source and then explicitly install individual
backported packages which you absolutely can not live without.

(see also "apt pinning")

What's the point of using an ultra-stable base distro like
Debian/stable if you move in and start installing the latest
greatest gadgets off the assembly line as soon as you get your
hands on it?

For similar reasons I am uncomfortable with the notion of using
Ubuntu as a main server OS. Their stated purpose and goal is to
promote ease of use over security and correctness. Debian's
stated purpose and goals are the opposite. Numerous examples of
the actual implementation of these policies can be found in the
final decisions taken in their respective bug trackers, justified
as such.

I've been maintaining Debian servers for years, and if there's
one thing I've learned it is not to try and out-smart the system
by installing gratuitous upgrades, custom modifications, or
bypassing the apt/dpkg system (if at all possible).

regards,
Hamish

7

Hamish wrote:

Alex Mandel wrote:

One note, we had to go with ext3 instead of ext4 because
the current Debian Stable kernels don't support ext4 well.
We may want to consider a different version of Debian/Ubuntu
for the download mirror later on to take advantage of the
speed increase of ext4 (I've heard it rivals XFS)

Martin Spott wrote:

I wonder why we then didn't go for XFS right from the
beginning, if XFS is the reference anyway !?

I agree with Martin. XFS is proven and well tested technology
which requires no special updates to bleeding-edge hotfixes.
Ext4 does not have the track record and does require custom
package installs (which should be avoided as much as possible).
Bugfixes and tuning are still dribbling into the newest Kernel
releases. In a year or two it could be the FS of choice but
for now, IMpersonalO it is still too immature.

a worthwhile read, more FS comparisons linked from the comments
section:

http://www.debian-administration.org/articles/388
(written for Etch == Lenny-1 aka "oldstable")

IMHO backports.org should be used as sparingly as possible, it
contains anything any DebDev decided to upload to it, has not
been through the QA gauntlet as such, and does not get formal
or timely security updates by the Debian Security Team (AFAIK).

The default should be to build the install using default stable
distro packages + security + volatile updates.

deb http://security.debian.org/ lenny/updates main
deb http://volatile.debian.org/debian-volatile lenny/volatile main

(volatile contains things like daylight savings database updates,
etc. which are not strictly security fixes)

After the install is complete, and only then, add backports.org
as a package source and then explicitly install individual
backported packages which you absolutely can not live without.

(see also "apt pinning")

What's the point of using an ultra-stable base distro like
Debian/stable if you move in and start installing the latest
greatest gadgets off the assembly line as soon as you get your
hands on it?

For similar reasons I am uncomfortable with the notion of using
Ubuntu as a main server OS. Their stated purpose and goal is to
promote ease of use over security and correctness. Debian's
stated purpose and goals are the opposite. Numerous examples of
the actual implementation of these policies can be found in the
final decisions taken in their respective bug trackers, justified
as such.

I've been maintaining Debian servers for years, and if there's
one thing I've learned it is not to try and out-smart the system
by installing gratuitous upgrades, custom modifications, or
bypassing the apt/dpkg system (if at all possible).

regards,
Hamish

Thanks for the feedback everyone, we found a reasonable compromise that
takes little to no time. The default will remain ext3 for large vms that
intend to hold large files, we will have a small 10G ext3 partition for
the OS and the rest will be a separate mount point XFS.
This will be implemented on Backup and on Download(when we get to that),
and other VMs if the need is identified.

In fact it's now done. So we are ready to move ahead with configuration
of Wiki, Secure and Backup. The XFS volume in backup is /dev/vdb and
will need to be mounted - suggestions for mount options are welcome.

Thanks,
Alex

8

Alex Mandel wrote:

Thanks for the feedback everyone, we found a reasonable compromise that
takes little to no time. The default will remain ext3 for large vms that
intend to hold large files, we will have a small 10G ext3 partition for
the OS and the rest will be a separate mount point XFS.
This will be implemented on Backup and on Download(when we get to that),
and other VMs if the need is identified.

Folks,

I have installed xfsprogs on backup.osgeo.osuosl.org formatted the big
partition and mounted it as /backup. No apparent problems though we
aren't actually using it for anything.

Over the next couple days I will start having this server rsync to mirror
the download server, and rsyncing backups from osgeo1 that are normally
pulled by osgeo2.

I'm willing to be more or less for the rsync based aspect of the backup
server. I'm assuming someone else will take care of the Backula aspect.

Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up | Frank Warmerdam, warmerdam@pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush | Geospatial Programmer for Rent

9

Alex wrote:

In fact it's now done. So we are ready to move ahead with configuration
of Wiki, Secure and Backup. The XFS volume in backup is /dev/vdb and
will need to be mounted - suggestions for mount options are welcome.

micro-wish: while yer at it, if you can figure out how to get the mediawiki
login page to accept https:// it would be really nice.

cheers,
Hamish