After chatting with Howard, I took a quick stab at spec'ing out a server
again just to get the ball rolling. I'm sure it's not perfect, so holler.
A couple things to note or uncertainties:
* AMD vs Intel ?
* I'm not familiar with SAS drives and RAID stuff much in general, so
not sure if I choose things properly there.
* Seemed like some 2U options didn't allow as many drives, but maybe 4U
is not needed?
* OSL prefers Dell, so the added cost of going with them will encourage
expedient service from OSL and in their relationship with Dell.
* Not sure if OSL can get a better deal.
Comments, changes? If anyone else is interested in specing out
something else, I won't be offended
On Tue, Jan 26, 2010 at 12:25:41PM -0800, Tyler Mitchell (OSGeo) wrote:
A couple things to note or uncertainties:
* AMD vs Intel ?
Personal favourites !?
* I'm not familiar with SAS drives and RAID stuff much in general, so
not sure if I choose things properly there.
Yes, getting a proper selection of RAID and disks is a PITA with Dell's
web configurator ....
I'm not too familiar with Dell's R* offers, but from my understanding
this is going to be a server with one CPU and one power supply (to my
experience a second PS is usually listed separately). Does this meet
your intentions ?
Cheers,
Martin.
--
Unix _IS_ user friendly - it's just selective about who its friends are !
--------------------------------------------------------------------------
I'm not too familiar with Dell's R* offers, but from my understanding
this is going to be a server with one CPU and one power supply (to my
experience a second PS is usually listed separately). Does this meet
your intentions ?
Thanks for the comments Martin - did you see the attached PDF? There is
more info about processor etc in there. Not sure if I chose a backup
power supply, but meant to.
This quote was for a 2x E7420 Xeon, four cores - if that makes sense
I've been lost in CPU world since they stopped calling them Pentium lol
On Tue, Jan 26, 2010 at 02:13:19PM -0800, Tyler Mitchell (OSGeo) wrote:
Thanks for the comments Martin - did you see the attached PDF? There is
more info about processor etc in there.
Ah, I see - I looked at the quantity, not realizing that "1" already
meant "2x" as written in the text. Sorry, looks a little like things
have changed since the last time I ordered a machine with Dell.
Cheers,
Martin.
--
Unix _IS_ user friendly - it's just selective about who its friends are !
--------------------------------------------------------------------------
I'm not too familiar with Dell's R* offers, but from my understanding
this is going to be a server with one CPU and one power supply (to my
experience a second PS is usually listed separately). Does this meet
your intentions ?
Thanks for the comments Martin - did you see the attached PDF? There is
more info about processor etc in there. Not sure if I chose a backup
power supply, but meant to.
This quote was for a 2x E7420 Xeon, four cores - if that makes sense
I've been lost in CPU world since they stopped calling them Pentium lol
Looking at the previous quotes, the base model was the R700 series and
not the R900 series. Inspecting even closer the processors on the
previous quotes look much better on paper, more features, lower power
consumption, higher freq and less money. That's the only thing I noticed
that seemed kinda strange.
About the AMD vs Intel, the only things I can find are that Opterons
consume much less power (up to 40% less) and Xeons perform around 10%
better for equal clock speeds. I'll poke around for more info.
You did go right on the SAS drives - the key I think is SCSI like
performance without the cost, aka the drives are 10K rpm.
You did go right on the SAS drives - the key I think is SCSI like
performance without the cost, aka the drives are 10K rpm.
From a laymans perspective who uses cheap big SATA drives, it seems
weird to request such small SAS drives (<200GB).
Ah, yes but that's because the goal was fast service of small amounts of
data. The impression I get from the others is that they aren't looking
to build a storage device, they want quick and reliable service.
We did have a previous quote with 300GB drives. Also note I think the
plan is for RAID 5 or 6 so the drives would be combined via
stripes/mirrors to be a very fast 500GB ish drive(rough estimate) where
any 1-2 drives could die and the data would still be safe.
If we also need space to house TB of data that should be a separate
server spec'd for storage space.
I've reworked another quote, matching more closely the original quote,
but with more RAM and disk. Our approved budget is $22k, so I'll assume
2 machines at ~10k each. More comments/concerns? I can scale back the
RAM and disk to line closer to the earlier quote if desired.
Missing anything?
Tyler
On 26/01/10 12:25 PM, Tyler Mitchell (OSGeo) wrote:
After chatting with Howard, I took a quick stab at spec'ing out a server
again just to get the ball rolling. I'm sure it's not perfect, so holler.
A couple things to note or uncertainties:
* AMD vs Intel ?
* I'm not familiar with SAS drives and RAID stuff much in general, so
not sure if I choose things properly there.
* Seemed like some 2U options didn't allow as many drives, but maybe 4U
is not needed?
* OSL prefers Dell, so the added cost of going with them will encourage
expedient service from OSL and in their relationship with Dell.
* Not sure if OSL can get a better deal.
Comments, changes? If anyone else is interested in specing out
something else, I won't be offended
Yes 72GB of ram may be a little overkill for the number of processor
cores available. 32 or 48 is probably quite reasonable. What are we
doing about offsite backup or is OSL handling that. The money save on
the ram from the machines should maybe be put towards a good offsite backup?
Alex
Tyler Mitchell (OSGeo) wrote:
I've reworked another quote, matching more closely the original quote,
but with more RAM and disk. Our approved budget is $22k, so I'll assume
2 machines at ~10k each. More comments/concerns? I can scale back the
RAM and disk to line closer to the earlier quote if desired.
Missing anything?
Tyler
On 26/01/10 12:25 PM, Tyler Mitchell (OSGeo) wrote:
After chatting with Howard, I took a quick stab at spec'ing out a server
again just to get the ball rolling. I'm sure it's not perfect, so holler.
A couple things to note or uncertainties:
* AMD vs Intel ?
* I'm not familiar with SAS drives and RAID stuff much in general, so
not sure if I choose things properly there.
* Seemed like some 2U options didn't allow as many drives, but maybe 4U
is not needed?
* OSL prefers Dell, so the added cost of going with them will encourage
expedient service from OSL and in their relationship with Dell.
* Not sure if OSL can get a better deal.
Comments, changes? If anyone else is interested in specing out
something else, I won't be offended
Yes 72GB of ram may be a little overkill for the number of processor
cores available. 32 or 48 is probably quite reasonable. What are we
doing about offsite backup or is OSL handling that. The money save on
the ram from the machines should maybe be put towards a good offsite backup?
Perhaps backup can be maintained using telascience. Or what other
options come to mind?
On Jan 28, 2010, at 4:52 PM, Tyler Mitchell (OSGeo) wrote:
Alex Mandel wrote:
Yes 72GB of ram may be a little overkill for the number of processor
cores available. 32 or 48 is probably quite reasonable. What are we
doing about offsite backup or is OSL handling that. The money save on
the ram from the machines should maybe be put towards a good offsite backup?
Perhaps backup can be maintained using telascience. Or what other
options come to mind?
Our bandwidth costs at OSL and TelaScience both inbound and outbound are essentially zero, right? I don't see any reason why that isn't still our offsite home.
I've reworked another quote, matching more closely the original quote,
but with more RAM and disk. Our approved budget is $22k, so I'll assume
2 machines at ~10k each. More comments/concerns? I can scale back the
RAM and disk to line closer to the earlier quote if desired.
Tyler,
As I recall the motion at the board level, 15K was approved for what would
likely be two servers. Where does the 22K number come from?
Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up | Frank Warmerdam, warmerdam@pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush | Geospatial Programmer for Rent
I've reworked another quote, matching more closely the original quote,
but with more RAM and disk. Our approved budget is $22k, so I'll assume
2 machines at ~10k each. More comments/concerns? I can scale back the
RAM and disk to line closer to the earlier quote if desired.
Tyler,
As I recall the motion at the board level, 15K was approved for what would
likely be two servers. Where does the 22K number come from?
Oops took it from the agenda from the board meeting, not from the final
motion. In the end the quote I got went down to ~17k for both.
I've reworked another quote, matching more closely the original quote,
but with more RAM and disk. Our approved budget is $22k, so I'll assume
2 machines at ~10k each. More comments/concerns? I can scale back the
RAM and disk to line closer to the earlier quote if desired.
Tyler,
As I recall the motion at the board level, 15K was approved for what would
likely be two servers. Where does the 22K number come from?
Oops took it from the agenda from the board meeting, not from the final
motion. In the end the quote I got went down to ~17k for both.
I didn't look into the quotes and the server requirements in detail, but for servers, I personally tend look for the best price/performance ratio (usually in the 3k$-5k$ range) and get multiple lower spec machines instead of a single higher end machine. In addition to giving you more processing power overall for your money, that gives you redundancy, and reduces the chances of a single service taking the whole infrastructure down.
Of course there are different schools of thought on that front, and I don't want to enter a religious debate, but I thought I'd share my point of view anyway.
I've reworked another quote, matching more closely the original quote,
but with more RAM and disk. Our approved budget is $22k, so I'll
assume
2 machines at ~10k each. More comments/concerns? I can scale back the
RAM and disk to line closer to the earlier quote if desired.
Tyler,
As I recall the motion at the board level, 15K was approved for what
would
likely be two servers. Where does the 22K number come from?
Oops took it from the agenda from the board meeting, not from the final
motion. In the end the quote I got went down to ~17k for both.
I didn't look into the quotes and the server requirements in detail, but
for servers, I personally tend look for the best price/performance ratio
(usually in the 3k$-5k$ range) and get multiple lower spec machines
instead of a single higher end machine. In addition to giving you more
processing power overall for your money, that gives you redundancy, and
reduces the chances of a single service taking the whole infrastructure
down.
Of course there are different schools of thought on that front, and I
don't want to enter a religious debate, but I thought I'd share my point
of view anyway.
I think at this point we've found good servers and can easily make the
price target with slight adjustments to the hard drives or ram. Those
quotes were with 500GBx8 drives and 72GB of ram per machine.
That's 3.5 TB RAID 5 and 3 TB RAID 6 compared with the original quotes
using 146 GB drives 1 TB, 876 GB
Moving down to 250 GB drives would probably cut $2-3 K and leave us with
1.7 TB or 1.46 TB. Or we could drop from 72GB or ram to 64GB of ram and
easily save the same amount.
Note all calculations were per server, so we are talking 2 TB across
both at minimum and up to 7 with both using the bigger of the drive options.
I'm sorry, I was not clear in my previous email. I knew about the plan to go with two servers and virtualization (and I am +10 on the need for virtualization on the next set of servers BTW), but I didn't want to get into specific details, I just wanted to share some general thoughts.
What I meant was that instead of two 7.5k$ machines (15k$) we could possibly get more power, storage and redundancy out of a cluster of four 3k$ machines (12k$), by going with (cheaper) SATA disks and sticking to 16-32GB RAM per box (using 2-4GB DIMMs) instead of going with the more expensive 8GB or 16GB DIMMs.
About a year ago we could get a SUN Fire X2200 server (2x Quad-Core AMD Opteron Model 2376, 2.3 GHz) with 8GB RAM and 2x 750GB HD for 2800$. I checked on SUN's site and the x2200 is discontinued now, but we could probably get even more power for the same price today.
My understanding is that many of our problems are caused by software related (e.g. bots or (mis)configured services taking down the servers), and my fear is that no matter how big the machine we get, there will always be something to take it down anyway. I don't claim to have all the answers, but my thinking is that it is not only hardware, but mostly virtualization that will help here by allowing us to set some limits on each VM and prevent a single service from taking down a complete box, while we work on trying to figure out the solution to each specific configuration issue that arises.
On Mon, Feb 01, 2010 at 11:26:56PM -0500, Daniel Morissette wrote:
What I meant was that instead of two 7.5k$ machines (15k$) we could
possibly get more power, storage and redundancy out of a cluster of four
3k$ machines (12k$), by going with (cheaper) SATA disks and sticking to
16-32GB RAM per box (using 2-4GB DIMMs) instead of going with the more
expensive 8GB or 16GB DIMMs.
I agree partially since nowadays you're already getting pretty nice
boxes for small money, for example at Sun Microsystems (or Dell,
whereas Sun would certainly be my personal favourite).
But please don't use cheap disks. Low latency at the disk subsystem is
a top priority for setups with pretty much 'random' load like here,
even more when you're going for virtualization, and making a bad
decision here is going to hurt you for years - without having much of a
handle to fix it.
Therefore I'd say that 10k or 15k SAS disks and no RAID6 !! (have you
ever seen a fast as well as reasonably priced RAID6 controller ?) is
almost a must.
Cheers,
Martin.
--
Unix _IS_ user friendly - it's just selective about who its friends are !
--------------------------------------------------------------------------
BTW, whoever is maintaining the Google spreadsheet, please count me in
for the MediaWiki on 'osgeo2', I've already been the one who set up the
current instance. And don't put that many MySQL databases there
In fact I think a single PostgreSQL instance for the Wiki stuff should
be sufficient.
Cheers,
Martin.
--
Unix _IS_ user friendly - it's just selective about who its friends are !
--------------------------------------------------------------------------
On Mon, Feb 01, 2010 at 11:26:56PM -0500, Daniel Morissette wrote:
What I meant was that instead of two 7.5k$ machines (15k$) we could
possibly get more power, storage and redundancy out of a cluster of four
3k$ machines (12k$), by going with (cheaper) SATA disks and sticking to
16-32GB RAM per box (using 2-4GB DIMMs) instead of going with the more
expensive 8GB or 16GB DIMMs.
I agree partially since nowadays you're already getting pretty nice
boxes for small money, for example at Sun Microsystems (or Dell,
whereas Sun would certainly be my personal favourite).
But please don't use cheap disks. Low latency at the disk subsystem is
a top priority for setups with pretty much 'random' load like here,
even more when you're going for virtualization, and making a bad
decision here is going to hurt you for years - without having much of a
handle to fix it.
Therefore I'd say that 10k or 15k SAS disks and no RAID6 !! (have you
ever seen a fast as well as reasonably priced RAID6 controller ?) is
almost a must.
Cheers,
Martin.
We are looking at either 10k or 15k SAS, no less.
Do you have any recommendations on a fast RAID6 controller.
RAID 5 does seem like a better choice since speed has been one of our
biggest issue, however the risk to the data concerns me.
This issue of disk i/o is actually why SATA is out of the running.
(Anyone know if the current systems have RAID x?)
On the current system quotes - we will opt for the Perc H700 to get the
6 GB/s throughput - this card can do most RAID configurations, still
looking for benchmarks. So it looks like we can make that decision after
the purchase.
I will have to check if the hosting cost for 3 or 4 machines is going to
be more than 2 machines. Power and people time can easily surpass the
difference in the cost of the machines in a couple of months. I would
also note that managing that many more physical machines is something to
be concerned about.
That said 3 servers might be a good option with the 3rd machine being
spec'd either as a storage device with slower drives for the less
critical apps/backup or handling things like LDAP with the other 2
servers doing the grunt work.
On Mon, Feb 01, 2010 at 11:26:56PM -0500, Daniel Morissette wrote:
What I meant was that instead of two 7.5k$ machines (15k$) we could
possibly get more power, storage and redundancy out of a cluster of four
3k$ machines (12k$), by going with (cheaper) SATA disks and sticking to
16-32GB RAM per box (using 2-4GB DIMMs) instead of going with the more
expensive 8GB or 16GB DIMMs.
I agree partially since nowadays you're already getting pretty nice
boxes for small money, for example at Sun Microsystems (or Dell,
whereas Sun would certainly be my personal favourite).
But please don't use cheap disks. Low latency at the disk subsystem is
a top priority for setups with pretty much 'random' load like here,
even more when you're going for virtualization, and making a bad
decision here is going to hurt you for years - without having much of a
handle to fix it.
Therefore I'd say that 10k or 15k SAS disks and no RAID6 !! (have you
ever seen a fast as well as reasonably priced RAID6 controller ?) is
almost a must.
Cheers,
Martin.
We are looking at either 10k or 15k SAS, no less.
Do you have any recommendations on a fast RAID6 controller.
RAID 5 does seem like a better choice since speed has been one of our
biggest issue, however the risk to the data concerns me.
This issue of disk i/o is actually why SATA is out of the running.
(Anyone know if the current systems have RAID x?)
On the current system quotes - we will opt for the Perc H700 to get the
6 GB/s throughput - this card can do most RAID configurations, still
looking for benchmarks. So it looks like we can make that decision after
the purchase.
I will have to check if the hosting cost for 3 or 4 machines is going to
be more than 2 machines. Power and people time can easily surpass the
difference in the cost of the machines in a couple of months. I would
also note that managing that many more physical machines is something to
be concerned about.
That said 3 servers might be a good option with the 3rd machine being
spec'd either as a storage device with slower drives for the less
critical apps/backup or handling things like LDAP with the other 2
servers doing the grunt work.
