On Fri, Feb 16, 2018 at 4:24 PM, Alex M <tech_dev@wildintellect.com> wrote:
On 02/16/2018 03:19 PM, Martin Spott wrote:
==8<---
WRT suggestions concerning "The New Hardware": Quite a few ideas have been
added to the agenda adressing I/O performance on the new machine.
Even though more I/O performance is always nice to have, I really wonder if
hardware limitations in this domain have been identified as current or
upcoming issue in OSGeo's own infrastructure.
==8<---
Cheers,
Martin.
There have been some performance issues with Trac/SVN, though that might
be solved with the updates and tweaks to apache. Download does need more
disk space along with the Foss4g archives.
I'm not sure if you should spend time testing the Debian8/9 upgrades as
all those VMs will be migrated to the new hardware in some fashion be it
new VMs, LXD, or other method. In which case it might be saner to start
with the newest and migrate the data and configuration over.
The migration of all things off OSGeo4 and then OSGeo3(with the new
hardware) will be a big place you can help, along with the SSL related
tickets, etc.
I am agnostic on the Optane card, the other options all seem within the
norm of what we do. I do understand the value of it to add caching
performance, which could remove all sorts of load from ever causing any
issues. Some of the choices are clearly based on mitigating risk on the
SSD drives. If the Optane prevents lots of wear on the disks that seems
worth it.
There does seems to still be plenty of debate on the precise
implementation, etc. But other than lots of ram I'm not sure ZFS
requires anything special. I also don't think this part of the debate
should hold up the purchase. Since it's all done during the install and
setup.
Thanks,
Alex
The rationale behind optimizing the i/o path is based on both the
mnuin data and observed service responsiveness.
Currently, latency appears to be moderate, but peaky, with disk
response times on the order of hundred millisecond being fairly
common. Write latency is generally higher than read (normal), but
occasionally appears to reach unacceptable levels for interactive use.
As load increases, the processors will be I/O bound very quickly if
the storage subsystem fails to provide sufficient read and write
throughput in the short-duration bursts typical of the types of
services provided. The machine will have plenty of ram, and few of the
services provided will have any significant computational overhead,
leaving the storage and network as the only remaining potential
bottlenecks -- and the built in networking hardware can easily handle
the entire available link provided I believe.
The reasoning behind Optane card specifically was twofold:
1 - physical considerations - The selected chassis has a very limited
number of locations to install storage; with 4HDDs in the hotswap bays
and 2SSDs in the remaining storage bays, the only location remaining
is the PCIe slot.
2 - performance and lifetime - Optane memory is both significantly
faster for random IO and more durable in terms of write cycles than
any existing SSD, which are ideal characteristics for a primary read
cache; With proper provisioning, the less expensive workstation grade
900p specified (vs DC P4800X) should have a service life greater than
7 years even with significantly higher loading than at current.
-- An alternative to utilize the available PCIe slot might be
something like this with a mirrored pair of m.2 NVMe drives:
https://www.supermicro.com/products/accessories/addon/AOC-SLG3-2M2.cfm
The hardware decisions are not contingent on the software details so
much as the intended application, so as long as the hardware appears
to be a good match to the services we expect to provide, there is no
reason to delay purchasing until a final software configuration has
been decided.
Take care,
~~~Chris~~~