[SAC] Temp disable of some sites on Projects

Committees SAC mailing list
1

We're still having I/O issues. Temp hack by Hamish and me to get some
stuff up:

The following sites are currently disabled:
projects:/etc/apache2/sites-enabled# a2dissite tilecache
projects:/etc/apache2/sites-enabled# a2dissite
community-review.foss4g.org.conf
projects:/etc/apache2/sites-enabled# a2dissite
community-review.foss4g.org.conf
projects:/etc/apache2/sites-enabled# a2dissite grasswiki.osgeo.org
projects:/etc/apache2/sites-enabled# a2dissite grassold.osgeo.org

Apache is going to be throttled to a lower number of processes 60 ( I
think it was 200)

Mapserver sphinx-build is being changed to once per day (we should look
to building it else-where and copying it over).

Selective log filtering is an option too, but not done yet.
http://2bits.com/drupal-performance/reducing-size-and-io-load-apaches-web-server-log-files.html

Next steps are to get osuosl to increase ram allocation, and verify VM
cache settings are optimum. After ram change we'd like projects to start
using memcache or apache inmemory caching where useful.

Long term (next few days) OSUOSL says we need replacement raid card
batteries. This could be affecting performance.
http://www.amazon.com/Dell-U8735-NU209-Battery-PowerEdge/dp/B0045ZNJJU
It's going to run ~$100 for 2 (both servers need it), depending on
shipping. Faster shipping encouraged. Can someone help Daniel get this
done quickly or offer to wait for reimbursement?

Thanks,
Alex

2

On 14-04-04 4:38 AM, Alex Mandel wrote:

Long term (next few days) OSUOSL says we need replacement raid card
batteries. This could be affecting performance.
http://www.amazon.com/Dell-U8735-NU209-Battery-PowerEdge/dp/B0045ZNJJU
It's going to run ~$100 for 2 (both servers need it), depending on
shipping. Faster shipping encouraged. Can someone help Daniel get this
done quickly or offer to wait for reimbursement?

Hi Alex,

I will order them now and have them shipped to OSUOSL with the same contact info as before. I will forward you the order confirmation in a separate email.

P.S. Please explicitly CC my email address when you need my attention for things like this to make sure I see the email since I do not read all SAC mails.

--
Daniel Morissette
T: +1 418-696-5056 #201
http://www.mapgears.com/
Provider of Professional MapServer Support since 2000

3

On 2014-04-04, 5:38 AM, Alex Mandel wrote:

Mapserver sphinx-build is being changed to once per day (we should look
to building it else-where and copying it over).

Sorry about that. I've reported this to the whole MapServer dev team,
we'll look at changing the process as you say.

-jeff

4

On Fri, Apr 04, 2014 at 01:38:48AM -0700, Alex Mandel wrote:

Long term (next few days) OSUOSL says we need replacement raid card
batteries. This could be affecting performance.

Definitely. Many HW RAID-controllers disable write-back cache when
battery backup is unavailable - this may result in a huge performance
it,

  Martin.
--
Unix _IS_ user friendly - it's just selective about who its friends are !
--------------------------------------------------------------------------

5

On 04/04/2014 07:24 AM, Jeff McKenna wrote:

On 2014-04-04, 5:38 AM, Alex Mandel wrote:

Mapserver sphinx-build is being changed to once per day (we should look
to building it else-where and copying it over).

Sorry about that. I've reported this to the whole MapServer dev team,
we'll look at changing the process as you say.

-jeff

While your at it, the logs from the builds need to be cleaned up, zipped
or configured with logrotate.

We can discuss other options, since Mapserver isn't the only project
with Sphinx build needs.

Thanks,
Alex

6

Update,

Thanks to Grant from OSM (Firefishy) I got some performance tuning tips
today.

echo 4096 | sudo tee /sys/block/vda/queue/read_ahead_kb ; echo noop |
sudo tee /sys/block/vda/queue/scheduler

Had OSUOSL double the ram (since Old backup VM is offline).

And played some more with Apache conf:
BufferedLogs (uses ram to queue batches of log writing)
Maxclients is now up to 250

Installed php-apc a caching tool for php.

Does anyone recall which if any sites had memory leak issues? And is
that why MaxRequestPerChild is low?

Performance seems decent now, and I'll start turning on sites that are
still off one by one later, checking after each for performance.

FYI, batteries for the raid cards are in the mail.

Thanks,
Alex

On 04/04/2014 01:38 AM, Alex Mandel wrote:

We're still having I/O issues. Temp hack by Hamish and me to get some
stuff up:

The following sites are currently disabled:
projects:/etc/apache2/sites-enabled# a2dissite tilecache
projects:/etc/apache2/sites-enabled# a2dissite
community-review.foss4g.org.conf
projects:/etc/apache2/sites-enabled# a2dissite
community-review.foss4g.org.conf
projects:/etc/apache2/sites-enabled# a2dissite grasswiki.osgeo.org
projects:/etc/apache2/sites-enabled# a2dissite grassold.osgeo.org

Apache is going to be throttled to a lower number of processes 60 ( I
think it was 200)

Mapserver sphinx-build is being changed to once per day (we should look
to building it else-where and copying it over).

Selective log filtering is an option too, but not done yet.
http://2bits.com/drupal-performance/reducing-size-and-io-load-apaches-web-server-log-files.html

Next steps are to get osuosl to increase ram allocation, and verify VM
cache settings are optimum. After ram change we'd like projects to start
using memcache or apache inmemory caching where useful.

Long term (next few days) OSUOSL says we need replacement raid card
batteries. This could be affecting performance.
http://www.amazon.com/Dell-U8735-NU209-Battery-PowerEdge/dp/B0045ZNJJU
It's going to run ~$100 for 2 (both servers need it), depending on
shipping. Faster shipping encouraged. Can someone help Daniel get this
done quickly or offer to wait for reimbursement?

Thanks,
Alex
_______________________________________________
Sac mailing list
Sac@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/sac

7

Alex wrote:

Mapserver sphinx-build is being changed to once per day (we should look
to building it else-where and copying it over).

Jeff wrote:

Sorry about that. I've reported this to the whole MapServer dev team,
we'll look at changing the process as you say.

for your consideration, I've added a commented out (completely untested)
test to the start of the /osgeo/mapserver.org/scripts/build-all-docs.sh
script, as follows:

# TODO: (ie untested)
#
#if [ `pgrep -c -f 'mapserver.org/scripts/build_docs.sh'` -gt 0 ] ; then
# echo "Sphinx docs build job for mapserver already running. Aborting."
# exit 1
#fi
#

For now I've disabled the twice-a-day sphinx doc build for the livedvd
on adhoc. Zillions of tiny files take their toll...

Hamish