[SAC] Webextra - internal php connections?

So Webextra has a huge load right now, and I'm trying to figure out why.
A couple of observations:

1. None of the apache site configs write to their own log files, so it's
quite hard to figure out which sites are responsible for the load.

As a standard practice I recommend each site or collection of sites
writes to a separate log file (ie all the foss4g sites can go to one).
We should probably apply this idea to all of our servers.

example:
ErrorLog /var/log/apache2/foss4g-error.log
CustomLog /var/log/apache2/foss4g-access_log combined

2. The one monster apache log is full of
::1 - - [05/Dec/2012:07:47:19 -0800] "OPTIONS * HTTP/1.0" 200 - "-"
"Apache/2.2.9 (Debian) PHP/5.2.6-1+lenny16 with Suhosin-Patch (internal
dummy connection)"

Anyone know what these php internal dummy connections are and why their
might be so many?

Speaking of Foss4g.org sites on webextra, anyone know why there's a
section outside of Virtualhost tags?

Thanks,
Alex

On 12/05/2012 09:30 AM, Alex Mandel wrote:

ErrorLog /var/log/apache2/foss4g-error.log
CustomLog /var/log/apache2/foss4g-access_log combined

Typo in the example, always end in .log (dot not _) on debian otherwise
logrotate won't find it.

ErrorLog /var/log/apache2/foss4g-error.log
CustomLog /var/log/apache2/foss4g-access.log combined

Thanks,
Alex

On Wed, Dec 5, 2012 at 6:34 PM, Alex Mandel <tech_dev@wildintellect.com> wrote:

On 12/05/2012 09:30 AM, Alex Mandel wrote:

ErrorLog /var/log/apache2/foss4g-error.log
CustomLog /var/log/apache2/foss4g-access_log combined

Typo in the example, always end in .log (dot not _) on debian otherwise
logrotate won't find it.

ErrorLog /var/log/apache2/foss4g-error.log
CustomLog /var/log/apache2/foss4g-access.log combined

Probably SAC should just rewrite the configs after waiting a few days?
I doubt that anybody of the site managers will react (sorry for being
pessimistic here)...

Markus

On 12-12-05 11:42 AM, Markus Neteler wrote:

On Wed, Dec 5, 2012 at 6:34 PM, Alex Mandel <tech_dev@wildintellect.com> wrote:

On 12/05/2012 09:30 AM, Alex Mandel wrote:

ErrorLog /var/log/apache2/foss4g-error.log
CustomLog /var/log/apache2/foss4g-access_log combined

Typo in the example, always end in .log (dot not _) on debian otherwise
logrotate won't find it.

ErrorLog /var/log/apache2/foss4g-error.log
CustomLog /var/log/apache2/foss4g-access.log combined

Probably SAC should just rewrite the configs after waiting a few days?
I doubt that anybody of the site managers will react (sorry for being
pessimistic here)...

Markus,

I think Alex has done so now. It looks like the main culprit on webextra
is the VMap Tile generation which was not documented in the wiki. I have
added it in:

   http://wiki.osgeo.org/wiki/WebExtraVM

It is using mapserver and some shapefiles to serve vmap0 tiles, presumably
mostly for openlayers, at the domain:

   vmap0.tiles.osgeo.org

The traffic is variable, but it is not yet clear that it is causing serious
problems for the VM. I think we should consider:

  1) Relaxing the munin load average alert.
  2) See if we can limit tile service traffic in some way to avoid bogging
     down the system serious if traffic continues to grow.
  3) Consider finding a new home for this service (I think it already was
     pushed here from the projects VM).

Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up | Frank Warmerdam, warmerdam@pobox.com
light and sound - activate the windows | http://home.gdal.org/warmerda
and watch the world go round - Rush | Geospatial Software Developer

On Wed, Dec 05, 2012 at 12:46:21PM -0800, Frank Warmerdam wrote:

3) Consider finding a new home for this service (I think it already was
    pushed here from the projects VM).

The "sphere.telascience.org" I use as MapServer for FlightGear Scenery
land cover has an adaption of this VMap0 tile service (the directory
structure is different) - but it's untested. If anyone would be
willing to select and describe a few reliable test cases, we might
consider moving it off the "webextra" VM or at least do some load
balancing.

Cheers,
  Martin.
--
Unix _IS_ user friendly - it's just selective about who its friends are !
--------------------------------------------------------------------------

On 12/05/2012 12:46 PM, Frank Warmerdam wrote:

On 12-12-05 11:42 AM, Markus Neteler wrote:

On Wed, Dec 5, 2012 at 6:34 PM, Alex Mandel
<tech_dev@wildintellect.com> wrote:

On 12/05/2012 09:30 AM, Alex Mandel wrote:

ErrorLog /var/log/apache2/foss4g-error.log
CustomLog /var/log/apache2/foss4g-access_log combined

Typo in the example, always end in .log (dot not _) on debian otherwise
logrotate won't find it.

ErrorLog /var/log/apache2/foss4g-error.log
CustomLog /var/log/apache2/foss4g-access.log combined

Probably SAC should just rewrite the configs after waiting a few days?
I doubt that anybody of the site managers will react (sorry for being
pessimistic here)...

Markus,

I think Alex has done so now. It looks like the main culprit on webextra
is the VMap Tile generation which was not documented in the wiki. I have
added it in:

  http://wiki.osgeo.org/wiki/WebExtraVM

I did for at least the webextra sites. We should email out to all those
using Projects and Adhoc about doing it to their stuff.

Thanks,
Alex

All,

On Wed, Dec 5, 2012 at 9:46 PM, Frank Warmerdam <warmerdam@pobox.com> wrote:
...

The traffic is variable, but it is not yet clear that it is causing serious
problems for the VM. I think we should consider:

1) Relaxing the munin load average alert.

+1

I think that 15 is ok, maybe even 20 since that seems to be typical (to
avoid too many useless notifications).

Markus

On Sat, Dec 8, 2012 at 12:16 AM, Markus Neteler <neteler@osgeo.org> wrote:

All,

On Wed, Dec 5, 2012 at 9:46 PM, Frank Warmerdam <warmerdam@pobox.com> wrote:
...

The traffic is variable, but it is not yet clear that it is causing serious
problems for the VM. I think we should consider:

1) Relaxing the munin load average alert.

+1

I think that 15 is ok, maybe even 20 since that seems to be typical (to
avoid too many useless notifications).

Changed now to

    load.warning 15
    load.critical 25

to reduce the email notification frequency.

Markus

On Sun, Dec 9, 2012 at 7:37 PM, Markus Neteler <neteler@osgeo.org> wrote:

On Sat, Dec 8, 2012 at 12:16 AM, Markus Neteler <neteler@osgeo.org> wrote:

...

Changed now to

    load.warning 15
    load.critical 25

to reduce the email notification frequency.

Bah, still not working:

osgeo.org :: webextra.osgeo.org :: Load average
        WARNINGs: load is 12.70 (outside range [:10]).

What's the trick to make munin avare of it? I restarted munin-node...

Markus

On 12/09/2012 02:34 PM, Markus Neteler wrote:

On Sun, Dec 9, 2012 at 7:37 PM, Markus Neteler <neteler@osgeo.org> wrote:

On Sat, Dec 8, 2012 at 12:16 AM, Markus Neteler <neteler@osgeo.org> wrote:

...

Changed now to

    load.warning 15
    load.critical 25

to reduce the email notification frequency.

Bah, still not working:

osgeo.org :: webextra.osgeo.org :: Load average
        WARNINGs: load is 12.70 (outside range [:10]).

What's the trick to make munin avare of it? I restarted munin-node...

Markus
_______________________________________________
Sac mailing list
Sac@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/sac

I don't think I had the syntax in munin.conf right:
changed from
load.warning to load.load.warning

FYI, munin-node service does not affect munin master which is a cron job.

Thanks,
Alex

On Wed, Dec 05, 2012 at 12:46:21PM -0800, Frank Warmerdam wrote:

I think Alex has done so now. It looks like the main culprit on webextra
is the VMap Tile generation which was not documented in the wiki. I have
added it in:

  http://wiki.osgeo.org/wiki/WebExtraVM

It is using mapserver and some shapefiles to serve vmap0 tiles, presumably
mostly for openlayers, at the domain:

  vmap0.tiles.osgeo.org

I've fixed the setup on "sphere.telascience.org" and changed the DNS
accordingly.
Please test - and let's wait a few days to see how things develop.

Cheers,
  Martin.
--
Unix _IS_ user friendly - it's just selective about who its friends are !
--------------------------------------------------------------------------