On Thu, Feb 2, 2012 at 10:03 AM, Hamish <hamish_b@yahoo.com> wrote:
ha! you'll enjoy this fix:
sudo apt-get install mysql-server
Wow.
The upgrade procedure of Debian is really convincing 
Fine, the GRASS Wiki is back, too...
Markus
OK, projectsVM is back online but the Apache
performance is terrible. But load average: 0.08.
I have already restarted the apache daemon.
Suggestions?
Markus
On Thu, Feb 2, 2012 at 10:22 AM, Markus Neteler <neteler@osgeo.org> wrote:
OK, projectsVM is back online but the Apache
performance is terrible. But load average: 0.08.I have already restarted the apache daemon.
Despite that I found
[Thu Feb 02 01:38:12 2012] [error] ap_proxy_connect_backend disabling
worker for (spatialreference.org)
[Thu Feb 02 01:38:16 2012] [error] proxy: HTTP: disabled connection
for (spatialreference.org)
[Thu Feb 02 01:38:23 2012] [error] proxy: HTTP: disabled connection
for (spatialreference.org)
[Thu Feb 02 01:38:29 2012] [error] proxy: HTTP: disabled connection
for (spatialreference.org)
[Thu Feb 02 01:38:29 2012] [error] proxy: HTTP: disabled connection
for (spatialreference.org)
...
No idea about this, the settings here should be as before
/etc/apache2/mods-enabled/proxy.conf
(compare
/backup_projects_etc/apache2/mods-enabled/proxy.conf
)
Markus
Hamish wrote:
> ha! you'll enjoy this fix:
> sudo apt-get install mysql-serverWow.
The upgrade procedure of Debian is really convincing
"mysql-server" is a dummy package which just depends on the latest
version of mysql-server-*.
I'm pretty sure what happened is that at the time of the original
install that package wasn't installed, and the deeper mysql-server-5.0
one was directly installed instead. so the upgrade knew that -5.0 wasn't
compatible with the new mysql libraries and it had to be removed*, but
because the generic "task" package wasn't installed it didn't know we
wanted a newer one.
[*] usually it complains a lot about that, but 'dist-upgrade' specifically
makes those complaints non-fatal and just goes for it.
suggestion for other VMs: make sure the "mysql-server" package is installed
if the "mysql-server-5.0" one is now.
see also /usr/share/doc/mysql-server-5.1/NEWS.Debian.gz
and look for "Table upgrade required" warnings in /var/log/daemon.log
Fine, the GRASS Wiki is back, too...
great. (fwiw I notice the oholo image on the wiki front-page isn't working
anymore, and it seems to be really struggling to load sub-pages)
thanks,
Hamish
On Thu, Feb 02, 2012 at 10:45:29AM +0100, Markus Neteler wrote:
[Thu Feb 02 01:38:12 2012] [error] ap_proxy_connect_backend disabling worker for (spatialreference.org)
[Thu Feb 02 01:38:16 2012] [error] proxy: HTTP: disabled connection for (spatialreference.org)
[Thu Feb 02 01:38:23 2012] [error] proxy: HTTP: disabled connection for (spatialreference.org)
[Thu Feb 02 01:38:29 2012] [error] proxy: HTTP: disabled connection for (spatialreference.org)
[Thu Feb 02 01:38:29 2012] [error] proxy: HTTP: disabled connection for (spatialreference.org)
Does anyone know why this Apache is proxying to a copy of itself behind
a different port on the same machine ?
Martin.
--
Unix _IS_ user friendly - it's just selective about who its friends are !
--------------------------------------------------------------------------
On Feb 2, 2012, at 1:39 PM, ext Martin Spott wrote:
On Thu, Feb 02, 2012 at 10:45:29AM +0100, Markus Neteler wrote:
[Thu Feb 02 01:38:12 2012] [error] ap_proxy_connect_backend disabling worker for (spatialreference.org)
[Thu Feb 02 01:38:16 2012] [error] proxy: HTTP: disabled connection for (spatialreference.org)
[Thu Feb 02 01:38:23 2012] [error] proxy: HTTP: disabled connection for (spatialreference.org)
[Thu Feb 02 01:38:29 2012] [error] proxy: HTTP: disabled connection for (spatialreference.org)
[Thu Feb 02 01:38:29 2012] [error] proxy: HTTP: disabled connection for (spatialreference.org)Does anyone know why this Apache is proxying to a copy of itself behind
a different port on the same machine ?
The software that runs spatialreference.org was regularly killing the apache
server it was running under. Rather than let it just continue to do that,
I moved it to a different server on the same machine (where when it dies,
at least it only kills itself, instead of everything else too.)
This isn't ideal, but it's better than spatialreference.org killing
everything on the server.
-- Chris
Martin.
--
Unix _IS_ user friendly - it's just selective about who its friends are !
--------------------------------------------------------------------------
_______________________________________________
Sac mailing list
Sac@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/sac
While it seems that all web servers on the projectsVM are back,
the apache performance (while the CPUs are lazy...) was very low.
I have tweaked the number of workers etc in apache2.conf, now
it seems to be fast again.
cheers
Markus
Two observations:
/var/log/apache2/
-> openlayers.log*
consumes many GB of space due to broken links. Perhaps some
of them could be fixed?
projects:/var/log/apache2# tail 2010.foss4g.org-access_log
58.22.135.104 - - [05/Feb/2012:06:28:56 -0800] "GET
http://ads.smowtion.com/ad.js?s=1953976&z=300x250 HTTP/1.0" 200 1459
"http://webhealthdoctor.com/Topical_Nail_Fungus_Treatment.htm"
"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
61.147.112.176 - - [05/Feb/2012:06:28:56 -0800] "GET
http://ad.yieldmanager.com/st?ad_type=ad&ad_size=300x250§ion=2739442
HTTP/1.0" 200 4511 "http://www.fcxo.com" "Mozilla/4.76 [en] (Win98;
U)"
77.120.196.2 - - [05/Feb/2012:06:28:55 -0800] "POST
http://pr.lifecs.sumy.ua/check.php HTTP/1.1" 200 759
"http://75c58d3615/" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT
5.1; SV1)"
61.147.112.176 - - [05/Feb/2012:06:28:56 -0800] "GET
http://ad.yieldmanager.com/st?ad_type=ad&ad_size=728x90§ion=2581834
HTTP/1.0" 200 4510 "http://www.cleanfox.com" "Mozilla/5.0 (Windows; U;
Windows NT 5.1; en-US; rv:1.7) Gecko/20040707 Firefox/0.8"
213.112.225.1 - - [05/Feb/2012:06:28:56 -0800] "POST
http://chek.zennolab.com/proxy.php HTTP/1.1" 200 351 "RefererString"
"Mozilla / 4.0"
216.24.193.224 - - [05/Feb/2012:06:28:56 -0800] "GET
http://ad.adserverplus.com/st?ad_type=ad&ad_size=160x600§ion=2690053
HTTP/1.0" 200 4511
"http://moonhealthylive.com/index.php?option=com_mailto&tmpl=component&link=8751d9c11d7ae8faabbb99c1328027612599f75e"
"Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:2.2a1pre) Gecko/20110324
Firefox/4.2a1pre"
...
--> is the conference server full of spam? 2010.foss4g.org-access_log* are
also extremely big and full of junk.
Markus
On Fri, Feb 3, 2012 at 10:59 PM, Markus Neteler <neteler@osgeo.org> wrote:
While it seems that all web servers on the projectsVM are back,
the apache performance (while the CPUs are lazy...) was very low.I have tweaked the number of workers etc in apache2.conf, now
it seems to be fast again.
... only temporarily
I have taken liberty to stop srorgapache, now the performance is sort of back.
Perhaps srorgapache should go to a different machine since it is slowing
down foss4g, gdal, geos, geotools, grass, grass wiki, mapbender, mapserver,
openlayers, remotesensing.org, and tilecache.
?
Markus
On 02/06/2012 01:44 AM, Markus Neteler wrote:
On Fri, Feb 3, 2012 at 10:59 PM, Markus Neteler <neteler@osgeo.org> wrote:
While it seems that all web servers on the projectsVM are back,
the apache performance (while the CPUs are lazy...) was very low.I have tweaked the number of workers etc in apache2.conf, now
it seems to be fast again.... only temporarily
I have taken liberty to stop srorgapache, now the performance is sort of back.
Perhaps srorgapache should go to a different machine since it is slowing
down foss4g, gdal, geos, geotools, grass, grass wiki, mapbender, mapserver,
openlayers, remotesensing.org, and tilecache.
?Markus
_______________________________________________
Sac mailing list
Sac@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/sac
It's quite a popular site so we can't leave it off indefinitely (It
appears to be on again). I did manage to figure out how to run the site
via gunicorn instead of apache but in testing it's seems the proxy by
the main apache is what's causing the most slow down and possibly load
on the main apache. I'm not sure the load on the cpu/ram is the culprit,
but I do have an idea to test it.
In the meantime I'd like to run an experiment for a week running sr.org
on gunicorn instead of the srorg apache to see if things behave any
better? Any objections, concerns or questions?
So maybe moving to another server is the answer, but we'd need to
identify what server would be good for that.
The long term interesting idea with running wsgi stuff behind gunicorn
or uWSGI is the potential to swap out apache for nginx (when needed) and
the possibility of using supervisord to keep servers prone to crashing
from requiring human intervention to restart web services.
Thanks,
Alex
On Tue, Feb 7, 2012 at 5:08 AM, Alex Mandel <tech_dev@wildintellect.com> wrote:
...
It's quite a popular site so we can't leave it off indefinitely
Sure.
(It appears to be on again). I did manage to figure out how to run the site
via gunicorn instead of apache but in testing it's seems the proxy by
the main apache is what's causing the most slow down and possibly load
on the main apache. I'm not sure the load on the cpu/ram is the culprit,
but I do have an idea to test it.In the meantime I'd like to run an experiment for a week running sr.org
on gunicorn instead of the srorg apache to see if things behave any
better? Any objections, concerns or questions?
Sounds good.
However, I noticed a few things:
- 2010.foss4g.org was active there but the true IP is a different one! so
I took it off
- likewise for community-review.foss4g.org which doesn't seem to exists
More importantly: the site is continuously bombed by Chinese IPs pointing
to ad.yieldmanager.com etc. Try the following:
tail -f /var/log/apache2/docs.geotools.org-access_log
So in fact the server performance is impressive, all is eaten by this junk!!
Any chance to block the IPs bombing the projectsVM with requests?
They always attack the alphabetically first server from
/etc/apache2/sites-enabled/ .
Markus
While trying to figure out why SSH public key authgentication doesn't
work for me on the "adhoc" and the "projects" VM after the dist-
upgrade, I checked the status of the installed packages - starting on
the "adhoc" VM (just because it was dist-upgraded earlier).
On both VM's there are still quite a few "Lenny"-packages which have
not been migrated/upgraded properly, but this mess really looks scary:
adhoc:~# ps -ef|grep bin\/postgres
postgres 1156 1 0 Jan06 ? 00:01:48 /usr/lib/postgresql/8.3/bin/postgres -D /var/lib/postgresql/8.3/main -c config_file=/etc/postgresql/8.3/main/postgresql.conf
jmckenna 7302 1 0 Jan09 ? 00:00:25 /osgeo/mapserver/fgs/bin/postgres -D /osgeo/mapserver/fgs/apps/pgsql/data -p 5432
postgres 17922 1 0 Jan29 ? 00:00:05 /usr/lib/postgresql/8.4/bin/postgres -D /var/lib/postgresql/8.4/main -c config_file=/etc/postgresql/8.4/main/postgresql.conf
Are these people serious !?
Cheers,
Martin.
--
Unix _IS_ user friendly - it's just selective about who its friends are !
--------------------------------------------------------------------------
On 02/07/2012 05:49 AM, Markus Neteler wrote:
On Tue, Feb 7, 2012 at 5:08 AM, Alex Mandel <tech_dev@wildintellect.com> wrote:
...It's quite a popular site so we can't leave it off indefinitely
Sure.
(It appears to be on again). I did manage to figure out how to run the site
via gunicorn instead of apache but in testing it's seems the proxy by
the main apache is what's causing the most slow down and possibly load
on the main apache. I'm not sure the load on the cpu/ram is the culprit,
but I do have an idea to test it.In the meantime I'd like to run an experiment for a week running sr.org
on gunicorn instead of the srorg apache to see if things behave any
better? Any objections, concerns or questions?Sounds good.
However, I noticed a few things:
- 2010.foss4g.org was active there but the true IP is a different one! so
I took it off
- likewise for community-review.foss4g.org which doesn't seem to existsMore importantly: the site is continuously bombed by Chinese IPs pointing
to ad.yieldmanager.com etc. Try the following:tail -f /var/log/apache2/docs.geotools.org-access_log
So in fact the server performance is impressive, all is eaten by this junk!!
Any chance to block the IPs bombing the projectsVM with requests?
They always attack the alphabetically first server from
/etc/apache2/sites-enabled/ .Markus
Anyone a pro at blacklist options? Something that wouldn't just block
all of Southeast asia by IP.
http://perishablepress.com/press/2009/02/03/eight-ways-to-blacklist-with-apaches-mod_rewrite/
It sounds like we can easily block by the referrer url if it's
ad.yieldmanager.com
Thanks,
Alex
On Tue, Feb 7, 2012 at 6:41 PM, Martin Spott <Martin.Spott@mgras.net> wrote:
While trying to figure out why SSH public key authgentication doesn't
work for me on the "adhoc" and the "projects" VM after the dist-
upgrade, I checked the status of the installed packages - starting on
the "adhoc" VM (just because it was dist-upgraded earlier).On both VM's there are still quite a few "Lenny"-packages which have
not been migrated/upgraded properly, but this mess really looks scary:adhoc:~# ps -ef|grep bin\/postgres
postgres 1156 1 0 Jan06 ? 00:01:48 /usr/lib/postgresql/8.3/bin/postgres -D /var/lib/postgresql/8.3/main -c config_file=/etc/postgresql/8.3/main/postgresql.conf
jmckenna 7302 1 0 Jan09 ? 00:00:25 /osgeo/mapserver/fgs/bin/postgres -D /osgeo/mapserver/fgs/apps/pgsql/data -p 5432
postgres 17922 1 0 Jan29 ? 00:00:05 /usr/lib/postgresql/8.4/bin/postgres -D /var/lib/postgresql/8.4/main -c config_file=/etc/postgresql/8.4/main/postgresql.conf
The content is:
postgres@projects: $ psql -l
List of databases
Name | Owner | Encoding
-------------------+-----------+----------
mapbender_2.6 | mapbender | UTF8
mapbender_demo | mapbender | UTF8
mapbender_metador | mapbender | UTF8
postgres | postgres | UTF8
review_foss4g2011 | postgres | UTF8
srorg | crschmidt | UTF8
template0 | postgres | UTF8
template1 | postgres | UTF8
template_postgis | postgres | UTF8
trunk | mapbender | UTF8
Are these people serious !?
It seems the owners do not care too much.
Markus
On Mon, Feb 06, 2012 at 08:08:02PM -0800, Alex Mandel wrote:
On 02/06/2012 01:44 AM, Markus Neteler wrote:
> I have taken liberty to stop srorgapache, now the performance is sort of back.
>
> Perhaps srorgapache should go to a different machine since it is slowing
> down foss4g, gdal, geos, geotools, grass, grass wiki, mapbender, mapserver,
> openlayers, remotesensing.org, and tilecache.
> ?
It's quite a popular site so we can't leave it off indefinitely (It
appears to be on again). I did manage to figure out how to run the site
via gunicorn instead of apache [...]
Oh, this sounds like a valuable progress !
Thanks,
Martin.
--
Unix _IS_ user friendly - it's just selective about who its friends are !
--------------------------------------------------------------------------
On 02/08/2012 02:33 AM, Martin Spott wrote:
On Mon, Feb 06, 2012 at 08:08:02PM -0800, Alex Mandel wrote:
On 02/06/2012 01:44 AM, Markus Neteler wrote:
I have taken liberty to stop srorgapache, now the performance is sort of back.
Perhaps srorgapache should go to a different machine since it is slowing
down foss4g, gdal, geos, geotools, grass, grass wiki, mapbender, mapserver,
openlayers, remotesensing.org, and tilecache.
?It's quite a popular site so we can't leave it off indefinitely (It
appears to be on again). I did manage to figure out how to run the site
via gunicorn instead of apache [...]Oh, this sounds like a valuable progress !
Thanks,
Martin.
Ok gunicorn is running on port 8092, if the test goes well I'm going to
restrict it to localhost so only the apache can see it and proxy it.
To restart gunicorn it's just like any other service /etc/init.d/gunicorn
The config to tweak workers etc is in /etc/gunicorn.d/
I left the srorgapache turned on for now, but swapped it in the
sr.org.conf in /etc/apache/site-available/
To change back just adjust the comments to turn on gunicorn and turn the
old proxy back on.
Thanks,
Alex
PS: I wonder if the proxy was slowing down because it had to resolve the
dns for every request? the new proxy uses 127.0.0.1 instead and seems to
work (possibly faster).
On Tue, Feb 07, 2012 at 06:41:38PM +0100, Martin Spott wrote:
While trying to figure out why SSH public key authgentication doesn't
work for me on the "adhoc" and the "projects" VM after the dist-
upgrade, [...]
Ok, now I know why: Some braindead jerk put the IP of my internet
connection into the /etc/hosts.deny file
Folks, this isn't fun any more, if you'd like to keep me out, simply
tell me and I'll leave !
Martin.
--
Unix _IS_ user friendly - it's just selective about who its friends are !
--------------------------------------------------------------------------
On Wed, Feb 8, 2012 at 8:01 PM, Martin Spott <Martin.Spott@mgras.net> wrote:
On Tue, Feb 07, 2012 at 06:41:38PM +0100, Martin Spott wrote:
While trying to figure out why SSH public key authgentication doesn't
work for me on the "adhoc" and the "projects" VM after the dist-
upgrade, [...]Ok, now I know why: Some braindead jerk put the IP of my internet
connection into the /etc/hosts.deny file
I believe that this is simply the culprit of
/usr/sbin/denyhosts
I got locked out my own server recently by using my own Sparkleshare instance.
Seems that denyhost is oversensitive in Debian squeeze. I eventually removed
it on my site. Since it does not help at all for the ongoing Apache bombing
with junk, remember to try
tail -f /var/log/apache2/docs.geotools.org-access_log
... I just think that we should remove denyhosts.
Best,
Markus
On Wed, Feb 08, 2012 at 09:46:42PM +0100, Markus Neteler wrote:
On Wed, Feb 8, 2012 at 8:01 PM, Martin Spott <Martin.Spott@mgras.net> wrote:
> Ok, now I know why: Some braindead jerk put the IP of my internet
> connection into the /etc/hosts.deny fileI believe that this is simply the culprit of
/usr/sbin/denyhosts
Actually that's pretty unlikely because there's just one single, quite
well maintained machine at this very fixed IP, being equipped with just
the minimum of what is required for an Internet gateway, having just a
single working shell account which permits to log into the OSGeo VM's -
and this particular account is having guaranteed working login into the
OSGeo VM's by using an SSH public key.
Of the five OSGeo VM's which have been updated to Squeeze so far, all
are running the same "denyhosts" configuration but just "adhoc" and
"projects" were denying SSH login from my IP.
Strange, isn't it !?
Cheers,
Martin.
--
Unix _IS_ user friendly - it's just selective about who its friends are !
--------------------------------------------------------------------------
Martin wrote:
> While trying to figure out why SSH public key
> authgentication doesn't
> work for me on the "adhoc" and the "projects" VM after
> the dist-upgrade, [...]Ok, now I know why: Some braindead jerk put the IP of my
internet connection into the /etc/hosts.deny file
hmm, portsentry can do that to you automatically if you hit the
wrong port, but it isn't installed on adhoc currently. fail2ban
is, but it only uses iptables to lock you out temporarily.
..most likely some firewall mechanism gone awry anyhow.
Hamish