[SAC] apache server lockup on osgeo1


Today at about 1:25pm (EST) I observed that web services from osgeo1
stopped responding. I logged in and found the machine was quiet, and
50 httpd processes seemed to be running though not doing anything.

I didn't wait very long to investigate (perhaps unfortunately). I did do
"/usr/sbin/apachectl status" which hung - presumably because it does a
fetch from the servers. /usr/sbin/apachectl restart worked fine.

I would add that the server "filled up" with 50+ requests from DanielM's
"server status check" script after restart. It is unclear whether they might
have played any role in the server lock up problem.

I sure wish I knew what was going on!

If we had evidence that the server was having all it's connections tied
up by some abuser I could put in the place the "max connections per ip#"
that I have put on download.osgeo.org to limit abuse. But currently I'm
not convinced that is the problem.

Note, this is apparently a different problem from what ever it is that causes
whole server lockups requiring remote power cycling.

Best regards,
I set the clouds in motion - turn up | Frank Warmerdam, warmerdam@pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush | President OSGeo, http://osgeo.org