FYI - I couldn't load any osgeo1 web sites a few minutes ago. I logged in (very slowly) with ssh and found that there was no available swap space (via top). I restarted httpd and it seemed to fix the problem. Swap availability is back to 1850828k free.
Tyler
Tyler Mitchell (OSGeo) wrote:
FYI - I couldn't load any osgeo1 web sites a few minutes ago. I logged in (very slowly) with ssh and found that there was no available swap space (via top). I restarted httpd and it seemed to fix the problem. Swap availability is back to 1850828k free.
Tyler,
This has occurred several times through the day. It seems something is
making some sort of huge request by http. When we last ran into this
problem it was huge changeset request from svn via trac. I didn't dig
into it today to see if it was the same thing. It's *really* a pain
in the ass for me to do the log ferret work on stuff like this due to
my high latency network access, so I'm hoping others will do some of this.
Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up | Frank Warmerdam, warmerdam@pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush | President OSGeo, http://osgeo.org
I just did a quick check of the logs and it may have been a few things.
the httpd server was restart around 15:54 yesterday by Tyler around that time
trac was doing a changeset in ossim
multiple robots where spidering, msnbot, slurp, twiciler, and teoma
the teoma (ask jeeves) robot was crawling through our mail archives
65.214.45.100 - - [04/Feb/2008:15:53:48 -0500] "GET /pipermail/grass-commit/2001-May/000154.html HTTP/1.0" 404 263 "-" "Mozilla/5.0 (compatible; Ask Jeeves/Teoma; +http://about.ask.com/en/docs/about/webmasters.shtml)
i suspect it was probably the combo, that pushed our load up.
shawn
"On 4-Feb-08, at 7:31 PM, Frank Warmerdam wrote:
Tyler Mitchell (OSGeo) wrote:
FYI - I couldn't load any osgeo1 web sites a few minutes ago. I logged in (very slowly) with ssh and found that there was no available swap space (via top). I restarted httpd and it seemed to fix the problem. Swap availability is back to 1850828k free.
Tyler,
This has occurred several times through the day. It seems something is
making some sort of huge request by http. When we last ran into this
problem it was huge changeset request from svn via trac. I didn't dig
into it today to see if it was the same thing. It's *really* a pain
in the ass for me to do the log ferret work on stuff like this due to
my high latency network access, so I'm hoping others will do some of this.Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up | Frank Warmerdam, warmerdam@pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush | President OSGeo, http://osgeo.org_______________________________________________
Sac mailing list
Sac@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/sac
sbarnes wrote:
I just did a quick check of the logs and it may have been a few things.
the httpd server was restart around 15:54 yesterday by Tyler around that time
trac was doing a changeset in ossim
multiple robots where spidering, msnbot, slurp, twiciler, and teomathe teoma (ask jeeves) robot was crawling through our mail archives
65.214.45.100 - - [04/Feb/2008:15:53:48 -0500] "GET /pipermail/grass-commit/2001-May/000154.html HTTP/1.0" 404 263 "-" "Mozilla/5.0 (compatible; Ask Jeeves/Teoma; +http://about.ask.com/en/docs/about/webmasters.shtml)
i suspect it was probably the combo, that pushed our load up.
Shawn,
Can you find any additional particulars on the ossim changeset request?
I stress that in the past the problem was *entirely* caused by huge
changeset requests from some spiders. One big changeset request causes
massive swapping as it is assembled in RAM and then the load average
spikes from normal other activity because nothing is getting completed
due to io contention.
We have a "spider trap" laid to try and identify spiders that are roaming
through trac changesets despite the robots.txt request not too. If the
spider hasn't fallen into the trap yet, but we can identify IP#'s we can
just add them to /var/www/trac/forbidden_ips.txt to block them.
I don't think we need to sweat about spiders in legal areas like the mailing
list archives.
Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up | Frank Warmerdam, warmerdam@pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush | President OSGeo, http://osgeo.org