#634: Trouble with awstats on download server
---------------------------+------------------------------------------------
Reporter: tmitchell | Owner: sac@…
Type: defect | Status: new
Priority: major | Milestone:
Component: Systems Admin | Keywords:
---------------------------+------------------------------------------------
Traditional url for stats seems dead, one example:
http://download.osgeo.org/cgi-
bin/awstats.pl?month=09&year=2007&output=main&config=mapguide.download.osgeo.org&framename=index
Checking on the server and the apache2 ScriptAlias seems good:
sites-enabled/000-default: ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
$ ls /usr/lib/cgi-bin/ -l
total 552
-rwxr-xr-x 1 root root 548960 2008-12-10 04:11 awstats.pl
Any ideas? Marking as major since awstats seems completely down to me
since the rebuild of the download server.
--
Ticket URL: <http://trac.osgeo.org/osgeo/ticket/634>
OSGeo <http://www.osgeo.org/>
OSGeo committee and general foundation issue tracker.
#634: Trouble with awstats on download server
---------------------------+------------------------------------------------
Reporter: tmitchell | Owner: sac@…
Type: defect | Status: closed
Priority: major | Milestone:
Component: Systems Admin | Resolution: fixed
Keywords: |
---------------------------+------------------------------------------------
Changes (by tmitchell):
* status: new => closed
* resolution: => fixed
Comment:
Thanks to Alex, I copied the scriptalias parameters for cgi-bin over from
000-default to download.... httpd config file. That seems to have done
the trick. Now how to backload all the past httpd logs which have been
set aside somehow.
--
Ticket URL: <http://trac.osgeo.org/osgeo/ticket/634#comment:1>
OSGeo <http://www.osgeo.org/>
OSGeo committee and general foundation issue tracker.
#634: Trouble with awstats on download server
---------------------------+------------------------------------------------
Reporter: tmitchell | Owner: sac@…
Type: defect | Status: reopened
Priority: major | Milestone:
Component: Systems Admin | Resolution:
Keywords: |
---------------------------+------------------------------------------------
Changes (by tmitchell):
* status: closed => reopened
* resolution: fixed =>
Comment:
Reopening as I struggle to ingest the previous awstats cache from
/osgeo/download/logs. It seems the default file naming is
awstatsmmddyy.txt instead of awstats072010.www.download.osgeo.org.txt, for
example from previous set up. Not sure how to change that. Help! 
--
Ticket URL: <http://trac.osgeo.org/osgeo/ticket/634#comment:2>
OSGeo <http://www.osgeo.org/>
OSGeo committee and general foundation issue tracker.
#634: Trouble with awstats on download server
---------------------------+------------------------------------------------
Reporter: tmitchell | Owner: sac@…
Type: defect | Status: reopened
Priority: major | Milestone:
Component: Systems Admin | Resolution:
Keywords: |
---------------------------+------------------------------------------------
Comment(by hamish):
Hi,
minor wish: could the awstats.pl cron job be run nice'd?
In the pre-osuosl days the awstats processes would dominate the xblade13's
single cpu for 30min-1 hour each day, which tended to slow everything else
down.
Out of curiosity I've restarted my log_cpu.sh 5 minute instrumenting on
xblade13; in the last month before the xblades went down I had stopped it
and posted the following to the SAC list, but it was a few kb over the
size limit so never made it & I never bothered to repost.
{{{
Hi,
fyi, before the Projects VM gets well underway
and with the auth delays now sorted out, I
though I'd wrap up my logging of loads on
xblade13 and 14.
over the period xblade13 had an average load
of 1%, and maxed out the CPU 1.5% of the
time. (5 minute averages, 5 minute sampling
period)
over the period xblade14 had an average load
of 52%, and maxed out the CPU 10.1% of the
time. (5 minute averages, 5 minute sampling
period)
logging has now ceased.
regards,
Hamish
}}}
chart plots can be seen here:
http://bambi.otago.ac.nz/hamish/osgeo/cpuload_5min.png
http://bambi.otago.ac.nz/hamish/osgeo/xblade_cpu_histogram.png
cheers,
Hamish
ps- if you have a choice, "yymmdd" instead of "ddmmyy" is nicer for sorted
directory listings..
--
Ticket URL: <http://trac.osgeo.org/osgeo/ticket/634#comment:3>
OSGeo <http://www.osgeo.org/>
OSGeo committee and general foundation issue tracker.
#634: Trouble with awstats on download server
---------------------------+------------------------------------------------
Reporter: tmitchell | Owner: sac@…
Type: defect | Status: reopened
Priority: major | Milestone:
Component: Systems Admin | Resolution:
Keywords: |
---------------------------+------------------------------------------------
Changes (by jmckenna):
* cc: jmckenna@… (added)
--
Ticket URL: <http://trac.osgeo.org/osgeo/ticket/634#comment:4>
OSGeo <http://www.osgeo.org/>
OSGeo committee and general foundation issue tracker.
#634: Trouble with awstats on download server
---------------------------+------------------------------------------------
Reporter: tmitchell | Owner: sac@…
Type: defect | Status: reopened
Priority: major | Milestone:
Component: Systems Admin | Resolution:
Keywords: |
---------------------------+------------------------------------------------
Comment(by jmckenna):
bump
--
Ticket URL: <http://trac.osgeo.org/osgeo/ticket/634#comment:5>
OSGeo <http://www.osgeo.org/>
OSGeo committee and general foundation issue tracker.
#634: Trouble with awstats on download server
---------------------------+------------------------------------------------
Reporter: tmitchell | Owner: sac@…
Type: defect | Status: reopened
Priority: major | Milestone:
Component: Systems Admin | Resolution:
Keywords: |
---------------------------+------------------------------------------------
Comment(by jmckenna):
Any ideas how we can move forward with this?
--
Ticket URL: <http://trac.osgeo.org/osgeo/ticket/634#comment:6>
OSGeo <http://www.osgeo.org/>
OSGeo committee and general foundation issue tracker.
#634: Trouble with awstats on download server
---------------------------+------------------------------------------------
Reporter: tmitchell | Owner: sac@…
Type: defect | Status: reopened
Priority: major | Milestone:
Component: Systems Admin | Resolution:
Keywords: |
---------------------------+------------------------------------------------
Comment(by neteler):
Did you try to run the setup (again)?
/usr/share/doc/awstats/examples/awstats_configure.pl
--
Ticket URL: <http://trac.osgeo.org/osgeo/ticket/634#comment:7>
OSGeo <http://www.osgeo.org/>
OSGeo committee and general foundation issue tracker.
#634: Trouble with awstats on download server
---------------------------+------------------------------------------------
Reporter: tmitchell | Owner: sac@…
Type: defect | Status: reopened
Priority: major | Milestone:
Component: Systems Admin | Resolution:
Keywords: |
---------------------------+------------------------------------------------
Comment(by wildintellect):
I haven't checked the server yet to see the current situation but here is
the todo list:
1. confirm the configuration for the download.osgeo.org domain is setup
correctly
2. create a cron job to run updates, it might be good to do it every hour
or two to minimize the amount of time/resources, as hamish stated it
should also be niced?(what level?)
3. stick a call to awstats into the logrotate script for apache to make
sure it catches data before it gets rotated out.
http://awstats.sourceforge.net/docs/awstats_faq.html#ROTATE
Now about the old data, best idea is to use a tool like mergelog to
1.combine all the old logs into 1 file,
2.then configure the awstats config to run this file
3.reconfigure awstats to point at the current file and update again
Then it would be up to date with full history. If I come across a better
way to get the history implemented I'll let everyone know, but I won't
have time to try this stuff for a couple of weeks.
--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/634#comment:8>
OSGeo <http://www.osgeo.org/>
OSGeo committee and general foundation issue tracker.
#634: Trouble with awstats on download server
---------------------------+------------------------------------------------
Reporter: tmitchell | Owner: sac@…
Type: defect | Status: reopened
Priority: major | Milestone:
Component: Systems Admin | Resolution:
Keywords: |
---------------------------+------------------------------------------------
Comment(by wildintellect):
Just an additional note, that awstats recommends an update timing of every
6 hours for a website with as much traffic as we have. This should reduce
the time and load quite a bit, and I agree the cron job should be nice -n
19
http://awstats.sourceforge.net/docs/awstats_benchmark.html
--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/634#comment:9>
OSGeo <http://www.osgeo.org/>
OSGeo committee and general foundation issue tracker.
#634: Trouble with awstats on download server
---------------------------+------------------------------------------------
Reporter: tmitchell | Owner: sac@…
Type: defect | Status: reopened
Priority: major | Milestone:
Component: Systems Admin | Resolution:
Keywords: |
---------------------------+------------------------------------------------
Comment(by wildintellect):
Read over the config. Looks ok (might want to change log type 1 instead of
4, can't remember why it's better right now).
The cron job was not actually executing awstats with the proper config
file. It was also set to run every 10 minutes which is a bit excessive.
I've set it to run every 4 hours for now nice'd at level 19. This should
be quick with a low load.
URL to access is http://download.osgeo.org/cgi-
bin/awstats.pl?config=download.osgeo.org
I'll look into processing old logs in the next couple of weeks since I
need to do this on some other servers too.
--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/634#comment:10>
OSGeo <http://www.osgeo.org/>
OSGeo committee and general foundation issue tracker.
#634: Trouble with awstats on download server
---------------------------+------------------------------------------------
Reporter: tmitchell | Owner: sac@…
Type: defect | Status: closed
Priority: major | Milestone:
Component: Systems Admin | Resolution: fixed
Keywords: |
---------------------------+------------------------------------------------
Changes (by wildintellect):
* status: reopened => closed
* resolution: => fixed
Comment:
Old Logs are now restored. Changed apache logging to combined (instead of
common) for more robust information. All up to date, not url is slightly
different than I thought in last note.
http://download.osgeo.org/cgi-bin/awstats.pl?config=www.download.osgeo.org
Please re-open if you notice anything isn't right.
--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/634#comment:11>
OSGeo <http://www.osgeo.org/>
OSGeo committee and general foundation issue tracker.
#634: Trouble with awstats on download server
---------------------------+------------------------------------------------
Reporter: tmitchell | Owner: sac@…
Type: defect | Status: reopened
Priority: major | Milestone:
Component: Systems Admin | Resolution:
Keywords: |
---------------------------+------------------------------------------------
Changes (by neteler):
* status: closed => reopened
* resolution: fixed =>
Comment:
I tried to see the entire year 2010 but
{{{
Error: Full year view has not been allowed from a browser
(AllowFullYearView should be set to 3).
Setup ('/etc/awstats/awstats.www.download.osgeo.org.conf' file, web server
or permissions) may be wrong.
Check config file, permissions and AWStats documentation (in 'docs'
directory).
}}}
Perhaps you could activate that?
--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/634#comment:12>
OSGeo <http://www.osgeo.org/>
OSGeo committee and general foundation issue tracker.
#634: Trouble with awstats on download server
---------------------------+------------------------------------------------
Reporter: tmitchell | Owner: sac@…
Type: defect | Status: closed
Priority: major | Milestone:
Component: Systems Admin | Resolution: fixed
Keywords: |
---------------------------+------------------------------------------------
Changes (by wildintellect):
* status: reopened => closed
* resolution: => fixed
--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/634#comment:13>
OSGeo <http://www.osgeo.org/>
OSGeo committee and general foundation issue tracker.
#634: Trouble with awstats on download server
---------------------------+------------------------------------------------
Reporter: tmitchell | Owner: sac@…
Type: defect | Status: closed
Priority: major | Milestone:
Component: Systems Admin | Resolution: fixed
Keywords: |
---------------------------+------------------------------------------------
Comment(by wildintellect):
Url is now http://download.osgeo.org/stats/ based on ticket:727
--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/634#comment:14>
OSGeo <http://www.osgeo.org/>
OSGeo committee and general foundation issue tracker.
