[SAC] [OSGeo] #2964: grasswiki and grass logrotates still broken

#2964: grasswiki and grass logrotates still broken
---------------------------+--------------------------------------
Reporter: robe | Owner: sac@…
     Type: defect | Status: new
Priority: normal | Milestone: Sysadmin Contract 2023-I
Component: Systems Admin | Keywords:
---------------------------+--------------------------------------
In looking at the grass-wiki and grass containers both now on osgeo8,
seems the logrotate is still not working.

Last log rotate for grasswiki was in May 20 and grass one not since I
migrated it to osgeo8
--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2964&gt;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#2964: grasswiki and grass logrotates still broken
---------------------------+---------------------------------------
Reporter: robe | Owner: sac@…
     Type: defect | Status: new
Priority: normal | Milestone: Sysadmin Contract 2023-I
Component: Systems Admin | Resolution:
Keywords: |
---------------------------+---------------------------------------
Comment (by neteler):

Anything I can do from my side?
--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2964#comment:1&gt;
OSGeo <https://osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#2964: grasswiki and grass logrotates still broken
---------------------------+---------------------------------------
Reporter: robe | Owner: sac@…
     Type: defect | Status: new
Priority: normal | Milestone: Sysadmin Contract 2023-I
Component: Systems Admin | Resolution:
Keywords: |
---------------------------+---------------------------------------
Comment (by robe):

I don't understand much about logrotate to be honest when the service
stops are starts.
I was going to look at the other systems where it's working to see if I
can see a pattern.

You are free to look yourself if you want. I'm trying to think if it
broke during the last OS upgrade.
--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2964#comment:2&gt;
OSGeo <https://osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#2964: grasswiki and grass logrotates still broken
---------------------------+---------------------------------------
Reporter: robe | Owner: sac@…
     Type: defect | Status: new
Priority: normal | Milestone: Sysadmin Contract 2023-I
Component: Systems Admin | Resolution:
Keywords: |
---------------------------+---------------------------------------
Comment (by neteler):

I just connected and tried:

{{{
/usr/sbin/logrotate /etc/logrotate.conf --debug
}}}

--> looks ok. But this doesn't:

{{{
systemctl status logrotate
● logrotate.service - Rotate log files
      Loaded: loaded (/lib/systemd/system/logrotate.service; static)
      Active: failed (Result: exit-code) since Sun 2023-08-13 00:00:00 UTC;
19h ago
TriggeredBy: ● logrotate.timer
        Docs: man:logrotate(8)
              man:logrotate.conf(5)
    Main PID: 368037 (code=exited, status=226/NAMESPACE)

Aug 13 00:00:00 grass systemd[1]: Starting Rotate log files...
Aug 13 00:00:00 grass systemd[368037]: logrotate.service: Failed to set up
mount namespacing: /run/systemd/unit-root/proc: Permission denied
Aug 13 00:00:00 grass systemd[368037]: logrotate.service: Failed at step
NAMESPACE spawning /usr/sbin/logrotate: Permission denied
Aug 13 00:00:00 grass systemd[1]: logrotate.service: Main process exited,
code=exited, status=226/NAMESPACE
Aug 13 00:00:00 grass systemd[1]: logrotate.service: Failed with result
'exit-code'.
Aug 13 00:00:00 grass systemd[1]: Failed to start Rotate log files.

ls -la /run/systemd/unit-root/proc
ls: cannot access '/run/systemd/unit-root/proc': No such file or directory
}}}

Some LXD trick needed?
--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2964#comment:3&gt;
OSGeo <https://osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#2964: grasswiki and grass logrotates still broken
---------------------------+---------------------------------------
Reporter: robe | Owner: sac@…
     Type: defect | Status: new
Priority: normal | Milestone: Sysadmin Contract 2023-I
Component: Systems Admin | Resolution:
Keywords: |
---------------------------+---------------------------------------
Comment (by robe):

Yap sounds like it. I'll research.
--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2964#comment:4&gt;
OSGeo <https://osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#2964: grasswiki and grass logrotates still broken
---------------------------+---------------------------------------
Reporter: robe | Owner: sac@…
     Type: defect | Status: new
Priority: normal | Milestone: Sysadmin Contract 2023-I
Component: Systems Admin | Resolution:
Keywords: |
---------------------------+---------------------------------------
Comment (by robe):

Yah has to do with the security.

I relaxed the security with

{{{
lxc config set grass security.nesting=true
}}}

and after that was able to start up the logrotate service
and was able to set that back to false and still restart logrotate.

But even though in restarting it starts fine, it still has

{{{
ls -la /run/systemd/unit-root/proc
ls: cannot access '/run/systemd/unit-root/proc': No such file or directory
}}}

if I check or perhaps that's only needed when it's in process of a
logrotate.

I'll keep this ticket open to see if it shuts down again.

Regarding these issues, I had simialr when upgrading the host to 22.04
something to do with
cgroup2 hierarchy not properly set with moving from 20.04 to 22.04, thou
as dicussed here https://discuss.linuxcontainers.org/t/2nd-system-
upgraded-from-ubuntu-20-04-w-working-lxd-to-ubuntu-22-04-lxd-again-not-
working/14009/29 but those were more serious with the containers not even
restarting and not being able to create older containers like centos 7.

but this might be some dangling left over from that.

Speaking of which -- you want to upgrade to bookworm :). Peertube I
upgraded last week to bookworm seems to be doing fine logrotate working
--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2964#comment:5&gt;
OSGeo <https://osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#2964: grasswiki and grass logrotates still broken
---------------------------+---------------------------------------
Reporter: robe | Owner: sac@…
     Type: defect | Status: new
Priority: normal | Milestone: Sysadmin Contract 2023-I
Component: Systems Admin | Resolution:
Keywords: |
---------------------------+---------------------------------------
Comment (by neteler):

An upgrade to bookworm is fine for me. Shall I trigger it or will you do
that?
--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2964#comment:6&gt;
OSGeo <https://osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#2964: grasswiki and grass logrotates still broken
---------------------------+---------------------------------------
Reporter: robe | Owner: sac@…
     Type: defect | Status: new
Priority: normal | Milestone: Sysadmin Contract 2023-I
Component: Systems Admin | Resolution:
Keywords: |
---------------------------+---------------------------------------
Comment (by robe):

I should do it. I usually take a snapshot from the host and if something
goes wrong I restore back to the snapshot. I'll test first on dev.
--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2964#comment:7&gt;
OSGeo <https://osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.