[SAC] [OSGeo] #3020: Creating backup of osgeo6 as lxc container on osgeo7

#3020: Creating backup of osgeo6 as lxc container on osgeo7
----------------------+--------------------------------------
Reporter: robe | Owner: sac@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2023-I
Component: SysAdmin | Keywords:
----------------------+--------------------------------------
Doing this so we can test upgrades and other stuff.
It went thru okay (first time I'm successfully using lxd-migration (a 5.0
version though, couldn't get the new one to compile as it complained about
dqlite, of which I had no success installing)

Steps are listed in
https://git.osgeo.org/gitea/sac/osgeo7/wiki/osgeo6-backup----IN-PROGRESS,
but I'll repeat here since nothing top secret about it

On osgeo6

{{{
sudo -i
screen #create new screen so if get disconnected will still be running

# worked this time 54.8GB copied
screen
./lxd-migrate

1) Url: https://osgeo7.osgeo.osuosl.org:8443
2) choose 1 - certificate token
    a) create on osgeo7 using (lxd config trust add (specify osgeo6 for
name), copy token

3) a) Choose create container
   b) name: osgeo6-2023-11-05
4) root path: /
5) Extra mounts: yes
6) mount: /var
7) mount: /var/lib/mailman
8) mount: /home
9) enter #will end list
9) 1) begin migration

Ctrl a d #exit out of screen and detach to keep it running
screen -r #pick the job # to reconnect

# after migration is complete
systemctl enable rsyslog
systemctl start rsyslog
}}}

Note I didn't bother copying /var/www, so sites will not start, most of
that stuff is deprecated, so I plan to rsync it over after the container
starts up.

# on osgeo7
{{{
lxc start osgeo6-2023-11-05
}}}

It took a really long time to start up, was on "Remapping filesystem for
long time (like 20 minutes)

At this point it has no ip, cause well old was hard-coded

# on osge7
{{{
lxc snapshot osgeo6-2023-11-05 before-changes
lxc exec osgeo6-2023-11-05 bash
}}}

# in osgeo6 backup container
{{{
#disable services
systemctl disable mailman
systemctl disable postfix #stop mail from sending
systemctl disable fail2ban
systemctl disable bacula-fd
systemctl stop mailman
systemctl enable rsyslog
systemctl stop cron.d
systemctl disable cron.d

#systemctl mask systemd-journald
systemctl start systemd-journald

systemctl disable munin-node #removed on osgeo6 so not needed to disable
any more
systemctl disable lvm2
systemctl stop lvm2

systemctl start rsyslog

nano /etc/hostname #change to osgeo6-backup
mv /etc/fstab /etc/fstab.orig
touch /etc/fstab #save blank

mv /etc/network/interfaces /etc/network/interfaces.orig
nano /etc/network/interfaces #change


 #contents after changing to static private
 auto lo
 iface lo inet loopback

 auto eth0
 iface eth0 inet dhcp

 --

reboot
}}}
--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/3020&gt;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3020: Creating backup of osgeo6 as lxc container on osgeo7
----------------------+---------------------------------------
Reporter: robe | Owner: sac@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2023-I
Component: SysAdmin | Resolution:
Keywords: |
----------------------+---------------------------------------
Description changed by robe:

Old description:

Doing this so we can test upgrades and other stuff.
It went thru okay (first time I'm successfully using lxd-migration (a 5.0
version though, couldn't get the new one to compile as it complained
about dqlite, of which I had no success installing)

Steps are listed in
https://git.osgeo.org/gitea/sac/osgeo7/wiki/osgeo6-backup----IN-PROGRESS,
but I'll repeat here since nothing top secret about it

On osgeo6

{{{
sudo -i
screen #create new screen so if get disconnected will still be running

# worked this time 54.8GB copied
screen
./lxd-migrate

1) Url: https://osgeo7.osgeo.osuosl.org:8443
2) choose 1 - certificate token
   a) create on osgeo7 using (lxd config trust add (specify osgeo6 for
name), copy token

3) a) Choose create container
  b) name: osgeo6-2023-11-05
4) root path: /
5) Extra mounts: yes
6) mount: /var
7) mount: /var/lib/mailman
8) mount: /home
9) enter #will end list
9) 1) begin migration

Ctrl a d #exit out of screen and detach to keep it running
screen -r #pick the job # to reconnect

# after migration is complete
systemctl enable rsyslog
systemctl start rsyslog
}}}

Note I didn't bother copying /var/www, so sites will not start, most of
that stuff is deprecated, so I plan to rsync it over after the container
starts up.

# on osgeo7
{{{
lxc start osgeo6-2023-11-05
}}}

It took a really long time to start up, was on "Remapping filesystem for
long time (like 20 minutes)

At this point it has no ip, cause well old was hard-coded

# on osge7
{{{
lxc snapshot osgeo6-2023-11-05 before-changes
lxc exec osgeo6-2023-11-05 bash
}}}

# in osgeo6 backup container
{{{
#disable services
systemctl disable mailman
systemctl disable postfix #stop mail from sending
systemctl disable fail2ban
systemctl disable bacula-fd
systemctl stop mailman
systemctl enable rsyslog
systemctl stop cron.d
systemctl disable cron.d

#systemctl mask systemd-journald
systemctl start systemd-journald

systemctl disable munin-node #removed on osgeo6 so not needed to disable
any more
systemctl disable lvm2
systemctl stop lvm2

systemctl start rsyslog

nano /etc/hostname #change to osgeo6-backup
mv /etc/fstab /etc/fstab.orig
touch /etc/fstab #save blank

mv /etc/network/interfaces /etc/network/interfaces.orig
nano /etc/network/interfaces #change


#contents after changing to static private
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet dhcp

--

reboot
}}}

New description:

Doing this so we can test upgrades and other stuff.
It went thru okay (first time I'm successfully using lxd-migration (a 5.0
version though, couldn't get the new one to compile as it complained about
dqlite, of which I had no success installing)

Steps are listed in
https://git.osgeo.org/gitea/sac/osgeo7/wiki/osgeo6-backup----IN-PROGRESS,
but I'll repeat here since nothing top secret about it

On osgeo6

{{{
sudo -i
screen #create new screen so if get disconnected will still be running

# worked this time 54.8GB copied
screen
./lxd-migrate

1) Url: https://osgeo7.osgeo.osuosl.org:8443
2) choose 1 - certificate token
    a) create on osgeo7 using (lxd config trust add (specify osgeo6 for
name), copy token

3) a) Choose create container
   b) name: osgeo6-2023-11-05
4) root path: /
5) Extra mounts: yes
6) mount: /var
7) mount: /var/lib/mailman
8) mount: /home
9) enter #will end list
9) 1) begin migration

Ctrl a d #exit out of screen and detach to keep it running
screen -r #pick the job # to reconnect

# after migration is complete
systemctl enable rsyslog
systemctl start rsyslog
}}}

Note I didn't bother copying /var/www, so sites will not start, most of
that stuff is deprecated, so I plan to rsync it over after the container
starts up.

# on osgeo7
{{{
lxc start osgeo6-2023-11-05
}}}

It took a really long time to start up, was on "Remapping filesystem for
long time (like 20 minutes)

At this point it has no ip, cause well old was hard-coded

# on osge7
{{{
lxc snapshot osgeo6-2023-11-05 before-changes
lxc exec osgeo6-2023-11-05 bash
}}}

# in osgeo6 backup container
{{{
#disable services
systemctl disable mailman
systemctl disable postfix #stop mail from sending
systemctl disable fail2ban
systemctl disable bacula-fd
systemctl stop mailman
systemctl enable rsyslog
systemctl stop cron.d
systemctl disable cron.d

#systemctl mask systemd-journald
systemctl start systemd-journald

systemctl disable munin-node #removed on osgeo6 so not needed to disable
any more
systemctl disable lvm2
systemctl stop lvm2

systemctl start rsyslog

nano /etc/hostname #change to osgeo6-backup
mv /etc/fstab /etc/fstab.orig
touch /etc/fstab #save blank

mv /etc/network/interfaces /etc/network/interfaces.orig
nano /etc/network/interfaces #change


 #contents after changing to static private
 auto lo
 iface lo inet loopback

 auto eth0
 iface eth0 inet dhcp

 --

reboot
}}}

#should have an ip at this point.
In osgeo7 take another snapshot

{{{
lxc snapshot osgeo6-2023-11-05 after-network-dhcp
}}}

--
--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/3020#comment:1&gt;
OSGeo <https://osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#3020: Creating backup of osgeo6 as lxc container on osgeo7
----------------------+---------------------------------------
Reporter: robe | Owner: sac@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2023-I
Component: SysAdmin | Resolution:
Keywords: |
----------------------+---------------------------------------
Comment (by robe):

Note that at this point

{{{
systemctl list-units --failed
}}}

shows these failed services

{{{
   UNIT LOAD ACTIVE SUB DESCRIPTION
● apache2.service loaded failed failed The Apache HTTP
Server
● smartd.service loaded failed failed Self Monitoring and
Reporting Technology (SMART) Daemon
● systemd-journald-audit.socket loaded failed failed Journal Audit Socket

}}}

apache2 is failed presumably cause I didn't bring over the /var/www
volume. I'll do that next.
The other stuff well smartd I think is hardware checker, which makes no
sense for an os container. I think systemd-journal-audit often fails
especially when no security.nesting=true is enabled on server. I'm going
to disable both, cause I don't think they are useful

{{{
systemctl disable smartd.service
systemctl disable systemd-journald-audit.socket
}}}
--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/3020#comment:2&gt;
OSGeo <https://osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#3020: Creating backup of osgeo6 as lxc container on osgeo7
----------------------+---------------------------------------
Reporter: robe | Owner: sac@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2023-I
Component: SysAdmin | Resolution:
Keywords: |
----------------------+---------------------------------------
Description changed by robe:

Old description:

Doing this so we can test upgrades and other stuff.
It went thru okay (first time I'm successfully using lxd-migration (a 5.0
version though, couldn't get the new one to compile as it complained
about dqlite, of which I had no success installing)

Steps are listed in
https://git.osgeo.org/gitea/sac/osgeo7/wiki/osgeo6-backup----IN-PROGRESS,
but I'll repeat here since nothing top secret about it

On osgeo6

{{{
sudo -i
screen #create new screen so if get disconnected will still be running

# worked this time 54.8GB copied
screen
./lxd-migrate

1) Url: https://osgeo7.osgeo.osuosl.org:8443
2) choose 1 - certificate token
   a) create on osgeo7 using (lxd config trust add (specify osgeo6 for
name), copy token

3) a) Choose create container
  b) name: osgeo6-2023-11-05
4) root path: /
5) Extra mounts: yes
6) mount: /var
7) mount: /var/lib/mailman
8) mount: /home
9) enter #will end list
9) 1) begin migration

Ctrl a d #exit out of screen and detach to keep it running
screen -r #pick the job # to reconnect

# after migration is complete
systemctl enable rsyslog
systemctl start rsyslog
}}}

Note I didn't bother copying /var/www, so sites will not start, most of
that stuff is deprecated, so I plan to rsync it over after the container
starts up.

# on osgeo7
{{{
lxc start osgeo6-2023-11-05
}}}

It took a really long time to start up, was on "Remapping filesystem for
long time (like 20 minutes)

At this point it has no ip, cause well old was hard-coded

# on osge7
{{{
lxc snapshot osgeo6-2023-11-05 before-changes
lxc exec osgeo6-2023-11-05 bash
}}}

# in osgeo6 backup container
{{{
#disable services
systemctl disable mailman
systemctl disable postfix #stop mail from sending
systemctl disable fail2ban
systemctl disable bacula-fd
systemctl stop mailman
systemctl enable rsyslog
systemctl stop cron.d
systemctl disable cron.d

#systemctl mask systemd-journald
systemctl start systemd-journald

systemctl disable munin-node #removed on osgeo6 so not needed to disable
any more
systemctl disable lvm2
systemctl stop lvm2

systemctl start rsyslog

nano /etc/hostname #change to osgeo6-backup
mv /etc/fstab /etc/fstab.orig
touch /etc/fstab #save blank

mv /etc/network/interfaces /etc/network/interfaces.orig
nano /etc/network/interfaces #change


#contents after changing to static private
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet dhcp

--

reboot
}}}

#should have an ip at this point.
In osgeo7 take another snapshot

{{{
lxc snapshot osgeo6-2023-11-05 after-network-dhcp
}}}

New description:

Doing this so we can test upgrades and other stuff.
It went thru okay (first time I'm successfully using lxd-migration (a 5.0
version though, couldn't get the new one to compile as it complained about
dqlite, of which I had no success installing)

Steps are listed in
https://git.osgeo.org/gitea/sac/osgeo7/wiki/osgeo6-backup----IN-PROGRESS,
but I'll repeat here since nothing top secret about it

On osgeo6

{{{
sudo -i
screen #create new screen so if get disconnected will still be running

# worked this time 54.8GB copied

./lxd-migrate

1) Url: https://osgeo7.osgeo.osuosl.org:8443
2) choose 1 - certificate token
    a) create on osgeo7 using (lxd config trust add (specify osgeo6 for
name), copy token

3) a) Choose create container
   b) name: osgeo6-2023-11-05
4) root path: /
5) Extra mounts: yes
6) mount: /var
7) mount: /var/lib/mailman
8) mount: /home
9) enter #will end list
9) 1) begin migration

Ctrl a d #exit out of screen and detach to keep it running
screen -r #pick the job # to reconnect

# after migration is complete
systemctl enable rsyslog
systemctl start rsyslog
}}}

Note I didn't bother copying /var/www, so sites will not start, most of
that stuff is deprecated, so I plan to rsync it over after the container
starts up.

# on osgeo7
{{{
lxc start osgeo6-2023-11-05
}}}

It took a really long time to start up, was on "Remapping filesystem for
long time (like 20 minutes)

At this point it has no ip, cause well old was hard-coded

# on osge7
{{{
lxc snapshot osgeo6-2023-11-05 before-changes
lxc exec osgeo6-2023-11-05 bash
}}}

# in osgeo6 backup container
{{{
#disable services
systemctl disable mailman
systemctl disable postfix #stop mail from sending
systemctl disable fail2ban
systemctl disable bacula-fd
systemctl stop mailman
systemctl enable rsyslog
systemctl stop cron.d
systemctl disable cron.d

#systemctl mask systemd-journald
systemctl start systemd-journald

systemctl disable munin-node #removed on osgeo6 so not needed to disable
any more
systemctl disable lvm2
systemctl stop lvm2

systemctl start rsyslog

nano /etc/hostname #change to osgeo6-backup
mv /etc/fstab /etc/fstab.orig
touch /etc/fstab #save blank

mv /etc/network/interfaces /etc/network/interfaces.orig
nano /etc/network/interfaces #change


 #contents after changing to static private
 auto lo
 iface lo inet loopback

 auto eth0
 iface eth0 inet dhcp

 --

reboot
}}}

#should have an ip at this point.
In osgeo7 take another snapshot

{{{
lxc snapshot osgeo6-2023-11-05 after-network-dhcp
}}}

--
--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/3020#comment:3&gt;
OSGeo <https://osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#3020: Creating backup of osgeo6 as lxc container on osgeo7
----------------------+---------------------------------------
Reporter: robe | Owner: sac@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2023-I
Component: SysAdmin | Resolution:
Keywords: |
----------------------+---------------------------------------
Description changed by robe:

Old description:

Doing this so we can test upgrades and other stuff.
It went thru okay (first time I'm successfully using lxd-migration (a 5.0
version though, couldn't get the new one to compile as it complained
about dqlite, of which I had no success installing)

Steps are listed in
https://git.osgeo.org/gitea/sac/osgeo7/wiki/osgeo6-backup----IN-PROGRESS,
but I'll repeat here since nothing top secret about it

On osgeo6

{{{
sudo -i
screen #create new screen so if get disconnected will still be running

# worked this time 54.8GB copied

./lxd-migrate

1) Url: https://osgeo7.osgeo.osuosl.org:8443
2) choose 1 - certificate token
   a) create on osgeo7 using (lxd config trust add (specify osgeo6 for
name), copy token

3) a) Choose create container
  b) name: osgeo6-2023-11-05
4) root path: /
5) Extra mounts: yes
6) mount: /var
7) mount: /var/lib/mailman
8) mount: /home
9) enter #will end list
9) 1) begin migration

Ctrl a d #exit out of screen and detach to keep it running
screen -r #pick the job # to reconnect

# after migration is complete
systemctl enable rsyslog
systemctl start rsyslog
}}}

Note I didn't bother copying /var/www, so sites will not start, most of
that stuff is deprecated, so I plan to rsync it over after the container
starts up.

# on osgeo7
{{{
lxc start osgeo6-2023-11-05
}}}

It took a really long time to start up, was on "Remapping filesystem for
long time (like 20 minutes)

At this point it has no ip, cause well old was hard-coded

# on osge7
{{{
lxc snapshot osgeo6-2023-11-05 before-changes
lxc exec osgeo6-2023-11-05 bash
}}}

# in osgeo6 backup container
{{{
#disable services
systemctl disable mailman
systemctl disable postfix #stop mail from sending
systemctl disable fail2ban
systemctl disable bacula-fd
systemctl stop mailman
systemctl enable rsyslog
systemctl stop cron.d
systemctl disable cron.d

#systemctl mask systemd-journald
systemctl start systemd-journald

systemctl disable munin-node #removed on osgeo6 so not needed to disable
any more
systemctl disable lvm2
systemctl stop lvm2

systemctl start rsyslog

nano /etc/hostname #change to osgeo6-backup
mv /etc/fstab /etc/fstab.orig
touch /etc/fstab #save blank

mv /etc/network/interfaces /etc/network/interfaces.orig
nano /etc/network/interfaces #change


#contents after changing to static private
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet dhcp

--

reboot
}}}

#should have an ip at this point.
In osgeo7 take another snapshot

{{{
lxc snapshot osgeo6-2023-11-05 after-network-dhcp
}}}

New description:

Doing this so we can test upgrades and other stuff.
It went thru okay (first time I'm successfully using lxd-migration (a 5.0
version though, couldn't get the new one to compile as it complained about
dqlite, of which I had no success installing)

Steps are listed in
https://git.osgeo.org/gitea/sac/osgeo7/wiki/osgeo6-backup----IN-PROGRESS,
but I'll repeat here since nothing top secret about it

On osgeo6

{{{
sudo -i
systemctl disable rsyslog
systemctl stop rsyslog

screen #create new screen so if get disconnected will still be running

# worked this time 54.8GB copied

./lxd-migrate

1) Url: https://osgeo7.osgeo.osuosl.org:8443
2) choose 1 - certificate token
    a) create on osgeo7 using (lxd config trust add (specify osgeo6 for
name), copy token

3) a) Choose create container
   b) name: osgeo6-2023-11-05
4) root path: /
5) Extra mounts: yes
6) mount: /var
7) mount: /var/lib/mailman
8) mount: /home
9) enter #will end list
9) 1) begin migration

Ctrl a d #exit out of screen and detach to keep it running
screen -r #pick the job # to reconnect

# after migration is complete
systemctl enable rsyslog
systemctl start rsyslog
}}}

Note I didn't bother copying /var/www, so sites will not start, most of
that stuff is deprecated, so I plan to rsync it over after the container
starts up.

# on osgeo7
{{{
lxc start osgeo6-2023-11-05
}}}

It took a really long time to start up, was on "Remapping filesystem for
long time (like 20 minutes)

At this point it has no ip, cause well old was hard-coded

# on osge7
{{{
lxc snapshot osgeo6-2023-11-05 before-changes
lxc exec osgeo6-2023-11-05 bash
}}}

# in osgeo6 backup container
{{{
#disable services
systemctl disable mailman
systemctl disable postfix #stop mail from sending
systemctl disable fail2ban
systemctl disable bacula-fd
systemctl stop mailman
systemctl enable rsyslog
systemctl stop cron.d
systemctl disable cron.d

#systemctl mask systemd-journald
systemctl start systemd-journald

systemctl disable munin-node #removed on osgeo6 so not needed to disable
any more
systemctl disable lvm2
systemctl stop lvm2

systemctl start rsyslog

nano /etc/hostname #change to osgeo6-backup
mv /etc/fstab /etc/fstab.orig
touch /etc/fstab #save blank

mv /etc/network/interfaces /etc/network/interfaces.orig
nano /etc/network/interfaces #change


 #contents after changing to static private
 auto lo
 iface lo inet loopback

 auto eth0
 iface eth0 inet dhcp

 --

reboot
}}}

#should have an ip at this point.
In osgeo7 take another snapshot

{{{
lxc snapshot osgeo6-2023-11-05 after-network-dhcp
}}}

--
--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/3020#comment:4&gt;
OSGeo <https://osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#3020: Creating backup of osgeo6 as lxc container on osgeo7
----------------------+---------------------------------------
Reporter: robe | Owner: sac@…
     Type: task | Status: closed
Priority: normal | Milestone: Sysadmin Contract 2023-I
Component: SysAdmin | Resolution: fixed
Keywords: |
----------------------+---------------------------------------
Changes (by robe):

* status: new => closed
* resolution: => fixed

--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/3020#comment:5&gt;
OSGeo <https://osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.