[OSGeo] #3395: Containerize osgeo6 on osgeo7

#3395: Containerize osgeo6 on osgeo7
----------------------+---------------------------
Reporter: robe | Owner: sac-tickets@…
     Type: task | Status: new
Priority: normal | Milestone: Unplanned
Component: SysAdmin | Keywords:
----------------------+---------------------------
Need to containerize osgeo6 so that we can reformat the server and move it
to new colo once new colo is available.
--
Ticket URL: <Making sure you're not a bot!;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3395: Containerize osgeo6 on osgeo7
----------------------+------------------------------------------------
Reporter: robe | Owner: sac-tickets@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2025-II (vicky)
Component: SysAdmin | Resolution:
Keywords: |
----------------------+------------------------------------------------
Changes (by robe):

* milestone: Unplanned => Sysadmin Contract 2025-II (vicky)

Comment:

Vicky I updated the instructions here

Making sure you're not a bot!
migration/README.md#lxd-migrate

Not sure the best place to put them, but I had put it here at a time when
osgeo7 was the only lxd host. See if you can follow to containerize
osgeo6.

I think it should be okay as an OS container but we also have the option
to make it a VM which would be heavier.
--
Ticket URL: <Making sure you're not a bot!;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3395: Containerize osgeo6 on osgeo9
----------------------+------------------------------------------------
Reporter: robe | Owner: sac-tickets@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2025-II (vicky)
Component: SysAdmin | Resolution:
Keywords: |
----------------------+------------------------------------------------
Changes (by robe):

* summary: Containerize osgeo6 on osgeo7 => Containerize osgeo6 on osgeo9

--
Ticket URL: <Making sure you're not a bot!;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3395: Containerize osgeo6 on osgeo9
----------------------+------------------------------------------------
Reporter: robe | Owner: sac-tickets@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2025-II (vicky)
Component: SysAdmin | Resolution:
Keywords: |
----------------------+------------------------------------------------
Comment (by cvvergara):

The container is on osgeo7, and its the backup.
--
Ticket URL: <Making sure you're not a bot!;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3395: Containerize osgeo6 on osgeo9
----------------------+------------------------------------------------
Reporter: robe | Owner: sac-tickets@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2025-II (vicky)
Component: SysAdmin | Resolution:
Keywords: |
----------------------+------------------------------------------------
Comment (by cvvergara):

> upgrade osgeo6 it spams Martin

Found the configuration file and change the mail to sysadmin.
(so the spam will go to sysadmin)
--
Ticket URL: <Making sure you're not a bot!;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3395: Containerize osgeo6 on osgeo9
----------------------+------------------------------------------------
Reporter: robe | Owner: sac-tickets@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2025-II (vicky)
Component: SysAdmin | Resolution:
Keywords: |
----------------------+------------------------------------------------
Comment (by cvvergara):

The container is done: osgeo6-2025-07-28

Snapshots:

{{{
+---------------------------+----------------------+------------+----------+
| NAME | TAKEN AT | EXPIRES AT | STATEFUL
|
+---------------------------+----------------------+------------+----------+
| snap0 | 2025/07/29 04:20 UTC | | NO
|
+---------------------------+----------------------+------------+----------+
| before-changes | 2025/07/29 14:23 UTC | | NO
|
+---------------------------+----------------------+------------+----------+
| after-change-to-dhcp | 2025/07/29 15:48 UTC | | NO
|
+---------------------------+----------------------+------------+----------+
| before-update-to-bullseye | 2025/07/30 15:52 UTC | | NO
|
+---------------------------+----------------------+------------+----------+
| before-bookworm | 2025/07/30 18:15 UTC | | NO
|
+---------------------------+----------------------+------------+----------+

}}}

Right now I am creating an image
{{{
lxc publish osgeo6-2025-07-28/before-update-to-bullseye --alias osgeo6
-update-to-bullseye
}}}

Where refinement of the update to bullseye will be done.
The before-bookworm snapshot was not showing bullseye.
So I started over, but as you mention in the wiki, with my rewording, many
things have to be experimented to make the upgrade work.
--
Ticket URL: <Making sure you're not a bot!;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3395: Containerize osgeo6 on osgeo9
----------------------+------------------------------------------------
Reporter: robe | Owner: sac-tickets@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2025-II (vicky)
Component: SysAdmin | Resolution:
Keywords: |
----------------------+------------------------------------------------
Comment (by cvvergara):

# osgeo7
* Container name `osge6-buster`
* Running and cron to `rsync` with osgeo6 at 1 and 13 hours

# osgeo9
* Container name `osge6-buster`
* Running and cron to `rsync` with osgeo6 at 2 and 14 hours
--
Ticket URL: <Making sure you're not a bot!;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3395: Containerize osgeo6 on osgeo9
----------------------+------------------------------------------------
Reporter: robe | Owner: sac-tickets@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2025-II (vicky)
Component: SysAdmin | Resolution:
Keywords: |
----------------------+------------------------------------------------
Comment (by cvvergara):

Next planned work on `osgeo9/osgeo6-buster`

* currently is running under dhcp
* create a staging DNS for the container
* test access

All work steps will be logged [Making sure you're not a bot!
/mailserver-container here]
--
Ticket URL: <Making sure you're not a bot!;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3395: Containerize osgeo6 on osgeo9
----------------------+------------------------------------------------
Reporter: robe | Owner: sac-tickets@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2025-II (vicky)
Component: SysAdmin | Resolution:
Keywords: |
----------------------+------------------------------------------------
Comment (by strk):

I've pushed a mail-servers branch in ansible-deployment where you can
deploy mail servers (production and staging) via:

> ansible-playbook deployment/deploy-mail-servers.yml

For now I've only put the osgeo9 container in the staging inventory, and
I'm using --check --diff to see what variables we'd need defined for
things to work transparently.
--
Ticket URL: <Making sure you're not a bot!;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3395: Containerize osgeo6 on osgeo9
----------------------+------------------------------------------------
Reporter: robe | Owner: sac-tickets@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2025-II (vicky)
Component: SysAdmin | Resolution:
Keywords: |
----------------------+------------------------------------------------
Comment (by strk):

I've merged mail-origin branch into master branch of ansible-deployment.,
as I think the variables are good enough.

Only parts missing seem to be:

  - Addition of 10.36.74.0/24 to postfix main.cf "mynetworks" directive
(why needed?)
  - Refrain from starting the postfix service (My understanding is that it
should be off in staging for now?)

The other changes are likely WANTED (differences in mail aliases) - I did
not deploy them.
--
Ticket URL: <Making sure you're not a bot!;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3395: Containerize osgeo6 on osgeo9
----------------------+------------------------------------------------
Reporter: robe | Owner: sac-tickets@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2025-II (vicky)
Component: SysAdmin | Resolution:
Keywords: |
----------------------+------------------------------------------------
Comment (by strk):

I've found the problem with network forwarding: kernel module br_netfilter
is required but is not loaded automatically.

Easy fix:
`lxc profile set default linux.kernel_modules br_netfilter`

I'd have ansible take care of that, using
https://docs.ansible.com/ansible/latest/collections/community/general/lxd_profile_module.html
--
Ticket URL: <Making sure you're not a bot!;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3395: Containerize osgeo6 on osgeo9
----------------------+------------------------------------------------
Reporter: robe | Owner: sac-tickets@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2025-II (vicky)
Component: SysAdmin | Resolution:
Keywords: |
----------------------+------------------------------------------------
Changes (by strk):

* cc: strk (added)

--
Ticket URL: <Making sure you're not a bot!;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3395: Containerize osgeo6 on osgeo9
----------------------+------------------------------------------------
Reporter: robe | Owner: sac-tickets@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2025-II (vicky)
Component: SysAdmin | Resolution:
Keywords: |
----------------------+------------------------------------------------
Comment (by strk):

Next thing that needs tweaking in Ansible is the various names that should
be considered local destinations (`mydestinations`) -- the current Ansible
does not include mail2.osgeo.org and mail2.osgeo.osuosl.org but there were
some mails in the queue with that destination. I'm not sure if we want to
handle those as a variable or have all servers accept mail for all
destinations, preparing for the switch
--
Ticket URL: <Making sure you're not a bot!;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3395: Containerize osgeo6 on osgeo9
----------------------+------------------------------------------------
Reporter: robe | Owner: sac-tickets@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2025-II (vicky)
Component: SysAdmin | Resolution:
Keywords: |
----------------------+------------------------------------------------
Comment (by strk):

The fix is now deployed to osgeo9 from ansible, see
Making sure you're not a bot!
deployment/commit/f2c0ec6db32150f9f4c968fac5a7e4979f442bd2

It isn't deployed to other lxd-hosts but would be on next deploy (NOTE
that --diff and --check don't work with the lxd_profile task, so it won't
be easy to tell something changed there -- see https://github.com/ansible-
collections/community.general/issues/10794)

So now I've deployed mail-servers in staging inventory and cleaned up the
/etc/hosts file inside nextcloud - tests welcome
--
Ticket URL: <Making sure you're not a bot!;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3395: Containerize osgeo6 on osgeo9
----------------------+------------------------------------------------
Reporter: robe | Owner: sac-tickets@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2025-II (vicky)
Component: SysAdmin | Resolution:
Keywords: |
----------------------+------------------------------------------------
Comment (by strk):

For the record: the web server in containerized osgeo6 is currently being
visited by openai bots from private IP address 10.36.74.151 -- I dunno how
it is possible to hit that container from a private address, but we should
be concerned because of a recent grant of trust to that network, see
Making sure you're not a bot!
deployment/commit/9f22b6a8bbf72f40c051ebf04e62ea06c8fedda7
--
Ticket URL: <Making sure you're not a bot!;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3395: Containerize osgeo6 on osgeo9
----------------------+------------------------------------------------
Reporter: robe | Owner: sac-tickets@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2025-II (vicky)
Component: SysAdmin | Resolution:
Keywords: |
----------------------+------------------------------------------------
Comment (by strk):

Updates:

1. ansible staging deploy will use staging hostnames for `mydestinations`
so won't accept mail for production hostnames. The only variables override
are for `myhostname` and `mydomain`

2. The containerized osgeo6 is currently unable to connect to the public
address we dedicate to the new mail server (140.211.15.14) so I've added
/etc/hosts lines to resolve the mail2/mail.staging/lists.staging names to
the loopback interface and configured postfix to use that file. This
second part is currently NOT DONE by ansible, so next deploy would revert
it (will revert the postfix config part, not the /etc/hosts)
--
Ticket URL: <Making sure you're not a bot!;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3395: Containerize osgeo6 on osgeo9
----------------------+------------------------------------------------
Reporter: robe | Owner: sac-tickets@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2025-II (vicky)
Component: SysAdmin | Resolution:
Keywords: |
----------------------+------------------------------------------------
Comment (by strk):

I've asked in the LXD forum (support category) about this problem with
network forward: https://discourse.ubuntu.com/t/what-are-the-differences-
between-lxc-config-device-add-instance-device-proxy-and-lxc-network-
forward/40650/9
--
Ticket URL: <Making sure you're not a bot!;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3395: Containerize osgeo6 on osgeo9
----------------------+------------------------------------------------
Reporter: robe | Owner: sac-tickets@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2025-II (vicky)
Component: SysAdmin | Resolution:
Keywords: |
----------------------+------------------------------------------------
Comment (by cvvergara):

Can not use lists.osgeo.org from osgeo4 anymore
--
Ticket URL: <Making sure you're not a bot!;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3395: Containerize osgeo6 on osgeo9
----------------------+------------------------------------------------
Reporter: robe | Owner: sac-tickets@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2025-II (vicky)
Component: SysAdmin | Resolution:
Keywords: |
----------------------+------------------------------------------------
Comment (by cvvergara):

Need to use lists.osgeo.org from staging/develop containers as
lists.staging will disappear and become production, and no lists.staging
will exist.
--
Ticket URL: <Making sure you're not a bot!;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3395: Containerize osgeo6 on osgeo9
----------------------+------------------------------------------------
Reporter: robe | Owner: sac-tickets@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2025-II (vicky)
Component: SysAdmin | Resolution:
Keywords: |
----------------------+------------------------------------------------
Comment (by strk):

Replying to [comment:17 cvvergara]:
> Can not use lists.osgeo.org from osgeo4 anymore

This is unrelated to the containerized osgeo6
--
Ticket URL: <Making sure you're not a bot!;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.