[SAC] [OSGeo] #2521: Change secure to use dedicated cert and set up script to copy the cert

#2521: Change secure to use dedicated cert and set up script to copy the cert
---------------------------+-----------------------
Reporter: robe | Owner: sac@…
     Type: task | Status: new
Priority: normal | Milestone: Unplanned
Component: Systems Admin | Keywords:
---------------------------+-----------------------
secure right now is using a letsencrypt wildcard cert which I never got
around to changing to a ldap.osgeo.org cert. So I've been renewing it
every 3 months which is not ideal.

My proposed:

Change to use ldap.osgeo.org
Setup a script on nginx that when it renews it copies the script to secure
container.

I do have a script already that copies the cert but it can be improved to
not require manual stuff. Something a savyor network admin like strk can
do inhis sleep.

I have what I am currently doing outlined here

https://git.osgeo.org/gitea/sac/osgeo7/wiki/NGinx-Proxy-container#user-
content-generating-wildcard-cert-for-osgeo-org

--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2521&gt;
OSGeo <https://osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#2521: Change secure to use dedicated cert and set up script to copy the cert
---------------------------+----------------------------------------
Reporter: robe | Owner: sac@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2020-II
Component: Systems Admin | Resolution:
Keywords: |
---------------------------+----------------------------------------
Changes (by robe):

* milestone: Unplanned => Sysadmin Contract 2020-II

--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2521#comment:1&gt;
OSGeo <https://osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#2521: Change secure to use dedicated cert and set up script to copy the cert
---------------------------+----------------------------------------
Reporter: robe | Owner: sac@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2020-II
Component: Systems Admin | Resolution:
Keywords: |
---------------------------+----------------------------------------

Comment (by strk):

I'm a bit confused by this ticket.
According to https://wiki.osgeo.org/wiki/SAC_Service_Status#NGINX_proxy
the nginx container proxies all traffic, then why should the cert be
installed in the underlying container ? Is ldap.osgeo.org out of the proxy
? What makes ldap.osgeo.org different from other containers ?

--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2521#comment:2&gt;
OSGeo <https://osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#2521: Change secure to use dedicated cert and set up script to copy the cert
---------------------------+----------------------------------------
Reporter: robe | Owner: sac@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2020-II
Component: Systems Admin | Resolution:
Keywords: |
---------------------------+----------------------------------------

Comment (by strk):

Answering myself: LDAP is different in that it's NOT using the certificate
in a web server but a LDAP server. We could have nginx indeed respond to
http requests for the LDAP server IP address for the sole purpose of doing
http validation for letsencrypt. About the copy I wonder if we should be
instead MOUNTING the certificate directory directly, instead of copying it
?

--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2521#comment:3&gt;
OSGeo <https://osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#2521: Change secure to use dedicated cert and set up script to copy the cert
---------------------------+----------------------------------------
Reporter: robe | Owner: sac@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2020-II
Component: Systems Admin | Resolution:
Keywords: |
---------------------------+----------------------------------------

Comment (by robe):

Mounting sounds like a cleaner approach. Only issue with that is it makes
it less portable.

--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2521#comment:4&gt;
OSGeo <https://osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#2521: Change secure to use dedicated cert and set up script to copy the cert
---------------------------+----------------------------------------
Reporter: robe | Owner: sac@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2020-II
Component: Systems Admin | Resolution:
Keywords: |
---------------------------+----------------------------------------

Comment (by strk):

The cronjob should be written in the new ansible deployment playbooks:
https://git.osgeo.org/gitea/sac/ansible-deployment

--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2521#comment:6&gt;
OSGeo <https://osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#2521: Change secure to use dedicated cert and set up script to copy the cert
---------------------------+----------------------------------------
Reporter: robe | Owner: sac@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2020-II
Component: Systems Admin | Resolution:
Keywords: |
---------------------------+----------------------------------------

Comment (by strk):

Cron job is now in place and deployed via ansible. It has to be tested
that it works correctly, including when errors arise, as I'm not sure if a
mail would ever be received on error (no MTA on osgeo7?)

--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2521#comment:7&gt;
OSGeo <https://osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#2521: Change secure to use dedicated cert and set up script to copy the cert
---------------------------+----------------------------------------
Reporter: robe | Owner: sac@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2020-II
Component: Systems Admin | Resolution:
Keywords: |
---------------------------+----------------------------------------

Comment (by strk):

Cron mails will arrive, as per #2536 - now it's to be tested if those
mails are too noisy, and then if renewal is effective.

--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2521#comment:8&gt;
OSGeo <https://osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#2521: Change secure to use dedicated cert and set up script to copy the cert
---------------------------+----------------------------------------
Reporter: robe | Owner: sac@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2020-II
Component: Systems Admin | Resolution:
Keywords: |
---------------------------+----------------------------------------

Comment (by strk):

I've added the ldap.osgeo.org virtual host configuration to the ansible
deployment repository with https://git.osgeo.org/gitea/sac/ansible-
deployment/commit/971e9a6fa5661353337f39194557644ac1609b54

Didn't add more virtual hosts but we should, in a separate ticket.

Only things left out of ansible yet is:

   - certbot configuration for ldap.osgeo.org
   - LXC container configuration to ensure a device listening on
ldap.osgeo.org is added to the 'nginx container

--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2521#comment:9&gt;
OSGeo <https://osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#2521: Change secure to use dedicated cert and set up script to copy the cert
---------------------------+----------------------------------------
Reporter: robe | Owner: sac@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2020-II
Component: Systems Admin | Resolution:
Keywords: |
---------------------------+----------------------------------------

Comment (by strk):

nginx container setup was added to ansible playbooks with
https://git.osgeo.org/gitea/sac/ansible-
deployment/commit/711a269093b1bb460a7180c5f5d39e2d6ade48fb

Only thin left out of ansible is now certbot configuration for
ldap.osgeo.org

--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2521#comment:10&gt;
OSGeo <https://osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#2521: Change secure to use dedicated cert and set up script to copy the cert
---------------------------+----------------------------------------
Reporter: robe | Owner: sac@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2020-II
Component: Systems Admin | Resolution:
Keywords: |
---------------------------+----------------------------------------
Changes (by strk):

* milestone: Sysadmin Contract 2020-III => Sysadmin Contract 2020-II

--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2521#comment:12&gt;
OSGeo <https://osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#2521: Change secure to use dedicated cert and set up script to copy the cert
---------------------------+----------------------------------------
Reporter: robe | Owner: sac@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2021-II
Component: Systems Admin | Resolution:
Keywords: |
---------------------------+----------------------------------------
Changes (by robe):

* milestone: Sysadmin Contract 2020-II => Sysadmin Contract 2021-II

Comment:

Okay this is till not working so reopening it.

When I do this:

Check if secure's cert expire date
{{{
  openssl s_client -servername ldap.osgeo.org -connect ldap.osgeo.org:636
2>/dev/null | openssl x509 -noout -dates

}}}
I get:

{{{
notBefore=Sep 29 09:16:39 2021 GMT
notAfter=Dec 28 09:16:38 2021 GMT
}}}

When I check the recently received cert on nginx:

{{{
  openssl s_client -servername ldap.osgeo.org -connect ldap.osgeo.org:443
2>/dev/null | openssl x509 -noout -dates
}}}

I get

{{{
notBefore=Nov 28 19:31:56 2021 GMT
notAfter=Feb 26 19:31:55 2022 GMT
}}}

If I restart slapd on osgeo7-secure

{{{
systemctl restart slapd
}}}

It does not fix the issue.

So it seems the cronjob is not working.

The cronjob on osgeo7 looks

{{{
  sudo systemctl status cron
}}}

shows this:

{{{
Dec 05 12:30:01 osgeo7 CRON[25890]: (tech_dev) CMD
(/usr/local/bin/copy_ldap_certs_to_secure.sh)
Dec 05 12:30:01 osgeo7 cron[4341]: sendmail: fatal: open
/etc/postfix/main.cf: Permission denied
Dec 05 12:30:01 osgeo7 CRON[25889]: (tech_dev) MAIL (mailed 109 bytes of
output but got status 0x004b from MTA
                                     )
Dec 05 12:30:01 osgeo7 CRON[25889]: pam_unix(cron:session): session closed
for user tech_dev
Dec 05 13:17:01 osgeo7 CRON[7005]: pam_unix(cron:session): session opened
for user root by (uid=0)
Dec 05 13:17:01 osgeo7 CRON[7009]: (root) CMD ( cd / && run-parts
--report /etc/cron.hourly)
Dec 05 13:17:01 osgeo7 CRON[7005]: pam_unix(cron:session): session closed
for user root
Dec 05 14:17:01 osgeo7 CRON[18083]: pam_unix(cron:session): session opened
for user root by (uid=0)
Dec 05 14:17:01 osgeo7 CRON[18084]: (root) CMD ( cd / && run-parts
--report /etc/cron.hourly)
Dec 05 14:17:01 osgeo7 CRON[18083]: pam_unix(cron:session): session closed
for user root

}}}

--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2521#comment:13&gt;
OSGeo <https://osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#2521: Change secure to use dedicated cert and set up script to copy the cert
---------------------------+----------------------------------------
Reporter: robe | Owner: sac@…
     Type: task | Status: closed
Priority: normal | Milestone: Sysadmin Contract 2021-II
Component: Systems Admin | Resolution: fixed
Keywords: |
---------------------------+----------------------------------------
Changes (by robe):

* status: new => closed
* resolution: => fixed

Comment:

okay looks like a permission issue.

I can't run under the osgeo7 tech_dev account

{{{
   /usr/local/bin/copy_ldap_certs_to_secure.sh
}}}

Get this
{{{
Error: open /var/lib/snapd/hostfs/etc/cron.d/fullchain.pem.new: permission
denied
tech_dev@osgeo7:/etc/cron.d$ sudo
/usr/local/bin/copy_ldap_certs_to_secure.sh
fullchain.pem.new fullchain.pem.current differ: byte 34, line 2

}}}

But if I do

{{{
  sudo /usr/local/bin/copy_ldap_certs_to_secure.sh
}}}

It works. I'm gong to change the cronjob to do sudo.

strk feel free to change again if you don't feel that is right.

--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2521#comment:14&gt;
OSGeo <https://osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.