[SAC] [OSGeo] #178: Need MySQL Backups on OSGeo1

#178: Need MySQL Backups on OSGeo1
--------------------------+-------------------------------------------------
Reporter: warmerdam | Owner: sac@lists.osgeo.org
    Type: task | Status: new
Priority: normal | Component: SAC
Keywords: backups mysql |
--------------------------+-------------------------------------------------
As far as I can tell the zrm backups were commented out of
/etc/cron.d/backup.cron many months ago, and we are now effectively not
doing backups of mysql except for whatever the magic tape backups done by
peer1 get which might not be in a consistent state.

This ticket, is to capture the task of re-enabling and verifying mysql
backups.

This is most critical for the various Drupal instances stored in mysql.

--
Ticket URL: <http://trac.osgeo.org/osgeo/ticket/178&gt;
OSGeo <http://www.osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#178: Need MySQL Backups on OSGeo1
---------------------------+--------------------
Reporter: warmerdam | Owner: sac@…
     Type: task | Status: new
Priority: normal | Milestone:
Component: Systems Admin | Resolution:
Keywords: backups mysql |
---------------------------+--------------------
Changes (by strk):

* cc: neteler (added)

Comment:

osgeo1 was retired. Web (drupal) was moved to osgeo3.

The Bacula backup for /var/lib/mysql does exist, although there isn't a
"dump" of the databases being taken, is that what this ticket is about ?
I'd note that for the PostgreSQL database on the TracSVN VM we do have a
full dump. Also that the same situation exists on the osgeo6 machine,
where GRASS is hosted.

--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/178#comment:1&gt;
OSGeo <http://www.osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#178: Need MySQL Backups on OSGeo1
---------------------------+--------------------
Reporter: warmerdam | Owner: sac@…
     Type: task | Status: new
Priority: normal | Milestone:
Component: Systems Admin | Resolution:
Keywords: backups mysql |
---------------------------+--------------------

Comment (by strk):

The Bacula config for osgeo6 has a commented-out call to a
"export_cms_grassgis.sh" script that is not in the expected location but
I've found in /var/www/grass/grass-cms/export_cms_grassgis.sh - the script
seems to require interaction (to enter a password) and seems to be run
manually by Markus (last time in Nov 13) -- Markus, could that script be
turned into a generic "dump each database" script ? If it outputs backups
under /osgeo/backup it'll be automatically included in the bacula backup
(this is more or less what's done on TracSVN).

--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/178#comment:2&gt;
OSGeo <http://www.osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#178: Need MySQL Backups on OSGeo1
---------------------------+--------------------
Reporter: warmerdam | Owner: sac@…
     Type: task | Status: new
Priority: normal | Milestone:
Component: Systems Admin | Resolution:
Keywords: backups mysql |
---------------------------+--------------------

Comment (by neteler):

Replying to [comment:2 strk]:
> The Bacula config for osgeo6 has a commented-out call to a
"export_cms_grassgis.sh" script that is not in the expected location but
I've found in /var/www/grass/grass-cms/export_cms_grassgis.sh

Yes, that is the script which I wrote.

> - the script seems to require interaction (to enter a password) and
seems to be run manually by Markus (last time in Nov 13)

Confirmed. And run again today.

> -- Markus, could that script be turned into a generic "dump each
database" script ? If it outputs backups under /osgeo/backup it'll be
automatically included in the bacula backup (this is more or less what's
done on TracSVN).

Sure, feel free to recycle it. And it needs to go into a cronjob as well.

--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/178#comment:3&gt;
OSGeo <http://www.osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#178: Need MySQL Backups on OSGeo1
---------------------------+--------------------
Reporter: warmerdam | Owner: sac@…
     Type: task | Status: new
Priority: normal | Milestone:
Component: Systems Admin | Resolution:
Keywords: backups mysql |
---------------------------+--------------------

Comment (by strk):

Bacula config can run it, no need to be in a cronjob.
This is what is done with PosgreSQL dumps
(machines have a PGDump.sh script in /usr/local/sbin that's
run before each Bacula update run)

--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/178#comment:4&gt;
OSGeo <http://www.osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#178: Need MySQL Backups on OSGeo1
---------------------------+---------------------
Reporter: warmerdam | Owner: sac@…
     Type: task | Status: closed
Priority: normal | Milestone:
Component: Systems Admin | Resolution: fixed
Keywords: backups mysql |
---------------------------+---------------------
Changes (by strk):

* status: new => closed
* resolution: => fixed

Comment:

So I've added an /osgeo/backup/mysql_backup.sh script that takes care of
creating a dump of each stored database. The script is called by
/osgeo/backup/bacula_before.sh, which is called just before backing up the
files (which include /osgeo directory). The script drops a tarball with
all the dumps under the /osgeo directory. A bacula_after.sh script, in the
same directory as bacula_before.sh, takes care of dropping the dump after
the bacup is completed.

Closing this as completed. Feel free to file open new ones for more backup
issues.

--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/178#comment:5&gt;
OSGeo <http://www.osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.