[SAC] rsync backups for download.osgeo.org

Folks,

Martin pointed out that the download.osgeo.org rsync backups have not been
running (for some time!). I have fixed download.osgeo.org so that rsyncd
is started from inetd and reenabled rsyncing from download.osgeo.org to
backup.osgeo.org again.

I'll try to check in that things are working smoothly in a few days.

Martin and I also briefly discussed what else is, or is not, getting
backed up on the new VMs and I think there are likely some holes. I think
he may follow up on some of those points but folks responsible for services
on the new VM might want to confirm that everything they need is getting
backed up in a way that we could recover fairly quickly if necessary.

Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up | Frank Warmerdam, warmerdam@pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush | Geospatial Programmer for Rent

On Fri, Nov 26, 2010 at 6:40 PM, Frank Warmerdam <warmerdam@pobox.com> wrote:

Folks,

Martin pointed out that the download.osgeo.org rsync backups have not been
running (for some time!). I have fixed download.osgeo.org so that rsyncd
is started from inetd and reenabled rsyncing from download.osgeo.org to
backup.osgeo.org again.

Thanks for your efforts.

I'll try to check in that things are working smoothly in a few days.

Martin and I also briefly discussed what else is, or is not, getting
backed up on the new VMs and I think there are likely some holes. I think
he may follow up on some of those points but folks responsible for services
on the new VM might want to confirm that everything they need is getting
backed up in a way that we could recover fairly quickly if necessary.

The backup of the "project" VM is quite important. Is there anything
known about that (and/or an updated Wiki page describing the state
of the art)?

thanks
Markus

On 11/26/2010 11:35 PM, Markus Neteler wrote:

On Fri, Nov 26, 2010 at 6:40 PM, Frank Warmerdam <warmerdam@pobox.com> wrote:

Folks,

Martin pointed out that the download.osgeo.org rsync backups have not been
running (for some time!). I have fixed download.osgeo.org so that rsyncd
is started from inetd and reenabled rsyncing from download.osgeo.org to
backup.osgeo.org again.

Thanks for your efforts.

I'll try to check in that things are working smoothly in a few days.

Martin and I also briefly discussed what else is, or is not, getting
backed up on the new VMs and I think there are likely some holes. I think
he may follow up on some of those points but folks responsible for services
on the new VM might want to confirm that everything they need is getting
backed up in a way that we could recover fairly quickly if necessary.

The backup of the "project" VM is quite important. Is there anything
known about that (and/or an updated Wiki page describing the state
of the art)?

thanks
Markus

Well there are 2 distinct steps.
1. a project needs to dump it's files/databases/configurations somewhere
easy to find and in an efficient format.
2. we need to configure a way to pull those to the Backup machine, most
likely using Bacula, but possibly just relying on rysnc

For step one I think a convention of /backup/projectname should make it
easy for us to sort through and grab backups with bacula.

Does that work well for others?

Thanks,
Alex

On Sat, Nov 27, 2010 at 9:05 AM, Alex Mandel <tech_dev@wildintellect.com> wrote:

On 11/26/2010 11:35 PM, Markus Neteler wrote:

On Fri, Nov 26, 2010 at 6:40 PM, Frank Warmerdam <warmerdam@pobox.com> wrote:

Folks,

Martin pointed out that the download.osgeo.org rsync backups have not been
running (for some time!). I have fixed download.osgeo.org so that rsyncd
is started from inetd and reenabled rsyncing from download.osgeo.org to
backup.osgeo.org again.

Thanks for your efforts.

I'll try to check in that things are working smoothly in a few days.

Martin and I also briefly discussed what else is, or is not, getting
backed up on the new VMs and I think there are likely some holes. I think
he may follow up on some of those points but folks responsible for services
on the new VM might want to confirm that everything they need is getting
backed up in a way that we could recover fairly quickly if necessary.

The backup of the "project" VM is quite important. Is there anything
known about that (and/or an updated Wiki page describing the state
of the art)?

thanks
Markus

Well there are 2 distinct steps.
1. a project needs to dump it's files/databases/configurations somewhere
easy to find and in an efficient format.

yes (I do this for GRASS occasionally).

2. we need to configure a way to pull those to the Backup machine, most
likely using Bacula, but possibly just relying on rysnc

So you mean that the project VM is not backup'ed at all?
oh.... since 9 servers are running on it... As of today
it contains:

docs.geotools.org.conf
geos.osgeo.org.conf
geotools.org.conf
mapserver.org.conf
projects.mapbender.osgeo.conf
sr.org.conf
tilecache.org.conf
www.gdal.org.conf
www.remotesensing.org.conf

oh.... since 9 servers are running on it...

For step one I think a convention of /backup/projectname should make it
easy for us to sort through and grab backups with bacula.

Does that work well for others?

Please set up a bacula backup soon.

thanks,
Markus

On Sat, Nov 27, 2010 at 9:05 AM, Alex Mandel <tech_dev@wildintellect.com> wrote:

Well there are 2 distinct steps.
1. a project needs to dump it's files/databases/configurations somewhere
easy to find and in an efficient format.

BTW: I used to copy over
http://gallery.osgeo.org/

but ssh is down:

gis:/srv/backup_grass.osgeo.org# ssh 198.202.74.219
ssh: connect to host 198.202.74.219 port 22: Connection refused

gis:/srv/backup_grass.osgeo.org# nslookup gallery.osgeo.org
Server: 213.133.100.100
Address: 213.133.100.100#53

Non-authoritative answer:
Name: gallery.osgeo.org
Address: 198.202.74.219

Does anyone backup the gallery?

Markus

On 11/27/2010 12:56 AM, Markus Neteler wrote:

On Sat, Nov 27, 2010 at 9:05 AM, Alex Mandel <tech_dev@wildintellect.com> wrote:

On 11/26/2010 11:35 PM, Markus Neteler wrote:

On Fri, Nov 26, 2010 at 6:40 PM, Frank Warmerdam <warmerdam@pobox.com> wrote:

Folks,

Martin pointed out that the download.osgeo.org rsync backups have not been
running (for some time!). I have fixed download.osgeo.org so that rsyncd
is started from inetd and reenabled rsyncing from download.osgeo.org to
backup.osgeo.org again.

Thanks for your efforts.

I'll try to check in that things are working smoothly in a few days.

Martin and I also briefly discussed what else is, or is not, getting
backed up on the new VMs and I think there are likely some holes. I think
he may follow up on some of those points but folks responsible for services
on the new VM might want to confirm that everything they need is getting
backed up in a way that we could recover fairly quickly if necessary.

The backup of the "project" VM is quite important. Is there anything
known about that (and/or an updated Wiki page describing the state
of the art)?

thanks
Markus

Well there are 2 distinct steps.
1. a project needs to dump it's files/databases/configurations somewhere
easy to find and in an efficient format.

yes (I do this for GRASS occasionally).

2. we need to configure a way to pull those to the Backup machine, most
likely using Bacula, but possibly just relying on rysnc

So you mean that the project VM is not backup'ed at all?
oh.... since 9 servers are running on it... As of today
it contains:

docs.geotools.org.conf
geos.osgeo.org.conf
geotools.org.conf
mapserver.org.conf
projects.mapbender.osgeo.conf
sr.org.conf
tilecache.org.conf
www.gdal.org.conf
www.remotesensing.org.conf

oh.... since 9 servers are running on it...

For step one I think a convention of /backup/projectname should make it
easy for us to sort through and grab backups with bacula.

Does that work well for others?

Please set up a bacula backup soon.

thanks,
Markus

Bacula is setup and currently backs up /etc on all of the new servers.
We have have not setup a backup for a /backup folder yet as far as I
know because we never agreed on the plan. If this idea works then we'll
have Martin setup the jobs on Bacula. It will be up to each of the above
mentioned projects though to create a script appropriate for dumping
their data appropriately.

We also need to decide on the frequency, full vs incremental and the
time schedule:
I would say every night(or 2), incremental, full every 2 weeks, maybe
right after the /etc backups happen.
An alternate if there is a lot of database dumping might be every week,
full every 4 weeks, sometime on the weekend.

Thoughts?

Thanks,
Alex