[SAC] Space on Backup

Committees SAC mailing list
1

Martin - Do we have a cap on how much space Bacula uses for it's storage
area?

All, adding gvSIG files on downloads will push the Backup server disk
space into the warning zone again once an rsync gets them. I'd like to
per-emptively avoid that. I see 3 options (2 easy):
1. move rsync somewhere else
2. limit bacula to rotate within a few less GB
3. Be more selective about what rsync and/or bacula pick up

On a related note, if you have an opinion about my previous email "Re:
[SAC] 2013 Agenda - was: Upgrade list for VMs" where I outlined possible
new hardware, please reply to that thread. This is the long term
solution to storage.

Thanks,
Alex

2

On Mon, Jan 28, 2013 at 10:31:08AM -0800, Alex Mandel wrote:

Martin - Do we have a cap on how much space Bacula uses for it's storage
area?

Disk full :slight_smile:

  Martin.
--
Unix _IS_ user friendly - it's just selective about who its friends are !
--------------------------------------------------------------------------

3

On 02/04/2013 01:01 AM, Martin Spott wrote:

On Mon, Jan 28, 2013 at 10:31:08AM -0800, Alex Mandel wrote:

Martin - Do we have a cap on how much space Bacula uses for it's storage
area?

Disk full :slight_smile:

  Martin.

It's easy enough to tell Bacula to only use X amount and recycle the
oldest volumes after that. We should probably do that in order to avoid
disk full which could happen at any point right now. I suggest we come
up with a reasonable number and set it for now while we work on figuring
out the whole 3rd server idea.

Thanks,
Alex

4

Hi Alex,

On Mon, Feb 04, 2013 at 01:21:48AM -0800, Alex Mandel wrote:

It's easy enough to tell Bacula to only use X amount and recycle the
oldest volumes after that. We should probably do that in order to avoid
disk full which could happen at any point right now. I suggest we come
up with a reasonable number and set it for now while we work on figuring
out the whole 3rd server idea.

a) I've already reduced the retention period at least twice during the
   past months, and, of course, it can be reduced even further. Yet,
   there's no more room for a "reasonable" number in the sense of
   having a backup you can rely on .... well, the entire idea of doing
   the backup on the same hardware which runs (at least some of) the
   machines you back up is, to put it mildly, slightly unreasonable :wink:
b) Disk full already happened during the past weekend.

Cheers,
  Martin.
--
Unix _IS_ user friendly - it's just selective about who its friends are !
--------------------------------------------------------------------------

5

On 02/04/2013 01:35 AM, Martin Spott wrote:

Hi Alex,

On Mon, Feb 04, 2013 at 01:21:48AM -0800, Alex Mandel wrote:

It's easy enough to tell Bacula to only use X amount and recycle the
oldest volumes after that. We should probably do that in order to avoid
disk full which could happen at any point right now. I suggest we come
up with a reasonable number and set it for now while we work on figuring
out the whole 3rd server idea.

a) I've already reduced the retention period at least twice during the
   past months, and, of course, it can be reduced even further. Yet,
   there's no more room for a "reasonable" number in the sense of
   having a backup you can rely on .... well, the entire idea of doing
   the backup on the same hardware which runs (at least some of) the
   machines you back up is, to put it mildly, slightly unreasonable :wink:

Most of the critical files are actually on the other server. But I agree
it's not ideal. I re-introduced the ideas I had on alleviating this
issue in a previous thread but I've gathered almost no response, so I'm
not sure where to go next.

I'm suggesting we simply put a cap on the number of volumes, regardless
of retention time. I hope we're not keeping more than one copy of most
things (I haven't looked at the rules in a while, so I can't recall).
Really we just need to reduced the influx of data from some of the less
critical applications. I'm not sure how to be more aggressive about
this, other than only backing up designated locations and telling
projects to dump critical data there.

b) Disk full already happened during the past weekend.

I'm aware, that's why I sent out the email to begin with.

Cheers,
  Martin.

Thanks,
Alex

6

On Mon, Feb 04, 2013 at 01:46:59AM -0800, Alex Mandel wrote:

On 02/04/2013 01:35 AM, Martin Spott wrote:

Most of the critical files are actually on the other server. But I agree
it's not ideal. I re-introduced the ideas I had on alleviating this
issue in a previous thread but I've gathered almost no response, so I'm
not sure where to go next.

It's probably my fault, but, to be honest, I'm unaware of anything
relevant to the discussion about OSGeo's future backup strategy which
hasn't already been said.

The latest planning state *I* am aware of is to buy a third server
focussing on as much disk space as possible and to find a home for it.
We had a short discssion about wether the backup machine should reside
off-site (which means not at OSL), making full backups difficult due to
network volume limitations or to install it at the same place, which
might result in trouble if the entire site would be affected by major
damage.
Anyhow, I'd say the second solution would still be better by magnitudes
compared to the current state. As far as I (vaguely) remember, you
were planning to negotiate with OSUOSL about hosting the third machine.

Without looking at the mail archives I'm pretty certain that the above
is just a repeating the result of past discussions. If the focus has
shifted, then I probably missed it.

Really we just need to reduced the influx of data from some of the less
critical applications. I'm not sure how to be more aggressive about
this, other than only backing up designated locations and telling
projects to dump critical data there.

Well, that's a communication issue. According to my knowledge,
projects had (without negotiations on a consent) been told to put
*everything* into /osgeo/, no matter how sensitive it was and I felt
really strong headwind when I was calling for a more nuanced directory
structure. Therefore I think I'm not going to join this discussion
again but instead leave it to others.

Cheers,
  Martin.
--
Unix _IS_ user friendly - it's just selective about who its friends are !
--------------------------------------------------------------------------

7

On Mon, Feb 4, 2013 at 1:46 AM, Alex Mandel <tech_dev@wildintellect.com> wrote:

Most of the critical files are actually on the other server. But I agree
it's not ideal. I re-introduced the ideas I had on alleviating this
issue in a previous thread but I've gathered almost no response, so I'm
not sure where to go next.

Alex,

I'm still supportive of us buying a third diskful server focused on
backup and possibly other disk-intensive needs. I have no strong
opinion on whether to get OSUOSL to purchase the machine or for us to
do it.

I would prefer to a diskful focus to one focused on big memory/cpu
suitable to running more VMs.

I'll keep out of the backula management topic.

Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up | Frank Warmerdam, warmerdam@pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush | Geospatial Software Developer

8

On Mon, Feb 04, 2013 at 09:28:52AM -0800, Frank Warmerdam wrote:

I'm still supportive of us buying a third diskful server focused on
backup and possibly other disk-intensive needs.

I'd like to point out that, this time, "backup" should be first and
"other disk-intensive needs" should be secondary. Otherwise the entire
idea of having a "backup" would, again, be rather pointless.

Cheers,
  Martin.
--
Unix _IS_ user friendly - it's just selective about who its friends are !
--------------------------------------------------------------------------

9

... We need to speed up this discussion:

On Wed, Mar 6, 2013 at 10:25 PM, <munin@webextra.osgeo.osuosl.org> wrote:

osgeo.org :: backup.osgeo.org :: Disk usage in percent
        WARNINGs: /backup is 96.91 (outside range [:92]).

Markus

10

On 03/06/2013 03:31 PM, Markus Neteler wrote:

... We need to speed up this discussion:

On Wed, Mar 6, 2013 at 10:25 PM, <munin@webextra.osgeo.osuosl.org> wrote:

osgeo.org :: backup.osgeo.org :: Disk usage in percent
         WARNINGs: /backup is 96.91 (outside range [:92]).

Markus
_______________________________________________
Sac mailing list
Sac@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/sac

I have ticket filed with OSUOSL about new equipment. Will bug them this week about it.

Thanks,
Alex

11

Note that we currently are proposing 5k for a backup server for the 2013
budget, and the budget will hopefully be approved by the Board shortly.

-jeff

On 13-03-06 10:04 PM, Alex Mandel wrote:

On 03/06/2013 03:31 PM, Markus Neteler wrote:

... We need to speed up this discussion:

On Wed, Mar 6, 2013 at 10:25 PM, <munin@webextra.osgeo.osuosl.org>
wrote:

osgeo.org :: backup.osgeo.org :: Disk usage in percent
         WARNINGs: /backup is 96.91 (outside range [:92]).

Markus
_______________________________________________
Sac mailing list
Sac@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/sac

I have ticket filed with OSUOSL about new equipment. Will bug them this
week about it.

Thanks,
Alex