[SAC] GitLab deploy update (runner, and out-of-space)

Today I've been adding continuous-integration support to
the experimental GitLab instance, and learning on the go.

I've added a shared runner (kind of a build slave)
that would execute builds inside a docker image
(used the same one used on gitlab.com by default, to simplify use)

Unfortunately, the machine (temporarely made available by Vincent Picavet)
is out of space, so the experiment cannot continue.

The machine currently runs:

- GitLab (with just a PostGIS clone)
- Mattermost
- GitLab-Runner (with docker)

It has only ~10GB of space in total.

Are there people still interested in this "git hosting" project ?
Is it possible to get more hardware resources to continue the
experiment ?

--strk;

On Thu, Oct 20, 2016 at 06:41:22PM +0200, Sandro Santilli wrote:

Unfortunately, the machine (temporarely made available by Vincent Picavet)
is out of space, so the experiment cannot continue.

[...]

It has only ~10GB of space in total.

I've found 1/3 of that space is taken by MongoDB:

  3148820 ./var/lib/mongodb/journal

Supposedly, GitLab doesn't use MongoDB.
Vincent, are you using it for anything else ?

--strk;

Hello,

On 20/10/2016 18:58, Sandro Santilli wrote:

On Thu, Oct 20, 2016 at 06:41:22PM +0200, Sandro Santilli wrote:

[..]

Supposedly, GitLab doesn't use MongoDB.
Vincent, are you using it for anything else ?

Nope, it's just for the gitlab instance.
Verify that one of the gitlab component does not rely on mongodb, it is
strange to have it take so much space.
Maybe Bjorn knows more.

I could probably get more space, would need some cash though.

Vincent

On Thu, Oct 20, 2016 at 07:04:22PM +0200, Vincent Picavet (ml) wrote:

I could probably get more space, would need some cash though.

My last motion (to call for a meeting) went unanswered, but maybe
you can try another motion to request the cash ?

--strk;

2016-10-20 18:41 GMT+02:00 Sandro Santilli <strk@kbt.io>:

···

Today I’ve been adding continuous-integration support to
the experimental GitLab instance, and learning on the go.

I’ve added a shared runner (kind of a build slave)
that would execute builds inside a docker image
(used the same one used on gitlab.com by default, to simplify use)

Unfortunately, the machine (temporarely made available by Vincent Picavet)
is out of space, so the experiment cannot continue.

The machine currently runs:

  • GitLab (with just a PostGIS clone)
  • Mattermost
  • GitLab-Runner (with docker)

It has only ~10GB of space in total.

Are there people still interested in this “git hosting” project ?
Is it possible to get more hardware resources to continue the
experiment ?

I’m still interested (not sure about community interest in general though) but been reluctant to put more time into it because of GitLab only being partially free with some needed features missing in the community edition, most importantly repository mirroring.

No idea what MongoDB is used for, cannot find it related to GitLab or Mattermost.

/Björn

–strk;


Sac mailing list
Sac@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/sac

On Thu, Oct 20, 2016 at 07:48:21PM +0200, Björn Harrtell wrote:

I'm still interested (not sure about community interest in general though)
but been reluctant to put more time into it because of GitLab only being
partially free with some needed features missing in the community edition,
most importantly repository mirroring.

Yeah, that's what did set me off too, but as we add more projects to
Gogs (I moved a couple postgis repos this week) I start being
concerned about the stability of Gogs. Just looking a bit forward.

Note that on the Gogs side I also spent some time drafting a trac2gogs
commandline: https://notabug.org/strk/trac2gogs :slight_smile:

No idea what MongoDB is used for, cannot find it related to GitLab or
Mattermost.

People in #gitlab say they are pretty sure MongoDB isn't used by
GitLab and recommend checking if it's not listening on public IP/ports
(as it would let anyone w/out any authentication to fill up the
databases). I've no experience with MongoDB to check this out.

--strk;

2016-10-20 21:24 GMT+02:00 Sandro Santilli <strk@kbt.io>:

On Thu, Oct 20, 2016 at 07:48:21PM +0200, Björn Harrtell wrote:

> I'm still interested (not sure about community interest in general
though)
> but been reluctant to put more time into it because of GitLab only being
> partially free with some needed features missing in the community
edition,
> most importantly repository mirroring.

Yeah, that's what did set me off too, but as we add more projects to
Gogs (I moved a couple postgis repos this week) I start being
concerned about the stability of Gogs. Just looking a bit forward.

Note that on the Gogs side I also spent some time drafting a trac2gogs
commandline: https://notabug.org/strk/trac2gogs :slight_smile:

> No idea what MongoDB is used for, cannot find it related to GitLab or
> Mattermost.

People in #gitlab say they are pretty sure MongoDB isn't used by
GitLab and recommend checking if it's not listening on public IP/ports
(as it would let anyone w/out any authentication to fill up the
databases). I've no experience with MongoDB to check this out.

Looks like the culprit is rocket.chat.

# mongo

show dbs

local 0.078GB
rocketchat 0.078GB

So, actual data is pretty small but the journal needs tweaking to stop it
growing uncontrollably. First time with MongoDB here so this is as far as I
can go without going into depth.

/Björn

--strk;

On Thu, Oct 20, 2016 at 09:37:41PM +0200, Björn Harrtell wrote:

Looks like the culprit is rocket.chat.

# mongo
> show dbs
local 0.078GB
rocketchat 0.078GB

A-ha!

So, actual data is pretty small but the journal needs tweaking to stop it
growing uncontrollably. First time with MongoDB here so this is as far as I
can go without going into depth.

I don't even remember having completed the setup of rocket.chat !
Maybe we should just kill it until someone finds the time to set it
up correctly.

--strk;

On Thu, Oct 20, 2016 at 09:46:42PM +0200, Sandro Santilli wrote:

On Thu, Oct 20, 2016 at 09:37:41PM +0200, Björn Harrtell wrote:

> Looks like the culprit is rocket.chat.
>
> # mongo
> > show dbs
> local 0.078GB
> rocketchat 0.078GB

A-ha!

> So, actual data is pretty small but the journal needs tweaking to stop it
> growing uncontrollably. First time with MongoDB here so this is as far as I
> can go without going into depth.

I don't even remember having completed the setup of rocket.chat !
Maybe we should just kill it until someone finds the time to set it
up correctly.

On re-reading, I now understand we don't need to kill rocket.chat
(which is not running) but tweak mongodb config.

See also
http://stackoverflow.com/questions/19533019/is-it-safe-to-delete-the-journal-file-of-mongodb

--strk;

On Thu, Oct 20, 2016 at 09:52:33PM +0200, Sandro Santilli wrote:

On re-reading, I now understand we don't need to kill rocket.chat
(which is not running) but tweak mongodb config.

Done, there are now 3GB available on the system.

--strk;

On 10/20/2016 10:06 AM, Sandro Santilli wrote:

On Thu, Oct 20, 2016 at 07:04:22PM +0200, Vincent Picavet (ml) wrote:

I could probably get more space, would need some cash though.

My last motion (to call for a meeting) went unanswered, but maybe
you can try another motion to request the cash ?

--strk;

If it seems like people are using it lets talk about a plan to move it OSGeo hardware.

Note Mongodb default installs tend to be insecure, please make sure to add authentication protection and to block it from external ip connections.

Thanks,
Alex

On Fri, Oct 21, 2016 at 08:25:55AM -0700, Alex Mandel wrote:

On 10/20/2016 10:06 AM, Sandro Santilli wrote:
>On Thu, Oct 20, 2016 at 07:04:22PM +0200, Vincent Picavet (ml) wrote:
>
>>I could probably get more space, would need some cash though.
>
>My last motion (to call for a meeting) went unanswered, but maybe
>you can try another motion to request the cash ?

If it seems like people are using it lets talk about a plan to move it OSGeo
hardware.

At the moment nobody is really using it.
Please correct me if I'm wrong.

Gogs is more used, togheter with Drone which is currently
hosted on my own server. Maybe we could move that one next ?

Note Mongodb default installs tend to be insecure, please make sure to add
authentication protection and to block it from external ip connections.

Thanks for the recommandation. I stopped the service as it's not
needed by either GitLab nor Mattermost at thismoment.

--strk;

For the record: I've removed the gitlab-ci runner as it took up
too much space anyway, for the builds and docker images needed
to build.

If anyone is interested in further tests, runners can be
executed from foreign machines (same as drone-agent).

I'll stick to Drone/Gogs for the time being.

--strk;

On Thu, Oct 20, 2016 at 10:09:24PM +0200, Sandro Santilli wrote:

On Thu, Oct 20, 2016 at 09:52:33PM +0200, Sandro Santilli wrote:

> On re-reading, I now understand we don't need to kill rocket.chat
> (which is not running) but tweak mongodb config.

Done, there are now 3GB available on the system.

--strk;