[GRASS-user] GRASS on SSHFS

[Daniel_Lee]

Dear list,

I’m working on a server that unfortunately does not have GRASS installed on it. In order to work on my large amounts of data there, I’ve mounted the external file system with ssh using SSHFS. This works fine and I can read and write data without problems.

I’ve made a GRASS location on that server, which also worked fine, and creating all of the locations and mapsets using the GRASS GUI worked without a hitch. But every time I try to perform any command within a mapset, I get the following error:

ERROR: MAPSET PERMANENT - permission denied

It doesn’t matter what the command is (g.region, g.list, g.gui) or whether the mapset is PERMANENT or another. I can create the mapsets, I just can’t work in them.

Just as an experiment, I changed the rights for the GRASS DB on the server to 777 recursively (there’s nothing in there anyway since I can’t do anything on it). This doesn’t make a difference, though.

Anybody know what the problem could be?

Best,
Daniel

[neteler]

Hi Daniel,

On Fri, Dec 7, 2012 at 11:38 AM, Daniel Lee <lee@isi-solutions.org> wrote:

Dear list,

I'm working on a server that unfortunately does not have GRASS installed on
it. In order to work on my large amounts of data there, I've mounted the
external file system with ssh using SSHFS. This works fine and I can read
and write data without problems.

(yes, did the same several times)

I've made a GRASS location on that server, which also worked fine, and
creating all of the locations and mapsets using the GRASS GUI worked without
a hitch. But every time I try to perform any command within a mapset, I get
the following error:

ERROR: MAPSET PERMANENT - permission denied

It doesn't matter what the command is (g.region, g.list, g.gui) or whether
the mapset is PERMANENT or another. I can create the mapsets, I just can't
work in them.

So, "touch /wherever/sshfs/grassloc/PERMANENT/blafile.txt" would work?

Just as an experiment, I changed the rights for the GRASS DB on the server
to 777 recursively (there's nothing in there anyway since I can't do
anything on it). This doesn't make a difference, though.

Anybody know what the problem could be?

Yet unclear to me. Could you send the permission list (ls -la) of the
parent directory?

Markus

[Glynn_Clements1]

Daniel Lee wrote:

I'm working on a server that unfortunately does not have GRASS installed on
it. In order to work on my large amounts of data there, I've mounted the
external file system with ssh using SSHFS. This works fine and I can read
and write data without problems.

I've made a GRASS location on that server, which also worked fine, and
creating all of the locations and mapsets using the GRASS GUI worked
without a hitch. But every time I try to perform any command within a
mapset, I get the following error:

ERROR: MAPSET PERMANENT - permission denied

It doesn't matter what the command is (g.region, g.list, g.gui) or whether
the mapset is PERMANENT or another. I can create the mapsets, I just can't
work in them.

Just as an experiment, I changed the rights for the GRASS DB on the server
to 777 recursively (there's nothing in there anyway since I can't do
anything on it). This doesn't make a difference, though.

Anybody know what the problem could be?

GRASS requires that you are the owner of the current mapset,
regardless of its permissions.

Ideally, you should try to persuade sshfs to maintain ownership
correctly (i.e. a newly-created directory should be owned by the user
who actually created it).

Failing that, you can modify G__mapset_permisssions() in
lib/gis/mapset_msc.c and re-compile.

--
Glynn Clements <glynn@gclements.plus.com>

[Daniel_Lee]

I accidentally just sent this to Glynn, rather than the list

···

2012/12/9 Glynn Clements <glynn@gclements.plus.com>

Daniel Lee wrote:

I’m working on a server that unfortunately does not have GRASS installed on
it. In order to work on my large amounts of data there, I’ve mounted the
external file system with ssh using SSHFS. This works fine and I can read
and write data without problems.

I’ve made a GRASS location on that server, which also worked fine, and
creating all of the locations and mapsets using the GRASS GUI worked
without a hitch. But every time I try to perform any command within a
mapset, I get the following error:

ERROR: MAPSET PERMANENT - permission denied

It doesn’t matter what the command is (g.region, g.list, g.gui) or whether
the mapset is PERMANENT or another. I can create the mapsets, I just can’t
work in them.

Just as an experiment, I changed the rights for the GRASS DB on the server
to 777 recursively (there’s nothing in there anyway since I can’t do
anything on it). This doesn’t make a difference, though.

Anybody know what the problem could be?

GRASS requires that you are the owner of the current mapset,
regardless of its permissions.

Ideally, you should try to persuade sshfs to maintain ownership
correctly (i.e. a newly-created directory should be owned by the user
who actually created it).

Failing that, you can modify G__mapset_permisssions() in
lib/gis/mapset_msc.c and re-compile.

–
Glynn Clements <glynn@gclements.plus.com>

[Glynn_Clements1]

Daniel Lee wrote:

Here's the output of ls -la:

ls -la
total 12
drwxrwxrwx 1 1002 9150 22 Dec 7 12:15 .
drwxrwxrwx 1 1002 9150 23 Dec 7 11:09 ..
drwxrwxrwx 1 1002 9150 129 Dec 7 12:14 PERMANENT

I think Glynn's right about the permissions being the problem. Even though
everybody has permissions here, the folders' owner is my SSH user, which is
different than the username on my own machine. It seems that's the source
of my troubles. Sadly, I don't see a way of getting around that except for
doing some really complicated SSH tunneling stuff or recompiling GRASS. The
easier way out is just contacting the system administrator of the server
I'm working on, which I've done - hopefully she'll have installed GRASS on
it by tomorrow, so that's one less thing to worry about.

Have you tried using the uid= and/or idmap=user mount options?

http://manpages.ubuntu.com/manpages/precise/man8/mount.fuse.8.html

idmap=user comes from sshfs (which doesn't appear to have any official
documentation), and is supposed to map UIDs.

That aside, though, is there a specific reason that GRASS requires the user
to own a mapset, rather than just have write permission?

Historically, there are two reasons:

1. To prevent multiple GRASS sessions from using the same mapset
directory. A lock file prevented a user from having more than one
session, and the ownership check prevented anyone other than the
directory's owner from using a directory as the current mapset.

2. Because otherwise people try to share mapsets by making them
group-writable, then discover that they can't replace maps due to
permission issues.

Specifically, if you allow someone else to write to your mapset
directory, and that user has a umask of 022 (or more), any directories
which they create (e.g. cell_misc/<mapname> directories) will be mode
755 (drwxr-xr-x) or less, meaning that you can't remove any files from
that directory, and thus won't be able to delete that directory. This
will prevent you from removing or overwriting the map using GRASS
commands.

Point #1 is no longer relevant, but point #2 is, at least on
multi-user systems.

On Windows, the checks are bypassed because Windows' stat() function
doesn't report the owner and Windows doesn't have getuid().

None of this would be a problem except for the fact that it's now
common to use non-Unix filesystems on Unix (where "non-Unix
filesystem" includes any filesystem where "touch file ; ls -l file"
reports an owner that doesn't match the output of "whoami").

--
Glynn Clements <glynn@gclements.plus.com>

[Daniel_Lee]

Have you tried using the uid= and/or idmap=user mount options?

http://manpages.ubuntu.com/manpages/precise/man8/mount.fuse.8.html

idmap=user comes from sshfs (which doesn't appear to have any official
documentation), and is supposed to map UIDs.

That sounds promising. I think I'll skip that though, since now we've got
GRASS running on the server and the server's a lot more comfortable for me
anyway due to the long calculations

Historically, there are two reasons:

1. To prevent multiple GRASS sessions from using the same mapset
directory. A lock file prevented a user from having more than one
session, and the ownership check prevented anyone other than the
directory's owner from using a directory as the current mapset.

2. Because otherwise people try to share mapsets by making them
group-writable, then discover that they can't replace maps due to
permission issues.

Specifically, if you allow someone else to write to your mapset
directory, and that user has a umask of 022 (or more), any directories
which they create (e.g. cell_misc/<mapname> directories) will be mode
755 (drwxr-xr-x) or less, meaning that you can't remove any files from
that directory, and thus won't be able to delete that directory. This
will prevent you from removing or overwriting the map using GRASS
commands.

Point #1 is no longer relevant, but point #2 is, at least on
multi-user systems.

On Windows, the checks are bypassed because Windows' stat() function
doesn't report the owner and Windows doesn't have getuid().

None of this would be a problem except for the fact that it's now
common to use non-Unix filesystems on Unix (where "non-Unix
filesystem" includes any filesystem where "touch file ; ls -l file"
reports an owner that doesn't match the output of "whoami").

Thanks for the explanation! That's good to know

Daniel

[Hamish]

Hi all, (online for just a few hours)

Glynn wrote:

Have you tried using the uid= and/or idmap=user mount
options?

http://manpages.ubuntu.com/manpages/precise/man8/mount.fuse.8.html

idmap=user comes from sshfs (which doesn't appear to have
any official documentation), and is supposed to map UIDs.

we use sshfs extensively on our heterogeneous computational cluster
instead of NFS (fun with MPI). It works very well, although our network
infrastructure is old enough not to make the extra ssh overhead the
limiting factor on I/O.

I always use 'sshfs -o idmap=user', the main trouble I've had with
newer releases is that if the remote dir is a symlink you must add a
trailing '/' or else it fails with a non-helpful error message. GRASS
over the sshfs link works fine, although my username is typically the
same both local and remote; while the UIDs are typically all over the
place due to the heterogeneity/provenance of the system.

I just tried with a local user name different that the remote one, GRASS
all works fine over the sshfs link, no permissions problem since the local
file system now thinks your local user owns all those files.

regards,
Hamish

[Daniel_Lee]

Hi there,

Alright, curiosity got the best of me and despite the fact that GRASS is now installed remotely, I fired it up locally and pointed it to the remote DB. Worked like a charm, I was able to see all of my data and do the things that I had wanted to before. Thanks for the tips! I think I’ll stick to working remotely, though, since I can leave processes running while I’m out and about with my computer. Nonetheless, it’s good to know

Best,
Daniel

···

2012/12/13 Hamish <hamish_b@yahoo.com>

Hi all, (online for just a few hours)

Glynn wrote:

Have you tried using the uid= and/or idmap=user mount
options?

http://manpages.ubuntu.com/manpages/precise/man8/mount.fuse.8.html

idmap=user comes from sshfs (which doesn’t appear to have
any official documentation), and is supposed to map UIDs.

we use sshfs extensively on our heterogeneous computational cluster
instead of NFS (fun with MPI). It works very well, although our network
infrastructure is old enough not to make the extra ssh overhead the
limiting factor on I/O.

I always use ‘sshfs -o idmap=user’, the main trouble I’ve had with
newer releases is that if the remote dir is a symlink you must add a
trailing ‘/’ or else it fails with a non-helpful error message. GRASS
over the sshfs link works fine, although my username is typically the
same both local and remote; while the UIDs are typically all over the
place due to the heterogeneity/provenance of the system.

I just tried with a local user name different that the remote one, GRASS
all works fine over the sshfs link, no permissions problem since the local
file system now thinks your local user owns all those files.

regards,
Hamish

[Markus_Metz]

On Tue, Dec 11, 2012 at 1:41 PM, Glynn Clements
<glynn@gclements.plus.com> wrote:

Daniel Lee wrote:

[...] is there a specific reason that GRASS requires the user
to own a mapset, rather than just have write permission?

Historically, there are two reasons:

1. To prevent multiple GRASS sessions from using the same mapset
directory. A lock file prevented a user from having more than one
session, and the ownership check prevented anyone other than the
directory's owner from using a directory as the current mapset.

2. Because otherwise people try to share mapsets by making them
group-writable, then discover that they can't replace maps due to
permission issues.

Specifically, if you allow someone else to write to your mapset
directory, and that user has a umask of 022 (or more), any directories
which they create (e.g. cell_misc/<mapname> directories) will be mode
755 (drwxr-xr-x) or less, meaning that you can't remove any files from
that directory, and thus won't be able to delete that directory. This
will prevent you from removing or overwriting the map using GRASS
commands.

Point #1 is no longer relevant, but point #2 is, at least on
multi-user systems.

Point #1 still applies in some environments, e.g. centralized GRASS
database, users can log in with the same user name and uid from
different machines. We are using such an environment.

On Windows, the checks are bypassed because Windows' stat() function
doesn't report the owner and Windows doesn't have getuid().

There is a new function G_owner(), r49747, which reports the owner and
works also on Windows. We could now activate lock files and ownership
tests for Windows.

Markus M

[Glynn_Clements1]

Markus Metz wrote:

> 1. To prevent multiple GRASS sessions from using the same mapset
> directory. A lock file prevented a user from having more than one
> session, and the ownership check prevented anyone other than the
> directory's owner from using a directory as the current mapset.

> Point #1 is no longer relevant, but point #2 is, at least on
> multi-user systems.

Point #1 still applies in some environments, e.g. centralized GRASS
database, users can log in with the same user name and uid from
different machines. We are using such an environment.

Point #1 is no longer relevant, as GRASS now puts the .gislock file in
the mapset directory. It used to put it in the user's home directory.
This prevented a single user from starting multiple sessions, but the
ownership check was required to prevent other users from starting
sessions in the same mapset directory.

> On Windows, the checks are bypassed because Windows' stat() function
> doesn't report the owner and Windows doesn't have getuid().

There is a new function G_owner(), r49747, which reports the owner and
works also on Windows. We could now activate lock files and ownership
tests for Windows.

On Unix, ownership checks should use the UID, not the username.

On Windows, the permission mechanism is sufficiently complicated that
I'd be inclined to disable ownership checks by default.

--
Glynn Clements <glynn@gclements.plus.com>

[Markus_Metz]

Glynn Clements wrote:

Markus Metz wrote:

There is a new function G_owner(), r49747, which reports the owner and
works also on Windows. We could now activate lock files and ownership
tests for Windows.

On Unix, ownership checks should use the UID, not the username.

G_owner is fixed accordingly in r54276.

On Windows, the permission mechanism is sufficiently complicated that
I'd be inclined to disable ownership checks by default.

I agree. If there is a GRASS developer willing to tackle the
intricacies of handling user IDs on Windows XP, Vista, 7, 8, Server
2003, Server 2008, Server 2012, we could give it a go.

Markus M