[GRASS-user] r.external across Windows and Linux

GRASS GRASS user
1

Hi,

The Institute I am working at has a Linux (Ubuntu) Server where we run GRASS 7 and Windows 7 as standard OS on workstations.

We are now about to move our GRASS DB to a network storage so it can be accessed from both sides. When I created a test-DB on the network storage I had trouble using external data (r.external.out). The links to the data seem to be different between Linux and Windows…

My question is: Is it in general possible to use external data linked to a GRASS DB which is accessed both from Windows and Linux boxes? Or is accessing one GRASS DB (or one mapset) both from Windows and Linux a problem in general?

Thanks in advance for helping!

Cheers

Stefan

2

Blumentrath, Stefan wrote:

We are now about to move our GRASS DB to a network storage so it can
be accessed from both sides. When I created a test-DB on the network
storage I had trouble using external data (r.external.out). The
links to the data seem to be different between Linux and Windows...

My question is: Is it in general possible to use external data
linked to a GRASS DB which is accessed both from Windows and Linux
boxes? Or is accessing one GRASS DB (or one mapset) both from
Windows and Linux a problem in general?

It should work, but I don't know how much testing r.external.out has
received, particlarly on Windows.

If you can report specific details of any issues you've encountered,
they will be looked into.

--
Glynn Clements <glynn@gclements.plus.com>

3

Hi Glynn,

Thanks for your reply. Here is a more detailed problem description:

First I created a new and empty GRASS Location "test" on the network storage (NFS) from the Linux Server. This GRASS DB is mounted on LINUX (through mount.cifs) to `/home/stefan/R_raw_data/test'. On Windows the "Network drive" is mapped to " R:\Raw_data\test "

Within this Location I crated a mapset "linux" (from the Linux Server) and a mapset "windows" from my Windows 7 workstation.

1st test case: Linux to Windows:
On the Linux server, and in the "linux" mapset I did:

r.external.out --verbose directory="/home/stefan/R_raw_data/test" extension=".tif" format="GTiff"
g.region -p n=1 s=0 e=1 w=0 res=0.1
r.mapcalc expression="linux_map=1"

On my Win 7 box I did:

d.rast map= linux_map @linux
Here I got the following error message:

Command 'd.rast map=linux_map@linux' failed
Details: ERROR 4: `/home/stefan/R_raw_data/test/linux_map.tif' does not exist in the file system, and is not recognised as a supported dataset name.

Then I tried relative paths:
On Linux:
r.external.out --verbose directory="../../" extension=".tif" format="GTiff"
r.mapcalc expression="linux_map=1" --o

Error message on Windows changes to:
Command 'd.rast map=linux_map@linux' failed
Details: ERROR 4: `/home/stefan/R_raw_data/test/newLocation/linux/../..//linux_map.tif' does not exist in the file system, and is not recognised as a supported dataset name.

2nd test case: Windows to Linux:
On my Windows box (and in the "windows" mapset) I did:
r.external.out --verbose directory="R:\Raw_data\test" extension=".tif" format="GTiff"
r.mapcalc expression=windows_map=1

Which results in the following error message:
ERROR: Unable to make mapset element R:\Raw_data\test (R:\Raw_data\test/newLocation/windows/R:\Raw_data\test): No such file or directory

Then I tried relative paths:
r.external.out --verbose directory="../../" extension=.tif format=GTiff

The relative paths work for the Windows side. However, back on Linux I tried:
d.rast map=windows_map@windows and got the following error message:

Command 'd.rast map=windows_map@windows' failed
Details: ERROR 4: `R:\Raw_data\test/newLocation/windows/../..//windows_map.tif' does not exist in the file system, and is not recognised as a supported dataset name.

Thanks for looking into this.
Should I open a ticket in trac?

Kind regards,
Stefan

4

Blumentrath, Stefan wrote:

Thanks for your reply. Here is a more detailed problem description:

First I created a new and empty GRASS Location "test" on the network
storage (NFS) from the Linux Server. This GRASS DB is mounted on
LINUX (through mount.cifs) to `/home/stefan/R_raw_data/test'. On
Windows the "Network drive" is mapped to " R:\Raw_data\test "

Within this Location I crated a mapset "linux" (from the Linux
Server) and a mapset "windows" from my Windows 7 workstation.

1st test case: Linux to Windows:
On the Linux server, and in the "linux" mapset I did:

r.external.out --verbose directory="/home/stefan/R_raw_data/test" extension=".tif" format="GTiff"
g.region -p n=1 s=0 e=1 w=0 res=0.1
r.mapcalc expression="linux_map=1"

On my Win 7 box I did:

d.rast map= linux_map @linux
Here I got the following error message:

Command 'd.rast map=linux_map@linux' failed
Details: ERROR 4: `/home/stefan/R_raw_data/test/linux_map.tif' does
not exist in the file system, and is not recognised as a supported
dataset name.

Then I tried relative paths:
On Linux:
r.external.out --verbose directory="../../" extension=".tif" format="GTiff"
r.mapcalc expression="linux_map=1" --o

Error message on Windows changes to:
Command 'd.rast map=linux_map@linux' failed
Details: ERROR 4:
`/home/stefan/R_raw_data/test/newLocation/linux/../..//linux_map.tif'
does not exist in the file system, and is not recognised as a
supported dataset name.

Okay, so the issue is the use of absolute paths which may be incorrect
if the database is accessed via a networked filesystem (or even a
local filesystem if the mount point changes).

Currently, if the directory given to r.external.out is relative
(doesn't begin with a "/"), it's converted to an absolute path
relative to the current mapset directory.

Thereafter, any created maps will have the absolute path in their GDAL
link (i.e. the cell_misc/<map>/gdal file).

Thus, maps created with r.external.out can't be read if the database
directory has "moved" relative to its location when the map was
created.

Should I open a ticket in trac?

Yes.

We should at least support paths relative to the mapset, and may need
additional options, e.g. the ability to use environment variables in
paths).

--
Glynn Clements <glynn@gclements.plus.com>