[GRASS-dev] Locking is not supported on Windows

[Marco_Pasetti]

Hi all,

there’s a thing that I never understood: why Locking is not supported on Windows?
Then:

1. what is exactly locking? I tried to find it on the programmer’s manual, but without success

2. I noticed that, when opening a location/mapset, GRASS doesn’t create the .gislock file, as done on linux; without the .gislock file it’s impossibel to change the working environment (that is the mapset…) by the g.mapset module

3. Could you please show me the related chapter/section in the programmer’s manual where I can find information about it?

Thanks,

Marco

[Paul_Kelly]

On Tue, 20 May 2008 marco.pasetti@alice.it wrote:

Hi all,

there's a thing that I never understood: why Locking is not supported on Windows?
Then:

1) what is exactly locking? I tried to find it on the programmer's manual, but without success

2) I noticed that, when opening a location/mapset, GRASS doesn't create the .gislock file, as done on linux; without the .gislock file it's impossibel to change the working environment (that is the mapset...) by the g.mapset module

Hello Marco,

If you start GRASS using grass63.bat (not grass63.sh), yes, you are correct that that won't work. I deliberately left it out of init.bat at the time I wrote it as it would have been a lot of extra work to implement it and it wasn't needed to get basic GRASS functionality working on Windows without the requirement for a Unix shell (which was my goal at the time). You can simply exit and restart GRASS if you want to change the mapset or location.

I suppose you could consider this as another example of the type of thing that will need to be fixed if WinGRASS is to move from alpha to beta-quality software.

3) Could you please show me the related chapter/section in the programmer's manual where I can find information about it?

I believe the facility to change mapset or location without exiting and restarting GRASS (and hence also this .gislock system) was added by Radim Blazek some time during the development of version 5.7 (which eventually became 6.0). So you might try searching for some old mailing list posts by him which might document it better. I'd risk a bet that it isn't documented in the programmers' manual, though it would be nice to be proved wrong

Paul

[Glynn_Clements1]

marco.pasetti@alice.it wrote:

there's a thing that I never understood: why Locking is not supported on Windows?
Then:

1) what is exactly locking? I tried to find it on the programmer's
manual, but without success

Locking is the process of first checking that the .gislock file
doesn't exist, then creating it.

This ensures that you cannot have two GRASS sessions with the same
current mapset.

2) I noticed that, when opening a location/mapset, GRASS doesn't
create the .gislock file, as done on linux;

The etc/lock program which creates the .gislock file doesn't work on
Windows. The program is created, but it just generates the "Locking is
not supported on Windows!" warning and exits without creating the lock
file.

AFAICT, this is excessive. The only part of that module which is
inherently non-portable is the check for a stale lock, i.e. if the
process which created the lock file no longer exists.

It should be possible to just skip that check on Windows, rather than
skipping locking altogether.

without the .gislock
file it's impossibel to change the working environment (that is the
mapset...) by the g.mapset module

Are you sure? AFAICT, etc/lock should always succeed on Windows, i.e.
it won't stop you from having two sessions with the same current
mapset. You will get a warning, but the mapset should still be
changed.

If g.mapset doesn't work, you can always change the GISDBASE,
LOCATION_NAME and MAPSET variables directly with g.gisenv.

Paul Kelly wrote:

If you start GRASS using grass63.bat (not grass63.sh), yes, you are
correct that that won't work. I deliberately left it out of init.bat at
the time I wrote it as it would have been a lot of extra work to implement
it

Where is the problem? Obtaining the PID?

In the worst case, you could just create a lock file with a PID of
zero. Assuming that the find_process() check is removed from etc/lock
(kill() doesn't exist on Windows), the actual PID written to the file
won't matter.

and it wasn't needed to get basic GRASS functionality working on
Windows without the requirement for a Unix shell (which was my goal at the
time). You can simply exit and restart GRASS if you want to change the
mapset or location.

Or use g.gisenv.

I believe the facility to change mapset or location without exiting and
restarting GRASS (and hence also this .gislock system) was added by Radim
Blazek some time during the development of version 5.7 (which eventually
became 6.0).

Actually, adding the ability to change the database/location/mapset
within a session was primarily an issue of changing scripts to get the
information from g.gisenv rather than relying upon the environment
variables being set.

Originally, Init.sh copied the GRASS variables into the environment,
and scripts just used the environment variables directly. This meant
that changing the variables with g.gisenv would only affect C modules
(which used $GISRC), but not scripts.

Radim then wrote g.mapset, which provides a more convenient mechanism
to change the variables (i.e. separate database= etc options rather
than using set=GISDBASE=...), as well as correctly handling locking
(i.e. obtaining the lock for the new mapset and releasing the existing
lock).

--
Glynn Clements <glynn@gclements.plus.com>

[Hamish]

Glynn:

Radim then wrote g.mapset, which provides a more convenient
mechanism to change the variables (i.e. separate database= etc
options rather than using set=GISDBASE=...), as well as correctly
handling locking (i.e. obtaining the lock for the new mapset and
releasing the existing

also it flushes the $MAPSET/.tmp/ dir on leaving, which you miss if you change mapsets with g.gisenv.

H

[Marco_Pasetti]

Hi,

thanks to all for the exhaustive replies.

Are you sure? AFAICT, etc/lock should always succeed on Windows, i.e.
it won’t stop you from having two sessions with the same current
mapset. You will get a warning, but the mapset should still be
changed.

sorry but not. I tried to find out why here http://trac.osgeo.org/grass/browser/grass/branches/develbranch_6/general/g.mapset/main.c but I still don’t know many things in GRASS code to understand it well (and my C is very very rusty). I just found that at lines 155-156 a conditional calls G_fatal_error if the GIS_LOCK env var is not set (var retrieved at line 154).

If g.mapset doesn’t work, you can always change the GISDBASE,
LOCATION_NAME and MAPSET variables directly with g.gisenv.

Yes, it doesn’t produce errors, but it doesn’t change the location/mapset!

Marco

[Glynn_Clements1]

marco.pasetti@alice.it wrote:

Hi,

thanks to all for the exhaustive replies.

>Are you sure? AFAICT, etc/lock should always succeed on Windows, i.e.
>it won't stop you from having two sessions with the same current
>mapset. You will get a warning, but the mapset should still be
>changed.

sorry but not. I tried to find out why here
http://trac.osgeo.org/grass/browser/grass/branches/develbranch_6/general/g.mapset/main.c
but I still don't know many things in GRASS code to
understand it well (and my C is very very rusty). I just found that at
lines 155-156 a conditional calls G_fatal_error if the GIS_LOCK env var
is not set (var retrieved at line 154).

Right. But in that case, it should fail with "Unable to read GIS_LOCK
enviroment variable". If it gets as far as etc/lock generating the
"Locking is not supported on Windows!" warning, that implies that
GIS_LOCK is set.

If GIS_LOCK isn't being set, you can just modify the init.bat script
to set it, e.g.:

  set GIS_LOCK=0

The value doesn't matter; it's only used to detect stale lock files,
and that part won't work on Windows any time soon.

>If g.mapset doesn't work, you can always change the GISDBASE,
>LOCATION_NAME and MAPSET variables directly with g.gisenv.

Yes, it doesn't produce errors, but it doesn't change the
location/mapset!

g.gisenv will definitely work, at least for the command line. It looks
as if the wxPython GUI may maintain its own separate GISRC file; if
that's the case, neither g.mapset nor g.gisenv will affect it.

--
Glynn Clements <glynn@gclements.plus.com>

[Marco_Pasetti]

Hi Glynn,

Right. But in that case, it should fail with “Unable to read GIS_LOCK
enviroment variable”.

as it does

g.gisenv will definitely work, at least for the command line

no, afraid. I also tried with the msys shell (after starting GRASS); for example, with GRASS opened with the demolocation (PERMANENT):

g.gisenv --v set=“LOCATION_NAME=GRASS-Tutorial”

it reports nothing as output, and the location is still the same.

Marco

[Glynn_Clements1]

marco.pasetti@alice.it wrote:

>Right. But in that case, it should fail with "Unable to read GIS_LOCK
enviroment variable".

as it does

In that case, set GIS_LOCK in init.bat.

>g.gisenv will definitely work, at least for the command line

no, afraid. I also tried with the msys shell (after starting GRASS);
for example, with GRASS opened with the demolocation (PERMANENT):

g.gisenv --v set="LOCATION_NAME=GRASS-Tutorial"

it reports nothing as output, and the location is still the same.

Are you saying that running g.gisenv with no arguments continues to
report the old value for LOCATION_NAME?

Or are you being mislead by the shell prompt? Note that whatever
g.gisenv reports for LOCATION_NAME *is* the current location. The
shell prompt is set when you start GRASS, and won't change if you
change the location with g.mapset, g.gisenv, or anything else.

Probably the easiest way to check the current location is to run
g.list.

--
Glynn Clements <glynn@gclements.plus.com>

[Marco_Pasetti]

Hi Glynn,

OK. I summarize:

if I launch GRASS with the wxpython GUI, g.gisenv works, if either launched from shell (MSYS) or from the module GUI; in fact, after the g.gisenv command, if I run g.list -f type=rast I have the list of all the rasters in the new location

if I launch GRASS with the tcltk GUI, g.gisenv doesn’t work, neither if launched from MSYS shell, nor from the module GUI: in fact, after it, if I run g.list it says that the location’s raster list is empty (as it should be, since I’m in the demolocation)

after adding the set GIS_LOCK=0 line in init.bat it works

another thing: the manual says:

g.gisenv [get=VARIABLE] [set=“VARIABLE=value”] [store=string]
[–verbose] [–quiet]

but it’s not correct: (in Windows?) should be:

g.gisenv [set=VARIABLE=value]

in fact, if I type, for example, from the demolocation:

g.gisenv set=“LOCATION_NAME=GRASS-Tutorial”

I have, as output from g.gisenv:

GISDBASE=‘C:/Documents and Settings/Marco/Documenti/GIS DataBase/GRASS Locations’;
LOCATION_NAME=‘demolocation’;
MAPSET=‘PERMANENT’;
“LOCATION_NAME='GRASS-Tutorial”';

While, if I use the suggested syntax I have:

GISDBASE=‘C:/Documents and Settings/Marco/Documenti/GIS DataBase/GRASS Locations’;
LOCATION_NAME=‘GRASS-Tutorial’;
MAPSET=‘PERMANENT’;

Thanks for your patience,

Marco

[Glynn_Clements1]

marco.pasetti@alice.it wrote:

OK. I summarize:

if I launch GRASS with the wxpython GUI, g.gisenv works, if either
launched from shell (MSYS) or from the module GUI; in fact, after the
g.gisenv command, if I run g.list -f type=rast I have the list of all
the rasters in the new location

if I launch GRASS with the tcltk GUI, g.gisenv doesn't work, neither if
launched from MSYS shell, nor from the module GUI: in fact, after it, if
I run g.list it says that the location's raster list is empty (as it
should be, since I'm in the demolocation)

after adding the set GIS_LOCK=0 line in init.bat it works

another thing: the manual says:

g.gisenv [get=VARIABLE] [set="VARIABLE=value"] [store=string]
   [--verbose] [--quiet]

but it's not correct: (in Windows?) should be:

g.gisenv [set=VARIABLE=value]

Right; the quotes in the example usage of set= shouldn't be there, as
quotation depends upon how the command is executed (e.g. via a shell,
in which case, which shell), not the command itself.

Fixed in CVS.

in fact, if I type, for example, from the demolocation:

g.gisenv set="LOCATION_NAME=GRASS-Tutorial"

I have, as output from g.gisenv:

GISDBASE='C:/Documents and Settings/Marco/Documenti/GIS DataBase/GRASS
Locations';
LOCATION_NAME='demolocation';
MAPSET='PERMANENT';
"LOCATION_NAME='GRASS-Tutorial"';

Right; it's taken the opening quote as part of the variable name, and
the closing quote as part of the value.

While, if I use the suggested syntax I have:

GISDBASE='C:/Documents and Settings/Marco/Documenti/GIS DataBase/GRASS
Locations';
LOCATION_NAME='GRASS-Tutorial';
MAPSET='PERMANENT';

Right, so g.gisenv is working.

--
Glynn Clements <glynn@gclements.plus.com>

[Paul_Kelly]

On Wed, 21 May 2008, Glynn Clements wrote:

Paul Kelly wrote:

If you start GRASS using grass63.bat (not grass63.sh), yes, you are
correct that that won't work. I deliberately left it out of init.bat at
the time I wrote it as it would have been a lot of extra work to implement
it

Where is the problem? Obtaining the PID?

I think I just didn't investigate the issues - thought I'd leave implementing that functionality until some later time. But clearly, now is that time.

In the worst case, you could just create a lock file with a PID of
zero. Assuming that the find_process() check is removed from etc/lock
(kill() doesn't exist on Windows), the actual PID written to the file
won't matter.

But does that not mean that if there is a stale lockfile for whatever reason, it can't be removed automatically and the user will never be able to use that mapset? (I'm thinking that Windows users are especially unlikely to go manually looking for the lockfile and deleting it.)

Paul

[Glynn_Clements1]

Paul Kelly wrote:

> In the worst case, you could just create a lock file with a PID of
> zero. Assuming that the find_process() check is removed from etc/lock
> (kill() doesn't exist on Windows), the actual PID written to the file
> won't matter.

But does that not mean that if there is a stale lockfile for whatever
reason, it can't be removed automatically and the user will never be able
to use that mapset? (I'm thinking that Windows users are especially
unlikely to go manually looking for the lockfile and deleting it.)

Correct.

One option would be to make etc/lock provide more detail if it fails
due to the lock being present, including the pathname of the lock
file, and a recommendation to manually delete it if you are certain
that no other GRASS sessions exist.

Another option is to change g.mapset to simply skip the locking step
on Windows. A user starts multiple sessions at their own risk.

Making locking actually work would require knowing how to check the
status of a process on Windows.

--
Glynn Clements <glynn@gclements.plus.com>

[Paul_Kelly]

On Thu, 22 May 2008, Glynn Clements wrote:

Paul Kelly wrote:

In the worst case, you could just create a lock file with a PID of
zero. Assuming that the find_process() check is removed from etc/lock
(kill() doesn't exist on Windows), the actual PID written to the file
won't matter.

But does that not mean that if there is a stale lockfile for whatever
reason, it can't be removed automatically and the user will never be able
to use that mapset? (I'm thinking that Windows users are especially
unlikely to go manually looking for the lockfile and deleting it.)

Correct.

Option 1:

One option would be to make etc/lock provide more detail if it fails
due to the lock being present, including the pathname of the lock
file, and a recommendation to manually delete it if you are certain
that no other GRASS sessions exist.

Option 2:

Another option is to change g.mapset to simply skip the locking step
on Windows. A user starts multiple sessions at their own risk.

Option 3:

Making locking actually work would require knowing how to check the
status of a process on Windows.

I think all three of these options are feasible. Option 2 is starting to appeal to me most. I thought of doing some research into Option 3, but then I thought it isn't really necessary, that the mapset locking is perhaps a bit over-protective. As far as I can think, it is possible to run multiple sessions in the same mapset anyway, if you use WIND_OVERRIDE or GRASS_REGION, so there is really no fundamental need to lock a mapset - it's just an extra encouragement/nagging to enforce good working practices. Or have I missed something?

If I changed this I would make $GISBASE/etc/lock issue a warning about locking of mapsets against concurrent use not being enforced on Windows.

Paul

[Glynn_Clements1]

Paul Kelly wrote:

I think all three of these options are feasible. Option 2 is starting to
appeal to me most. I thought of doing some research into Option 3, but
then I thought it isn't really necessary, that the mapset locking is
perhaps a bit over-protective. As far as I can think, it is possible to
run multiple sessions in the same mapset anyway, if you use WIND_OVERRIDE
or GRASS_REGION, so there is really no fundamental need to lock a mapset -
it's just an extra encouragement/nagging to enforce good working
practices. Or have I missed something?

You also need to ensure that you don't try to modify a map (etc) from
two sessions concurrently.

OTOH, even with two sessions using different mapsets, you can
theoretically run into problems with one session reading a map which
is being modified by the other session. But I have yet to hear of
anyone actually being bitten by it.

But, yes, WIND is the main issue, and WIND_OVERRIDE should deal with
that (except that the wxPython GUI reads the WIND file directly, which
is a pretty serious bug).

The VAR file could potentially be an issue, as could the CURGROUP and
CURSUBGROUP files. There may be other files which I have overlooked.

Personally, I would like to see both WIND and CUR[SUB]GROUP moved into
$GISRC.

The database stuff in VAR should also be able to be overridden by
$GISRC, with VAR being used as a fallback (similar to DEFAULT_WIND for
the region).

--
Glynn Clements <glynn@gclements.plus.com>

[Hamish]

Glynn:

Personally, I would like to see both WIND and CUR[SUB]GROUP
moved into $GISRC.

The database stuff in VAR should also be able to be
overridden by $GISRC, with VAR being used as a fallback (similar
to DEFAULT_WIND for the region).

these things are all mapset specific, so do you specifically mean moved into /tmp/grass6-$USER-$PID/gisrc not ~/.gisrc, correct?

e.g. last used region obviously changes between ll and utm locations; it is nice to remember the last ones used per mapset between sessions; for VAR, some datasets prefer a heavier DB by default without imposing that as the default on all others, etc.

just checking,
Hamish

[Glynn_Clements1]

Hamish wrote:

> Personally, I would like to see both WIND and CUR[SUB]GROUP
> moved into $GISRC.
>
> The database stuff in VAR should also be able to be
> overridden by $GISRC, with VAR being used as a fallback (similar
> to DEFAULT_WIND for the region).

these things are all mapset specific, so do you specifically mean
moved into /tmp/grass6-$USER-$PID/gisrc not ~/.gisrc, correct?

Yes. ~/.grassrc6 is only used by Init.sh; everything else uses $GISRC.

e.g. last used region obviously changes between ll and utm locations;
it is nice to remember the last ones used per mapset between sessions;

Define "last used" in the case when you are using both the command
line and the GUI within the same mapset, but with different regions.

for VAR, some datasets prefer a heavier DB by default without imposing
that as the default on all others, etc.

If you are going to record per-mapset defaults, the contents should be
copied into $GISRC when it is created, when you switch mapsets, etc,
and the contents of $GISRC should be used thereafter. Library
functions shouldn't be reading these files directly.

Essentially, $GISRC should contain the entire session state. You can
have multiple processes with separate values for $GISRC, but you can't
have multiple WIND/VAR/CUR[SUB]GROUP files in one mapset.

If you want to store defaults in a mapset directory, there should be a
single file (e.g. VAR) for everything (i.e. replace the WIND and
CUR[SUB]GROUP files with VAR entries).

--
Glynn Clements <glynn@gclements.plus.com>