On Tue, Jun 2, 2015 at 7:50 AM, Moritz Lennert <mlennert@club.worldonline.be> wrote:
On 02/06/15 03:05, Vaclav Petras wrote:
Hi all,
I need opinions on how to resolve the question whether GRASS session can
be started from another GRASS session.
Here we want to use GISBASE from environment instead of build constant
if available:
https://trac.osgeo.org/grass/changeset/45576
However, here we cause error when GISBASE already exists:
http://trac.osgeo.org/grass/changeset/59210
But the check was removed in:
http://trac.osgeo.org/grass/changeset/59220
because of:
http://trac.osgeo.org/grass/ticket/2213
The point of r59210 was to make impossible to start another GRASS
session within an existing session.
Even though I see this happen regularly amongst my students,
Interesting. I actually haven’t seen it in this way. I’m concerned about scripting, other applications and advanced users.
I’m not sure that I consider this such a problem to justify significantly changing the existing logic.
… and apparently it is not trivial (r59210 and r59220).
Have you come across any serious issues related to this ?
I don’t know about any problems and even the commit r59210 which did the check and failed with present GISBASE doesn’t disclose any motivation. Maybe Martin remembers… Markus is saying that it is asking for trouble. Perhaps, but on the other hand, a new session overrides a lot of variables and surely allows to at least use different data (but MANPATH and PYTHONPATH should be OK too). But also, there is a lot of variables which are not overridden (e.g. GRASS_HTML_BROWSER which is OK, but also discussed GISBASE which is a catastrophe if the version/installation is different).
The whole notion of a “session” is a bit misleading as all a “session” is (IIUC) is a specific state of environment variables.
You are right that whole notion of a “session” is a mess. GRASS session has two parts.
First, runtime environment for modules which is a set of variables such as PATH, LD_LIBRARY_PATH or PYTHONPATH.
Second, data linked using GISRC file and variable. In a GRASS session, data, specifically Mapset, is locked to prevent other GRASS sessions from modifying it. There is a “gisrc” file somewhere which contains information about current GRASS Database directory, Location and Mapset (basically full path to mapset) and optionally other variables. Path to this file is stored in GISRC variable and it does not change during the GRASS session, although subprocess may change GISRC variable to do some special things (e.g. Mapset-based parallelization).
GRASS session by itself can change which data (Mapset) are linked/connected but it cannot (should not) change the GISRC variable (“gisrc” file path). Runtime environment cannot be changed by the session itself which is both technical and design limitation.
If one just sets up the required variables, there is not locking of the Mapset nor cleaning of temporary directory in a Mapset (other managed temporary file is for example “gisrc” file). GRASS session can be understood as a process which starts and ends with all these steps besides providing the right environment.
Then perhaps it seems that we can allow having runtime environment in the system all the time and the only thing where we may (or may not) need to limit nesting is a fully working environment which means that GISRC environment variable is present. But the latter test also does have to be present unless we want to really ban interactive session in another session.
This would leave use just with the issue of getting GISBASE in case of OSGeo4W.
So if someone wants to switch GRASS version during a working session, this should remain possible by just changing these environment variables, i.e. after having launched trunk, switch to the grass70 release branch:
export GISBASE=/data/home/mlennert/SRC/GRASS/grass70_release/dist.x86_64-unknown-linux-gnu/
export LD_LIBRARY_PATH=/data/home/mlennert/SRC/GRASS/grass70_release/dist.x86_64-unknown-linux-gnu/lib/
export PATH=/data/home/mlennert/SRC/GRASS/grass70_release/dist.x86_64-unknown-linux-gnu/bin/:/data/home/mlennert/SRC/GRASS/grass70_release/dist.x86_64-unknown-linux-gnu/scripts/:/home/mlennert/.grass7/addons/bin:/home/mlennert/.grass7/addons/scripts:/home/mlennert/.grass7/addons/:/home/mlennert/BIN:/home/mlennert/BIN:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/opt/libreoffice/program:/home/mlennert/SRC/FWTools-2.0.6/bin_safe
or similar (there might be some other variables that need to be changed).
Why should this be bad ? But I see your argument that (in this case) the grass70 starting (better name would be “env_setup”) script should actually force setting the GISBASE (and the others) to the path where grass70 is installed, instead of taking the existing setting from the environment, thus allowing easy switching of versions.
Exactly, but OSGeo4W requires setting of GISBASE from outside because GISBASE from compilation is wrong. I’m not sure if this is possible on some other platforms or if there is some other way around than setting a variable.
I still can see only two possible solutions. Use GRASS_GISBASE (or whatever name) variable when present in cases that the distribution cannot ensure the right path in grass.py. GRASS_GISBASE would be used only in a distribution custom script which subsequently invokes grass.py. I hope that GRASS_GISBASE wouldn’t be used in scripts because the need for it would be already satisfied in whatever is behind grass7 command (which should be used in the scripts instead or with the actual GISBASE).
The second solution is to use existing GISBASE variable as a fallback when the path from compile time does not exist. I’m not sure how reliable this can be tested. Is it possible that the directly will exist but is unusable (and user indeed wants to use the existing GISBASE) or there are some strange permission issues and it would say it does not exist just because we cannot write there or something? Pseudo-code for both solutions is in my previous email to Markus.
However, r45576 is using exactly
same check (GISBASE in env) to see if GISBASE is forced from outside.
The two places for it are here:
https://trac.osgeo.org/grass/browser/grass/trunk/lib/init/grass.py?annotate=blame&rev=65348#L48
https://trac.osgeo.org/grass/browser/grass/trunk/lib/init/grass.py?annotate=blame&rev=65348#L1633
I think starting non-interactive session (batch/exec mode) should be
possible. Perhaps interactive session (–text & --gui) should have the
check.
Non interactive sessions should always be possible since you can set all the env variables in the script and IIUC these will only be valid during the execution of the script.
By non-interactive session I mean an actual run of grass7 command* but without jumping to the command line. This can be now achieved by --exec or BATCH_MODE. Perhaps interactive is not a right word since you can actually start GUI using the --exec flag, so what you need and then close it. The difference is in the presence of a prompt and execution of a process specified when starting the session.
Manual setting of the environment with or without the help of grass.script.setup.init function should be always possible. However, I think it is a pain and I would like to see it replaced by usage of “grass71 --exec” interface as it is easier, safer and more reliable (e.g. locking, ctypes works). However, I also see that changing the environment directly in the script has it’s advantages, most importantly you can use grass.script interface without creating another file, so I’m also trying to simplify, or at least not complicate, this procedure.
- I really wonder how should I call that. I would like to see all distributions to use just grass, so one don’t have to think if it is 70 or 71 and write specific number only if necessary (like with Python). And it is a command, program or an app?
If the check is present (even just for the interactive sessions), the
OSGeo4W version (and probably standalone winGRASS too) is going to break
(needs to be changed somehow).
IIRC, this is just due to the check being in grass.py, but wingrass setting the env variable in the bat startup script.
I think that the bat script just calls the grass.py file:
https://trac.osgeo.org/grass/browser/grass/trunk/mswindows/osgeo4w/grass.bat.tmpl
If we don’t check for existing session using GISBASE or we use one of the solutions above for getting the right GISBASE. there should be no problem.
Moreover, this would disallow setting
some GISBASE to system for scripting purposes (“without starting
explictly”) and then starting a standard session.
The issue with the current state is that you cannot run different
version of GRASS from a current session.
AFAIU, you can. You just have to set the env variables correctly.
If we would use GISBASE to check for an existing session, you wouldn’t be able to put GISBASE to your system environment and then start GRASS using grass7 command. If check applied also for --exec, you would have to unset GISBASE in any script which would use --exec and which should be portable. This would complicate the manual setup which is not acceptable.
More serious issue is that you
cannot start another version if you have GISBASE in your system
environment for scripting purposes (as mentioned above).
If you don’t want to have this problem, just don’t set a system-wide / user-session wide GISBASE. You can always set it explicitely in your scripts.
True, but there are people who want to use GRASS modules directly in the command line and I also don’t want to have complicated scripts with a lot of hardcoded paths (preferably no hardcoded paths). The alternative for scripts is to have grass7 command/program on path and then get GISBASE from it. I like that but I think that it is still quite long which may be the reason why I haven’t seen it used much. The other reason might be that grass is not on path on MS Windows by default, nor is the possibility advertised in doc/wiki such as here:
http://grasswiki.osgeo.org/wiki/Working_with_GRASS_without_starting_it_explicitly
The current
behavior may also differ depending on a platform.
I’m not really sure what were the original intentions but I would prefer
to switch the logic in r45576 and use GISBASE only if the path from
compile time does not exist.
This would mean as I said above that calling the “starting” script of a specific version would always by default set the relevant variables to that version. I can agree to that.
This allows to override current session
(GISBASE set in some way) but allows to supply the GISBASE when there is
something wrong with compile time path.
You mean that we would have some fallback mechanism allowing to overwrite the default from the command line (i.e. something like this: grass70 --use-env-variables) ?
Yes, we even already have it but because of usage of GISBASE variable there is great potential for problems as discussed above. I now suggest that either invalid GISBASE from compile time or presence of another variable (e.g. GRASS_GISBASE) would use it. However, I’m not sure what are the issues with either of these solutions. I wonder if there is some other solution (like no GISBASE path from compilation, better distribution/install process, …). The solution with a parameter (–use-env-variables or --gisbase “/the/path”) also crossed my mind and there already was a similar interface for Location and Mapset (to --use-env-variables) but it seems to me that environmental variable is more appropriate interface in this case considering the use cases (distribution, system-driven invocation) as opposed to user/programmer interface which should be as clean as possible (and --gisbase “/the/path” is not nice, even just in --help doc).
If environmental variable has a special name (such as GRASS_GIS which does not overlap with in-session name), there is no need for --use-env-variables but this kind of parameter could be helpful when you would like to use GRASS default for GRASS_HTML_BROWSER although you have something set in the system but I don’t think that this useful (and if you need it then I think is OK to do things like unset the variable).
The fallback is need for OSGeo4W and perhaps other distributions as the actual (runtime) path may not be known during compilation.
I think this discussion is relevant, but please do not go through any premature changes before we haven’t thought this through.
This is my intention exactly. Thanks for the comments so far, it was very helpful for me in sorting out things and thinking about solutions. There is a lot of moving peaces. I also believe that I cleaned up grass.py significantly, so I invite anybody who is interested to examine actual code and commend on it.
Vaclav
