[GRASS-user] can I access mapset outside of grass, by using python?

Dear friends,

I can run some simple scripts on python if I start
Msys → grass64 → python myscrypt.py

But now I would like to know if I can build a python
code outside of a msys/grass session, and get
access on a mapset withour stay running grass.

thanks a lot,

milton

Milton Cezar Ribeiro wrote:

I can run some simple scripts on python if I start
Msys -> grass64 -> python myscrypt.py

But now I would like to know if I can build a python
code outside of a msys/grass session, and get
access on a mapset withour stay running grass.

The GRASS libraries require certain environment variables to be set. A
"GRASS session" is just a set of processes (e.g. a shell and/or GUI)
which have the necessary environment settings.

Specifically:

GISBASE needs to be set to the top-level directory of the GRASS
installation.

GISRC needs to contain the absolute path to a file containing settings
for GISDBASE, LOCATION_NAME and MAPSET.

PATH needs to include $GISBASE/bin and $GISBASE/scripts.

If the GRASS libraries are shared libraries, the loader needs to be
able to find them. This normally means that LD_LIBRARY_PATH (Linux,
Solaris), DYLD_LIBRARY_PATH (MacOSX) or PATH (Windows) need to contain
$GISBASE/lib, although there are other means to the same end (e.g. on
Linux, putting $GISBASE/lib (with $GISBASE replaced by its actual
value) into /etc/ld.so.conf then running ldconfig).

Some libraries and modules use other variables. See the file
$GISBASE/docs/html/variables.html for most of them (the display
libraries used by d.* commands use additional variables, which are
documented along with the individual drivers).

Personally, I have the following run from my ~/.bash_profile script:

  export GISBASE=/usr/local/src/grass/svn/dist.i686-pc-linux-gnu
  export GRASS_GNUPLOT='gnuplot -persist'
  export GRASS_WIDTH=640
  export GRASS_HEIGHT=480
  export GRASS_HTML_BROWSER=firefox
  export GRASS_PAGER=cat
  export GRASS_WISH=wish
  export GRASS_PYTHON=python
  export GRASS_MESSAGE_FORMAT=silent
  export GRASS_TRUECOLOR=TRUE
  export GRASS_TRANSPARENT=TRUE
  export GRASS_PNG_AUTO_WRITE=TRUE
  
  export PATH="$GISBASE/bin:$GISBASE/scripts:$PATH"
  export LD_LIBRARY_PATH="$GISBASE/lib"
  export GRASS_LD_LIBRARY_PATH="$LD_LIBRARY_PATH"
  export PYTHONPATH="$GISBASE/etc/python:$PYTHONPATH"
  export MANPATH=$MANPATH:$GISBASE/man
  
  export GIS_LOCK=$$
  export GRASS_VERSION="7.0.svn"
  
  tmp=/tmp/grass6-"`whoami`"-$GIS_LOCK
  export GISRC="$tmp/gisrc"
  mkdir "$tmp"
  cp ~/.grassrc6 "$GISRC"

This allows GRASS commands to be used anywhere (my ~/.Xsession sources
the bash startup scripts, so the settings aren't limited to
interactive shells, but also work for e.g. M-! in XEmacs). Each
interactive shell gets a separate "session" (i.e. a separate $GISRC
file), while GUI programs share a common session.

The above is for GRASS 7, and uses the version from the dist.<arch>
directory in the GRASS source tree rather than an installed version.

The only time I ever start GRASS explicitly is if I need to test
changes to the startup scripts, or test a 6.x version.

--
Glynn Clements <glynn@gclements.plus.com>

Hi Glynn!

I've set-up my system based on your detailed descriptions/instructions
below (old post) using grass64.

1. Would you mind sharing in addition the way you change between
mapsets/locations? Do you start a new shell session after re-defining
somehow the variables LOCATION_NAME or/and MAPSET? Use just g.gisenv?

2. Also, I have compiled (but _not_ installed) grass65 and grass70 and
run them from "dist" directory, i.e. I launch grass70 via
"../grass_trunk/dist.x86_64-unknown-linux-gnu/grass70.tmp".

Now, I launch grass70 for testing, I try "d.mon" and I get:
---
Incompatible library version for module. You need to rebuild GRASS
       or untangle multiple installations.
---

I would expect the module not to be found at all since it doesn't exist
in grass70. Could this be a problem? Is there a way to lock-out all
other versions of grass-modules from being detectable when I am already
inside a grass70 session?

Thanks, Nikos

/----------------------------------------------------------------------/

On Fri, 2009-08-07 at 19:54 +0100, Glynn Clements wrote:

The GRASS libraries require certain environment variables to be set. A
"GRASS session" is just a set of processes (e.g. a shell and/or GUI)
which have the necessary environment settings.

Specifically:

GISBASE needs to be set to the top-level directory of the GRASS
installation.

GISRC needs to contain the absolute path to a file containing settings
for GISDBASE, LOCATION_NAME and MAPSET.

PATH needs to include $GISBASE/bin and $GISBASE/scripts.

If the GRASS libraries are shared libraries, the loader needs to be
able to find them. This normally means that LD_LIBRARY_PATH (Linux,
Solaris), DYLD_LIBRARY_PATH (MacOSX) or PATH (Windows) need to contain
$GISBASE/lib, although there are other means to the same end (e.g. on
Linux, putting $GISBASE/lib (with $GISBASE replaced by its actual
value) into /etc/ld.so.conf then running ldconfig).

Some libraries and modules use other variables. See the file
$GISBASE/docs/html/variables.html for most of them (the display
libraries used by d.* commands use additional variables, which are
documented along with the individual drivers).

Personally, I have the following run from my ~/.bash_profile script:

  export GISBASE=/usr/local/src/grass/svn/dist.i686-pc-linux-gnu
  export GRASS_GNUPLOT='gnuplot -persist'
  export GRASS_WIDTH=640
  export GRASS_HEIGHT=480
  export GRASS_HTML_BROWSER=firefox
  export GRASS_PAGER=cat
  export GRASS_WISH=wish
  export GRASS_PYTHON=python
  export GRASS_MESSAGE_FORMAT=silent
  export GRASS_TRUECOLOR=TRUE
  export GRASS_TRANSPARENT=TRUE
  export GRASS_PNG_AUTO_WRITE=TRUE
  
  export PATH="$GISBASE/bin:$GISBASE/scripts:$PATH"
  export LD_LIBRARY_PATH="$GISBASE/lib"
  export GRASS_LD_LIBRARY_PATH="$LD_LIBRARY_PATH"
  export PYTHONPATH="$GISBASE/etc/python:$PYTHONPATH"
  export MANPATH=$MANPATH:$GISBASE/man
  
  export GIS_LOCK=$$
  export GRASS_VERSION="7.0.svn"
  
  tmp=/tmp/grass6-"`whoami`"-$GIS_LOCK
  export GISRC="$tmp/gisrc"
  mkdir "$tmp"
  cp ~/.grassrc6 "$GISRC"

This allows GRASS commands to be used anywhere (my ~/.Xsession sources
the bash startup scripts, so the settings aren't limited to
interactive shells, but also work for e.g. M-! in XEmacs). Each
interactive shell gets a separate "session" (i.e. a separate $GISRC
file), while GUI programs share a common session.

The above is for GRASS 7, and uses the version from the dist.<arch>
directory in the GRASS source tree rather than an installed version.

The only time I ever start GRASS explicitly is if I need to test
changes to the startup scripts, or test a 6.x version.

Νίκος Αλεξανδρής wrote:

I've set-up my system based on your detailed descriptions/instructions
below (old post) using grass64.

1. Would you mind sharing in addition the way you change between
mapsets/locations? Do you start a new shell session after re-defining
somehow the variables LOCATION_NAME or/and MAPSET? Use just g.gisenv?

Use g.mapset.

2. Also, I have compiled (but _not_ installed) grass65 and grass70 and
run them from "dist" directory, i.e. I launch grass70 via
"../grass_trunk/dist.x86_64-unknown-linux-gnu/grass70.tmp".

Now, I launch grass70 for testing, I try "d.mon" and I get:
---
Incompatible library version for module. You need to rebuild GRASS
       or untangle multiple installations.
---

I would expect the module not to be found at all since it doesn't exist
in grass70. Could this be a problem? Is there a way to lock-out all
other versions of grass-modules from being detectable when I am already
inside a grass70 session?

The grassXY scripts prepend the GRASS directories to PATH,
LD_LIBRARY_PATH, etc. They won't remove any entries which are already
there.

I only use the grassXY scripts if I actually need to test the startup
process. To switch versions, I use the following script:

  strippath()
  {
      (
      oldpath="$1"
      newpath=
      IFS=:
      for dir in $oldpath ; do
    case "${dir}" in
    *grass*)
      ;;
    *)
      newpath="$newpath:$dir"
      ;;
    esac
      done
      echo "${newpath#:}"
      )
  }
  
  PATH=`strippath $PATH`
  
  export GISBASE=$PWD/dist.i686-pc-linux-gnu
  export PATH="$GISBASE/bin:$GISBASE/scripts:$PATH"
  export LD_LIBRARY_PATH="$GISBASE/lib"
  export PYTHONPATH="$GISBASE/etc/python"

Note: the above script needs to be "source"d; executing it won't work.

--
Glynn Clements <glynn@gclements.plus.com>

Glynn,
thank you so much for sharing :slight_smile:

I understand that this "way" of using grass isn't recommended for (let
us say) beginners. Nevertheless, my aim was/is to put most of this
thread in a wiki-page because I consider it as very practical.

Before doing so I want to test more to look for "gaps" and ask for
possible problems. More below...

Νίκος wrote:

> 1. Would you mind sharing in addition the way you change between
> mapsets/locations? Do you start a new shell session after re-defining
> somehow the variables LOCATION_NAME or/and MAPSET? Use just g.gisenv?

Glynn wrote:

Use g.mapset.

Right!

> Is there a way to lock-out all other versions of grass-modules from
> being detectable when I am already inside a grass70 session?

The grassXY scripts prepend the GRASS directories to PATH,
LD_LIBRARY_PATH, etc. They won't remove any entries which are already
there.

I only use the grassXY scripts if I actually need to test the startup
process. To switch versions, I use the following script:

[...]

Note: the above script needs to be "source"d; executing it won't work.

Didn't test that yet. Trying now...

.

The thing that troubles me most currently is: using normal grass
sessions you get entries recorded in
"/grassdb/location/mapset/.bash_history" while working with "pure" bash
shell entries go in "~/.bash_history".

- Merging existing "grass-bash_history(-ies)" with "bash_history" would
probably be NOT a good idea.

- Trying to separate (somehow) grass-commands history-entries and other
shell actions related to a specific grass-project (speak
grassdb/project/location/mapset) from other grass-projects or
non-grass-projects is, I think, non-sense.

So, when working on a project, it is best practice to stick with
grass-sessions?

Sorry if I insist so much about this,
Nikos

Nikos wrote:

So, when working on a project, it is best practice to stick
with grass-sessions?

There will be a number of opinions on this, and there are probably a number
of valid answers depending on the use-case and expertise level of the user.
Mine is that it is useful to use init.sh sessions. You mention the
.bash_history eg: often I will look at the $MAPSET/.bash_history months
later to figure out what I did last time.

Hamish

Nikos wrote:

> So, when working on a project, it is best practice to stick
> with grass-sessions?

Hamish wrote:

There will be a number of opinions on this, and there are probably a number
of valid answers depending on the use-case and expertise level of the user.

( ...I guess there is no thing like "The Truth" :wink:

Mine is that it is useful to use init.sh sessions. You mention the
.bash_history eg: often I will look at the $MAPSET/.bash_history months
later to figure out what I did last time.

Right! I was doing that today. I could not remember a thing of what I
did back in November09 :open_mouth:

Nikos Alexandris wrote:

The thing that troubles me most currently is: using normal grass
sessions you get entries recorded in
"/grassdb/location/mapset/.bash_history" while working with "pure" bash
shell entries go in "~/.bash_history".

- Merging existing "grass-bash_history(-ies)" with "bash_history" would
probably be NOT a good idea.

- Trying to separate (somehow) grass-commands history-entries and other
shell actions related to a specific grass-project (speak
grassdb/project/location/mapset) from other grass-projects or
non-grass-projects is, I think, non-sense.

So, when working on a project, it is best practice to stick with
grass-sessions?

You can control where history entries are written via $HISTFILE and
the "history" command. It's up to the user how they want to manage
their shell history.

--
Glynn Clements <glynn@gclements.plus.com>

Νίκος:

Is there a way to lock-out all
> other versions of grass-modules from being detectable when I am already
> inside a grass70 session?

Glynn:

The grassXY scripts prepend the GRASS directories to PATH,
LD_LIBRARY_PATH, etc. They won't remove any entries which are already
there.

I only use the grassXY scripts if I actually need to test the startup
process. To switch versions, I use the following script:

I get the following error:
--%<---
/geo/osgeo/src/grass6_devel/dist.x86_64-unknown-linux-gnu/etc/prompt.sh:
2: g.gisenv: not found
/geo/osgeo/src/grass6_devel/dist.x86_64-unknown-linux-gnu/etc/prompt.sh:
2: g.gisenv: not found
/geo/osgeo/src/grass6_devel/dist.x86_64-unknown-linux-gnu/etc/prompt.sh:
2: g.gisenv: not found
--%<---

So I did some modifications (see below). Sorry for the noob questions
(and editing).

-- code --

  strippath()
  {
      (
      oldpath="$1"
      newpath=
      IFS=:
      for dir in $oldpath ; do
    case "${dir}" in
    *grass*)
      ;;
    *)
      newpath="$newpath:$dir"
      ;;
    esac
      done
      echo "${newpath#:}"

--> here I added "unset IFS": wrong, right or not required?

      )
  }
  
  PATH=`strippath $PATH`

I changed that to:
         PATH="$GISBASE/bin:$GISBASE/scripts:`strippath $PATH`"

  export GISBASE=$PWD/dist.i686-pc-linux-gnu

-(a)-> changed the 2nd part to "dist.x86_64-unknown-linux-gnu"
-(b)-> I don't understand the logic behind "$PWD" here. From within a
grass65 or grass70 session why need to re-define $GISBASE? It's already
set.

  export PATH="$GISBASE/bin:$GISBASE/scripts:$PATH"

(moved up to the "PATH=...")

  export LD_LIBRARY_PATH="$GISBASE/lib"
  export PYTHONPATH="$GISBASE/etc/python"

-- end of code --

Note: the above script needs to be "source"d; executing it won't work.

OK! It works like that for me when sourcing inside a grass65, grass70
session. Paths point to wanted grass version. Or am I in the wrong
direction?

Thanks, Nikos

Nikos Alexandris wrote:

> The thing that troubles me most currently is: using normal grass
> sessions you get entries recorded in
> "/grassdb/location/mapset/.bash_history" while working with "pure" bash
> shell entries go in "~/.bash_history".
>
> - Merging existing "grass-bash_history(-ies)" with "bash_history" would
> probably be NOT a good idea.
>
> - Trying to separate (somehow) grass-commands history-entries and other
> shell actions related to a specific grass-project (speak
> grassdb/project/location/mapset) from other grass-projects or
> non-grass-projects is, I think, non-sense.
>
> So, when working on a project, it is best practice to stick with
> grass-sessions?

Glynn Clements:

You can control where history entries are written via $HISTFILE and
the "history" command. It's up to the user how they want to manage
their shell history.

I've already have re-touched this trying to unify history across all
used shells. The relevant part of my .bashrc looks like:

--%<---
# append to the history file, don't overwrite it
shopt -s histappend

# ... or force ignoredups and ignorespace
HISTCONTROL=erasedups:ignoreboth # does this work?

# HISTIGNORE
HISTIGNORE='&:ls:ll:[bf]g:cd ~:cd ..:history *:clear:exit'

# for setting history length see HISTSIZE and HISTFILESIZE in bash(1)
HISTSIZE=10000
HISTFILESIZE=15000

# check the window size after each command and, if necessary,
# update the values of LINES and COLUMNS.
shopt -s checkwinsize

# multi-line commands
shopt -s cmdhist
--%<---

I think it's going to be really hard to mess up this more (?). Anyhow,
thank you for the feedback so far - no need to take this thread further,
I guess :slight_smile:

Nikos

Apologies for a second post with the code. Just to make it easier to
read/ understand what I've changed.

Νίκος Αλεξανδρής wrote:

> Is there a way to lock-out all other versions of grass-modules from
> being detectable when I am already inside a grass70 session?

Glynn Clements wrote:

The grassXY scripts prepend the GRASS directories to PATH,
LD_LIBRARY_PATH, etc. They won't remove any entries which are already
there.

I only use the grassXY scripts if I actually need to test the startup
process. To switch versions, I use the following script:

  strippath()
  {
      (
      oldpath="$1"
      newpath=
      IFS=:
      for dir in $oldpath ; do
    case "${dir}" in
    *grass*)
      ;;
    *)
      newpath="$newpath:$dir"
      ;;
    esac
      done
      echo "${newpath#:}"
      )
  }
  
  PATH=`strippath $PATH`
  
  export GISBASE=$PWD/dist.i686-pc-linux-gnu
  export PATH="$GISBASE/bin:$GISBASE/scripts:$PATH"
  export LD_LIBRARY_PATH="$GISBASE/lib"
  export PYTHONPATH="$GISBASE/etc/python"

Note: the above script needs to be "source"d; executing it won't work.

-- slightly modified code --

strippath()
{
    (
    oldpath="$1"
    newpath=
    IFS=:
    for dir in $oldpath ; do
        case "${dir}" in
        *grass*)
                ;;
        *)
                newpath="$newpath:$dir"
                ;;
        esac
    done
    echo "${newpath#:}"
    )
    unset IFS
}

export PATH="$GISBASE/bin:$GISBASE/scripts:`strippath $PATH`"
export LD_LIBRARY_PATH="$GISBASE/lib"
export PYTHONPATH="$GISBASE/etc/python"

-- end of slightly modified code --

Nikos

Nikos Alexandris wrote:

> Is there a way to lock-out all
> > other versions of grass-modules from being detectable when I am already
> > inside a grass70 session?

Glynn:
> The grassXY scripts prepend the GRASS directories to PATH,
> LD_LIBRARY_PATH, etc. They won't remove any entries which are already
> there.
>
> I only use the grassXY scripts if I actually need to test the startup
> process. To switch versions, I use the following script:

I get the following error:
--%<---
/geo/osgeo/src/grass6_devel/dist.x86_64-unknown-linux-gnu/etc/prompt.sh:
2: g.gisenv: not found

So I did some modifications (see below). Sorry for the noob questions
(and editing).

--> here I added "unset IFS": wrong, right or not required?

Not required; the IFS setting is with a subshell, for this reason.

> )
> }
>
> PATH=`strippath $PATH`
I changed that to:
         PATH="$GISBASE/bin:$GISBASE/scripts:`strippath $PATH`"

Wait until after $GISBASE has been set.

> export GISBASE=$PWD/dist.i686-pc-linux-gnu
-(a)-> changed the 2nd part to "dist.x86_64-unknown-linux-gnu"

Right; that would explain the "g.gisenv: not found" errors. These
scripts were constructed for personal use, so they reflect my system.

-(b)-> I don't understand the logic behind "$PWD" here. From within a
grass65 or grass70 session why need to re-define $GISBASE? It's already
set.

The scripts exist to change the "active" version of GRASS. I set all
of the environment variables from a ~/.bashrc.grass file which is
sourced by my ~/.bash_profile file, so GRASS commands work everywhere.
The environment variables are initially set to use 7.0-svn.

But each version of GRASS which I have checked out has a copy of the
scripts. "source"ing the script will cause that particular version of
GRASS to become "active", i.e. $GISBASE, $PATH, $LD_LIBRARY_PATH and
$PYTHONPATH will all refer to the copy of GRASS in $PWD/dist.i686-pc-linux-gnu.

--
Glynn Clements <glynn@gclements.plus.com>

Nikos Alexandris wrote:

> > Is there a way to lock-out all
> > > other versions of grass-modules from being detectable when I am already
> > > inside a grass70 session?

Glynn Clements wrote:

> > The grassXY scripts prepend the GRASS directories to PATH,
> > LD_LIBRARY_PATH, etc. They won't remove any entries which are already
> > there.
> >
> > I only use the grassXY scripts if I actually need to test the startup
> > process. To switch versions, I use the following script:
>
> I get the following error:
> --%<---
> /geo/osgeo/src/grass6_devel/dist.x86_64-unknown-linux-gnu/etc/prompt.sh:
> 2: g.gisenv: not found

> So I did some modifications (see below). Sorry for the noob questions
> (and editing).

> --> here I added "unset IFS": wrong, right or not required?

Glynn Clements wrote:

Not required; the IFS setting is with a subshell, for this reason.

> > )
> > }
> >
> > PATH=`strippath $PATH`

> I changed that to:
> PATH="$GISBASE/bin:$GISBASE/scripts:`strippath $PATH`"

Wait until after $GISBASE has been set.

> > export GISBASE=$PWD/dist.i686-pc-linux-gnu
> -(a)-> changed the 2nd part to "dist.x86_64-unknown-linux-gnu"

Right; that would explain the "g.gisenv: not found" errors.

Actually the error insists and _only_ after I've used the above "PATH"
setting it went away... (?).

The piece of info that I did not provide and makes perhaps the
difference is that I use grass64 as default (see below). I could
describe all of the details if wanted, but I guess you are probably busy
with more important stuff.

These
scripts were constructed for personal use, so they reflect my system.

> -(b)-> I don't understand the logic behind "$PWD" here. From within a
> grass65 or grass70 session why need to re-define $GISBASE? It's already
> set.

The scripts exist to change the "active" version of GRASS. I set all
of the environment variables from a ~/.bashrc.grass file which is
sourced by my ~/.bash_profile file, so GRASS commands work everywhere.
The environment variables are initially set to use 7.0-svn.

Well, I've used your instructions in order to have grass64 set as
default (e.g.:

- in ~/.bashrc I have:

-- code ---
# ~/.bash_profile
if [ -f ~/.bash_profile ] ; then
    . ~/.bash_profile
fi
-- end of code ---

- in ~/.bash_profile I have (after your instructions, 2nd post, current
thread):

-- code ---
export GISBASE=/usr/local/grass-6.4.0svn
# and all of the of the other grass-Variables
[...]
export GRASS_VERSION="6.4.0svn"
[...]
-- end of code ---

Maybe this explains it better.

But each version of GRASS which I have checked out has a copy of the
scripts. "source"ing the script will cause that particular version of
GRASS to become "active", i.e. $GISBASE, $PATH, $LD_LIBRARY_PATH and
$PYTHONPATH will all refer to the copy of GRASS in $PWD/dist.i686-pc-linux-gnu.

I think I understand the concept and I adjusted it to my needs correctly
(don't see any errors and checking the paths I get what I expect to
see). I'll stick with this configuration as it's very flexible.

Nevertheless, whenever I'll _work_ on a project I'll prefer a
grass-session (to have a per-session history) or when I want to use
grass6_devel/grass70, again I'll use a session and source the strippath
function.

Thank you Glynn,
Nikos

On Mon, 2010-03-08 at 01:20 +0100, Nikos Alexandris wrote:

I understand that this "way" of using grass isn't recommended for (let
us say) beginners. Nevertheless, my aim was/is to put most of this
thread in a wiki-page because I consider it as very practical.

--%<---

http://grass.osgeo.org/wiki/Working_with_GRASS_without_starting_it_explicitly

More details and issues to be added...
Nikos